WHAM

Most human motion capture from video gives you pose relative to the camera — the person walks, but the world doesn't move. WHAM solves the harder problem: recovering full 3D body motion in world coordinates from a single moving-camera video. It lifts 2D keypoint sequences to 3D SMPL parameters with a motion encoder trained on AMASS, folds in image context over time, and disentangles camera motion from human motion using SLAM (or gyro data) plus foot-contact reasoning to kill foot sliding.

The CVPR 2024 paper reports state-of-the-art results on 3DPW and EMDB for both pose and world-coordinate trajectory accuracy, and the MIT-licensed code runs fast enough for practical use — there's even a Colab demo. Real-world users have run it on everything from tennis footage to drone video. Just know its limits up front: it learns motion from AMASS, so out-of-distribution moves (cycling, skateboarding) degrade, and you'll need to register for the SMPL body models separately.

• Paper (arXiv 2312.07531): state-of-the-art on 3DPW and EMDB for world-coordinate trajectory estimation
• CVPR 2024; paperswithcode SOTA badges for 3D human pose estimation on 3DPW/EMDB