Performance Animation from Low-dimensional Control Signals

Real-time animation and control of three-dimensional human motions using low-cost and non-intrusive devices

Project description

The ability to accurately reconstruct a user's motion in real time would allow the intuitive control of characters in computer games, the control of avatars for virtual reality or electronically mediated communication, and the rapid prototyping of character animations. This project introduces an approach to performance animation that employs video cameras and a small set of retro-reflective markers to create a low-cost, easy-to-use system that might someday be practical for home use. The low-dimensional control signals from the user's performance are supplemented by a database of pre-recorded human motion. At run time, the system automatically learns a series of local models from a set of motion capture examples that are a close match to the marker locations captured by the cameras. These local models are then used to reconstruct the motion of the user as a full-body animation.

We demonstrate the power and flexibility of this approach by having users control six behaviors in real time without significant latency: walking, running, hopping, jumping, boxing, and Kendo (Japanese sword art). The reconstructed motion is based on a single large human motion database. Our experiments indicate that this approach scales well with the size and heterogeneity of the database and is robust to variations in kinematics between users. The resulting animation also captures the individual style of the user's motion through spatial-temporal interpolation of the data. Finally, we assess the quality of the reconstructed motion by comparing against ground truth data simultaneously captured with a full marker set in a commercial motion capture system.

Publications

·         Performance Animation from Low-dimensional Control Signals, ACM Transactions on Graphics (SIGGRAPH 2005)

PDF (1.6M); final video (83Mb mov clip with audio)

Videos, data and slides

  • Walking  from 6 Markers and two cameras
  • Running from 6 markers and two cameras
  • Jumping  from 6 markers and two cameras
  • Leaping  from 6 markers and two cameras
  • Boxing from 6 markers and two cameras
  • Kendo from 6 markers and two cameras
  • Transitions between different motion types
  • Animations from one camera
  • Testing on different users

Project members

Related projects

 

Acknowledgement

 

Supported in part by the NSF under Grant IIS-0205224 and IIS-0326322.


Jinxiang Chai
Last Updated:
May 12, 2005