Manipulation from Demonstration

Adapting captured human motion for use with animated characters and humanoid robots
human demonstrator character robot

Project Description

Captured human motion can be a rich source of examples of manipulation tasks such as tumbling, sliding, or pivoting. However, adapting these examples for use with an animated character or robot can be challenging. One difficulty is that the problem to be solved by the character or robot is never exactly the same as that in the original demonstration. The robot will be a different size and have different degrees of freedom; its workspace may be more constrained. We may wish the character to manipulate objects of a wide variety of sizes, shapes, and frictional properties.

Despite these differences, we would like to preserve the manner in which the task is performed by the human demonstrator. It should appear to an observer that the person, the animated character, and the robot are all doing the same thing in the same way. Preserving the "feel" of the motion is important for allowing a person to act as a director, indicating the style in which a character should perform a motion. It is also important when a naive user is coaching a robot. The user may wish not only to control the task outcome, but also to control exactly how the task is performed.

Adapting an Example to New Object Geometries

We have begun by looking at the problem of adapting an example to new object geometries. We assume the task is to effect a change in object state (e.g. tumble the object 90 degrees to the right). We identify contacts and their roles, and adapt the example to the new object geometry in a way that preserves those roles. As a result of our contruction technique (described in the WAFR 2002 paper referenced below), forces applied to the new object should appear similar to those required for the demonstrated motion. In the figure below, compare the force vectors for the different objects in each column.

forces for all objects are similar


Adapting a demonstration to new geometries is a first step, but there is still much to be done. Achieving realistic reaching motion for a broader range of character and object geometries is one challenge, as is natural control of gaze direction. Perhaps more important for real robots is creating control policies that are robust to uncertainties in object placement and object properties such as mass, center of mass, and coefficient of friction. We are exploring these areas in ongoing research.



Project Team

Nancy Pollard
Last Updated: September 3, 2003