|
|
||||||
| Home | People | Publications | Courses | Software/Data | Sponsors | |
Image-based ShavingMinh Hoai Nguyen, Jean-François Lalonde, Alexei A. Efros, and Fernando de la Torre AbstractMany categories of objects, such as human faces, can be naturally viewed as a composition of several different layers. For example, a bearded face with glasses can be decomposed into three layers: a layer for glasses, a layer for the beard and a layer for other permanent facial features. While modeling such a face with a linear subspace model could be very difficult, layer separation allows for easy modeling and modification of some certain structures while leaving others unchanged. In this paper, we present a method for automatic layer extraction and its applications to face synthesis and editing. Layers are automatically extracted by utilizing the differences between subspaces and modeled separately. We show that our method can be used for tasks such beard removal (virtual shaving), beard synthesis, and beard transfer, among others. CitationMinh Hoai Nguyen, Jean-François Lalonde, Alexei A. Efros, and Fernando de la Torre. Image-based shaving. Computer Graphics Forum Journal (Eurographics 2008), 27(2), 2008. [BiBTeX] Links
Laziness is a virtue: Motion stitching using effort minimizationLei Li, James McCann, Christos Faloutsos, and Nancy Pollard AbstractGiven two motion-capture sequences that are to be stitched together, how can we assess the goodness of the stitching? The straightforward solution, Euclidean distance, permits counter-intuitive results because it ignores the effort required to actually make the stitch. The main contribution of our work is that we propose an intuitive, first-principles approach, by computing the effort that is needed to do the transition (laziness-effort, or 'L-score'). Our conjecture is that, the smaller the effort, the more natural the transition will seem to humans. Moreover, we propose the elastic L-score which allows for elongated stitching, to make a transition as natural as possible. We present preliminary experiments on both artificial and real motions which show that our L-score approach indeed agrees with human intuition, it chooses good stitching points, and generates natural transition paths. CitationLei Li, James McCann, Christos Faloutsos, and Nancy Pollard. Laziness is a virtue: Motion stitching using effort minimization. In Short Papers Proceedings of EUROGRAPHICS, 2008. [BiBTeX] Links
Real-Time Gradient-Domain PaintingJames McCann and Nancy S. Pollard AbstractWe present an image editing program which allows artists to paint in the gradient domain with real-time feedback on megapixel-sized images. Along with a pedestrian, though powerful, gradient-painting brush and gradient-clone tool, we introduce an edge brush designed for edge selection and replay. These brushes, coupled with special blending modes, allow users to accomplish global lighting and contrast adjustments using only local image manipulations -- e.g. strengthening a given edge or removing a shadow boundary. Such operations would be tedious in a conventional intensity-based paint program and hard for users to get right in the gradient domain without real-time feedback. The core of our paint program is a simple-to-implement GPU multigrid method which allows integration of megapixel-sized full-color gradient fields at over 20 frames per second on modest hardware. By way of evaluation, we present example images produced with our program and characterize the iteration time and convergence rate of our integration method. CitationJames McCann and Nancy S. Pollard. Real-time gradient-domain painting. ACM Transactions on Graphics (SIGGRAPH 2008), 27(3), August 2008. [BiBTeX] Links
FMDistance: A fast and effective distance function for motion capture dataKensuke Onuma, Christos Faloutsos, and Jessica K. Hodgins AbstractGiven several motion capture sequences, of similar (but not identical) length, what is a good distance function? We want to find similar sequences, to spot outliers, to create clusters, and to visualize the (large) set of motion capture sequences at our disposal. We propose a set of new features for motion capture sequences. We experiment with numerous variations (112 feature-sets in total, using variations of weights, logarithms, dimensionality reduction), and we show that the appropriate combination leads to near-perfect classification on a database of 226 actions with twelve different categories, and it enables visualization of the whole database as well as outlier detection. CitationKensuke Onuma, Christos Faloutsos, and Jessica K. Hodgins. FMDistance: A fast and effective distance function for motion capture data. In Short Papers Proceedings of EUROGRAPHICS, 2008. [BiBTeX] Links
Action Capture with AccelerometersRonit Slyper and Jessica Hodgins AbstractWe create a performance animation system that leverages the power of low-cost accelerometers, readily available motion capture databases, and construction techniques from e-textiles. Our system, built with only off-the-shelf parts, consists of five accelerometers sewn into a comfortable shirt that streams data to a computer. The accelerometer readings are continuously matched against accelerations computed from existing motion capture data, and an avatar is animated with the closest match. We evaluate our system visually and using simultaneous motion and accelerometer capture. CitationRonit Slyper and Jessica Hodgins. Action capture with accelerometers. In 2008 ACM SIGGRAPH / Eurographics Symposium on Computer Animation, July 2008. [BiBTeX] Links
IM2GPS: Estimating geographic information from a single imageJames Hays and Alexei A. Efros AbstractEstimating geographic information from an image is an excellent, difficult high-level computer vision problem whose time has come. The emergence of vast amounts of geographically-calibrated image data is a great reason for computer vision to start looking globally — on the scale of the entire planet! In this paper, we propose a simple algorithm for estimating a distribution over geographic locations from a single image using a purely data-driven scene matching approach. For this task, we will leverage a dataset of over 6 million GPS-tagged images from the Internet. We represent the estimated image location as a probability distribution over the Earth's surface. We quantitatively evaluate our approach in several geolocation tasks and demonstrate encouraging performance (up to 30 times better than chance). We show that geolocation estimates can provide the basis for numerous other image understanding tasks such as population density estimation, land cover estimation or urban/rural classification. CitationJames Hays and Alexei A. Efros. Im2gps: Estimating geographic information from a single image. In Proceedings of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2008. [BiBTeX] Links
Method for determining kinematic parameters of the in vivo thumb carpometacarpal jointLillian Y. Chang and Nancy S. Pollard AbstractThe mobility of the thumb carpometacarpal (CMC) joint is critical for functional grasping and manipulation tasks. We present an optimization technique for determining from surface marker measurements a subject-specific kinematic model of the in vivo CMC joint that is suitable for measuring mobility. Our anatomy-based cost metric scores a candidate joint model by the plausibility of the corresponding joint angle values and kinematic parameters rather than only the marker trajectory reconstruction error. The proposed method repeatably determines CMC joint models with anatomically-plausible directions for the two dominant rotational axes and a lesser range of motion (RoM) for the third rotational axis. We formulate a low-dimensional parameterization of the optimization domain by first solving for joint axis orientation variables that then constrain the search for the joint axis location variables. Individual CMC joint models were determined for 24 subjects. The directions of the flexion-extension (FE) axis and adduction-abduction (AA) axis deviated on average by 9 degrees and 22 degrees, respectively, from the mean axis direction. The average RoM for FE, AA, and pronation-supination (PS) joint angles were 76 degrees, 43 degrees, and 23 degrees for active CMC movement. The mean separation distance between the FE and AA axes was 4.6 mm, and the mean skew angle was 87 degrees from the positive flexion axis to the positive abduction axis. CitationLillian Y. Chang and Nancy S. Pollard. Method for determining kinematic parameters of the in vivo thumb carpometacarpal joint. IEEE Transactions on Biomedical Engineering, 55(7):1897–1906, July 2008. [BiBTeX] Links
Accelerometer-based User Interfaces for the Control of a Physically Simulated CharacterTakaaki Shiratori and Jessica K. Hodgins AbstractIn late 2006, Nintendo released a new game controller, the Wiimote, which included a three-axis accelerometer. Since then, a large variety of novel applications for these controllers have been developed by both independent and commercial developers. We add to this growing library with three performance interfaces that allow the user to control the motion of a dynamically simulated, animated character through the motion of his or her arms, wrists, or legs. For comparison, we also implement a traditional joystick/button interface. We assess these interfaces by having users test them on a set of tracks containing turns and pits. Two of the interfaces (legs and wrists) were judged to be more immersive and were better liked than the joystick/button interface by our subjects. All three of the Wiimote interfaces provided better control than the joystick interface based on an analysis of the failures seen during the user study. CitationTakaaki Shiratori and Jessica K. Hodgins. Accelerometer-based user interfaces for the control of a physically simulated character. ACM Transactions on Graphics (SIGGRAPH Asia 2008), 27(5), December 2008. [BiBTeX] Links
Six-DoF Haptic Rendering of Contact between Geometrically Complex Reduced Deformable ModelsJernej Barbič and Doug L. James AbstractReal-time evaluation of distributed contact forces between rigid or deformable 3D objects is a key ingredient of 6-DoF force-feedback rendering. Unfortunately, at very high temporal rates, there is often insufficient time to resolve contact between geometrically complex objects. We propose a spatially and temporally adaptive approach to approximate distributed contact forces under hard real-time constraints. Our method is CPU-based and supports contact between rigid or reduced deformable models with complex geometry. We propose a contact model that uses a point-based representation for one object and a signed-distance field for the other. This model is related to the Voxmap-PointShell (VPS) method, but gives continuous contact forces and torques, enabling stable rendering of stiff penalty-based distributed contacts. We demonstrate that stable haptic interactions can be achieved by point-sampling offset surfaces to input "polygon soup" geometry using particle repulsion. We introduce a multiresolution nested pointshell construction that permits level-of-detail contact forces and enables graceful degradation of contact in close-proximity scenarios. Parametrically deformed distance fields are proposed for contact between reduced deformable objects. We present several examples of 6-DoF haptic rendering of geometrically complex rigid and deformable objects in distributed contact at real-time kilohertz rates. CitationJernej Barbič and Doug L. James. Six-dof haptic rendering of contact between geometrically complex reduced deformable models. IEEE Transactions on Haptics, 1(1):39–52, 2008. [BiBTeX] Links
What does the sky tell us about the camera?Jean-François Lalonde, Srinivasa G. Narasimhan, and Alexei A. Efros AbstractAs the main observed illuminant outdoors, the sky is a rich source of information about the scene. However, it is yet to be fully explored in computer vision because its appearance depends on the sun position, weather conditions, photometric and geometric parameters of the camera, and the location of capture. In this paper, we propose the use of a physically-based sky model to analyze the information available within the visible portion of the sky, observed over time. By fitting this model to an image sequence, we show how to extract camera parameters such as the focal length, and the zenith and azimuth angles. In short, the sky serves as a geometric calibration target. Once the camera parameters are recovered, we show how to use the same model in two applications: 1) segmentation of the sky and cloud layers, and 2) data-driven sky matching across different image sequences based on a novel similarity measure defined on sky parameters. This measure, combined with a rich appearance database, allows us to model a wide range of sky conditions. CitationJean-François Lalonde, Srinivasa G. Narasimhan, and Alexei A. Efros. What does the sky tell us about the camera? In European Conference on Computer Vision, 2008. [BiBTeX] Links
A Finite Element Method for Animating Large Viscoplastic FlowAdam W. Bargteil, Chris Wojtan, Jessica K. Hodgins, and Greg Turk AbstractWe present an extension to Lagrangian finite element methods to allow for large plastic deformations of solid materials. These behaviors are seen in such everyday materials as shampoo, dough, and clay as well as in fantastic gooey and blobby creatures in special effects scenes. To account for plastic deformation, we explicitly update the linear basis functions defined over the finite elements during each simulation step. When these updates cause the basis functions to become ill-conditioned, we remesh the simulation domain to produce a new high-quality finite-element mesh, taking care to preserve the original boundary. We also introduce an enhanced plasticity model that preserves volume and includes creep and work hardening/softening. We demonstrate our approach with simulations of synthetic objects that squish, dent, and flow. To validate our methods, we compare simulation results to videos of real materials. CitationAdam W. Bargteil, Chris Wojtan, Jessica K. Hodgins, and Greg Turk. A finite element method for animating large viscoplastic flow. ACM Transactions on Graphics (SIGGRAPH 2007), 26(3), August 2007. [BiBTeX] Links
Constrained least-squares optimization for robust estimation of center of rotationLillian Y. Chang and Nancy S. Pollard AbstractThis paper presents a new direct method for estimating the average center of rotation (CoR). An existing least-squares (LS) solution has been shown by previous works to have reduced accuracy for data with small range of motion (RoM). Alternative methods proposed to improve the CoR estimation use iterative algorithms. However, in this paper we show that with a carefully chosen normalization scheme, constrained least-squares solutions can perform as well as iterative approaches, even for challenging problems with significant noise and small RoM. In particular, enforcing the normalization constraint avoids poor fits near plane singularities that can affect the existing LS method. Our formulation has an exact solution, accounts for multiple markers simultaneously, and does not depend on manually-adjusted parameters. Simulation tests compare the method to four published CoR estimation techniques. The results show that the new approach has the accuracy of the iterative methods as well as the short computation time and repeatability of a least-squares solution. In addition, application of the new method to experimental motion capture data of the thumb carpometacarpal (CMC) joint yielded a more plausible CoR location compared to the previously reported LS solution and required less time than all four alternative techniques. CitationLillian Y. Chang and Nancy S. Pollard. Constrained least-squares optimization for robust estimation of center of rotation. Journal of Biomechanics, 40(6):1392–1400, 2007. [BiBTeX] Links
Constraint-based Motion Optimization Using A Statistical Dynamic ModelJinxiang Chai and Jessica K. Hodgins AbstractWe present a technique for generating animation from a variety of user-defined constraints. We pose constraint-based motion synthesis as a maximum a posterior (MAP) problem and develop an optimization framework that generates natural motion satisfying user constraints. The system automatically learns a statistical dynamic model from motion capture data and then enforces it as a motion prior. This motion prior, together with user-defined constraints, comprises a trajectory optimization problem. Solving this problem in the low-dimensional space yields optimal natural motion that achieves the goals specified by the user. CitationJinxiang Chai and Jessica K. Hodgins. Constraint-based motion optimization using a statistical dynamic model. ACM Transactions on Graphics (SIGGRAPH 2007), 26(3), August 2007. [BiBTeX] Links
Construction and optimal search of interpolated motion graphsAlla Safonova and Jessica K. Hodgins AbstractMany compelling applications would become feasible if novice users had the ability to synthesize high quality human motion based only on a simple sketch and a few easily specified constraints. We approach this problem by representing the desired motion as an interpolation of two time-scaled paths through a motion graph. The graph is constructed to support interpolation and pruned for efficient search. We use an anytime version of A* search to find a globally optimal solution in this graph that satisfies the user's specification. Our approach retains the natural transitions of motion graphs and the ability to synthesize physically realistic variations provided by interpolation. We demonstrate the power of this approach by synthesizing optimal or near optimal motions that include a variety of behaviors in a single motion. CitationAlla Safonova and Jessica K. Hodgins. Construction and optimal search of interpolated motion graphs. ACM Transactions on Graphics (SIGGRAPH 2007), 26(3), August 2007. [BiBTeX] Links
Data driven grasp synthesis using shape matching and task-based pruningYing Li, Jiaxin L. Fu, and Nancy S. Pollard AbstractHuman grasps, especially whole-hand grasps, are difficult to animate because of the high number of degrees of freedom of the hand and the need for the hand to conform naturally to the object surface. Captured human motion data provides us with a rich source of examples of natural grasps. However, for each new object, we are faced with the problem of selecting the best grasp from the database and adapting it to that object. This paper presents a data-driven approach to grasp synthesis. We begin with a database of captured human grasps. To identify candidate grasps for a new object, we introduce a novel shape matching algorithm that matches hand shape to object shape by identifying collections of features having similar relative placements and surface normals. This step returns many grasp candidates, which are clustered and pruned by choosing the grasp best suited for the intended task. For pruning undesirable grasps, we develop an anatomically based grasp quality measure specific to the human hand. Examples of grasp synthesis are shown for a variety of objects not present in the original database. This algorithm should be useful both as an animator tool for posing the hand and for automatic grasp synthesis in virtual environments. CitationYing Li, Jiaxin L. Fu, and Nancy S. Pollard. Data driven grasp synthesis using shape matching and task-based pruning. IEEE Transactions on Visualization and Computer Graphics, 2007. In press. [BiBTeX] Links
Face Poser: Interactive Modeling of 3D Facial Expressions Using Model PriorsManfred Lau, Jin-Xiang Chai, Ying-Qing Xu, and Heung-Yeung Shum AbstractIn this paper, we present an intuitive interface for interactively posing 3D facial expressions. The user can create and edit facial expressions by drawing freeform strokes, or by directly dragging facial points in 2D screen space. Designing such an interface for face modeling and editing is challenging because many unnatural facial expressions might be consistent with the ambiguous user input. The system automatically learns a model prior from a prerecorded facial expression database and uses it to remove the ambiguity. We formulate the problem in a maximum a posteriori (MAP) framework by combining the prior with user-defined constraints. Maximizing the posterior allows us to generate an optimal and natural facial expression that satisfies the user-defined constraints. Our system is interactive; it is also simple and easy to use. A first-time user can learn to use the system and start creating a variety of natural face models within minutes. We evaluate the performance of our approach with cross validation tests, and by comparing with alternative techniques. CitationManfred Lau, Jin-Xiang Chai, Ying-Qing Xu, and Heung-Yeung Shum. Face poser: Interactive modeling of 3d facial expressions using model priors. In 2007 ACM SIGGRAPH / Eurographics Symposium on Computer Animation, August 2007. [BiBTeX] Links
Feature Selection for Grasp Recognition from Optical MarkersLillian Y. Chang, Nancy Pollard, Tom Mitchell, and Eric P. Xing AbstractAlthough the human hand is a complex biomechanical system, only a small set of features may be necessary for observation learning of functional grasp classes. We explore how to methodically select a minimal set of hand pose features from optical marker data for grasp recognition. Supervised feature selection is used to determine a reduced feature set of surface marker locations on the hand that is appropriate for grasp classification of individual hand poses. Classifiers trained on the reduced feature set of five markers retain at least 92% of the prediction accuracy of classifiers trained on a full feature set of thirty markers. The reduced model also generalizes better to new subjects. The dramatic reduction of the marker set size and the success of a linear classifier from local marker coordinates recommend optical marker techniques as a practical alternative to data glove methods for observation learning of grasping. CitationLillian Y. Chang, Nancy Pollard, Tom Mitchell, and Eric P. Xing. Feature selection for grasp recognition from optical markers. In Proceedings of the 2007 IEEE/RSJ Intl. Conference on Intelligent Robots and Systems (IROS 2007), pages 2944–2950, October 2007. [BiBTeX] Links
Legendre Fluids: A Unified Framework for Analytic Reduced Space Modeling and Rendering of Participating MediaMohit Gupta and Srinivasa G. Narasimhan AbstractIn this paper, we present a unified framework for reduced space modeling and rendering of dynamic and non-homogenous participating media, like snow, smoke, dust and fog. The key idea is to represent the 3D spatial variation of the density, velocity and intensity fields of the media using the same analytic basis. In many situations, natural effects such as mist, outdoor smoke and dust are smooth (low frequency) phenomena, and can be compactly represented by a small number of coefficients of a Legendre polynomial basis. We derive analytic expressions for the derivative and integral operators in the Legendre coefficient space, as well as the triple product integrals of Legendre polynomials. These mathematical results allow us to solve both the Navier-Stokes equations for fluid flow and light transport equations for single scattering efficiently in the reduced Legendre space. Since our technique does not depend on volume grid resolution, we can achieve computational speedups as compared to spatial domain methods while having low memory and pre-computation requirements as compared to data-driven approaches. Also, analytic definition of derivatives and integral operators in the Legendre domain avoids the approximation errors inherent in spatial domain finite difference methods. We demonstrate many interesting visual effects resulting from particles immersed in fluids as well as volumetric scattering in non-homogenous and dynamic participating media, such as fog and mist. CitationMohit Gupta and Srinivasa G. Narasimhan. Legendre fluids: A unified framework for analytic reduced space modeling and rendering of participating media. In 2007 ACM SIGGRAPH / Eurographics Symposium on Computer Animation, August 2007. [BiBTeX] Links
Many-Worlds Browsing for Control of Multibody DynamicsChristopher D. Twigg and Doug L. James AbstractAnimation techniques for controlling passive simulation are commonly based on an optimization paradigm: the user provides goals a priori, and sophisticated numerical methods minimize a cost function that represents these goals. Unfortunately, for multibody systems with discontinuous contact events these optimization problems can be highly nontrivial to solve, and many-hour offline optimizations, unintuitive parameters, and convergence failures can frustrate end-users and limit usage. On the other hand, users are quite adaptable, and systems which provide interactive feedback via an intuitive interface can leverage the user's own abilities to quickly produce interesting animations. However, the online computation necessary for interactivity limits scene complexity in practice. We introduce Many-Worlds Browsing, a method which circumvents these limits by exploiting the speed of multibody simulators to compute numerous example simulations in parallel (offline and online), and allow the user to browse and modify them interactively. We demonstrate intuitive interfaces through which the user can select among the examples and interactively adjust those parts of the scene that don't match his requirements. We show that using a combination of our techniques, unusual and interesting results can be generated for moderately sized scenes with under an hour of user time. Scalability is demonstrated by sampling much larger scenes using modest offline computations. CitationChristopher D. Twigg and Doug L. James. Many-worlds browsing for control of multibody dynamics. ACM Transactions on Graphics (SIGGRAPH 2007), 26(3), August 2007. [BiBTeX] Links
Photo Clip ArtJean-François Lalonde, Derek Hoiem, Alexei A. Efros, Carsten Rother, John Winn, and Antonio Criminisi AbstractWe present a system for inserting new objects into existing photographs by querying a vast image-based object library, precomputed using a publicly available Internet object database. The central goal is to shield the user from all of the arduous tasks typically involved in image compositing. The user is only asked to do two simple things: 1) pick a 3D location in the scene to place a new object; 2) select an object to insert using a hierarchical menu. We pose the problem of object insertion as a data-driven, 3D-based, context-sensitive object retrieval task. Instead of trying to manipulate the object to change its orientation, color distribution, etc. to fit the new image, we simply retrieve an object of a specified class that has all the required properties (camera pose, lighting, resolution, etc) from our large object library. We present new automatic algorithms for improving object segmentation and blending, estimating true 3D object size and orientation, and estimating scene lighting conditions. We also present an intuitive user interface that makes object insertion fast and simple even for the artistically challenged. CitationJean-François Lalonde, Derek Hoiem, Alexei A. Efros, Carsten Rother, John Winn, and Antonio Criminisi. Photo clip art. ACM Transactions on Graphics (SIGGRAPH 2007), 26(3), August 2007. [BiBTeX] Links
Responsive Characters from Motion FragmentsJames McCann and Nancy S. Pollard AbstractIn game environments, animated character motion must rapidly adapt to changes in player input -- for example, if a directional signal from the player's gamepad is not incorporated into the character's trajectory immediately, the character may blithely run off a ledge. Traditional schemes for data-driven character animation lack the split-second reactivity required for this direct control; while they can be made to work, motion artifacts will result. We describe an on-line character animation controller that assembles a motion stream from short motion fragments, choosing each fragment based on current player input and the previous fragment. By adding a simple model of player behavior we are able to improve an existing reinforcement learning method for precalculating good fragment choices. We demonstrate the efficacy of our model by comparing the animation selected by our new controller to that selected by existing methods and to the optimal selection, given knowledge of the entire path. This comparison is performed over real-world data collected from a game prototype. Finally, we provide results indicating that occasional low-quality transitions between motion segments are crucial to high-quality on-line motion generation; this is an important result for others crafting animation systems for directly-controlled characters, as it argues against the common practice of transition thresholding. CitationJames McCann and Nancy S. Pollard. Responsive characters from motion fragments. ACM Transactions on Graphics (SIGGRAPH 2007), 26(3), August 2007. [BiBTeX] Links
Robust estimation of dominant axis of rotationLillian Y. Chang and Nancy Pollard AbstractA simple method is developed for robustly estimating a fixed dominant axis of rotation (AoR) of anatomical joints from surface marker data. Previous approaches which assume a model of circular marker trajectories use plane-fitting to estimate the direction of the AoR. However, when there is limited joint range of motion and rotation due to a second degree of freedom, minimizing only the planar error can give poor estimates of the AoR direction. Optimizing a cost function which includes the error component within a plane, instead of only the component orthogonal to a plane, leads to improved estimates of the AoR direction for joints which exhibit additional rotational motion from a second degree of freedom. Results from synthetic data validation show the ranges of motion where the new method has lower estimation error compared to plane-fitting techniques. Estimates of the flexion-extension AoR from empirical motion capture data of the knee and index finger joints were also more anatomically plausible. CitationLillian Y. Chang and Nancy Pollard. Robust estimation of dominant axis of rotation. Journal of Biomechanics, 40(12):2707–2715, 2007. [BiBTeX] Links
Scene Completion Using Millions of PhotographsAbstractWhat can you do with a million images? In this paper we present a new image completion algorithm powered by a huge database of photographs gathered from the Web. The algorithm patches up holes in images by finding similar image regions in the database that are not only seamless but also semantically valid. Our chief insight is that while the space of images is effectively infinite, the space of semantically differentiable scenes is actually not that large. For many image completion tasks we are able to find similar scenes which contain image fragments that will convincingly complete the image. Our algorithm is entirely data-driven, requiring no annotations or labelling by the user. Unlike existing image completion methods, our algorithm can generate a diverse set of results for each input image and we allow users to select among them. We demonstrate the superiority of our algorithm over existing image completion approaches. CitationJames Hays and Alexi Efros. Scene completion using millions of photographs. ACM Transactions on Graphics (SIGGRAPH 2007), 26(3), August 2007. [BiBTeX] Links
Time-critical distributed contact for 6-DoF haptic rendering of adaptively sampled reduced deformable modelsJernej Barbič and Doug L. James AbstractReal-time evaluation of distributed contact forces for rigid or deformable 3D objects is important for providing multi-sensory feedback in emerging real-time applications, such as 6-DoF haptic force-feedback rendering. Unfortunately, at very high temporal rates (1 kHz for haptics), there is often insufficient time to resolve distributed contact between geometrically complex objects. In this paper, we present a spatially and temporally adaptive sample-based approach to approximate contact forces under hard real-time constraints. The approach is CPU based, and supports contact between a rigid and a reduced deformable model with complex geometry. Penalty-based contact forces are efficiently resolved using a multi-resolution point-based representation for one object, and a signed-distance field for the other. Hard real-time approximation of distributed contact forces uses multi-level progressive point-contact sampling, and exploits temporal coherence, graceful degradation and other optimizations. We present several examples of 6-DoF haptic rendering of geometrically complex rigid and deformable objects in distributed contact at real-time kilohertz rates. CitationJernej Barbič and Doug L. James. Time-critical distributed contact for 6-dof haptic rendering of adaptively sampled reduced deformable models. In 2007 ACM SIGGRAPH / Eurographics Symposium on Computer Animation, August 2007. [BiBTeX] Links
Using Color Compatibility for Assessing Image RealismJean-François Lalonde and Alexei A. Efros AbstractWhy does placing an object from one photograph into another often make the colors of that object suddenly look wrong? One possibility is that humans prefer distributions of colors that are often found in nature; that is, we find pleasing these color combinations that we see often. Another possibility is that humans simply prefer colors to be consistent within an image, regardless of what they are. In this paper, we explore some of these issues by studying the color statistics of a large dataset of natural images, and by looking at differences in color distribution in realistic and unrealistic images. We apply our findings to two problems: 1) classifying composite images into realistic vs. non-realistic, and 2) recoloring image regions for realistic compositing. CitationJean-François Lalonde and Alexei A. Efros. Using color compatibility for assessing image realism. IEEE International Conference on Computer Vision, 2007. [BiBTeX] Links
Anthropomorphism influences perception of computer-animated characters' actionsThierry Chaminade, Jessica Hodgins, and Mitsuo Kawato AbstractComputer-animated characters are common in popular culture and have begun to be used as experimental tools in social cognitive neurosciences. Here we investigated how appearance of these characters' influences perception of their actions. Subjects were presented with different characters animated either with motion data captured from human actors or by interpolating between poses (keyframes) designed by an animator, and were asked to categorize the motion as biological or artificial. The response bias towards 'biological', derived from the Signal Detection Theory, decreases with characters' anthropomorphism, while sensitivity is only affected by the simplest rendering style, point-light displays. fMRI showed that the response bias correlates positively with activity in the mentalizing network including left temporoparietal junction and anterior cingulate cortex, and negatively with regions sustaining motor resonance. The absence of significant effect of the characters on the brain activity suggests individual differences in the neural responses to unfamiliar artificial agents. While computer-animated characters are invaluable tools to investigate the neural bases of social cognition, further research is required to better understand how factors such as anthropomorphism affect their perception, in order to optimize their appearance for entertainment, research or therapeutic purposes. CitationThierry Chaminade, Jessica Hodgins, and Mitsuo Kawato. Anthropomorphism influences perception of computer-animated characters' actions. Social Cognitive and Affective Neuroscience, May 2007. [BiBTeX] Links
Interactive Tensor Field Design and Visualization on SurfacesEugene Zhang, James Hays, and Greg Turk AbstractDesigning tensor fields in the plane and on surfaces is a necessary task in many graphics applications, such as painterly rendering, pen-and-ink sketch of smooth surfaces, and anisotropic remeshing. In this paper, we present an interactive design system that allows a user to create a wide variety of surface tensor fields with control over the number and location of degenerate points. Our system combines basis tensor fields to make an initial tensor field that satisfies a set of userspecifications. However, such a field often contains unwanted degenerate points that cannot always be eliminated due to topological constraints of the underlying surface. To reduce the artifacts caused by these degenerate points, our system allows the user to move a degenerate point or to cancel a pair of degenerate points that have opposite tensor indices. CitationEugene Zhang, James Hays, and Greg Turk. Interactive tensor field design and visualization on surfaces. IEEE Transactions on Visualization and Computer Graphics, 13(1):94–107, January 2007. [BiBTeX] Links
A Texture Synthesis Method for Liquid AnimationsAdam W. Bargteil, Funshing Sin, Jonathan E. Michaels, Tolga G. Goktekin, and James F. O'Brien AbstractIn this paper we present a method for synthesizing textures on animated liquid surfaces generated by a physically based fluid simulation system. Rather than advecting texture coordinates on the surface, we synthesize a new texture for every frame. We synthesize the texture with an optimization procedure which attempts to match the surface texture to an input sample texture. By synthesizing a new texture for every frame, our method is able to overcome the discontinuities and distortions of an advected parameterization. We achieve temporal coherence by initializing the surface texture with color values advected from the surface at the previous frame and including these colors in the energy function used during optimization. CitationAdam W. Bargteil, Funshing Sin, Jonathan E. Michaels, Tolga G. Goktekin, and James F. O'Brien. A texture synthesis method for liquid animations. In 2006 ACM SIGGRAPH / Eurographics Symposium on Computer Animation, September 2006. [BiBTeX] Links
Physics-Based Motion RetimingJames McCann, Nancy S. Pollard, and Siddhartha S. Srinivasa AbstractBy changing only the playback timing of a motion sequence, an animator can achieve a variety of effects that alter our perception of an event. In some scenarios, it may be important to consider physical properties of the motion when retiming (e.g., to preserve physical plausibility). However, existing retiming solutions can be quite time consuming when physical parameters are considered. This paper presents an interactive method for creating optimal motion retimings that takes into account physically based constraints and objective functions. We achieve fast performance through a precomputation phase where constraints are projected into the two-dimensional space of velocities and accelerations along the input motion path. Unlike previous approaches, our precomputation technique allows for rapid computation of plausible contact forces that result from retiming, and it also accommodates changing physical parameters. We demonstrate our approach by creating physically plausible results for changes in motion duration, manipulations of the gravity vector, and modifications of character limb masses. CitationJames McCann, Nancy S. Pollard, and Siddhartha S. Srinivasa. Physics-based motion retiming. In 2006 ACM SIGGRAPH / Eurographics Symposium on Computer Animation, September 2006. [BiBTeX] Links
Precomputed Search Trees: Planning for Interactive Goal-Driven AnimationManfred Lau and James J. Kuffner AbstractWe present a novel approach for interactively synthesizing motions for characters navigating in complex environments. We focus on the runtime efficiency for motion generation, thereby enabling the interactive animation of a large number of characters simultaneously. The key idea is to precompute search trees of motion clips that can be applied to arbitrary environments. Given a navigation goal relative to a current body position, the best available solution paths and motion sequences can be efficiently extracted during runtime through a series of table lookups. For distant start and goal positions, we first use a fast coarse-level planner to generate a rough path of intermediate sub-goals to guide each iteration of the runtime lookup phase. We demonstrate the efficiency of our technique across a range of examples in an interactive application with multiple autonomous characters navigating in dynamic environments. Each character responds in real-time to arbitrary user changes to the environment obstacles or navigation goals. The runtime phase is more than two orders of magnitude faster than existing planning methods or traditional motion synthesis techniques. Our technique is not only useful for autonomous motion generation in games, virtual reality, and interactive simulations, but also for animating massive crowds of characters offline for special effects in movies. CitationManfred Lau and James J. Kuffner. Precomputed search trees: Planning for interactive goal-driven animation. In 2006 ACM SIGGRAPH / Eurographics Symposium on Computer Animation, pages 299–308, September 2006. [BiBTeX] Links
Acquiring Scattering Properties of Participating Media by DilutionSrinivasa Narasimhan, Mohit Gupta, Craig Donner, Ravi Ramamoorthi, Shree K. Nayar, and Henrik Wann Jensen AbstractThe visual world around us displays a rich set of volumetric effects due to participating media. The appearance of these media is governed by several physical properties such as particle densities, shapes and sizes, which must be input (directly or indirectly) to a rendering algorithm to generate realistic images. While there has been significant progress in developing rendering techniques (for instance, volumetric Monte Carlo methods and analytic approximations), there are very few methods that measure or estimate these properties for media that are of relevance to computer graphics. In this paper, we present a simple device and technique for robustly estimating the properties of a broad class of participating media that can be either (a) diluted in water such as juices, beverages, paints and cleaning supplies, or (b) dissolved in water such as powders and sugar/salt crystals, or (c) suspended in water such as impurities. The key idea is to dilute the concentrations of the media so that single scattering effects dominate and multiple scattering becomes negligible, leading to a simple and robust estimation algorithm. Furthermore, unlike previous approaches that require complicated or separate measurement setups for different types or properties of media, our method and setup can be used to measure media with a complete range of absorption and scattering properties from a single HDR photograph. Once the parameters of the diluted medium are estimated, a volumetric Monte Carlo technique may be used to create renderings of any medium concentration and with multiple scattering. We have measured the scattering parameters of forty commonly found materials, that can be immediately used by the computer graphics community. We can also create realistic images of combinations or mixtures of the original measured materials, thus giving the user a wide flexibility in making realistic images of participating media. CitationSrinivasa Narasimhan, Mohit Gupta, Craig Donner, Ravi Ramamoorthi, Shree K. Nayar, and Henrik Wann Jensen. Acquiring scattering properties of participating media by dilution. ACM Transactions on Graphics (SIGGRAPH 2006), 25(3), August 2006. [BiBTeX] Links
Capturing and Animating Skin Deformation in Human MotionSang Il Park and Jessica K. Hodgins AbstractDuring dynamic activities, the surface of the human body moves in many subtle but visually significant ways: bending, bulging, jiggling, and stretching. We present a technique for capturing and animating those motions using a commercial motion capture system and approximately 350 markers. Although the number of markers is significantly larger than that used in conventional motion capture, it is only a sparse representation of the true shape of the body. We supplement this sparse sample with a detailed, actor-specific surface model. The motion of the skin can then be computed by segmenting the markers into the motion of a set of rigid parts and a residual deformation (approximated first as a quadratic transformation and then with radial basis functions). We demonstrate the power of this approach by capturing flexing muscles, high frequency motions, and abrupt decelerations on several actors. We compare these results both to conventional motion capture and skinning and to synchronized video of the actors. CitationSang Il Park and Jessica K. Hodgins. Capturing and animating skin deformation in human motion. ACM Transactions on Graphics (SIGGRAPH 2006), 25(3), August 2006. [BiBTeX] Links
Hierarchical Simplification of City Models to Maintain Urban LegibilityRemco Chang, Thomas Butkiewicz, Caroline Ziemkiewicz, Zachary Wartell, Nancy S. Pollard, and William Ribarsky AbstractFor 3D global visualization systems such as Google Earth, it is important to be able to render city-sized collections of relatively simple building models at fast speeds without losing spatial coherence. Since traditional mesh simplification algorithms are not designed for collections of simple models, we introduce a method of simplification through merging of similar objects. We incorporate the concept of "urban legibility" from architecture and city-planning as a guideline for simplifying city models. Our algorithm can be broken down into five steps. Hierarchical clustering, cluster merging, polyline simplification, and hierarchical texturing are performed during pre-processing, while at runtime, the levels-of-detail (LOD) process selects the appropriate models to render. It is our belief that many applications can benefit from our algorithm. Google Earth (and other 3D geographical information systems) as well as any spatial data visualization applications (including scatter plots) can all use logical, simplified clusters to represent large amounts of spatial information. CitationRemco Chang, Thomas Butkiewicz, Caroline Ziemkiewicz, Zachary Wartell, Nancy S. Pollard, and William Ribarsky. Hierarchical simplification of city models to maintain urban legibility. In Proceedings of the SIGGRAPH 2006 Conference on Sketches & Applications. ACM Press, August 2006. [BiBTeX] Links
Mesh Ensemble Motion Graphs: Data-driven Mesh Animation with ConstraintsDoug L. James, Christopher D. Twigg, Andrew Cove, and Robert Y. Wang AbstractWe describe a technique for using space-time cuts to smoothly transition between stochastic mesh animation clips involving numerous deformable mesh groups while subject to physical constraints. These transitions are used to construct Mesh Ensemble Motion Graphs for interactive data-driven animation of high-dimensional mesh animation datasets, such as those arising from expensive physical simulations of deformable objects blowing in the wind. We formulate the transition computation as an integer programming problem, and introduce a novel randomized algorithm to compute transitions subject to geometric noninterpenetration constraints. CitationDoug L. James, Christopher D. Twigg, Andrew Cove, and Robert Y. Wang. Mesh ensemble motion graphs: Data-driven mesh animation with constraints. In Proceedings of the SIGGRAPH 2006 Conference on Sketches & Applications. ACM Press, August 2006. [BiBTeX] Links
Precomputed Acoustic Transfer: Output-sensitive, accurate sound generation for geometrically complex vibration sourcesDoug L. James, Jernej Barbič, and Dinesh K. Pai AbstractSimulating sounds produced by realistic vibrating objects is challenging because sound radiation involves complex diffraction and interreflection effects that are very perceptible and important. These wave phenomena are well understood, but have been largely ignored in computer graphics due to the high cost and complexity of computing them at audio rates. We describe a new algorithm for real-time synthesis of realistic sound radiation from rigid objects. We start by precomputing the linear vibration modes of an object, and then relate each mode to its sound pressure field, or acoustic transfer function, using standard methods from numerical acoustics. Each transfer function is then approximated to a specified accuracy using low-order multipole sources placed near the object. We provide a low-memory, multilevel, randomized algorithm for optimized source placement that is suitable for complex geometries. At runtime, we can simulate new interaction sounds by quickly summing contributions from each modes equivalent multipole sources. We can efficiently simulate global effects such as interreflection and changes in sound due to listener location. The simulation costs can be dynamically traded-off for sound quality. We present several examples of sound generation from physically based animations. CitationDoug L. James, Jernej Barbič, and Dinesh K. Pai. Precomputed acoustic transfer: Output-sensitive, accurate sound generation for geometrically complex vibration sources. ACM Transactions on Graphics (SIGGRAPH 2006), 25(3), August 2006. [BiBTeX] Links
Quantitative Evaluation on Near Regular Texture SynthesisWen-Chieh Lin, James H. Hays, Chenyu Wu, Vivek Kwatra, and Yanxi Liu AbstractNear regular textures are pervasive in man-made and natural world. Their global regularity and local randomness pose new difficulties to the state of the art texture analysis and synthesis algorithms. We carry out a systematic comparison study on the performance of four texture synthesis algorithms on near-regular textures. Our results confirm that faithful near-regular texture synthesis remains a challenging problem for the state of the art general purpose texture synthesis algorithms. In addition, we provide comparison of human perception with computer evaluations on the quality of the texture synthesis results. CitationWen-Chieh Lin, James H. Hays, Chenyu Wu, Vivek Kwatra, and Yanxi Liu. Quantitative evaluation on near regular texture synthesis. In Computer Vision and Pattern Recognition Conference (CVPR '06), volume 1, pages 427–434, June 2006. [BiBTeX] Links
Discovering Texture Regularity as a Higher-Order Correspondence ProblemJames H. Hays, Marius Leordeanu, Alexei A. Efros, and Yanxi Liu AbstractUnderstanding texture regularity in real images is a challenging computer vision task. We propose a higher-order feature matching algorithm to discover the lattices of near-regular textures in real images. The underlying lattice of a near-regular texture identifies all of the texels as well as the global topology among the texels. A key contribution of this paper is to formulate lattice-finding as a correspondence problem. The algorithm finds a plausible lattice by iteratively proposing texels and assigning neighbors between the texels. Our matching algorithm seeks assignments that maximize both pair-wise visual similarity and higher-order geometric consistency. We approximate the optimal assignment using a recently developed spectral method. We successfully discover the lattices of a diverse set of unsegmented, real-world textures with significant geometric warping and large appearance variation among texels. CitationJames H. Hays, Marius Leordeanu, Alexei A. Efros, and Yanxi Liu. Discovering texture regularity as a higher-order correspondence problem. In 9th European Conference on Computer Vision, May 2006. [BiBTeX] Links
Learning Silhouette Features for Control of Human MotionLiu Ren, Gregory Shakhnarovich, Jessica K. Hodgins, Hanspeter Pfister, and Paul Viola AbstractWe present a vision-based performance interface for controlling animated human characters. The system combines information about the user's motion contained in silhouettes from several viewpoints with domain knowledge contained in a motion capture database to interactively produce a high quality animation. Such an interactive system will be useful for authoring, teleconferencing, or as a control interface for a character in a game. In our system, the user performs in front of three video cameras; the resulting silhouettes are used to estimate his or her orientation and body configuration based on a set of discriminative local features. Those features are selected by a machine learning algorithm during a preprocessing step. Sequences of motions that approximate the user's actions are extracted from the motion database and scaled in time to match the speed of the user's motion. We use swing dancing, an example of complex human motion, to demonstrate the effectiveness of our approach and compare the results obtained with discriminative local features to those obtained with global features, Hu moments, and to ground truth measurement from a motion capture system. CitationLiu Ren, Gregory Shakhnarovich, Jessica K. Hodgins, Hanspeter Pfister, and Paul Viola. Learning silhouette features for control of human motion. ACM Transactions on Graphics, 24(4), October 2005. [BiBTeX] Links
Oral FixationsJames Duesing, Jessica K. Hodgins, Bum Lee, Moshe Mahler, Jay Oberski, Sang Il Park, and David Tinapple AbstractOral Fixations is a single channel video installation that evolves over a seven hour time period. The project is a darkly humorous look at a habit of endless consumption and the resulting accumulation of waste. A narrative gradually emerges from the on-screen action that depicts a large mouthed character who dances while flossing its one protruding tooth. A conveyor belt regularly delivers factory-farm fresh hams that the character delights in taking one large bite from and then tossing aside. Over the duration of the piece the hams begin to pile up in the room until at the end of seven hours the room is filled with the refuse of this gluttony. The viewer is encouraged to revisit the piece periodically throughout the day and see how the discarded hams build an oddly humorous environment of waste around the character. CitationJames Duesing, Jessica K. Hodgins, Bum Lee, Moshe Mahler, Jay Oberski, Sang Il Park, and David Tinapple. Oral fixations. In SIGGRAPH '05: ACM SIGGRAPH 2005 Electronic Art and Animation Catalog, pages 118–119, New York, NY, USA, 2005. ACM Press. [BiBTeX] Links
A Data-Driven Approach to Quantifying Natural Human MotionLiu Ren, Alton Patrick, Alexei A. Efros, Jessica K. Hodgins, and James M. Rehg AbstractIn this paper, we investigate whether it is possible to develop a measure that quantifies the naturalness of human motion (as defined by a large database). Such a measure might prove useful in verifying that a motion editing operation had not destroyed the naturalness of a motion capture clip or that a synthetic motion transition was within the space of those seen in natural human motion. We present an ensemble approach to attack this problem. In our approach, we first hierarchically decompose human motion into its constituent parts (individual joints, limbs, and full body) and build a statistical model of each one using existing machine learning techniques such as mixture of Gaussians (MoG), hidden Markov models (HMM), and switching linear dynamic systems (SLDS). We then combine these models into an ensemble model for classification of the motion as natural or unnatural. We also implement a Naive Bayes (NB) model for a baseline comparison. We test these techniques on motion capture data held out from a database, keyframed motions, edited motions, motions with noise added, and synthetic motion transitions. We present the results as receiver operating characteristic (ROC) curves and compare the results to the judgments made by subjects in a user study. CitationLiu Ren, Alton Patrick, Alexei A. Efros, Jessica K. Hodgins, and James M. Rehg. A data-driven approach to quantifying natural human motion. ACM Transactions on Graphics (SIGGRAPH 2005), 24(3), August 2005. [BiBTeX] Links
A Practical Analytic Single Scattering Model for Real Time RenderingBo Sun, Ravi Ramamoorthi, Srinivasa Narasimhan, and Shree K. Nayar AbstractWe consider real-time rendering of scenes in participating media, capturing the effects of light scattering in fog, mist and haze. While a number of sophisticated approaches based on Monte Carlo and finite element simulation have been developed, those methods do not work at interactive rates. The most common real-time methods are essentially simple variants of the OpenGL fog model. While easy to use and specify, that model excludes many important qualitative effects like glows around light sources, the impact of volumetric scattering on the appearance of surfaces such as the diffusing of glossy highlights, and the appearance under complex lighting such as environment maps. In this paper, we present an alternative physically based approach that captures these effects while maintaining real-time performance and the ease-of-use of the OpenGL fog model. Our method is based on an explicit analytic integration of the single scattering light transport equations for an isotropic point light source in a homogeneous participating medium. We can implement the model in modern programmable graphics hardware using a few small numerical lookup tables stored as texture maps. Our model can also be easily adapted to generate the appearances of materials with arbitrary BRDFs, environment map lighting, and precomputed radiance transfer methods, in the presence of participating media. Hence, our techniques can be widely used in real-time rendering applications. CitationBo Sun, Ravi Ramamoorthi, Srinivasa Narasimhan, and Shree K. Nayar. A practical analytic single scattering model for real time rendering. ACM Transactions on Graphics (SIGGRAPH 2005), 24(3), August 2005. [BiBTeX] LinksAnalyzing the Physical Correctness of Interpolated Human MotionAlla Safonova and Jessica K. Hodgins AbstractTwo human motions can be linearly interpolated to produce a new motion, giving the animator control over the length of a jump, the speed of walking, or the height of a kick. Over the past ten years, this simple technique has been shown to produce surprisingly natural looking results. In this paper, we analyze the motions produced by this technique for physical correctness and suggest small modifications to the standard interpolation technique that in some circumstances will produce significantly more natural looking motion. CitationAlla Safonova and Jessica K. Hodgins. Analyzing the physical correctness of interpolated human motion. In 2005 ACM SIGGRAPH / Eurographics Symposium on Computer Animation, August 2005. [BiBTeX] Links
Automatic Photo Pop-upDerek Hoiem, Alexei A. Efros, and Martial Hebert AbstractOur system automatically constructs simple "pop-up" 3D models, like those one would find in a children's book, out of a single outdoor image. The labels each region of an outdoor image as "ground", "vertical", or "sky". Line segments fitted to the ground-vertical boundary in the image and an estimate of the horizon's position provide the necessary information to determine where to "cut" and "fold" in the image. After cutting and folding, the model is popped up, and the image is texture mapped onto the model. CitationDerek Hoiem, Alexei A. Efros, and Martial Hebert. Automatic photo pop-up. ACM Transactions on Graphics (SIGGRAPH 2005), 24(3), August 2005. [BiBTeX] Links
Behavior Planning for Character AnimationManfred Lau and James J. Kuffner AbstractThis paper explores a behavior planning approach to automatically generate realistic motions for animated characters. Motion clips are abstracted as high-level behaviors and associated with a behavior finite-state machine (FSM) that defines the movement capabilities of a virtual character. During runtime, motion is generated automatically by a planning algorithm that performs a global search of the FSM and computes a sequence of behaviors for the character to reach a user-designated goal position. Our technique can generate interesting animations using a relatively small amount of data, making it attractive for resource-limited game platforms. It also scales efficiently to large motion databases, because the search performance is primarily dependent on the complexity of the behavior FSM rather than on the amount of data. Heuristic cost functions that the planner uses to evaluate candidate motions provide a flexible framework from which an animator can control character preferences for certain types of behavior. We show results of synthesized animations involving up to one hundred human and animal characters planning simultaneously in both static and dynamic environments. CitationManfred Lau and James J. Kuffner. Behavior planning for character animation. In 2005 ACM SIGGRAPH / Eurographics Symposium on Computer Animation, pages 271–280, August 2005. [BiBTeX] Links
Performance Animation from Low-dimensional Control SignalsJinxiang Chai and Jessica K. Hodgins AbstractThe ability to accurately reconstruct a user's motion in real time would allow the intuitive control of characters in computer games, the control of avatars for virtual reality or electronically mediated communication, and the rapid prototyping of character animations. This project introduces an approach to performance animation that employs video cameras and a small set of retro-reflective markers to create a low-cost, easy-to-use system that might someday be practical for home use. The low-dimensional control signals from the user's performance are supplemented by a database of pre-recorded human motion. At run time, the system automatically learns a series of local models from a set of motion capture examples that are a close match to the marker locations captured by the cameras. These local models are then used to reconstruct the motion of the user as a full-body animation. We demonstrate the power and flexibility of this approach by having users control six behaviors in real time without significant latency: walking, running, hopping, jumping, boxing, and Kendo (Japanese sword art). The reconstructed motion is based on a single large human motion database. Our experiments indicate that this approach scales well with the size and heterogeneity of the database and is robust to variations in kinematics between users. The resulting animation also captures the individual style of the user's motion through spatial-temporal interpolation of the data. Finally, we assess the quality of the reconstructed motion by comparing against ground truth data simultaneously captured with a full marker set in a commercial motion capture system. CitationJinxiang Chai and Jessica K. Hodgins. Performance animation from low-dimensional control signals. ACM Transactions on Graphics (SIGGRAPH 2005), 24(3), August 2005. [BiBTeX] Links
Physically Based Grasping Control from ExampleNancy S. Pollard and Victor B. Zordan AbstractAnimated human characters in everyday scenarios must interact with the environment using their hands. Captured human motion can provide a database of realistic examples. However, examples involving contact are difficult to edit and retarget; realism can suffer when a grasp does not appear secure or when an apparent impact does not disturb the hand or the object. Physically based simulations can preserve plausibility through simulating interaction forces. However, such physical models must be driven by a controller, and creating effective controllers for new motion tasks remains a challenge. In this project, we present a controller for physically based grasping that draws from motion capture data. Our controller explicitly includes passive and active components to uphold compliant yet controllable motion, and it adds compensation for movement of the arm and for gravity to make the behavior of passive and active components less dependent on the dynamics of arm motion. Given a set of motion capture grasp examples, our system solves for all but a small set of parameters for this controller automatically. We demonstrate results for tasks including grasping and two-hand interaction and show that a controller derived from a single motion capture example can be used to form grasps of different object geometries. CitationNancy S. Pollard and Victor B. Zordan. Physically based grasping control from example. In 2005 ACM SIGGRAPH / Eurographics Symposium on Computer Animation, August 2005. [BiBTeX] Links
Real-Time Subspace Integration for St. Venant-Kirchhoff Deformable ModelsJernej Barbič and Doug L. James AbstractIn this paper, we present an approach for fast subspace integration of reduced-coordinate nonlinear deformable models that is suitable for interactive applications in computer graphics and haptics. Our approach exploits dimensional model reduction to build reduced-coordinate deformable models for objects with complex geometry. We exploit the fact that model reduction on large deformation models with linear materials (as commonly used in graphics) result in internal force models that are simply cubic polynomials in reduced coordinates. Coefficients of these polynomials can be precomputed, for efficient runtime evaluation. This allows simulation of nonlinear dynamics using fast implicit Newmark subspace integrators, with subspace integration costs independent of geometric complexity. We present two useful approaches for generating low-dimensional subspace bases: modal derivatives and an interactive sketching technique. Mass-scaled principal component analysis (mass-PCA) is suggested for dimensionality reduction. Finally, several examples are given from computer animation to illustrate high performance, including force-feedback haptic rendering of a complicated object undergoing large deformations. CitationJernej Barbič and Doug L. James. Real-time subspace integration for St. Venant-Kirchhoff deformable models. ACM Transactions on Graphics (SIGGRAPH 2005), 24(3):982–990, August 2005. [BiBTeX] Links
Skinning Mesh AnimationsDoug L. James and Christopher D. Twigg AbstractWe extend approaches for skinning characters to the general setting of skinning deformable mesh animations. We provide an automatic algorithm for generating progressive skinning approximations, that is particularly efficient for pseudo-articulated motions. Our contributions include the use of nonparametric mean shift clustering of high-dimensional mesh rotation sequences to automatically identify statistically relevant bones, and robust least squares methods to determine bone transformations, bone-vertex influence sets, and vertex weight values. We use a low-rank data reduction model defined in the undeformed mesh configuration to provide progressive convergence with a fixed number of bones. We show that the resulting skinned animations enable efficient hardware rendering, rest pose editing, and deformable collision detection. Finally, we present numerous examples where skins were automatically generated using a single set of parameter values. CitationDoug L. James and Christopher D. Twigg. Skinning mesh animations. ACM Transactions on Graphics (SIGGRAPH 2005), 24(3), August 2005. [BiBTeX] Links
Vortex Fluid for Gaseous PhenomenaSang Il Park and Myoung Jun Kim AbstractIn this paper, we present a method for visual simulation of gaseous phenomena based on the vortex method. Using a localized vortex flow as a basic building block, the whole flow field is described. As a result, we generate more dynamic swirling fluid flows while achieving computational efficiency by concentrating only on the localized vorticity region during the simulation. Based on the lagrangian framework, we consider various boundary conditions. By exploiting the panel method, we satisfy the no-through boundary condition in a Lagrangian way. A simple and effective way of handling the no-slip boundary condition is also presented. In treating the no-slip boundary condition, we allow a user to assign the roughness of the boundary surface, which further improves visual realism. CitationSang Il Park and Myoung Jun Kim. Vortex fluid for gaseous phenomena. In 2005 ACM SIGGRAPH / Eurographics Symposium on Computer Animation, August 2005. [BiBTeX] Links
Hardware-Accelerated Adaptive EWA Volume SplattingWei Chen, Liu Ren, Matthias Zwicker, and Hanspeter Pfister Abstractwe present a hardware-accelerated adaptive EWA(elliptical weighted average) volume splatting algorithm. EWA splatting combines a Gaussian reconstruction kernel with a low-pass image filter for high image quality without aliasing artifacts or excessive blurring. We introduce a novel adaptive filtering scheme to reduce the computational cost of EWA splatting. We show how this algorithm can be efficiently implemented on modern graphics processing units (GPUs). Our implementation includes interactive classification and fast lighting. To accelerate the rendering we store splat geometry and 3D volume data locally in GPU memory. We present results for several rectilinear volume datasets that demonstrate the high image quality and interactive rendering speed of our method. CitationWei Chen, Liu Ren, Matthias Zwicker, and Hanspeter Pfister. Hardware-accelerated adaptive EWA volume splatting. In Proceedings of IEEE Visualization 2004, October 2004. [BiBTeX] Links
Autonomous Behaviors for Interactive Vehicle AnimationsJared Go, Thuc Vu, and James J. Kuffner AbstractWe present a method for synthesizing animations of autonomous space, water, and land-based vehicles in games or other interactive simulations. Controlling the motion of such vehicles to achieve a desirable behavior is difficult due to the constraints imposed by the system dynamics. We combine real-time path planning and a simplified physics model to automatically compute control actions to drive a vehicle from an input state to desirable output states based on a behavior cost function. Both offline trajectory preprocessing and online search are used to build an animation framework suitable for interactive vehicle simulations. We demonstrate synthesized animations of spacecraft performing a variety of autonomous behaviors, including Seek, Pursue, Avoid, Avoid Collision, and Flee. We also explore several enhancements to the basic planning algorithm and examine the resulting tradeoffs in runtime performance and quality of the generated motion. CitationJared Go, Thuc Vu, and James J. Kuffner. Autonomous behaviors for interactive vehicle animations. In 2004 ACM SIGGRAPH / Eurographics Symposium on Computer Animation, August 2004. [BiBTeX] Links
Evaluating Motion Graphs for Character NavigationPaul S. A. Reitsma and Nancy S. Pollard AbstractRealistic and directable humanlike characters are an ongoing goal in animation. Motion graph data structures hold much promise for achieving this goal. However, the quality of the results obtained from a motion graph may not be easy to predict from the input motion segments. This paper introduces the idea of assessing a data structure such as a motion graph for its utility in a particular application. We focus on navigation tasks and define metrics for evaluating expected path quality and coverage for a given environment. One key to evaluating a motion graph for navigation tasks is to first embed it into the environment in a way that captures all possible paths that might result from playing back the motion graph within that environment. This paper describes an algorithm for accomplishing this embedding that preserves the flexibility of the original motion graph. We use the metrics defined in this paper to compare motion datasets and to highlight areas where these datasets could be improved. CitationPaul S. A. Reitsma and Nancy S. Pollard. Evaluating motion graphs for character navigation. In 2004 ACM SIGGRAPH / Eurographics Symposium on Computer Animation, August 2004. [BiBTeX] Links
Flow-based Video Synthesis and EditingKiran S. Bhat, Steven M. Seitz, Jessica K. Hodgins, and Pradeep K. Khosla AbstractThis paper presents a novel algorithm for synthesizing and editing video of natural phenomena that exhibit continuous flow patterns. The algorithm analyzes the motion of textured particles in the input video along user-specified flow lines, and synthesizes seamless video of arbitrary length by enforcing temporal continuity along a second set of user-specified flow lines. The algorithm is simple to implement and use. We used this technique to edit video of waterfalls, rivers, flames, and smoke. CitationKiran S. Bhat, Steven M. Seitz, Jessica K. Hodgins, and Pradeep K. Khosla. Flow-based video synthesis and editing. ACM Transactions on Graphics (SIGGRAPH 2004), 23(3), August 2004. [BiBTeX] Links
Learning Silhouette Features for Control of Human MotionLiu Ren, Gregory Shakhnarovich, Jessica K. Hodgins, Hanspeter Pfister, and Paul Viola AbstractWe present a vision-based performance interface for controlling animated human characters. The system combines information about the user's motion contained in silhouettes from several viewpoints with domain knowledge contained in a motion capture database to interactively produce a high quality animation. Such an interactive system will be useful for authoring, teleconferencing, or as a control interface for a character in a game. In our system, the user performs in front of three video cameras; the resulting silhouettes are used to estimate his or her orientation and body configuration based on a set of discriminative local features. Those features are selected by a machine learning algorithm during a preprocessing step. Sequences of motions that approximate the user's actions are extracted from the motion database and scaled in time to match the speed of the user's motion. We use swing dancing, an example of complex human motion, to demonstrate the effectiveness of our approach and compare the results obtained with discriminative local features to those obtained with global features, Hu moments, and to ground truth measurement from a motion capture system. CitationLiu Ren, Gregory Shakhnarovich, Jessica K. Hodgins, Hanspeter Pfister, and Paul Viola. Learning silhouette features for control of human motion. In Proceedings of the SIGGRAPH 2004 Conference on Sketches & Applications. ACM Press, August 2004. [BiBTeX] Links
Markerless Human Motion TransferGerman Cheung, Simon Baker, Jessica K. Hodgins, and Takeo Kanade CitationGerman Cheung, Simon Baker, Jessica K. Hodgins, and Takeo Kanade. Markerless human motion transfer. In Proceedings of the SIGGRAPH 2004 Conference on Sketches & Applications. ACM Press, August 2004. [BiBTeX] 
Near-Regular Texture Analysis and ManipulationYanxi Liu, Wen-Chieh Lin, and James Hays AbstractA near-regular texture deviates geometrically and photometrically from a regular congruent tiling. Although near-regular textures are ubiquitous in the man-made and natural world, they present computational challenges for state of the art texture analysis and synthesis algorithms. Using regular tiling as our anchor point, and with user-assisted lattice extraction, we can explicitly model the deformation of a near-regular texture with respect to geometry, lighting and color. We treat a deformation field both as a function that acts on a texture and as a texture that is acted upon, and develop a multimodal framework where each deformation field is subject to analysis, synthesis and manipulation. Using this formalization, we are able to construct simple parametric models to faithfully synthesize the appearance of a near-regular texture and purposefully control its regularity. CitationYanxi Liu, Wen-Chieh Lin, and James Hays. Near-regular texture analysis and manipulation. ACM Transactions on Graphics (SIGGRAPH 2004), 23(3), August 2004. [BiBTeX] Links
Squashing Cubes: Automating Deformable Model Construction for GraphicsDoug L. James, Jernej Barbič, and Christopher D. Twigg AbstractThe vast majority of geometric meshes used in computer graphics are optimized for rendering, and not deformable object simulation. Despite tools for volume (or surface) (re)meshing of geometric models to support physical simulation, in practice, the construction of physically based deformable models from arbitrary graphical models remains a tedious process for animators. Squashing Cubes automates the construction of physically based deformable objects from arbitrary geometric models. During preprocess, the geometric model (typically a surface mesh) is voxelized into tiny elastic cubes, i.e., the (squashing cubes model). Second, a generic deformable object simulator is used to deform the squashing cubes model. Finally, the resulting deformations are interpolated back onto the original model, thus producing the final animation. CitationDoug L. James, Jernej Barbič, and Christopher D. Twigg. Squashing cubes: Automating deformable model construction for graphics. In Proceedings of the SIGGRAPH 2004 Conference on Sketches & Applications. ACM Press, August 2004. [BiBTeX] Links
Synthesizing Animations of Human Manipulation TasksKatsu Yamane, James J. Kuffner, and Jessica K. Hodgins AbstractEven such simple tasks as placing a box on a shelf are difficult to animate, because the animator must carefully position the character to satisfy geometric and balance constraints while creating motion to perform the task with a natural-looking style. In this paper, we explore an approach for animating characters manipulating objects that combines the power of path planning with the domain knowledge inherent in data-driven, constraint-based inverse kinematics. A path planner is used to find a motion for the object such that the corresponding poses of the character satisfy geometric, kinematic, and posture constraints. The inverse kinematics computation of the character's pose resolves redundancy by biasing the solution toward natural-looking poses extracted from a database of captured motions. Having this database greatly helps to increase the quality of the output motion. The computed path is converted to a motion trajectory using a model of the velocity profile. We demonstrate the effectiveness of the algorithm by generating animations across a wide range of scenarios that cover variations in the geometric, kinematic, and dynamic models of the character, the manipulated object, and obstacles in the scene. CitationKatsu Yamane, James J. Kuffner, and Jessica K. Hodgins. Synthesizing animations of human manipulation tasks. ACM Transactions on Graphics (SIGGRAPH 2004), 23(3), August 2004. [BiBTeX] Links
Synthesizing Physically Realistic Human Motion in Low-Dimensional, Behavior-Specific SpacesAlla Safonova, Jessica K. Hodgins, and Nancy S. Pollard AbstractOptimization is an appealing way to compute the motion of an animated character because it allows the user to specify the desired motion in a sparse, intuitive way. The difficulty of solving this problem for complex characters such as humans is due in part to the high dimensionality of the search space. The dimensionality is an artifact of the problem representation because most dynamic human behaviors are intrinsically low dimensional with, for example, legs and arms operating in a coordinated way. We describe a method that exploits this observation to create an optimization problem that is easier to solve. Our method utilizes an existing motion capture database to find a low-dimensional space that captures the properties of the desired behavior. We show that when the optimization problem is solved within this low-dimensional subspace, a sparse sketch can be used as an initial guess and full physics constraints can be enabled. We demonstrate the power of our approach with examples of forward, vertical, and turning jumps; with running and walking; and with several acrobatic flips. CitationAlla Safonova, Jessica K. Hodgins, and Nancy S. Pollard. Synthesizing physically realistic human motion in low-dimensional, behavior-specific spaces. ACM Transactions on Graphics (SIGGRAPH 2004), 23(3), August 2004. [BiBTeX] Links
Team Teaching Animation Art and TechnologyJames Duesing and Jessica K. Hodgins AbstractIn this paper we report on an interdisciplinary course, "Animation Art and Technology," which we have taught for the past two years at Carnegie Mellon University. Faculty and teaching assistants from computer science and art teach the class as a team and the students are an interdisciplinary mix. This class is a project-based course in which teams of students produce 4-5 animations. Most of the animations have a substantive technical component and the students are challenged to consider innovation with content to be equal with the technical. In this paper, we describe the structure of the class and assess the elements that have worked well and those that require improvement. CitationJames Duesing and Jessica K. Hodgins. Team teaching animation art and technology. In Educators Program from the 31st Annual Conference on Computer Graphics and Interactive Techniques. ACM Press, August 2004. [BiBTeX] Links
BD-Tree: Output-Sensitive Collision Detection for Reduced Deformable ModelsDoug L. James and Dinesh K. Pai AbstractWe introduce the Bounded Deformation Tree, or BD-Tree, which can perform collision detection with reduced deformable models at costs comparable to collision detection with rigid objects. Reduced deformable models represent complex deformations as linear superpositions of arbitary displacement fields, and are used in a variety of applications of interactive computer graphics. The BD-Tree is a bounding sphere hierarchy for output-sensitive collision detection with such models. Its bounding spheres can be updated after deformation in any order, and at a cost independent of the geometric complexity of the model; in fact the cost can be as low as one multiplication and addition per tested sphere, and at most linear in the number reduced deformation coordinates. We show that the BD-Tree is also extremely simple to implement, and performs well in practice for a variety of real-time and complex off-line deformable simulation examples. CitationDoug L. James and Dinesh K. Pai. BD-Tree: Output-sensitive collision detection for reduced deformable models. ACM Transactions on Graphics (SIGGRAPH 2004), 23(3), August 2004. [BiBTeX] Links
Modeling Physical Variability for Synthetic MOUT AgentsGita Sukthankar, Michael Mandel, Katia Sycara, and Jessica K. Hodgins AbstractGenerating behavioral variability is an important prerequisite in the development of synthetic MOUT (Military Operations in Urban Terrain) agents for military simulations. Agents that lack variability are predictable and ineffective as opponents and teammates for human trainees. Along with cognitive differences, physical differences contribute towards behavioral variability. In this paper, we describe a novel method for modeling physical variability in MOUT soldiers using motion capture data acquired from human subjects. Motion capture data is commonly used to create animated characters since it retains the nuances of the original human movement. We build a cost model over the space of agent actions by creating and stochastically sampling motion graphs constructed from human data. Our results demonstrate how different cost models can induce variable behavior that remains consistent with military doctrine. CitationGita Sukthankar, Michael Mandel, Katia Sycara, and Jessica K. Hodgins. Modeling physical variability for synthetic mout agents. In Proceedings of 2004 Conference on Behavior Representation in Modeling and Simulation, May 2004. [BiBTeX] Links
Segmenting Motion Capture Data into Distinct BehaviorsJernej Barbič, Alla Safonova, Jia-Yu Pan, Christos Faloutsos, Jessica K. Hodgins, and Nancy S. Pollard AbstractMuch of the motion capture data used in animations, commercials, and video games is carefully segmented into distinct motions either at the time of capture or by hand after the capture session. As we move toward collecting more and longer motion sequences, however, automatic segmentation techniques will become important for processing the results in a reasonable time frame. We have found that straightforward, easy to implement segmentation techniques can be very effective for segmenting motion sequences into distinct behaviors. In this paper, we present three approaches for automatic segmentation. The first two approaches are online, meaning that the algorithm traverses the motion from beginning to end, creating the segmentation as it proceeds. The first assigns a cut when the intrinsic dimensionality of a local model of the motion suddenly increases. The second places a cut when the distribution of poses is observed to change. The third approach is a batch process and segments the sequence where consecutive frames belong to different elements of a Gaussian mixture model. We assess these three methods on fourteen motion sequences and compare the performance of the automatic methods to that of transitions selected manually. CitationJernej Barbič, Alla Safonova, Jia-Yu Pan, Christos Faloutsos, Jessica K. Hodgins, and Nancy S. Pollard. Segmenting Motion Capture Data into Distinct Behaviors. In Proceedings of Graphics Interface 2004, pages 185–194, May 2004. [BiBTeX] Links
Efficient Synthesis of Physically Valid Human MotionAnthony C. Fang and Nancy S. Pollard AbstractOptimization is a promising way to generate new animations from a minimal amount of input data. Physically based optimization techniques, however, are difficult to scale to complex animated characters, in part because evaluating and differentiating physical quantities becomes prohibitively slow. Traditional approaches often require optimizing or constraining parameters involving joint torques; obtaining first derivatives for these parameters is generally an O(D2) process, where D is the number of degrees of freedom of the character. In this paper, we describe a set of objective functions and constraints that lead to linear time analytical first derivatives. The surprising finding is that this set includes constraints on physical validity, such as ground contact constraints. Considering only constraints and objective functions that lead to linear time first derivatives results in fast per-iteration computation times and an optimization problem that appears to scale well to more complex characters. We show that qualities such as squash-and-stretch that are expected from physically based optimization result from our approach. Our animation system is particularly useful for synthesizing highly dynamic motions, and we show examples of swinging and leaping motions for characters having from 7 to 22 degrees of freedom. CitationAnthony C. Fang and Nancy S. Pollard. Efficient synthesis of physically valid human motion. ACM Transactions on Graphics (SIGGRAPH 2003), 22(3):417–426, July 2003. [BiBTeX] Links
Estimating Cloth Simulation Parameters from VideoKiran S. Bhat, Christopher D. Twigg, Jessica K. Hodgins, Pradeep K. Khosla, Zoran Popović, and Steven M. Seitz AbstractCloth simulations are notoriously difficult to tune due to the many parameters that must be adjusted to achieve the look of a particular fabric. In this paper, we present an algorithm for estimating the parameters of a cloth simulation from video data of real fabric. A perceptually motivated metric based on matching between folds is used to compare video of real cloth with simulation. This metric compares two video sequences of cloth and returns a number that measures the differences in their folds. Simulated annealing is used to minimize the frame by frame error between the metric for a given simulation and the real-world footage. To estimate all the cloth parameters, we identify simple static and dynamic calibration experiments that use small swatches of the fabric. To demonstrate the power of this approach, we use our algorithm to find the parameters for four different fabrics. We show the match between the video footage and simulated motion on the calibration experiments, on new video sequences for the swatches, and on a simulation of a full skirt. CitationKiran S. Bhat, Christopher D. Twigg, Jessica K. Hodgins, Pradeep K. Khosla, Zoran Popović, and Steven M. Seitz. Estimating cloth simulation parameters from video. In 2003 ACM SIGGRAPH / Eurographics Symposium on Computer Animation, pages 37–51, July 2003. [BiBTeX] Links
Perceptual Metrics for Character Animation: Sensitivity to Errors in Ballistic MotionPaul S. A. Reitsma and Nancy S. Pollard AbstractMotion capture data and techniques for blending, editing, and sequencing that data can produce rich, realistic character animation; however, the output of these motion processing techniques sometimes appears unnatural. For example, the motion may violate physical laws or reflect unreasonable forces from the character or the environment. While problems such as these can be fixed, doing so is not yet feasible in real time environments. We are interested in developing ways to estimate perceived error in animated human motion so that the output quality of motion processing techniques can be better controlled to meet user goals. This paper presents results of a study of user sensitivity to errors in animated human motion. Errors were systematically added to human jumping motion, and the ability of subjects to detect these errors was measured. We found that users were able to detect motion with errors, and noted some interesting trends: errors in horizontal velocity were easier to detect than errors in vertical velocity, and added accelerations were easier to detect than added decelerations. On the basis of our results, we propose a perceptually based metric for measuring errors in ballistic human motion. CitationPaul S. A. Reitsma and Nancy S. Pollard. Perceptual metrics for character animation: Sensitivity to errors in ballistic motion. ACM Transactions on Graphics (SIGGRAPH 2003), 22(3):537–542, July 2003. [BiBTeX] Links
Precomputing Interactive Dynamic Deformable ScenesDoug L. James and Kayvon Fatahalian AbstractWe present an approach for precomputing data-driven models of interactive physically based deformable scenes. The method permits real-time hardware synthesis of nonlinear deformation dynamics, including self-contact and global illumination effects, and supports real-time user interaction. We use data-driven tabulation of the system's deterministic state space dynamics, and model reduction to build efficient low-rank parameterizations of the deformed shapes. To support runtime interaction, we also tabulate impulse response functions for a palette of external excitations. Although our approach simulates particular systems under very particular interaction conditions, it has several advantages. First, parameterizing all possible scene deformations enables us to precompute novel reduced coparameterizations of global scene illumination for low-frequency lighting conditions. Second, because the deformation dynamics are precomputed and parameterized as a whole, collisions are resolved within the scene during precomputation so that runtime self-collision handling is implicit. Optionally, the data-driven models can be synthesized on programmable graphics hardware, leaving only the low-dimensional state space dynamics and appearance data models to be computed by the main CPU. CitationDoug L. James and Kayvon Fatahalian. Precomputing interactive dynamic deformable scenes. ACM Transactions on Graphics (SIGGRAPH 2003), 22(3):879–887, July 2003. [ |