Automatic Editing of Footage
from Multiple Social Cameras
*Ido Arev1,3, *Hyun Soo Park2, Yaser Sheikh2,3, Jessica Hodgins2,3, and Ariel Shamir1,3
1The Interdisciplinary Center Herzliya 2Carnegie Mellon University 3Disney Research Pittsburgh
* Both authors contributed equally to this work.
A visualization of three shots from a coherent video cut of a social event created by our algorithm. In this case, seven social cameras recorded a basketball game. Their views are shown at three different times. The 3D positions of the cameras, the 3D joint attention estimate (blue dots), and the line-of-action are shown in the 3D view.
Using cinematographic guidelines, quality of the footage, and joint attention estimation, our algorithm chooses times to cut from one camera to another (from the blue camera to purple and then to green). Abstract We present an approach that takes multiple videos captured by
social cameras---cameras that are carried or worn by members
of the group involved in an activity---and produces a coherent
"cut" video of the activity.
Footage from social cameras contains an intimate, personalized view
that reflects the part of an event that was of importance to the
camera operator. We leverage the insight that social cameras share
the focus of attention of the people carrying (or wearing) them.
We use this insight to determine where the important ``content''
in a scene is taking place, and use it in conjunction with
cinematographic guidelines to select which cameras to cut to and to
determine the timing of those cuts. A trellis graph formulation is
used to optimize an objective function that maximizes coverage of
the important content in the scene, while respecting cinematographic
guidelines such as the 180-degree rule and avoiding jump cuts. We
demonstrate cuts of the videos in various styles and lengths for a
number of scenarios, including sports games, street performance,
family activities, and social get-togethers. We evaluate our
results through an in-depth analysis of the cuts in the resulting
videos and through comparison with videos produced by a professional
editor and existing commercial solutions. Paper *Ido Arev, *Hyun Soo Park, Yaser Sheikh, J. Hodgins, and Ariel Shamir, "Automatic Editing of Footage from Multiple Social Cameras " ACM Transactions on Graphics (SIGGRAPH), 2014 [PDF (24 Mb), Video (218 Mb), Bibtex] Video