Photo Clip Art
|Image credits: M. Caimary|
Before you begin, have a look at the tutorial.
Then try our live java applet demo!
If the demo is offline, please let us know and we'll put it back online asap.
- Jean-François Lalonde (Carnegie Mellon University)
- Derek Hoiem (Carnegie Mellon University)
- Alexei A. Efros (Carnegie Mellon University)
- Carsten Rother (Microsoft Research Cambridge)
- John Winn (Microsoft Research Cambridge)
- Antonio Criminisi (Microsoft Research Cambridge)
We present a system for inserting new objects into existing photographs by querying a vast image-based object library, precomputed using a publicly available Internet object database. The central goal is to shield the user from all of the arduous tasks typically involved in image compositing. The user is only asked to do two simple things: 1) pick a 3D location in the scene to place a new object; 2) select an object to insert using a hierarchical menu. We pose the problem of object insertion as a data-driven, 3D-based, context-sensitive object retrieval task. Instead of trying to manipulate the object to change its orientation, color distribution, etc. to fit the new image, we simply retrieve an object of a specified class that has all the required properties (camera pose, lighting, resolution, etc) from our large object library. We present new automatic algorithms for improving object segmentation and blending, estimating true 3D object size and orientation, and estimating scene lighting conditions. We also present an intuitive user interface that makes object insertion fast and simple even for the artistically challenged.
|Jean-François Lalonde, Derek Hoeim, Alexei A. Efros, Carsten Rother, John Winn and Antonio Criminisi. Photo Clip Art, ACM Transactions on Graphics (SIGGRAPH 2007), August 2007, Vol 26. No. 3. [PDF] [BibTeX]|
TalkDownload the slides from the talk given at SIGGRAPH 2007 in the following formats:
- [MS Powerpoint, 25.8MB]
- Accompanying videos: [MOV, 211.8MB][MOV, 44.2MB]
- [PDF, 27.3MB] (no videos)
- [Apple Keynote, 274.4MB], original version used at SIGGRAPH 2007.
View additional results here. Click on the images for high resolution versions.
|Image credits: Mack Scogin Merrill Elam Architects|
This project has been getting quite a bit a media coverage. Here's a summary of the most notable places where Photo Clip Art was mentioned (if I've missed some, please let me know).
- Watch a video of our demo at the Microsoft Tech Fest
2008 (under the name of "In-depth image editing"), presented by
Carsten Rother, showing our live
Java applet demo:
[WMV 9, 320x240, 46.4MB], © Channel 10.
- CMU website: Image editing made easier
- BBC News: Photo tool could fix bad images
- CNET News.com: Researchers try Google approach to understanding photos
- News.fr (french): Le photomontage pour les nuls
- USAToday: Researchers try Google photo tactic
- C't magazine (german): Fotomontage im Handumdrehen
- Press release
Acknowledgements and Funding
This work would not have been possible without the efforts of Bryan Russell, Antonio Torralba and the rest of the LabelMe team in designing the labeling tool and maintaining the database.
This research is supported by:
- NSF CCF-0541230
- NSF CAREER IIS-0546547
- Microsoft Corp.