What Makes Paris Look like Paris?

teaserImage

People

Abstract

Given a large repository of geotagged imagery, we seek to automatically find visual elements, e.g. windows, balconies, and street signs, that are most distinctive for a certain geo-spatial area, for example the city of Paris. This is a tremendously difficult task as the visual features distinguishing architectural elements of different places can be very subtle. In addition, we face a hard search problem: given all possible patches in all images, which of them are both frequently occurring and geographically informative? To address these issues, we propose to use a discriminative clustering approach able to take into account the weak geographic supervision. We show that geographically representative image elements can be discovered automatically from Google Street View imagery in a discriminative manner. We demonstrate that these elements are visually interpretable and perceptually geo-informative. The discovered visual elements can also support a variety of computational geography tasks, such as mapping architectural correspondences and influences within and across cities, finding representative elements at different geo-spatial scales, and geographically-informed image retrieval.

Paper & Presentation

SIGGRAPH Paper (preprint, pdf, 71MB)
Reduced-size SIGGRAPH Paper (preprint, pdf, 9MB)

SIGGRAPH talk slides (pptx, 238MB)

Citation
 
 Carl Doersch, Saurabh Singh, Abhinav Gupta, Josef Sivic, and Alexei A. Efros. What Makes Paris Look like Paris? ACM Transactions on Graphics (SIGGRAPH 2012), August 2012, vol. 31, No. 3. [Show BibTex]

CACM Citation
 
 Carl Doersch, Saurabh Singh, Abhinav Gupta, Josef Sivic, and Alexei A. Efros. What Makes Paris Look like Paris? ACM Transactions on Graphics (Communications of the ACM 2012), December 2015, vol. 59, No. 12. [Show BibTex]

Video






Related Papers

S. Singh, A. Gupta and A. A. Efros. 2012. Unsupervised discovery of mid-level discriminative patches. In ECCV 2012.

C. Doersch, A. Gupta and A. A. Efros. 2013. Mid-Level Visual Element Discovery as Discriminative Mode Seeking. In NIPS 2013.

Code

Current Release

Version 4.5 tar.gz (1.6 MB; released Jan. 29 2012)

Older Releases

Version 3.0 tar.gz (1.5 MB; released Sept. 13 2012)

Additional Materials

Full Results: Paris Elements
An example of the results obtained entirely automatically, using our algorithm to find the unique visual elements of Paris. Each row shows the most confident detections in Paris for one of these elements. WARNING: This page contains about 5000 patches; don't click the link unless your browser can handle it!

Take the Paris-NonParis test!
This test presents 100 random street-level images: 50 from Paris, 50 from 11 other cities. Guess whether each one is from Paris (try not to look at the text...) and we'll score you at the end!

Artist Materials
For one experiment, we had an artist sketch an image, and then later re-sketch it with help from our discovered elements. See the resulting sketches, plus the materials we showed our artist.

Popular Press

Funding

This research is supported by: