http://graphics.cs.cmu.edu/projects/crossDomainMatching/

Data-driven Visual Similarity for Cross-domain Image Matching

Presented at SIGGRAPH Asia, 2011

People

• Abhinav Shrivastava
• Tomasz Malisiewicz
• Abhinav Gupta
• Alexei A. Efros

A data-driven technique to find visual similarity which does not depend on any particular image domain or feature representation. Visit the webpage to see some cool results and applications.

Abstract

The goal of this work is to find visually similar images even if they appear quite different at the raw pixel level. This task is particularly important for matching images across visual domains, such as photos taken over different seasons or lighting conditions, paintings, hand-drawn sketches, etc. We propose a surprisingly simple method that estimates the relative importance of different features in a query image based on the notion of "data-driven uniqueness". We employ standard tools from discriminative object detection in a novel way, yielding a generic approach that does not depend on a particular image representation or a specific visual domain. Our approach shows good performance on a number of difficult cross-domain visual tasks e.g., matching paintings or sketches to real photographs. The method also allows us to demonstrate novel applications such as Internet re-photography, and painting2gps.

Featuring Articles

Paper

Supplementary Material

Video

Watch in HD!

More Videos (Visual-Memex Traversal)

Presentation

Data Used

Software

Source code for the basic infrastructure used in this paper (Exemplar-SVM infrastructure for large-scale training using a cluster, fast detection, etc.) is available for download:

Exemplar-SVM tarball
Exemplar-SVM zipfile

You can also directly navigate to the Exemplar-SVM Github project page, which has download instructions, a wiki, and additional starter-guides.

**Instructions specific to the this project**

Related Papers

Ensemble of Exemplar-SVMs for Object Detection and Beyond, Tomasz Malisiewicz, Abhinav Gupta, Alexei A. Efros, in International Conference on Computer Vision (ICCV), 2011

Exemplar-SVMs for Visual Object Detection, Label Transfer and Image Retrieval, Tomasz Malisiewicz, Abhinav Shrivastava, Alexei A. Efros, Invited Applications Paper in International Conference on Machine Learning (ICML), 2012 (PDF)

Beyond Categories: The Visual Memex Model for Reasoning About Object Relationships, Tomasz Malisiewicz, Alexei A. Efros, In NIPS, December 2009

BibTeX

@article{shrivastava-sa11,
 author = {Shrivastava, Abhinav and Malisiewicz, Tomasz and Gupta, Abhinav and Efros, Alexei A.},
 title = {Data-driven Visual Similarity for Cross-domain Image Matching}, 
 journal = {ACM Transaction of Graphics (TOG) (Proceedings of ACM SIGGRAPH ASIA)},
 year = {2011},
 volume = {30},
 number = {6},
}

Funding

This research is supported by:

• ONR Grant N000141010766
• MSR-CMU Center for Computational Thinking
• Google

Comments, questions to Abhinav Shrivastava

Content © 2011 Abhinav Shrivastava. All Rights Reserved.
Template © 2011-12 Abhinav Shrivastava. All Rights Reserved.