Context as Supervisory Signal:
Discovering Objects with Predictable Context

teaserImage

Objects Discovered Without Supervision From The Full PASCAL VOC 2011 Dataset.

Presented at ECCV 2014

People

Abstract

This paper addresses the well-established problem of unsupervised object discovery with a novel method inspired by weakly-supervised approaches. In particular, the ability of an object patch to predict the rest of the object (its context) is used as supervisory signal to help discover visually consistent object clusters. The main contributions of this work are: 1) framing unsupervised clustering as a leave-one-out context prediction task; 2) evaluating the quality of context prediction by statistical hypothesis testing between thing and stuff appearance models; and 3) an iterative region prediction and context alignment approach that gradually discovers a visual object cluster together with a segmentation mask and fine-grained correspondences. The proposed method outperforms previous unsupervised as well as weakly-supervised object discovery approaches, and is shown to provide correspondences detailed enough to transfer keypoint annotations.

Paper & Presentation

ECCV paper (pdf, 2.6MB)
Update 8/24/2014: expand sec. A.5 and fix a typo.

Example Slides (pptx, 4.4MB)

Citation
 
 Carl Doersch, Abhinav Gupta, and Alexei A. Efros. Context as Supervisory Signal: Discovering Objects with Predictable Context. In ECCV 2014. [Show BibTex]

Code

This code includes both the full object discovery pipeline as well as a quick demo that lets you visualize the context prediction procedure for verifying a single patch as a member of a cluster. The quick demo should take less than 5 minutes to set up and run. Note that the code for this project has been modified from the version used in the paper. The qualitative behavior should be the same, but the output may not be identical to the results in the paper.

Code available on Github

Additional Materials

Full Results: PASCAL VOC 2011
An example of the results obtained entirely automatically, using our unsupervised object discovery algorithm. Each row shows the most confident oatcges for one discovered object. Also, see this page, which shows the same clusters but ranks each row by the Exemplar-LDA score rather than our algorithm, to convince yourself that our algorithm is helping. WARNING: These pages contains about 6000 patches; don't click the link unless your browser can handle it!

Funding

This research was supported by:

Comments, questions to Carl Doersch