Constrained Semi-Supervised Learning using Attributes and Comparative Attributes

teaserImage

Presented at ECCV, 2012

People

...

Abstract

We consider the problem of semi-supervised bootstrap learning for scene categorization. Existing semi-supervised approaches are typically unreliable and face semantic drift because the learning task is under-constrained. This is primarily because they ignore the strong interactions that often exist between scene categories, such as the common attributes shared across categories as well as the attributes which make one scene different from another. The goal of this paper is to exploit these relationships and constrain the semi-supervised learning problem. For example, the knowledge that an image is an auditorium can improve labeling of amphitheaters by enforcing constraints that an amphitheater image should have more circular structures than an auditorium image. We propose constraints based on mutual exclusion, binary attributes and comparative attributes and show that they help us to constrain the learning problem and avoid semantic drift. We demonstrate the effectiveness of our approach through extensive experiments, including results on a very large dataset of one million images.

Talk


Constrained Semi-Supervised Learning using Attributes and Comparative Attributes
Abhinav Shrivastava

Paper

paper
Download pdf (7.9MB)

BibTeX

@inproceedings{shrivastava_eccv12,
   author="Abhinav Shrivastava and Saurabh Singh and Abhinav Gupta",
   title="Constrained Semi-Supervised Learning using Attributes and Comparative Attributes",
   booktitle="European Conference on Computer Vision",
   year="2012"
}

Vocabulary Used (Annotations)

Table 1 represents the shorthand notation used for defining binary and comparative attributes in Table 2 and Table 3 resp.

Table 1: Shorthand notation for 15 Scene categories used in the paper.
Scene Categories
Amphitheater (Am) Auditorium (Au) Banquet Hall (BH) Barn (Bn) Bedroom (Be)
Bowling Alley (BA) Bridge (Br) Casino Indoor (CI) Cemetery (Ce) Church Outdoor (CO)
Coast (C) Conference Room (CR) Desert Sand (DS) Field Cultivated (FC) Restaurant (R)

Table 2: Binary Class-Attribute constraints (BA) used in our experiments. (✓ stands for the attribute being present for a class.)
Binary Class-Attributes (BA)
Am Au BH Bn Be BA Br CI Ce CO C CR DS FC R
Horizon Visible ✓✓✓
Indoor ✓✓✓✓✓✓✓
Has Water ✓✓
Has Building ✓✓✓
Has Seat Rows ✓✓
Has People ✓✓✓✓
Has Grass ✓✓✓
Has Clutter ✓✓
Has Chairs & Tables ✓✓✓✓
Is Man Made ✓✓✓✓✓✓✓✓✓✓✓
Eating Place ✓✓
Fun Place ✓✓
Made of Stone ✓✓✓✓
Formal Meeting Place ✓✓
Livable ✓
Part of House ✓
Relaxing Place ✓
Animal-Related Places ✓
Crowd-Related Places ✓✓✓✓

Table 3: Comparative-Attribute constraints (CA) used in our experiments. ’≻’ is an operator that induces partial ordering using the corresponding comparative attributes. For example, Is more Open — ’Br ≻ Ce’ means that Bridge is more open then Cemetery. Some relations are coupled in groups in this table for compact display.
Comparative Attributes
Is more open AM ≻ {Bn, CO}, Br ≻ Ce, C ≻ {Am, Bn, Br, CO, Ce}, DS ≻ {Am, Bn, Br, Ce, CO},
FC ≻ {Bn, CO, Ce}
Has more open space (outdoor) FC ≻ {Am, Bn, Ce}, C ≻ {Am, Bn, Br, Ce, FC}, DS ≻ {Am, Bn, Br, Ce, FC}
Has more open space (indoor)Au ≻ {BH, Be, CI, CR, R}, {BH, BA, CI, R} ≻ Be
Has more seating spaceAu ≻ {BH, Be, CR, R}, CR ≻ Be, BH ≻ {Be, R}
Has larger structures{Am, Bn, Br, CO} ≻ Ce, Ce≻ FC
Has taller structuresBn ≻ Ce, CO ≻ Ce
Has horizontally longer structuresBr ≻ {Bn, Ce, CO} , Am ≻ {Ce, Bn, CO}
Has more waterBr ≻ {Am, Bn, Ce, CO, DS, FC} , C ≻ {Bn, Br, Ce, DS, FC}
Has more sandDS ≻ {Am, Bn, CO, Ce, C, FC}
Has more greeneryBn ≻ CO, FC ≻ {Am, Bn, Br, Ce, CO, C, DS}

Presentation

Approach Overview

teaserImage

Supplementary Material

Coming soon...

Software

Coming soon on GitHub!

Funding

This research is supported by:

Comments, questions to Abhinav Shrivastava