Description: [SCS dragon logo]

15-463 (15-862): Computational Photography
Computer Science Department
Carnegie Mellon University

INSTRUCTOR: Alexei (Alyosha) Efros (Office hours: Tuesdays 5pm-6pm, EDSH 225)
TA: Marynel Vázquez  (Office hours: Wednesdays 10am-11am, NSH 4227)
UNIVERSITY UNITS: 12
SEMESTER: Fall 2012
WEB PAGE: http://graphics.cs.cmu.edu/courses/15-463/
Q&A: Piazza Course Website
LOCATION: GHC 5222
TIME
: T R 12:00--1:20 PM

COURSE OVERVIEW:
Computational Photography is an emerging new field created by the convergence of computer graphics, computer vision and photography. Its role is to overcome the limitations of the traditional camera by using computational techniques to produce a richer, more vivid, perhaps more perceptually meaningful representation of our visual world.

The aim of this advanced undergraduate course is to study ways in which samples from the real world (images and video) can be used to generate compelling computer graphics imagery. We will learn how to acquire, represent, and render scenes from digitized photographs. Several popular image-based algorithms will be presented, with an emphasis on using these techniques to build practical systems. This hands-on emphasis will be reflected in the programming assignments, in which students will have the opportunity to acquire their own images of indoor and outdoor scenes and develop the image analysis and synthesis tools needed to render and view the scenes on the computer.

TOPICS TO BE COVERED:

  • Cameras, Image Formation
  • Visual Perception
  • Image and Video Processing (filtering, anti-aliasing, pyramids)
  • Image Manipulation (warping, morphing, mosaicing, matting, compositing)
  • Modeling and Synthesis with Visual Big Data
  • High Dynamic Range Imaging and Tone Mapping
  • Image-Based Lighting
  • Image-Based Rendering
  • Non-photorealistic Rendering

PREREQUISITES:
Programming experience and familiarity with linear algebra and calculus is assumed.  Some background in computer graphics, computer vision, or image processing is helpful.  This class does not significantly overlap with 15-462 and can be taken concurrently.
PhD Students: a small number of PhD students will be allowed to take the graduate version of this course (15-862) with the permission of the instructor. Students taking 15-862 will be required to do more substantial assignments as well as a research-level final paper.
Note: if the system doesn?t let you sign up, or puts you on the waitlist, do talk to me.

PROGRAMMING ASSIGNMENTS:       

Project 1: Images of the Russian Empire -- colorizing the Prokudin-Gorskii photo collection
Description: http://www.cs.cmu.edu/afs/andrew/scs/cs/15-463/pub/www/images/3-strip-1.jpgDescription: http://www.cs.cmu.edu/afs/andrew/scs/cs/15-463/pub/www/images/3-8086-left.jpg

See student responses here.
Class Choice Awards: Yan Gu and Junyan Zhu.

Project 2: Hybrid Images


See student responses here (includes responses to 15-862 project).
Class Choice Awards: Michael Kamm (Hybrid Images), and Yan Gu (Eulerian Magnification).

Project 2: Eulerian Video Magnification (for the 15-862 graduate class)

Project 3: Image Resizing by Seam Carving


See student responses here (includes responses to 15-862 project).
Class Choice Awards: Michael Kamm and Nicolas Feltman.

Project 3: Gradient Domain Editing (for the 15-862 graduate class)

Project 4: Face morphing and modeling photo collection

See student responses here.
Watch morphing results in YouTube.
Class Choice Award: Feng Zhou.

Project 5: Building a Pinhole Camera

See student responses here.
Class Choice Awards: Krystyna Genser.

Project 6: [Auto]Stitching Photo Mosaics

See student responses here.

Final Project

TEXT:
There is now a textbook that covers most (if not all) of the topics related to Computational Photography.  This will be the primary reference for the course:

            Computer Vision: Algorithms and Applications, Richard Szeliski, 2010

There is a number of other fine texts that you can use for general reference:

Photography (8th edition), London and Upton, (a great general guide to taking pictures)
Vision Science: Photons to Phenomenology, Stephen Palmer (great book on human visual perception)
Digital Image Processing, 2nd edition, Gonzalez and Woods (a good general image processing text)
The Art and Science of Digital Compositing, Ron Brinkmann (everything about compositing)
Multiple View Geometry in Computer Vision, Hartley & Zisserman (a bible on recovering 3D geometry)
The Computer Image, Watt and Policarpo (a nice "vision for graphics" text, somewhat dated)
3D Computer Graphics (3rd Edition), Watt (a good general graphics text)
Fundamentals of Computer Graphics, Peter Shirley (another good general graphics text)
Linear Algebra and its Applications, Gilbert Strang (a truly wonderful book on linear algebra)

CLASS NOTES
The instructor is extremely grateful to a large number of researchers for making their slides available for use in this course.  Steve Seitz and Rick Szeliski have been particularly kind in letting me use their wonderful lecture notes.  In addition, I would like to thank Paul Debevec, Stephen Palmer, Paul Heckbert, David Forsyth, Steve Marschner and others, as noted in the slides.  The instructor gladly gives permission to use and modify any of the slides for academic and research purposes. However, please do also acknowledge the original sources where appropriate.

   

CLASS SCHEDULE:

CLASS DATE

TOPICS

Material

Tu Aug 28

Introduction

Th Aug 30

Capturing Light... in man and machine

Tu Sept 4

Sampling and Reconstruction

  • Slides (ppt)
  • Start Szeliski Ch 3

Th Sept 6

Sampling and Reconstruction

  • Continue slides (ppt)
  • Continue Szeliski Ch 3

Tu & Th
Sept 11/13

The Frequency Domain and Filtering

  • Slides (ppt)
  • Continue Szeliski Ch 3

Tu Sept 18

Image Blending and Compositing

Th Sept 20

Point Processing and Image Warping

  • Point Processing Slides (ppt)
  • Warping Slides (ppt)
  • Continue Szeliski Ch 3

Tu Sept 25

Image Morphing

  • Slides (ppt)
  • Continue Szeliski Ch 3

Th Sept 27
Tu Oct 2

Data-driven Methods: Faces

Tu Oct 9

Data-driven Methods: Visual Data on the Internet (part 1)

Th Oct 11

Data-driven Methods: Visual Data on the Internet (part 2)

Tu Oct 16

Data-driven Methods: Visual Data on the Internet (part 3)

Th Oct 18

The Camera

  • Slides (ppt)
  • Szeliski Ch. 2

Tu Oct 23

Modeling Light

Th Nov 1st

Homographies and Mosaics

  • Slides (ppt)
  • Szeliski Ch 9

Tu Nov 6th

More Mosaic Madness

Tu Nov 13th

Automatic Alignment

Th Nov 15th

Single View Reconstruction

Tu Nov 20th

Multi-perspective Panoramas

Tu Nov 27th

High Dynamic Range Images

Th Nov 29th

Image-based Lighting

Tu Dec 4th

Image-based Lighting II

Th Dec 6th

What makes a great picture?

CAMERAS:
Although it is not required, students are highly encouraged to obtain a digital camera for use in the course.

METHOD OF EVALUATION:
Grading will be based on a set of programming and written assignments (60%), an exam (20%) and a final project (20%).  For the programming assignments, students will be allowed a total of 5 (five) late days per semester; each additional late day will incur a 10% penalty.

Students taking 15-862 will also be required to submit a conference-style paper describing their final project.

MATLAB:
Students will be encouraged to use Matlab (with the Image Processing Toolkit) as their primary computing platform.  Besides being a great prototyping environment, Matlab is particularly well-suited for working with image data and offers tons of build-in image processing functions.  Here is a link to some useful Matlab resources

PREVIOUS OFFERINGS OF THIS COURSE:
Previous offerings of this course can be found here.

SIMILAR COURSES IN OTHER UNIVERSITIES:

 

Page design courtesy of Doug James