[SCS dragon logo]

15-463 (15-862): Computational Photography
(formerly: Rendering and Image Processing)
Computer Science Department
Carnegie Mellon University

INSTRUCTOR: Alexei (Alyosha) Efros (Office hours: Tu 4:45-5:45pn)
UNIVERSITY UNITS: 12
SEMESTER: Fall 2005
NEWSGROUP: cmu.cs.class.cs463 (read this for important information!)
WEB PAGE: http://graphics.cs.cmu.edu/courses/15-463/
LOCATION: WeH 5419C
TIME: TR 10:30--11:50 AM

COURSE OVERVIEW:
Computational Photography is an emerging new field created by the convergence of computer graphics, computer vision and photography. Its role is to overcome the limitations of the traditional camera by using computational techniques to produce a richer, more vivid, perhaps more perceptually meaningful representation of our visual world.

The aim of this advanced undergraduate course is to study ways in which samples from the real world (images and video) can be used to generate compelling computer graphics imagery. We will learn how to acquire, represent, and render scenes from digitized photographs. Several popular image-based algorithms will be presented, with an emphasis on using these techniques to build practical systems. This hands-on emphasis will be reflected in the programming assignments, in which students will have the opportunity to acquire their own images of indoor and outdoor scenes and develop the image analysis and synthesis tools needed to render and view the scenes on the computer.

TOPICS TO BE COVERED:

  • Cameras, Image Formation
  • Image and Video Processing (filtering, anti-aliasing, pyramids)
  • Image Manipulation (warping, morphing, mosaicing, matting, compositing)
  • Data-driven Synthesis
  • Visual Perception
  • High Dynamic Range Imaging and Tone Mapping
  • Image-Based Lighting
  • Image-Based Rendering
  • Non-photorealistic Rendering

PREREQUISITES:
An introductory course in either Computer Graphics or Computer Vision or a solid knowledge of linear algebra and permission of the instructor.
Graduate Students: a small number of graduate students will be allowed to take the graduate version of this course (15-862) with the permission of the instructor. Students taking 15-862 will be required to do more substratial assignments as well as a research-level final paper.
Note: if the system doesn’t let you sign up, or puts you on the waitlist, do talk to me.

PROGRAMMING ASSIGNMENTS:

·  1. Images of the Russian Empire -- colorizing the Prokudin-Gorskii photo collection:

Project Results!

· 2. Face morphing and modeling:

Project Results!

·  3. Stitching Photo Mosaics

Project Results!

·  4. Feature Matching for Autostitching
    
Project Results!

·  5. Single-View Reconstruction (Tour into the Picture):

Project Results!

·  Final Projects!

TEXT:
There is no required text. Various course notes and papers will be made available.  Furthermore, there is an optional textbook that you might find helpful.  It will be placed on reserve at the Wean Hall library:

Computer Vision: The Modern Approach, Forsyth and Ponce

There is a number of other fine texts that you can use for general reference:

Photography (8th edition), London and Upton, (a great general guide to taking pictures)
Vision Science: Photons to Phenomenology, Stephen Palmer (great book on human visual perception)
Digital Image Processing, 2nd edition, Gonzalez and Woods (a good general image processing text)
The Art and Science of Digital Compositing, Ron Brinkmann (everything about compositing)
Multiple View Geometry in Computer Vision, Hartley & Zisserman (a bible on recovering 3D geometry)
The Computer Image, Watt and Policarpo (a nice “vision for graphics” text, somewhat dated)
3D Computer Graphics (3rd Edition), Watt (a good general graphics text)
Fundamentals of Computer Graphics, Peter Shirley (another good general graphics text)
Linear Algebra and its Applications, Gilbert Strang (a truly wonderful book on linear algebra)

CLASS SCHEDULE:

CLASS DATE

TOPICS

Material

Tu Aug 30

Introduction

Th Sept 1

Camera
 

Tu Sept 6

Capturing Light… in man and machine

Th Sept 8

Image Processing


 

Tu Sept 13

Image Warping
     

Th Sept 15

Image Morphing

Tu Sept 20

Face Modeling

Th Sept 22

Fourier Analysis without tears

Tu Sept 27

Convolution and Edge Detection
 

Slides (ppt, pdf)

Th Sept 29

Image Pyramids and Blending

Tu Oct 4

Modeling Light

Slides (ppt, pdf)

Th Oct 6

Homographies and Mosaics

 

 

Tu Oct 11

More Mosaic Madness

·       Slides (ppt, pdf)

·       Additional Reading: Rick Szeliski, Image Alignment and Stitching, A Tutorial (DRAFT)

Th Oct 13

Midterm Review

 

Tu Oct 18

Midterm

 

Th Oct 20

Post-midterm Nirvana – no class

 

Tu Oct 25

Direct Automatic Alignment (Optical Flow)

 

Th Oct 27

Feature-Based Automatic Alignment

 

Tu Nov 1

Feature Matching and RANSAC

 

Th Nov 3

Video Texture

 

Tu Nov 8

Texture Synthesis

 

Th Nov 10

Background Subtraction and Matting
  

 

Th Nov 15

High Dynamic Range Images

 

Th Nov 17

Single View Modeling
 

 

Tu Nov 22

More Single View Geometry
    

 

Tu Nov 29

Image-Based Lighting

Th Dec 1

Image-Based Lighting II

Tu Dec 6

3D from two and more views

Slides (ppt, pdf)

 

Th Dec 8

Wrap-up.  Final Project Presentations

Slides (ppt, pdf)

 

Mon Dec 19, 8:30am
in DH 2302

Final Project Presentations

 

CAMERAS:
Although it is not required, students are highly encouraged to obtain a digital camera for use in the course (one can get a pretty good camera for under $300). A couple cameras with tripods will be available on loan from the instructor.

METHOD OF EVALUATION:
Grading will be based on a set of programming assignments (60%), a midterm (15%) and a final project presentation (25%).  There will be no final exam.  For the programming assignments, students will be allowed a total of 5 (five) late days; each additional late day will incur a 10% penalty.

Students taking 15-862 will also be required to submit a conference-style paper describing their final project.

COMPUTING FACILITIES:
All students will have access to the graphics cluster in Wean Hall 5336. You will need to setup your Andrew account in the right way before you can longin.  Follow these instructions (replacing 462 with 463).  A universal memory card reader will be installed so that you can download the data from your Compact Flash, Memory Stick, or other card.

MATLAB:
Students will be encouraged to use Matlab (with the Image Processing Toolkit) as their primary computing platform.  Besides being a great prototyping environment, Matlab is particularly well-suited for working with image data and offers tons of build-in image processing functions.  Here is a link to some useful Matlab resources

PREVIOUS OFFERINGS OF THIS COURSE:
Previous offerings of this course can be found here.

SIMILAR COURSES IN OTHER UNIVERSITIES:

 

Page design courtesy of Doug James