|
|
|
|
|
15-463 (15-862): Computational |
![Description: [SCS dragon logo]](463_files/scslogo.gif)
INSTRUCTOR:
Alexei (Alyosha) Efros (Office hours:
Tuesdays 5pm-6pm, EDSH 225)
TA: Marynel Vázquez (Office hours: Wednesdays 10am-11am, NSH 4227)
UNIVERSITY UNITS: 12
SEMESTER: Fall 2012
WEB PAGE: http://graphics.cs.cmu.edu/courses/15-463/
Q&A: Piazza Course Website
LOCATION: GHC 5222
TIME: T R 12:00--1:20
PM
COURSE
OVERVIEW:
Computational Photography is an emerging new field created by the convergence
of computer graphics, computer vision and photography. Its role is to overcome
the limitations of the traditional camera by using computational techniques to
produce a richer, more vivid, perhaps more perceptually meaningful
representation of our visual world.
The
aim of this advanced undergraduate course is to study ways in which samples
from the real world (images and video) can be used to generate compelling
computer graphics imagery. We will learn how to acquire, represent, and render
scenes from digitized photographs. Several popular image-based algorithms will
be presented, with an emphasis on using these techniques to build practical
systems. This hands-on emphasis will be reflected in the programming
assignments, in which students will have the opportunity to acquire their own
images of indoor and outdoor scenes and develop the image analysis and
synthesis tools needed to render and view the scenes on the computer.
TOPICS
TO BE COVERED:
- Cameras,
Image Formation
- Visual
Perception
- Image and
Video Processing (filtering, anti-aliasing, pyramids)
- Image
Manipulation (warping, morphing, mosaicing,
matting, compositing)
- Modeling and
Synthesis with Visual Big Data
- Image-Based
Lighting
- Image-Based Rendering
- Non-photorealistic Rendering
PREREQUISITES:
Programming experience and familiarity with linear algebra and calculus is
assumed. Some background in computer
graphics, computer vision, or image processing is helpful. This class does not significantly overlap
with 15-462 and can be taken concurrently.
PhD Students: a small number of PhD students will be allowed to take the
graduate version of this course (15-862) with the permission of the instructor.
Students taking 15-862 will be required to do more substantial assignments as
well as a research-level final paper.
Note: if the system doesn?t let you
sign up, or puts you on the waitlist, do talk to me.
PROGRAMMING ASSIGNMENTS:
|
Project 1: Images of the
Russian Empire -- colorizing the Prokudin-Gorskii
photo collection
|
|
|
See student responses here (includes responses to 15-862 project).  Class Choice Awards: Michael Kamm (Hybrid Images), and Yan Gu (Eulerian Magnification).
|
Project 2: Eulerian Video Magnification (for the 15-862 graduate class)
|
|
Project 3: Image Resizing by Seam Carving
See student responses here (includes responses to 15-862 project). Class Choice Awards: Michael Kamm and Nicolas Feltman.
|
Project 3: Gradient Domain Editing (for the 15-862 graduate class)
|
|
Project 4: Face morphing and modeling
photo collection
|
|
|
Project 5: Building a Pinhole Camera
|
|
|
Project 6: [Auto]Stitching Photo Mosaics
|
|
TEXT:
There is now a textbook that covers most (if not all) of the
topics related to Computational Photography. This will be the primary reference for
the course:
Computer
Vision: Algorithms and Applications, Richard Szeliski,
2010
There
is a number of other fine texts that you can use for
general reference:
Photography
(8th edition), London and Upton, (a
great general guide to taking pictures)
Vision Science: Photons to Phenomenology, Stephen
Palmer (great book on human visual
perception)
Digital
Image Processing, 2nd edition, Gonzalez and Woods (a good general image processing text)
The Art and Science of Digital Compositing, Ron Brinkmann
(everything about compositing)
Multiple View Geometry in Computer Vision, Hartley
& Zisserman (a
bible on recovering 3D geometry)
The Computer Image, Watt and Policarpo (a nice "vision for graphics" text, somewhat
dated)
3D Computer Graphics (3rd Edition), Watt (a
good general graphics text)
Fundamentals of Computer Graphics, Peter Shirley (another good general graphics text)
Linear Algebra and its Applications, Gilbert Strang
(a truly wonderful book on linear
algebra)
CLASS NOTES
The instructor is extremely grateful to a large number of researchers for
making their slides available for use in this course. Steve Seitz and Rick Szeliski
have been particularly kind in letting me use their wonderful lecture
notes. In addition, I would like to
thank Paul Debevec,
Stephen Palmer, Paul Heckbert, David Forsyth, Steve Marschner
and others, as noted in the slides. The
instructor gladly gives permission to use and modify any of the slides for academic
and research purposes. However, please do also acknowledge the original sources
where appropriate.
CLASS SCHEDULE:
|
CLASS DATE |
TOPICS |
Material |
|
Tu Aug 28 |
Introduction |
|
|
Th Aug 30 |
Capturing Light... in man and machine |
|
|
Tu Sept 4 |
Sampling and Reconstruction |
|
|
Th Sept 6 |
Sampling and Reconstruction |
|
|
Tu & Th |
The Frequency Domain and Filtering |
|
|
Tu Sept 18 |
Image Blending and Compositing |
|
|
Th Sept 20 |
Point Processing and Image Warping |
|
|
Tu Sept 25 |
Image Morphing |
|
|
Th Sept 27 |
Data-driven Methods: Faces |
|
|
Tu Oct 9 |
Data-driven Methods: Visual Data on the Internet (part 1) |
|
|
Th Oct 11 |
Data-driven Methods: Visual Data on the Internet (part 2) |
|
|
Tu Oct 16 |
Data-driven Methods: Visual Data on the Internet (part 3) |
|
|
Th Oct 18 |
The Camera |
|
|
Tu Oct 23 |
Modeling Light |
|
|
Th Nov 1st |
Homographies and Mosaics |
|
|
Tu Nov 6th |
More Mosaic Madness |
|
|
Tu Nov 13th |
Automatic Alignment |
|
|
Th Nov 15th |
Single View Reconstruction |
|
|
Tu Nov 20th |
Multi-perspective Panoramas |
|
|
Tu Nov 27th |
High Dynamic Range Images |
|
|
Th Nov 29th |
Image-based Lighting |
|
|
Tu Dec 4th |
Image-based Lighting II |
|
|
Th Dec 6th |
What makes a great picture? |
|
CAMERAS:
Although it is not required, students are highly encouraged to obtain a digital
camera for use in the course.
METHOD OF EVALUATION:
Grading will be based on a set of programming and written assignments (60%), an
exam (20%) and a final project (20%).
For the programming assignments, students will be allowed a total of 5
(five) late days per semester; each additional late day will incur a 10%
penalty.
Students taking 15-862 will also be required to
submit a conference-style paper describing their final project.
MATLAB:
Students will be encouraged to use Matlab
(with the Image Processing Toolkit) as their primary computing platform. Besides being a great prototyping
environment, Matlab is particularly well-suited for
working with image data and offers tons of build-in image processing
functions. Here is a link to some useful Matlab resources
PREVIOUS OFFERINGS OF THIS COURSE:
Previous offerings of this course can be found here.
SIMILAR COURSES IN OTHER UNIVERSITIES:
- Computational Photography (Hoiem, UIUC)
- Computational Photography (Hays,
Brown)
- Digital and Computational Photography
(Durand, MIT)
- Computer Vision (Seitz &
Szeliski, UWashington)
Page design
courtesy of Doug James