|
|
|
|
|
15-463 (15-862):
Computational |
![[SCS dragon logo]](scslogo.gif)
INSTRUCTOR: Alexei
(Alyosha) Efros (Office hours: Thursdays 2:30-3:30, NSH 4207)
TA: Jim McCann (Office hours: Tuesdays
5-6, NSH 4228)
UNIVERSITY UNITS: 12
SEMESTER: Fall 2007
NEWSGROUP: cmu.cs.class.cs463 (read this for important information!)
WEB PAGE: http://graphics.cs.cmu.edu/courses/15-463/
LOCATION: WeH 5312
TIME: T R 12:00--1:20 PM
COURSE OVERVIEW:
Computational Photography is an emerging new field created by the convergence
of computer graphics, computer vision and photography. Its role is to overcome
the limitations of the traditional camera by using computational techniques to
produce a richer, more vivid, perhaps more perceptually meaningful
representation of our visual world.
The aim of this advanced
undergraduate course is to study ways in which samples from the real world (images
and video) can be used to generate compelling computer graphics imagery. We
will learn how to acquire, represent, and render scenes from digitized
photographs. Several popular image-based algorithms will be presented, with an
emphasis on using these techniques to build practical systems. This hands-on
emphasis will be reflected in the programming assignments, in which students
will have the opportunity to acquire their own images of indoor and outdoor
scenes and develop the image analysis and synthesis tools needed to render and
view the scenes on the computer.
TOPICS TO BE COVERED:
- Cameras, Image Formation
- Visual Perception
- Image and Video Processing (filtering,
anti-aliasing, pyramids)
- Image Manipulation (warping, morphing, mosaicing, matting, compositing)
- Modeling and Synthesis using Lots of Data
- Image-Based Lighting
- Image-Based
Rendering
- Non-photorealistic
Rendering
PREREQUISITES:
Programming experience and familiarity with linear algebra and calculus is
assumed. Some background in computer
graphics, computer vision, or image processing is helpful. This class does not significantly overlap
with 15-462 and can be taken concurrently.
Graduate Students: a small number of graduate students will be allowed
to take the graduate version of this course (15-862) with the permission of the
instructor. Students taking 15-862 will be required to do more substantial
assignments as well as a research-level final paper.
Note: if the system doesn’t
let you sign up, or puts you on the waitlist, do talk to me.
PROGRAMMING ASSIGNMENTS:
· Project
0: The Vertigo Shot
|
|
·
Project 1: Images of the Russian Empire -- colorizing the Prokudin-Gorskii photo collection Results |
|
·
Project 2: Image Resizing by Seam Carving |
|
· Project 3: Face morphing and modeling: Results Class movie |
|
·
Project 4: Stitching Photo Mosaics (including autostitching) |
|
· Project 5: Single-View Reconstruction(Tour into the Picture): |
|
Final Projects!
Microstructural Grain Modeling Image
Deformation using Moving Least Squares Flickrbooks: Flipbooks using millions of Flickr images |
TEXT:
Since Computational Photography is such a new discipline, no comprehensive
textbook exists for use in the class. Therefore, there is no required text.
Various course notes and papers will be made available. Furthermore, there is an optional textbook
that you might find helpful. It will be
placed on reserve at the Wean Hall library:
Computer
Vision: The Modern Approach, Forsyth and
There is a number of other fine
texts that you can use for general reference:
Photography (8th
edition), London and Upton, (a great
general guide to taking pictures)
Vision Science: Photons to
Phenomenology, Stephen Palmer (great book
on human visual perception)
Digital Image Processing, 2nd edition, Gonzalez and
Woods (a good general image processing
text)
The Art and Science of Digital Compositing, Ron Brinkmann
(everything about compositing)
Multiple View Geometry in
Computer Vision, Hartley & Zisserman (a bible on recovering 3D geometry)
The Computer Image, Watt and Policarpo (a nice “vision for graphics”
text, somewhat dated)
3D Computer Graphics (3rd Edition), Watt (a
good general graphics text)
Fundamentals of Computer Graphics, Peter Shirley (another good general graphics text)
Linear Algebra and its Applications, Gilbert Strang
(a truly wonderful book on linear
algebra)
CLASS NOTES
The instructor is extremely grateful to a large number of researchers for making
their slides available for use in this course.
Steve Seitz and Rick Szeliski
have been particularly kind in letting me use their wonderful lecture
notes. In addition, I would like to
thank Paul Debevec,
Stephen Palmer, Paul Heckbert, David Forsyth, Steve Marschner
and others, as noted in the slides. The
instructor gladly gives permission to use and modify any of the slides for
academic and research purposes. However, please do also acknowledge the
original sources where appropriate.
CLASS SCHEDULE:
|
CLASS DATE |
TOPICS |
Material |
|
Tu
Aug 28 |
Introduction |
|
|
Th Aug 30 |
The Camera |
|
|
Tu Sept 4 |
Capturing Light… in man and machine |
|
|
Th
Sept 6 |
Sampling and Reconstruction |
|
|
Tu Sept 11 |
The Frequency Domain |
|
|
Th Sept 13 |
Image Processing (Jim) |
|
|
Tu Sept 18 |
Image Blending and Compositing |
Slides (ppt, pdf) Additional |
|
Th Sept 20 |
Image Warping |
|
|
Tu Sept 25 |
Image Morphing |
|
|
Th Sept 27 |
Data-driven Methods: Faces |
Slides (ppt, pdf) · Additional
|
|
Friday Sept 28, 3:30pm, NSH 1305 |
Special Guest Lecture : From
Images To Insights (Wojciech Matusik) |
|
|
Tu Oct 2 |
Data-driven Methods: Video |
|
|
Th Oct 4 |
Data-driven Methods: Textures and Images |
|
|
Tu Oct 9 |
Modeling Light |
|
|
Th Oct 11 |
Midtern review |
|
|
Tu Oct 16 |
Midterm Exam (Jim) |
|
|
Th Oct 18 |
Midterm Overview (Jim) |
|
|
Tu Oct 23 |
Homographies and Mosaics |
|
|
Th Oct 25 |
More Mosaic Madness |
|
|
Friday Oct 26, 3:30pm, NSH 1305 |
Special Guest Lecture : Paul Debevec |
|
|
Tu Oct 30 |
Automatic Alignment |
|
|
Th Nov 1 |
Multi-perspective Panoramas |
|
|
Tu Nov 6 |
Single View Reconstruction |
· |
|
Th Nov 8 |
More Single View Geometry |
· Additional |
|
Tu Nov 13 |
|
|
|
Th Nov 15 |
Image-based Lighting |
Slides (ppt, pdf) |
|
Tu Nov 20 |
Image-based Lighting II |
Slides (ppt, pdf) |
|
Tu Nov 27 |
Taking Great Pictures (guest lecture by Yan Ke) |
Slides (ppt, pdf) |
|
Th Nov 29 |
Matting |
|
|
Tu Dec 4 |
Multi-view Geometry |
|
|
Th Dec 6 |
Navigating the World’s Photographs (virtual
guest lecture by Steve
Seitz) |
CAMERAS:
Although it is not required, students are highly encouraged to obtain a digital
camera for use in the course (one can get a pretty
good camera for under $150). A camera might be available on load from the
instructor.
METHOD OF EVALUATION:
Grading will be based on a set of programming and written assignments (60%), an
exam (20%) and a final project (20%).
For the programming assignments, students will be allowed a total of 5
(five) late days per semester; each additional late day will incur a 10% penalty.
Students taking 15-862 will also be required to submit a
conference-style paper describing their final project.
COMPUTING
FACILITIES:
All students will have access to
the graphics cluster in Wean Hall 5336. You will need to setup your Andrew
account in the right way before you can login.
MATLAB:
Students will be encouraged to
use Matlab (with the Image Processing Toolkit) as
their primary computing platform.
Besides being a great prototyping environment, Matlab
is particularly well-suited for working with image data and offers tons of
build-in image processing functions.
Here is a link to some useful Matlab
resources
PREVIOUS OFFERINGS OF THIS
COURSE:
Previous offerings of this course can be found here.
SIMILAR
COURSES IN OTHER UNIVERSITIES:
- Computational Photography
SIGGRAPH Course (Raskar & Tumblin)
- Computational
Photography (Raskar, Northeastern)
- Digital
and Computational Photography (Durand & Freeman, MIT)
- Computational
Photography (
- Computational
Photography (Levoy & Wilburn, Stanford)
- Computer Vision (Seitz & Szeliski, UWashington)
- Instroduction
to Visual Computing and Visual Modeling
(Kutulakos, UToronto)
- Topics in Image-based
Modeling and Rendering (Kriegman, UCSD)
- Symposium on Computational Photography and
Video (May 2005, MIT)
Page design courtesy of Doug James