|
|
|
|
|
15-463 (15-862): Computational |
![[SCS dragon logo]](scslogo.gif)
INSTRUCTOR: Alexei (Alyosha) Efros (Office hours: T
Th 1:30-2pm, NSH 4207)
TA: Mohit Gupta (Office
hours: W 11-12, EDSH 208)
UNIVERSITY UNITS: 12
SEMESTER: Fall 2006
NEWSGROUP: cmu.cs.class.cs463 (read this for important information!)
WEB PAGE: http://graphics.cs.cmu.edu/courses/15-463/
LOCATION: WeH 5312
TIME: TR 12:00--1:20 PM
COURSE
OVERVIEW:
Computational Photography is an emerging new field created by the convergence
of computer graphics, computer vision and photography. Its role is to overcome
the limitations of the traditional camera by using computational techniques to
produce a richer, more vivid, perhaps more perceptually meaningful
representation of our visual world.
The
aim of this advanced undergraduate course is to study ways in which samples
from the real world (images and video) can be used to generate compelling
computer graphics imagery. We will learn how to acquire, represent, and render
scenes from digitized photographs. Several popular image-based algorithms will
be presented, with an emphasis on using these techniques to build practical
systems. This hands-on emphasis will be reflected in the programming
assignments, in which students will have the opportunity to acquire their own
images of indoor and outdoor scenes and develop the image analysis and
synthesis tools needed to render and view the scenes on the computer.
TOPICS
TO BE COVERED:
- Cameras, Image Formation
- Image and Video
Processing (filtering, anti-aliasing, pyramids)
- Image Manipulation
(warping, morphing, mosaicing, matting, compositing)
- Data-driven Synthesis
- Visual Perception
- Image-Based Lighting
- Image-Based Rendering
- Non-photorealistic Rendering
PREREQUISITES:
Programming experience and familiarity with linear algebra and calculus is
assumed. Some background in computer
graphics, computer vision, or image processing is helpful. This class does not significantly overlap
with 15-462 and can be taken concurrently.
Graduate Students: a small number of graduate students will be allowed
to take the graduate version of this course (15-862) with the permission of the
instructor. Students taking 15-862 will be required to do more substratial
assignments as well as a research-level final paper.
Note: if the system doesn’t
let you sign up, or puts you on the waitlist, do talk to me.
PROGRAMMING
ASSIGNMENTS:
|
· 1. Images of the Russian Empire
-- colorizing the Prokudin-Gorskii photo collection: |
|
· 5. Single-View Reconstruction (Tour into the Picture): |
TEXT:
Since Computational Photography is such a new discipline, no comprehensive
textbook exists for use in the class. Therefore, there is no required text.
Various course notes and papers will be made available. Furthermore, there is an optional textbook
that you might find helpful. It will be
placed on reserve at the Wean Hall library:
Computer Vision: The Modern Approach, Forsyth and
There
is a number of other fine texts that you can use for general reference:
Photography
(8th edition), London and Upton, (a
great general guide to taking pictures)
Vision Science: Photons to Phenomenology, Stephen Palmer (great book on human visual perception)
Digital Image
Processing, 2nd edition, Gonzalez and Woods (a
good general image processing text)
The Art and Science of Digital Compositing, Ron Brinkmann (everything about compositing)
Multiple View Geometry in Computer Vision, Hartley & Zisserman (a bible on recovering 3D geometry)
The Computer Image, Watt and Policarpo (a
nice “vision for graphics” text, somewhat dated)
3D Computer Graphics (3rd Edition), Watt (a
good general graphics text)
Fundamentals of Computer Graphics, Peter Shirley (another good general graphics text)
Linear Algebra and its Applications, Gilbert Strang (a truly wonderful book on linear algebra)
CLASS
NOTES
The instructor is extremely grateful to a large number of researchers for
making their slides available for use in this course. Steve Seitz and Rick Szeliski have been
particularly kind in letting me use their wonderful lecture notes. In addition, I would like to thank Paul Debevec, Stephen Palmer, Paul Heckbert, David Forsyth, and others, as noted
in the slides. The instructor gladly
gives permission to use and modify any of the slides for academic and research
purposes. However, please do also acknowledge the original sources where
appropriate.
CLASS
SCHEDULE:
|
CLASS DATE |
TOPICS |
Material |
|
Tu
Aug 29 |
Introduction |
|
|
Th Aug 31 |
Camera |
|
|
Tu Sept 5 |
Capturing
Light… in man and machine |
|
|
Th Sept 7 |
Image
Processing |
|
|
Tu Sept 12 |
Image
Warping |
|
|
Th Sept 14 |
Image
Morphing |
|
|
Tu Sept 19 |
Face
Modeling |
|
|
Th Sept 21 |
Guest
Lecture by Mohit |
Slides forthcoming |
|
Tu Sept 26 |
No
class |
|
|
Th Sept 28 |
Fourier
Analysis without tears |
|
|
Tu oct 3 |
Convolution,
Edge Detection, and Sampling |
|
|
Th Oct 5 |
Modeling
Light |
Bob
Shaw, “Light
of Other Days” |
|
Tu Oct 10 |
Homographies
and Mosaics |
|
|
Th Oct 12 |
Image
Pyramids and Blending |
|
|
Tu Oct 17 |
More
Mosaic Madness |
|
|
Tu Oct 24 |
Direct
Automatic Alignment (Optical Flow) |
|
|
Th Oct 26 |
Feature-Based
Alignment and RANSAC |
|
|
Tu Oct 31 |
Video
Texture |
|
|
Th Nov 2 |
Texture
Synthesis |
|
|
Tu Nov 7 |
Background
Subtraction and Matting |
|
|
Th Nov 9 |
Midterm |
|
|
Tu Nov 14 |
Single
View Modeling |
|
|
Th Nov 16 |
More
Single View Geometry |
|
|
Tu Nov 21 |
Triangulation
and Environment Matting |
|
|
Tu Nov 28 |
|
|
|
Th Nov 30 |
Image-Based
Lighting |
|
|
Tu Dec 5 |
Image-Based
Lighting II |
|
|
Th Dec 7 |
3D
from two and more views |
|
CAMERAS:
Although it is not required, students are highly encouraged to obtain a digital
camera for use in the course (one can get a pretty
good camera for under $300). A couple cameras with tripods will be
available on loan from the instructor.
METHOD
OF EVALUATION:
Grading will be based on a set of programming assignments (60%), a midterm
(15%) and a final project presentation (25%).
There will be no final exam. For
the programming assignments, students will be allowed a total of 5 (five) late
days; each additional late day will incur a 10% penalty.
Students taking 15-862
will also be required to submit a conference-style paper describing their final
project.
COMPUTING FACILITIES:
All students will have access to the graphics cluster in Wean Hall
5336. You will need to setup your Andrew account in the right way before you
can longin. Follow these instructions
(replacing 462 with 463). A universal
memory card reader will be installed so that you can download the data from
your Compact Flash, Memory Stick, or other card.
MATLAB:
Students will be encouraged to use Matlab (with the Image Processing
Toolkit) as their primary computing platform.
Besides being a great prototyping environment, Matlab is particularly
well-suited for working with image data and offers tons of build-in image
processing functions. Here is a link to
some useful Matlab resources
PREVIOUS
OFFERINGS OF THIS COURSE:
Previous offerings of this course can be found here.
SIMILAR COURSES IN OTHER
UNIVERSITIES:
- Computational
Photography (Raskar, Northeastern)
- Digital
and Computational Photography (Durand & Freeman, MIT)
- Computational
Photography (
- Computational
Photography (Levoy & Wilburn, Stanford)
- Computer Vision (Seitz & Szeliski, UWashington)
- Instroduction
to Visual Computing and Visual Modeling
(Kutulakos, UToronto)
- Topics
in Image-based Modeling and Rendering (Kriegman, UCSD)
- Symposium
on Computational Photography and Video (May 2005, MIT)
Page design courtesy of Doug
James