|
|
|
|
|
15-463 Rendering and Image Processing |
![[SCS dragon logo]](scslogo.gif)
INSTRUCTOR: Alexei Efros (Office hours: Tu 5:00-6:00, NSH 4207)
TA: James
Hays (Office hours: F 4:00-5:00, NSH 4228)
UNIVERSITY UNITS: 12
SEMESTER: Fall 2004
NEWSGROUP: cmu.cs.class.cs463 (read this for important information!)
WEB PAGE:
463.html
LOCATION: Porter Hall A20A
TIME: TR 10:30--11:50 AM
PREREQUISITES:
The prerequisite will be 15-462 Computer Graphics I (or equivalent
undergraduate course) or permission of the instructor.
COURSE
OVERVIEW:
Realistic image synthesis is a central goal of computer graphics. Yet, even the
best computer-generated feature films like "Final Fantasy" or "Shrek" are a far cry from what we might consider
"real". Curiously, the problem here is generally NOT with computer
graphics being unable to model the physics of the real world -- the problem is
with the real world itself! It's just too complex, too noisy, too rich and
vivid to be recreated from scratch by even the most skilled animators. On the
other hand, special effects in movies like "The Matrix" demonstrate
thrilling possibilities -- graphical models that look and move so realistically
that they integrate seamlessly with live action footage. Their trick is exactly
that -- instead of recreating the entire world from scratch, they carefully
combine synthetic and real data together, letting the real world provide visual
complexity while the synthesized world handles controllability.
The
aim of this advanced undergraduate graphics course is to study ways in which
samples from the real world (mainly images and video) can be used to generate
compelling computer graphics imagery. This course will teach how to acquire,
represent, and render scenes from digitized photographs. Several image-based
approaches will be presented, with an emphasis on how to use these techniques
to build practical systems. This hands-on emphasis will be reflected in the
programming assignments, in which students will have the opportunity to acquire
their own images of indoor and outdoor scenes and develop the image analysis
and synthesis tools needed to render and view the scenes on the computer.
TOPICS
TO BE COVERED:
- Image Formation
- Image and Video
Processing (filtering, antialiasing, pyramids)
- Image Manipulation
(warping, morphing, mosaicing, matting,
compositing)
- Data-driven Synthesis
- Image-Based Rendering
- Image-Based Lighting
- Visual Perception
- 3D Data Acquisition
- Global Illumination
- Non-photorealistic
Rendering
PROGRAMMING
ASSIGNMENTS (tentitive):
TEXT:
There is no required text. Various course notes and papers will be made
available. Furthermore, there are two
optional textbooks that you might find helpful.
They will be placed on reserve at the Wean Hall library:
Digital Image Processing, 2nd edition, Gonzalez
and Woods
Computer Vision: The Modern Approach, Forsyth and
There
is a number of other fine texts that you can use for general reference:
The Computer Image, Watt and Policarpo
(a nice “vision for graphics”
text, somewhat dated)
3D Computer Graphics (3rd Edition), Watt (a
good general graphics text)
Fundamentals of Computer Graphics, Peter Shirley (another good general graphics text)
Realistic Image Synthesis Using Photon Mapping, Henrik
Wann Jensen (latest
developments in Global Illumination)
Realistic Ray Tracing, 2nd Edition, Peter Shirley, R. Keith Morley (how to write a perfect ray tracer)
Multiple View Geometry in Computer Vision, Hartley & Zisserman
(a bible on recovering 3D geometry)
CAMERAS:
Although it is not required, students are highly encouraged to obtain a digital
camera for use in the course (one can get a pretty
good camera for under $200). A couple cameras with tripods will be
available on loan from the instructor.
METHOD
OF EVALUATION:
Grading will be based on a set of programming assignments (60%), a midterm
(15%) and a final project presentation (25%).
There will be no final exam. For
the programming assignments, students will be allowed a total of 5 (five) late
days; each additional late day will incur a 10% penalty.
COMPUTING FACILITIES:
All students will have access to the graphics cluster in Wean Hall
5336. You will need to setup your Andrew account in the right way before you
can longin.
Follow these instructions
(replacing 462 with 463). A universal
memory card reader will be installed so that you can download the data from
your Compact Flash, Memory Stick, or other card.
MATLAB:
Students will be encouraged to use Matlab
(with the Image Processing Toolkit) as their primary computing platform. Besides being a great prototyping
environment, Matlab is particularly well-suited for
working with image data and offers tons of build-in image processing
functions. Here is a link to some useful Matlab
resources.
CLASS SCHEDULE:
|
CLASS DATE |
TOPICS |
Material |
|
Tu Aug 31 |
Introduction
|
|
|
Th Sep 2 |
Modeling
Light
|
|
|
Tu Sep 7 |
Image
Processing I
|
|
|
Th Sep 9 |
Color
Processing
|
|
|
Tu Sep 14 |
Neighborhood
Processing (filtering) |
|
|
Th Sep 16 |
Fourier
Analysis |
|
|
Tu Sep 21 |
Sampling
and Image Pyramids |
· Burt and Adelson,
A multiresolution spline with
application to image mosaics, ACM ToG (1983) |
|
Th Sep 23 |
Image
Warping and Mosaicing
|
· Paul Heckbert,
Projecting
Mappings for Image Warping · Rick Szeliski,
Image Alignment and Stitching, A
Tutorial (DRAFT)
|
|
Tu Sep 28 |
Cylindrical
Mosaics and Camera Calibration
|
|
|
Th Sep 30 |
Image
Blending and Compositing
|
|
|
Tu Oct 5 |
Automatic
Image Alignment
|
|
|
Th Oct 7 |
Image
Morphing
|
|
|
Tu Oct 12 |
Blob
Processing
|
|
|
Th Oct 14 |
Image
Stacks
|
|
|
Tu Oct 19 |
Homework
discussion and Midtern Review
|
|
|
Th Oct 21 |
Midterm
|
Location: NSH 1507 |
|
Tu Oct 19 |
Background
Subtraction (and answers to midterm)
|
|
|
Tu Oct 26 |
Modeling
with Weather (guest lecture by Srinivasa Narasimhan)
|
Slides
coming… |
|
Tu Nov 2 |
Faces
and PCA
|
Additional
Rowland,
Ferrett, “Manipulating Facial Appearance through
Shape and Color”,
CG&A, 1995 |
|
Th Nov 4 |
Texture
Synthesis
|
Additional
Schold, et all, “Video
Textures”, SIGGRAPH 2000 |
|
Tu Nov 9 |
Matting
|
Additional
Smith
& Blinn, “Blue
Screen Matting”, SIGGRAPH 1996 |
|
Th Nov 11 |
Shadow
Matting and Flash/No Flash
|
Additional
Petschnigg et all, “Digital
Photography with Flash and No-Flash Image Pairs”, SIGGRAPH 2004 Eisemann and Durand, “Flash
Photography Enhancement via Intrinsic Relighting”, SIGGRAPH
2004 |
|
Tu Nov 16 |
|
Additional
Debevec & Malik, “Recovering
High Dynamic Range Radiance Maps from Photographs”, SIGGRAPH 1997 |
|
Th Nov 18 |
Image-Based
Lighting
|
Additional
Resources: |
|
Tu Nov 23 |
Image-Based
Lighting II
|
|
|
Tu Nov 30 |
Scene
Reconstruction from Single View
|
Additional
Materials: “Tour
into the Picture”, SIGGRAPH ‘97 |
|
Th Dec 2 |
More
Single View Geometry™
|
Additional
Materials: |
|
Tu Nov 30 |
3D
from two and more views
|
|
|
Tu Nov 30 |
Wrap-up
|
|
PREVIOUS
OFFERINGS:
Previous offerings of this course can be found here.
Page design courtesy of Doug
James