15-463 (15-663, 15-862)
Computational Photography, Fall 2018
Time: Mondays, Wednesdays 12:00 PM - 1:20 PM
Location: TBD
Instructor: Ioannis (Yannis) Gkioulekas
Teaching Assistant: TBD
Course Description

Computational photography is the convergence of computer graphics, computer vision and imaging. Its role is to overcome the limitations of the traditional camera, by combining imaging and computation to enable new and enhanced ways of capturing, representing, and interacting with the physical world.

This advanced undergraduate course provides a comprehensive overview of the state of the art in computational photography. At the start of the course, we will study modern image processing pipelines, including those encountered on mobile phone and DSLR cameras, and advanced image and video editing algorithms. Then we will proceed to learn about the physical and computational aspects of tasks such as 3D scanning, coded photography, lightfield imaging, time-of-flight imaging, VR/AR displays, and computational light transport. Near the end of the course, we will discuss active research topics, such as creating cameras that capture video at the speed of light, cameras that look around walls, or cameras that can see through tissue.

The course has a strong hands-on component, in the form of seven homework assignments and a final project. In the homework assignments, students will have the opportunity to implement many of the techniques covered in the class, by both acquiring their own images of indoor and outdoor scenes and developing the computational tools needed to extract information from them. For their final projects, students will have the choice to use modern sensors provided by the instructors (lightfield cameras, time-of-flight cameras, depth sensors, structured light systems, etc.).

Prerequisites

This course requires familarity with linear algebra, calculus, programming, and doing computations with images. In particular, either of the following courses can serve as proof that you satisfy these prerequisites:

If you have not taken any of the above courses but want to enroll, please contact the instructor — exceptions will be made on a case-by-case basis. The course does not require prior experience with photography or imaging.

Textbook

Readings will be assigned from the following textbook (available online for free):

Additional readings will be assigned from relevant papers. Readings will be posted at the last slide of each lecture.

The following textbooks can also be useful references but are not required:

Evaluation

Your final grade will be made up from:

Homework assignments: All homework assignments will have both a photography and a programming component, where students will use DSLR cameras to capture and process their own images. The programming component of all assignments will be done in Matlab. The first assignment will serve as a short tutorial for Matlab and DSLR cameras. We have collected a few useful Matlab resources here.

Late days: For the homework assignments, students will be allowed a total of five free late days. Any additional late days will each incur a 10% penalty. No more than three late days may be used per each assignment.

Final project: More details about the final project will be provided after the class begins.

15-663, 15-862: Students taking 15-663 or 15-862 will be required to do a more substantial final project, as well as submit a longer paper describing their project.

Submitting homeworks: We use Canvas (link TBD) for submitting and grading homeworks.

Cameras

Students are highly encouraged to obtain a digital camera for use in the course. Any digital camera with manual controls should work. A few cameras will be provided by the instructors.

Email, Office Hours, and Discussion

Email: Please use [15463] in the title when emailing the teaching staff!

Office hours: TBD.

Discussion: We use Piazza (link TBD) for class discussions and announcements.

(Tentative) Syllabus

Slides will be updated on this website after each lecture.

DateTopicsSlidesAssignments
TBDIntroduction
TBDPinholes, cameras, and exposure
TBDSensors and noise
TBDDigital photography pipeline
TBDHigh dynamic range imaging and tonemapping
TBDColor
TBDModern image filtering
TBDImage warping
TBDImage blending
TBDGradient-domain image processing
TBDPyramids and local Laplacian filtering
TBDHigh performance image processing
TBDLenses and other optical elements
TBDFocal stacks and depth from defocus
TBDLightfields
TBDDeconvolution
TBDCoded photography
TBDCompressive imaging
TBDHyperspectral imaging
TBDCamera models and geometric calibration
TBDStructure from motion
TBDStereo and structured light
TBDLight transport
TBDComputational light transport
TBDTime-of-flight imaging
TBDNon-line-of-sight imaging
TBDFourier optics
TBDInterferometry
TBDModern sensors
TBDTime-lapse and hyper-lapse
TBDData-driven methods
Previous Course Offerings
Similar Courses at CMU and Elsewhere
Special Thanks

These lecture notes have been pieced together from many different people and places. Special thanks TBD.