15-463, 15-663, 15-862
Computational Photography, Fall 2019
|Time:||Mondays, Wednesdays 12:00 PM - 1:20 PM|
|Instructor:||Ioannis (Yannis) Gkioulekas|
|Teaching Assistant:||Tiancheng Zhi|
Computational photography is the convergence of computer graphics, computer vision, optics and imaging. Its role is to overcome the limitations of traditional cameras, by combining imaging and computation to enable new and enhanced ways of capturing, representing, and interacting with the physical world.
This advanced undergraduate course provides a comprehensive overview of the state of the art in computational photography. At the start of the course, we will study modern image processing pipelines, including those encountered on mobile phone and DSLR cameras, and advanced image and video editing algorithms. Then we will continue to learn about the physical and computational aspects of tasks such as 3D scanning, coded photography, lightfield imaging, time-of-flight imaging, VR/AR displays, and computational light transport. Near the end of the course, we will discuss active research topics, such as creating cameras that capture video at the speed of light, cameras that look around walls, or cameras that can see below skin.
The course has a strong hands-on component, in the form of seven homework assignments and a final project. In the homework assignments, students will have the opportunity to implement many of the techniques covered in the class, by both acquiring their own images of indoor and outdoor scenes and developing the computational tools needed to extract information from them. Example homeworks include building end-to-end HDR imaging pipelines and structured light scanners. For their final projects, students will have the choice to use modern sensors and other optical instrumentation provided by the instructors (lightfield cameras, time-of-flight sensors, projectors, laser sources, and so on).
Cross-listing: This class is cross-listed as 15-463 (for undergraduate students), 15-663 (for Master's students), and 15-862 (for PhD students). Please make sure to register for the section of the class that matches your current enrollment status.
This course requires familarity with linear algebra, calculus, programming, and doing computations with images. In particular, either of the following courses can serve as proof that you satisfy these prerequisites:
If you have not taken any of the above courses but want to enroll, please contact the instructor — exceptions will be made on a case-by-case basis. The course does not require prior experience with photography or imaging.
Readings will be assigned primarily from relevant papers. Readings will be posted at the last slide of each lecture.
Additionally readings may be assigned from the following textbooks, which can also be useful references in general. All of them are available online from the CMU library:
Your final grade will be made up of:
Homework assignments: All homework assignments will have a programming component and a photography component, where students will use a DSLR camera to capture and process their own images. The programming component of all assignments be done in Matlab. The first assignment will serve as a short tutorial in Matlab and DSLR cameras. We have collected a few useful Matlab resources here.
Bonus homework assignment: There will be a seventh optional homework assignment that can be used to make up for lost credit.
Late days: For the homework assignments, students will be allowed a total of five free late days. Any additional late days will each incur a 10% penalty.
Collaboration policy: Students are encouraged to work in groups but each student must submit their own work. This includes: writing your own code, building your own imaging setups, taking your own photographs, and producing your own writeup. If you work as a group, include the names of your collaborators in your writeup. You absolutely should not share or copy code. Additionally, you should not use any external code unless explicitly permitted. Plagiarism is strongly prohibited and will lead to failure of this course.
Submitting homeworks: We will use Canvas for submitting and grading homeworks.
Final project: Details about the final project are available here.
15-663, 15-862: Students taking 15-663 or 15-862 will be required to do a more substantial final project, as well as submit a longer paper describing their project.
Students are encouraged to obtain a digital camera for use in the course — any digital camera with manual controls should work. However, this is not a requirement, and about 20 cameras will be provided by the instructors for students who do not have one.
Email: Please use  in the title when emailing the teaching staff!
Office hours: Teaching staff have regular office hours at the following times.
Feel free to email us about scheduling additional office hours.
Discussion: We will use Piazza for class discussion and announcements.
We are actively looking for students at all levels (undergraduates, MS, PhD) to help in projects on various aspects of computational photography, imaging, rendering, and graphics in general. If you are interested, please send Yannis an email (or talk to him in person in class).
You should also take a look at the imaging group website, to find more information about related projects active here at CMU.
Finally, you are welcome to attend the imaging group meetings, every Friday, 3:00 - 4:00 PM, at WEH 5421. If you want to receive announcements about the group meetings, as well as other emails regarding photography and imaging research, ask Yannis to add you to the imaging group's mailing list.
Slides will be updated on this website after each lecture.
|M, Aug 26||Introduction||pdf, pptx|
|W, Aug 28||Digital photography pipeline||pdf, pptx|
|F, Aug 30||HW1 out|
|M, Sep 2||No class (Labor day)|
|W, Sep 4||Pinholes and lenses||pdf, pptx|
|M, Sep 9||Photographic optics and aberrations||pdf, pptx|
|W, Sep 11||Exposure and high-dynamic-range imaging||pdf, pptx|
|F, Sep 13||HW1 due, HW2 out|
|M, Sep 16||Noise modeling and calibration||pdf, pptx|
|W, Sep 18||Color||pdf, pptx|
|M, Sep 23||Bilateral and edge-aware filtering||pdf, pptx|
|W, Sep 25||Gradient-domain image processing||pdf, pptx|
|F, Sep 27||HW2 due, HW3 out|
|M, Sep 30||Focal stacks and lighfields||pdf, pptx|
|W, Oct 2||Focal stacks and lightfields (continued)||pdf, pptx|
|M, Oct 7||Deconvoluion||pdf, pptx|
|W, Oct 9||Coded photography||pdf, pptx|
|F, Oct 11||HW3 due, HW4 out|
|M, Oct 14||Geometric camera models and calibration||pdf, pptx|
|W, Oct 16||No class|
|M, Oct 21||Radiometry and reflectance||pdf, pptx|
|W, Oct 23||Photometric stereo||pdf, pptx|
|F, Oct 25||HW4 due|
|M, Oct 28||Two-view geometry||pdf, pptx|
|W, Oct 30||Stereo and structured light||pdf, pptx||HW5 out|
|M, Nov 4||Global illumination||pdf, pptx|
|W, Nov 6||Guest lecture: Aswin Sankaranarayanan|
|M, Nov 11||Computational light transport||pdf, pptx|
|W, Nov 13||Time-of-flight imaging||pdf, pptx||HW5 due|
|M, Nov 18||Fourier Optics||pdf, pptx||HW6 out|
|W, Nov 20||Non-line-of-sight imaging|
|M, Nov 25||Special topics|
|W, Nov 27||No class (Thanksgiving)|
|M, Dec 2||Special topics||HW7 out|
|W, Dec 4||Special topics|
|F, Dec 6||HW6 due, HW7 due|
These lecture notes have been pieced together from many different people and places. Special thanks TBD.