Programming Project #4g (graduate students only) 15-463: Computational Photography

Tour into the Picture: Single View Modeling

Due Date: by 11:59pm, Friday, November 12

The goal of this project is to create a simple, planar 3D scene from a single photograph. The project will follow the description in Tour into the Picture by Horry et al. in modeling the scene as a 3D axis-parallel box. First, we will let the user specify simple constraints on that box (the back wall plus the vanishing point). Then, it’s just a matter of extracting the coordinates of the box in 3D, and texture-mapping the faces of the box. The paper has a rather poor description of the process, so consult the lecture notes.

In this project, you are free to use any programming language you like (but you can do everything in Maltab!). I have provided sample starting code here. It includes a (rather messy) interface for specifying the user constraints. It also shows how to setup texture-mapped surfaces in Matlab. You part of the project is actually pretty straightforward: you will need to compute 3D coordinates of each vertex of each of the five planes. Then you will define 3D geometry corresponding to these planes. Finally, you will use your homography warping code from previous projects to rectify the textures for the planes and then texturemap them onto the 3D model. You will then be able to move and rotate the camera and look at the scene from different viewpoints. The only other parameter that you will need to worry about is the focal length f. You can just guess it (a wrong f will simply mean that your scene is too deep or too shallow). Alternatively, you can get f from the EXIF data (it’s not exactly correct focal length, but it’s in the right ballpark).

Submit Your Results

You will need to submit all your code. Capture at least 3 photos conforming to the assumptions of the TIP model. For each, show the input photo and a couple of novel viewpoints.

Bells & Whistles

• (10 points) Follow the paper to add foreground objects to your models. You should use the “alpha” function to set part of the foreground rectangles transparent.
• (15 points) Produce a movie fly-through of the 3D scene. (check out functions like camdolly and getframe). You can do fun Escher-like things, like flying into a copy of the photo recursively, etc.
• (15 points) Feeling altruistic? Produce a better TIP_GUI interface for future generations of Computational Photography students. Consider using “callback” functions. If you want to see how a particular function works in Matlab, type “type functionname”.
• (20 points) Implement a more general, two-point perspective version of TIP. Let the user specify the horizon line and the floor/wall poly-line. You can handle the ceiling any way you like (e.g. throw it away).