Programming Project #5 15-463: Computational Photography

Tour into the Picture: Single View Modeling

Due Date: by 11:59pm, Thursday, Dec 1

The goal of this project is to create a simple, planar 3D scene from a single photograph. The project will follow the description in Tour into the Picture by Horry et al. in modeling the scene as a 3D axis-parallel box. First, we will let the user specify simple constraints on that box (the back wall plus the vanishing point). Then, it’s just a matter of extracting the coordinates of the box in 3D, and texture-mapping the faces of the box.

In this project, you are free to use any programming language you like (but you can do everything in Maltab!). I have provided sample starting code here. It includes a (rather messy) interface for specifying the user constraints. It also shows how to setup texture-mapped surfaces in Matlab. You part of the project is actually pretty straightforward: you will need to compute 3D coordinates of each vertex of each of the five planes. Then you will define 3D geometry corresponding to these planes. Finally, you will use your homography warping code from previous projects to rectify the textures for the planes and then texturemap them onto the 3D model. You will then be able to move and rotate the camera and look at the scene from different viewpoints.

The only other parameter that you will need to worry about is the focal length f. You can just guess it (a wrong f will simply mean that your scene is too deep or too shallow). Alternatively, you can get f from the EXIF data (it’s not exactly correct focal length, but it’s in the right ballpark). From David Martin: We will take advantage of the fact that cameras write some of their intrinsic paramters into the EXIF metadata inside JPEG and TIFF image files. In particular, the focal length and pixel size are stored in the EXIF tags. Use this code to read EXIF tags in MATLAB. You'll need to compile the exif function with the command mex exif.c. Use this exifFocalLength.m function to get the camera focal length in pixels from an image file.

Submit Your Results

You will need to submit all your code. Capture at least 3 photos conforming to the assumptions of the TIP model. For each, show the input photo and a couple of novel viewpoints.

Bells & Whistles

• (10 points) Follow the paper to add foreground objects to your models. You should use the “alpha” function to set part of the foreground rectangles transparent.
• (10 points) Produce a movie fly-through of the 3D scene. (check out functions like camdolly and getframe). You can do fun Escher-like things, like flying into a copy of the photo recursively, etc.
• (10 points) Feeling altruistic? Produce a better TIP_GUI interface for future generations of Computational Photography students. Consider using “callback” functions. If you want to see how a particular function works in Matlab, type “type functionname”.