Due Date: April 30th (Last Day of Classes) 2010 11:59 pm What to submit: Code + webpage write up Project 5 (a.k.a Meta Project 5) is an alternative to doing a Final Project. Since it is META it requires you to implement 2 (TWO) out of the list of options below:
SEAM CARVING One of the (not so) recent (anymore) papers at SIGGRAPH that made news was
Seam
Carving for Content-Aware Image Resizing
by Shai Avidan and Ariel Shamir. In this assignment, you'll be
implementing the basic algorithm presented therein. To wit, you'll be
designing a program which can shrink an image (either horizontally or
vertically) to a given dimension. For a brief overview of the algorithm
and some inspiring results, check out the video.
Project 2 from CS 15463 fall 2008 has more details.
SINGLE VIEW MODELING The goal of this
project is to create a simple, planar 3D scene from a single
photograph. The project will follow the description in Tour into the Picture by Horry et al.
in modeling the scene as a 3D axis-parallel box. First, we will
let the user specify simple constraints on that box (the back wall plus
the vanishing point). Then, it’s just a matter of extracting the
coordinates of the box in 3D, and texture-mapping the faces of the box.
The paper has a rather poor description of the process, so consult the
lecture notes or section 2.2 of this thesis for details. Some sample starting code can be found here.
It includes a (rather messy) interface for specifying the user
constraints. It also shows how to setup texture-mapped surfaces
in Matlab. You part of the project is actually pretty
straightforward: you will need to compute 3D coordinates of each vertex
of each of the five planes. Then you will define 3D geometry
corresponding to these planes. Finally, you will use your
homography warping code from previous projects to rectify the textures
for the planes and then texturemap them onto the 3D model.
You will then be able to move and rotate the camera and look at the
scene from different viewpoints. The only other parameter that you will
need to worry about is the focal length f. You can just guess it
(a wrong f will simply mean that your scene is too deep or too
shallow). Alternatively, you can get f from the EXIF data (it’s
not exactly correct focal length, but it’s in the right
ballpark). Project 5 from CS 15463 fall 2008 has more details.
TRIANGULATION MATTING AND COMPOSITING
Pick an
interesting, semitransparent object and produce an alpha matte by
shooting the object against two different backgrounds. That is,
given the four images C1, C2 (object against two backgrounds) and Ck1,
Ck2 (just the backgrounds), compute the image of the object Co and the
corresponding alpha matte. This can be done simply by solving a
system of linear equations on Slide 15 of the Matting lecture.
The only trick to remember is that in solving for the unknowns, it’s
very useful here to consider the colors of Co to be premultiplied by
the alpha, e.g. Co = [aR, aG, aB]. While the
backgrounds can be any images at all, we advise to use uniform colors
since the refraction (which we are not modeling) in some transparent
objects can change the pixel correspondence between foreground and
background shots. Another consideration is having enough light in
the scene to avoid noise in the resulting images (but be careful not to
create too many shadows). Make sure that you take all the images
under the same settings (set camera to manual mode). After
successfully acquiring an object with its matte, you will composite the
object into a novel (and interesting!) scene. For example, you
can put a giant coke bottle at the entrance to Wean Hall. Since
the orientation of the camera will most likely be different for your
object and the novel scene, you will need to compute a homography and
warp one of the images using code from the Mosaicing project. Put
a square somewhere when you capture your object, and another one on the
ground of the target composite scene (or use one of the square-like
tiles on the ground). Use these 4->4 points to estimate a
homography
VIDEO TEXTURES
Implement a
simple version of the Video Textures paper. Acquire a video sequence of
some repeatable phenomenon, and compute transitions between frames of
this sequence so that it can be run forever. Don’t worry about
the fancy stuff like blending/warping and Q-learning for detecting
dead-ends. Just precompute a set of good transitions and jump
between them randomly (but always take the last good transition of the
sequence to avoid dead ends). The
most challenging part of this project is dealing with video, which is
not very easy. Generally, pick a simple video texture for which
you don’t need a lot of video data (30 sec. at most). Processing
video in Matlab is a bit tricky. Theoretically, there is aviread
but, under linux, it will only read uncompressed AVIs. Most
current digital cameras produce video in DV AVI format. One way
to deal with this is to splice up the video into individual frames and
then read them into Matlab one by one. On the graphics cluster,
you can do (some variant of) the following to produce the frames from a
video:
Also note
that handling video is a time-consuming thing (not just for you, but
for the computer as well). If you shoot a minute of video, that’s
already 60*30=1800 images! So, start early and don’t be afraid to
let Matlab crunch numbers overnight.
RECOVERING HIGH DYNAMIC RANGE IMAGES
Follow
the procedure outlined in the Debevec paper to recover a HDR image from
a set of images with varying exposure. You will need to carefully
read and completely understand the paper first but then the
implementation is straightforward (you can use the Matlab code in the
paper and shown in lecture). Your result should be a radiance map of
the image. When acquiring the images, make sure that you only
vary the shutter speed and not the aperture (switch the camera to fully
manual mode). Also, try to experiment with some of the simple
tone-mapping approaches, like global scaling, taking the first 0…255
values, the global operator presented in class (L/(1+L)). Which
one works best for you?
![[SCS dragon logo]](http://graphics.cs.cmu.edu/courses/15-463/2010_spring/scslogo.gif)