


Programming Project #2 15463: Computational
Photography 
FACE MORPHING
Due Date: by 11:59pm, Tu, Sept 27
In this assignment you
will produce a "morph" animation of your face into another student's face.
Each of you will generate two seconds of animation, which we will record onto
video for you. The final video will thus be a metamorphosis through each of the
faces in the class.
A morph is a
simultaneous warp of the image shape and a crossdissolve of the image colors.
The crossdissolve is the easy part; controlling and doing the warp is the hard
part. The warp is controlled by defining a correspondence between the two
pictures. The correspondence should map eyes to eyes, mouth to mouth, chin to chin,
ears to ears, etc., to get the smoothest transformations possible.
We will take pictures
of everyone in the class. Each student
will then get a starting image ("picture A") and an ending image
("picture B") for the animation sequence. Picture A will be your
face. Picture B will be another student or other primate. Your pictures will be available here:
http://www.cs.cmu.edu/afs/andrew/scs/cs/15463/pub/www/projects/proj2/
Use your photo as Picture A and the next person’s photo (in the directory
listing) as Picture B.
You'll morph still
picture A into still picture B and produce 61 frames of animation numbered
060, where frame 0 must be identical to picture A and frame 60 must be
identical to picture B. In the video, each frame will be displayed for 1/30 of
a second. Name your frames morph??.jpg where ?? is 00 to 60.
DEFINING CORRESPONDENCES
First, you will need to
define pairs of corresponding points on the two images by hand (the more
points, the better the morph, generally).
The simplest way is probably to use the cpselect tool or write your own little
tool using ginput and plot commands (with hold on and hold off).
Now, you need to provide a triangulation of these points that will be
used for morphing. You can compute a
triangulation any way you like, or even define it by hand. A Delaunay
triangulation (see dalaunay and related functions) is a good choice since it does
not produce overly skinny triangles. You
can compute the Delaunay triangulation on either of
the point sets (but not both – the triangulation has to be the same
throughout the morph!). But the best
approach would probably be to compute the triangulation at midway shape (i.e.
mean of the two point sets) to lessen the potential triangle deformations.
THE MORPH
You need to write a
function:
morphed_im = morph(im1, im2, im1_pts, im2_pts, tri, warp_frac,
dissolve_frac);
that produces a warp between im1 and im2 using
point correspondences defined in im1_pts and im2_pts (which
are both nby2 matrices of (x,y) locations) and the
triangulation structure tri.. The parameters warp_frac
and dissolve_frac control shape warping and crossdissolve,
respectively. In particular, images im1 and im2 are
first warped into an intermediate shape configuration controlled by warp_frac, and then crossdissolved
according to dissolve_frac. For interpolation, both
parameters lie in the range [0,1]. They are the only
parameters that will vary from frame to frame in the animation. For your
starting frame, they will both equal 0, and for your ending frame, they will
both equal 1.
Given a new intermediate shape, the
main task is implementing an affine warp for each triangle in the triangulation
from the original images into this new shape.
This will involve computing an affine transformation matrix A between
two triangles:
A = computeAffine(tri1_pts,tri2_pts)
A set
of these transformation matrices will then need to be used to implement an
inverse warp (as discussed in class) of all pixels. Functions tsearch and interp2 can
come very handy here. But note that you
are not allowed to use Matlab’s buildin
offerings for computing transformations, (e.g. imtransform,
cp2tform, maketform, etc).
COMPUTING THE “MEAN FACE”
Several
fun things are possible with our new morpher. For example, we can compute the mean face of
15463 students. This would involve: 1)
computing the average shape, 2) warping all faces into that shape, and 3)
averaging the colors together. However,
this would also require a consistent labeling of all the faces. So, what we will do
is ask everyone to label their own face (and maybe a few others of their own
choosing), but do it in a consistent manner as shown in the following two
images: points, point_labels.
For each face image, you should put on your website a text file with
coordinates of x,y
positions, one per line (42 lines total).
You can read/write such files with save
–ascii and load –ascii
commands. If everyone does this in a
timely manner, then everyone can use all of the date to compute their mean
image. Show the mean image that you
got, as well as 1) your face warped into the average geometry, and 2) the
average face warped into your geometry.
SCORING
To get
full credit, you will need to implement a warping algorithm and turn in a video
morph. You will also need to compute the mean face and show results of warping
your face into the mean face. Doing some
Bells & Whistles will earn you extra points.