See under "Computing the Mean Face": e-mail me a .txt with your face points. Update -- face points are now available.
In this assignment you will produce a "morph" animation of your face
into another student's face. If all goes well, I'll stitch together the
results into a morph through the faces of the whole class.
A morph is a simultaneous warp of the image shape and a cross-dissolve of the image colors. The cross-dissolve is the easy part; controlling and doing the warp is the hard part. The warp is controlled by defining a correspondence between the two pictures. The correspondence should map eyes to eyes, mouth to mouth, chin to chin, ears to ears, etc., to get the smoothest transformations possible.
We will take pictures of everyone in the class. Each student will then get a starting image ("picture A") and an ending image ("picture B") for the animation sequence. Picture A will be the photo of your face. Picture B will be the student whose image name is after yours in alphabetical order (by last name). Your pictures are available here (Updated with 2 missing pics Saturday, March 16!).
You'll morph still picture A into still picture B and produce 61 frames of animation numbered 0-60, where frame 0 must be identical to picture A and frame 60 must be identical to picture B. In the video, each frame will be displayed for 1/30 of a second. Name your frames morph_name1_name2_??.jpg where ?? is 00 to 60 (do use two digits, even for 01, 02, etc.) and name1 and name2 are the basenames of the two images (e.g., "berg").
Here is a workflow summary for the creating a morph:
Now, we describe these steps in more detail.
First, you will need to define pairs of corresponding points on the two
images by hand. The more points, the better the morph, generally. I
suggest using the standard correspondence format described under
"Computing the Mean Face", as you will need to provide this for your own
face anyway. The simplest way is to use the cpselect tool or to write your own little tool using
ginput and plot commands (with hold on
and hold off).
Now, you need to provide a triangulation of these points that will be used for morphing. You can compute a triangulation any way you like, or even define it by hand. A Delaunay triangulation (see dalaunay and related functions) is a good choice since it does not produce overly skinny triangles. You can compute the Delaunay triangulation on either of the point sets (but not both -- the triangulation has to be the same throughout the morph!). But the best approach would probably be to compute the triangulation at midway shape (i.e. mean of the two point sets) to lessen the potential triangle deformations.
You need to write a function:
morphed_im = morph(im1, im2, im1_pts, im2_pts, tri, warp_frac, dissolve_frac);
that produces a warp between im1 and im2
using point correspondences defined in im1_pts and im2_pts
(which are both n-by-2 matrices of (x,y) locations) and the triangulation structure
tri. Note that tri can be computed using delaunay from one set of points, as mentioned earlier.
The parameters warp_frac and dissolve_frac
control shape warping and cross-dissolve, respectively.
In particular, images im1 and im2 are
first warped into an intermediate shape configuration controlled by
warp_frac, and then cross-dissolved according to dissolve_frac.
For interpolation, both parameters lie in the range [0,1]. They are the
only parameters that will vary from frame to frame in the animation. For
your starting frame, they will both equal 0, and for your ending frame,
they will both equal 1.
Given a new intermediate shape, the main task is implementing an affine warp for each triangle in the triangulation from the original images into this new shape. This will involve computing an affine transformation matrix A between two triangles:
A = computeAffine(tri1_pts,tri2_pts)
of these transformation matrices will then need to be used to implement an
inverse warp (as discussed in class) of all pixels. Functions
tsearch and interp2 can come very handy here.
Note, to get full credit for your implementation you should not use
Matlab's build-in offerings for computing transformations, (e.g.
imtransform, cp2tform, maketform -- if you use these
functions, you can get partial credit).
Note, however, that
assumes that your triangulation is always Delaunay.
In our case, this might not always be true --
you may start with a Delaunay triangulation, but through the course of
the morph it might produce skinny triangles and stop being Delaunay. David Martin from Boston College
has kindly given access to his versions of
tsearch that work on any triangulation: mytsearch.m.
Don't worry too much about using fancier interpolation methods, just mapping to the nearest pixel should work ok.
Several fun things are possible with our new morpher.
For example, we can compute the mean face of CP students.
This would involve: 1) computing the average
shape, 2) warping all faces into that shape, and 3) averaging the colors
However, this would also
require a consistent labeling of all the faces.
So, what we will do is ask everyone to label
their own face, but do it in a consistent
manner as shown in the following two images:
For each face image, you should put on your
website a text file with coordinates of x,y positions, one per line (43
lines total). Please name the file with the same name as your picture,
but with a ".txt" extension.
You can read/write such files
pts = [x(:) y(:)];
save -ascii ./berg.txt pts;
If everyone does this in a timely manner, then everyone can use all of the data to compute their mean image. Show the mean image that you got, as well as 1) your face warped into the average geometry, and 2) the average face warped into your geometry. To get you started, here are the faces and points from other classes. Note that these may be of a different scale and translation, and a couple of the annotations might be wrong, so you may want to check them using cpselect before using them and automatically rescale and translate.
Labeled Face images for our class are now available here (updated to add additional faces on 3/22).
To turn in your assignment, email your commented code, readme, and results to firstname.lastname@example.org. If your images are too large, you may use links to an online version, but please send small versions via email!
Use both words and images to show us what you've done. Please:
The core assignment is worth 100 points, as follows:
Thanks to Alexei Efros for creating this cool assignment and giving permission to use it.