Description
1 Theory Questions: Epipolar Geometry [29 points]
1. (14 points.) Consider the case of two cameras viewing an object such
that the second camera di↵ers from the first by a pure translation parallel
to the image plane. Show that the epipolar lines in the two cameras are
parallel.
2. (15 points.) Suppose two cameras fixate on a point P (see figure 1) in
space such that their principal axes intersect at that point. Show that if
the image coordinates are normalized so that the coordinate origin (0, 0)
coincides with the principal point, the F33 element of the fundamental
matrix is zero.
Figure 1: C1 and C2 are the optical centers. The principal axes intersect at
point P.
2 3D Reconstruction [70 points]
In this problem, you will estimate the fundamental matrix from a pair of images.
Then you will compute the camera matrices and triangulate the 2D points to
obtain the 3D scene structure. The data for this problem can be found in the
temple/ directory, where there are two images of a temple.
Figure 2: Two novel views of an object reconstructed in 3D from two images.
2.1 The Eight Point Algorithm [20 points]
FILE TO EDIT: eightpoint.m
FILE TO RUN: Q2 1.m
ADDITIONAL DELIVERABLES: Q2 1.mat, screenshot(s) in writeup
The 8-point algorithm discussed in class and outlined in Section 10.1 of
Forsyth & Ponce is arguably the simplest method for estimating the fundamental matrix. For this section, you can use the provided correspondences from
temple/some corresp.mat. Pass them to your implementation of the 8-point
algorithm.
For this portion of the assignment, fill in the function in eightpoint.m with
the following signature:
function F = eightpoint(X,Y,M);
where X and Y are each N ⇥ 2 matrices with coordinates that constitute correspondences in the first and second image respectively (the format returned
by cpselect). M is the scaling factor. Remember that the first coordinate of
a point in the image is its column entry, and the second coordinate is the row
entry. Note: 8-point is just a figurative name. Your algorithm should use an
overdetermined system.
You should scale the data by dividing each coordinate by M, the largest image dimension. After computing F, you will have to “unscale” the fundamental
matrix.
To visualize the correctness of your estimated F, use the provided function
displayEpipolarF. In your writeup, include a screenshot of the output
of displayEpipolarF showing three points and the epipolar lines in the
other image, demonstrating that the corresponding points are on the
epipolar lines in the second image. This image should look like the
one below (Figure 3).
Also save F, M, and your points as Q2 1.mat.
Figure 3: Points on the epipolar lines.
2.2 Metric Reconstruction [20 points]
FILE TO EDIT AND RUN: Q2 2.m
ADDITIONAL DELIVERABLES: screenshot(s) in writeup
The fundamental matrix can be used to determine the camera matrices of
the two views used to generate F. In general, M1 and M2 will be projective
camera matrices, i.e., they are related to each other by a projective transform.
For details on recovering M2, see 7.2 in Szeliski. Thankfully, we have provided
you with the function camera2(F, K1, K2, pts1, pts2) that will find M2
given F , K1, and K2 and a number of correspondence points. Load and use
the correspondence points provided in some corresp.mat. Load and use the
intrinsic matrices Ki from temple/intrinsics.mat.
We have also provided a function to triangulate a set of 2D coordinates in
the image to a set of 3D points with the signature:
function P = triangulate(M1, pts 1, M2, pts 2)
where pts 1 and pts 2 are the 2D image coordinates and P is a matrix with
the corresponding 3D points, per row. Make sure to multiply your M1, M2 by
the camera matrices!
Included in the homework folder is a file many corresp.mat which contains
288 hand-selected points from im1 saved in the variables x1 and y1.
Now, we can determine the 3D locations of these point correspondences using
the triangulate function. These 3D point locations can then be plotted using
the MATLAB function scatter3. The resulting figure can be rotated using the
Rotate 3D tool, which can be accessed through the figure menubar.
Please take six screenshots showing di↵erent views of the 3D visualization,
and include them in your writeup.
2.3 3D Correspondence [30 points]
FILE TO EDIT: epipolarCorrespondence.m
FILE TO RUN: Q2 3.m
ADDITIONAL DELIVERABLE: Q2 3.mat, screenshot(s) in writeup
To conclude this project, you will calculate correspondences using 3D information via epipolar geometry.
In epipolarCorrespondence.m, fill in the function
function[x2, y2] = epipolarCorrespondence(im1, im2, F, x1, y1)
and associated helper functions getWindow and computeDifference.
epipolarCorrespondence takes in the x and y coordinates of a pixel on
im1, and your fundamental matrix F, and returns the coordinates of the pixel
on im2 which correspond to the input point. Fortunately, we know the fundamental matrix F. Therefore, instead of searching for our matching point at
every possible location in im2, we can simply search over the set of pixels that
lie along the epipolar line (recall that the epipolar line passes through a single
point in im2 which corresponds to the point (x1, y1) in im1). Slide a window
along the epipolar line and find the one that matches most closely to the window around the point in im1. There are various ways to compute the window
similarity. For this assignment, simple methods such as the Euclidean or Manhattan distances between the intensity of pixels should suce. Implementation
hints:
• Experiment with various window sizes.
• It may help to use a Gaussian weighting of the window, so that the center
has greater influence than the periphery.
• Since the two images only di↵er by a small amount, it might be beneficial
to consider matches for which the distance from (x1, y1) to (x2, y2) is
small.
To test your function epipolarCorrespondence, we have included a GUI:
function[coordsIM1, coordsIM2] = epipolarMatchGUI(im1, im2, F)
This script allows you to click on a point in im1, and will use your function to
display the corresponding point in im2. The process repeats until you rightclick in the figure, and the sets of matches will be returned. Use your function
epipolarCorrespondence to calculate the corresponding points in im2 from
chosen points in im1. Save your points and F to Q2 3.mat.
Also include as part of your writeup a screenshot of the output
of epipolarMatchGUI demonstrating three identified correspondences
across the two temple images.
3 Questionnaire [1 point]
Approximately how many hours did you spend on this homework? Please answer
this question in your writeup.
