## Description

I. Theoretical Problems (Total 8 points)

Write a report on your solutions for the theoretical problems. This report can be handed in be

by adding to the electronic pdf/Word report of the theoretical part (preferred), or handed in on

paper during the class lecture on Tue Mar 22, or

Problem 1: Epipolar Geometry with 3 Cameras (2 pts)

We discussed epipolar geometry betwen two cameras, which limits a search for correspondence to

an epipolar line. The Java demo (http://www.ai.sri.com/ luong/research/Meta3DViewer/EpipolarGeo.html)

also illustrates the epipolar geometry for 3 camera views. Given the fact that for two cameras,

the constraint for finding a corresponding point in the right camera for each point in the left

camera is a line (i.e. still one degree of freedom), discuiss the situation with 3 cameras. Could

it be that with 3 cameras, point to point correspondences are uniquely defined? Why or why

not?

Figure 1: Epipolar geometry with 3 camera views.

Problem 2: Epiploar geometry and disparity forward translating camera (4pts)

In our course lecture, we discussed the epipolar geometry and fundamental matrix of a forward

translating camera.

• Given the definition of the epipolar plane as the plane defined by the two optical centers

and a world point, explain why the geometry for a forward moving camera results in a set

of radially oriented intersecting planes as shown in the following figure.

• Given geometric considerations of a forward moving camera, develop a framework to calculate depth of world points from disparity as observed in consecutive frames. Make use

of the definition of disparity as presented for the case of a stereo camera setup.

Figure 2: Epipolar geometry with 3 camera views.

I. Practical Problems

Problem 4: 3D from stereo image pairs (total 24 pts)

This objective of this assignment is to take a stereo pair of images, and explore the epiploar

gemeotry and the ability to recover depth from a pair of images. For this assignment, you

will need two stereo pairs of your choice, one with arbitrary translation and rotation of the

two camera views, and one where you simulate a translated camera by shifting one camera

horizontally and by recording the distance of the baseline.

4a: Epipolar geometry from F-matrix (12 points)

We discussed the essential and fundamental matrices and its properties to establish a relationship

between corresponding points in the left and right cameras, and relationship of a points and

their associated epipolar lines.

Following the examples discussed in our slides (CS6643-CV-S2016-chap11-multiple-views-partII-animated.pdf, Fig. 3 and chapter 7.1 in our textbook (7.1 new book, 10/11 old book), you

will explore the following steps:

• Shoot a stereo-pair of a scene of your choice with your camera, where you take a left and

a right picture where you translate and rotate the camera.

• Specify a set of corresponding pixel landmark pairs in the left and right camera views.

• Calculate the F-matrix using instructions as discussed in the slides.

• Choose a point in the left image and calculate the epipolar line in the right image. Display

the point and the line as overlays in your images.

• Do the same for a point in the right image and epipolar line in the left image.

• Calculate the position of the epipole of the left camera (see last two slides for instructions).

Discuss if the calculated position seems reasonable.

Figure 3: Concept of epipolar point to line transformation via F matrix.

4b: 3D Object geometry via triangulation (12 pts)

Using a setup with cameras with parallel image planes, take a stereo picture of an object with

simple geometry, e.g. a cube (see Fig. 4. Such objects are defined by a small set of landmarks

(here 3D corners) and can be reconstructed by displaying a set of lines joining these key points.

• Define a small set of corresponding key locations by either manual definition of landmarks

or a correlation-based image processing method (with selection of only the major key

points).

• Calculate horizontal and vertical disparity in pixel units and mm-units, and from those

the 3D point coordinates (X,Y,Z).

• Use Matlab or your software to display the 3D points and edge lines for the reconstructed

object (only those visible in your images). Choose display viewpoints different from the

camera views to verify the quality of 3D reconstruction.

• Discuss your results.

Figure 4: Two pictures of a simple geometric object (here vertical stereo) and its 3D reconstruction with a wireframe illustration.