Description
1 Convolutions and Correlations
The file you will work on in this section is filters.m.
1.1 Implementation [10 points]
In this section, you will implement your own convolution and correlation in
MATLAB. You should implement the following two functions in filters.m:
function y = my conv2d(I,f)
function y = my corr2d(I,f)
Both of them take an image I and a filter f as inputs and output an image
y that has the same shape as the input image. Your implementation will be
compared with MATLAB function conv2 and filter2. Show an input image
and two corresponding output images in your write-up. You are provided with
example images in folder img to test your implementation, but you are free to
include any image you like in your write-up.
Note: The convolution and correlation formulas showed in the slides are not
exactly the same as the built-in functions in MATLAB. You may need to add
an offset to the original image like this:
h[m, n] = X
k,l
f[k, l]I[m − k + ∆, n − l + ∆]
1.2 Commutative Property [5 points]
Recall that the convolution of an image f : R2 → R and a kernel h : R2 → R
is defined as follows:
(f ∗ h)[m, n] = X∞
i=−∞
X∞
j=−∞
f[i, j] · h[m − i, n − j]
Or equivalently,
(f ∗ h)[m, n] = X∞
i=−∞
X∞
j=−∞
h[i, j] · f[m − i, n − j] = (h ∗ f)[m, n]
Show that this is true (i.e. prove that the convolution operator is commutative:
f ∗ h = h ∗ f). Please answer it in your write-up.
1.3 Linear and Shift Invariance [5 points]
Let f be a function R2 → R. Consider a system f
S
−→ g, where g = (f ∗ h) with
some kernel h : R2 → R. Show that S defined by any kernel h is a Linear Shift
Invariant (LSI) system. In other words, for any h, show that S satisfies both of
the following:
• S[a · f1 + b · f2] = a · S[f1] + b · S[f2]
2
• If f[m, n]
S
−→ g[m, n], then f[m − m0, n − n0]
S
−→ g[m − m0, n − n0]
Please answer it in your write-up.
2 Fourier Transform
The file you will work on in this section is frequency.m.
2.1 1D Fourier Series [3 points]
In this question, you are expected to compute the 1D Fourier series to convert an input function from spatial domain to frequency domain by filling out
the corresponding part of frequency.m. The Fourier transform needs to be
rearranged by shifting the zero-frequency component to the center of the array.
You are allowed to call built-in MATLAB functions. Figure 1 shows the example results for function cos(
2πx
10 ). Please save your resulting figure for function
sin(
2πx
5
) + cos(
2πx
10 ) and include it in your write-up.
Figure 1: 1D Fourier Transform for cos(
2πx
10 )
3
2.2 2D Fourier Transform [3 points]
Now you will implement 2D Fourier transform to convert an image from spatial
domain to frequency domain by filling out necessary lines in frequency.m. The
Fourier transform needs to be rearranged by shifting the zero-frequency component to the center of the array. You are allowed to call built-in MATLAB
functions. Figure 2 shows the example results for image dog.jpg. Pick an arbitrary image other than the example one, save the resulting figure produced by
your code, and include it in your write-up.
Figure 2: 2D Fourier Transform for dog.jpg
2.3 Sampling and Aliasing [8 points]
In the lecture of filters and frequency analysis, we have introduced how to subsample an image and fix aliasing. In this question, you are asked to first downsample an image, then fix the aliasing issue using the algorithm introduced in
the class. (See slides P84-P93 for reference.) The implementation can be done
by filling out the corresponding part in frequency.m. You are not allowed to
call imresize() directly, but you can use other built-in MATLAB functions.
Run your code on zebra.jpg and show the resulting figure in your report.
2.4 Low-pass Filtering and High-pass Filtering [12 points]
In this question, you will implement a procedure to do low-pass filtering and
high-pass filtering on an input image. Both of the low-pass filter and highpass filter should be applied in frequency domain. (See slides P76-P80 for
reference) One low-pass filter you are recommended to try is Gaussian filter.
Images after filtering are shown in spatial domain. You are allowed to call
built-in MATLAB functions to do Fourier transform or create filters. Run your
code on an image and include the resulting figure in your write-up.
4
3 Local Feature Descriptors and Matching
The file you will work on in this section is local feature.m.
In this part, you will be implementing keypoint detector, feature descriptors
and feature matching. Keypoint detectors find particularly salient points, such
as corners, in an image upon which we can apply a feature descriptor. Once we
have descriptors, we can use them to extract local features for each keypoint,
and match keypoints between images by measuring feature similarities. The file
you will work on in this section is local feature.m.
3.1 Harris Corner Detector and Feature Matching [25 points]
1. Complete the implementation of function harris corners(I), which takes
an image and outputs a response map that has the same shape as the input
image. The response map is computed according to the corner response
function.
2. Design and implement your own simple feature descriptor. One design
choice you are recommended to try is to build a gradient histogram for
the local patch.
First, you need to complete the function simple descriptor(patch).
Then, implement describe keypoints(I, keypoints) to get local feature descriptors for all keypoints. (simple descriptor() is called in
describe keypoints())
3. Complete the function match descriptors(desc1, desc2). In this function, you will match the keypoints according to your descriptors. You can
use any distance function mentioned in the class. Run your code on the
provided images img/building 1.jpg and img/building 2.jpg and include the visualization results in your write-up.
3.2 SURF Features [4 points]
In the class, we have talked about SIFT feature descriptor. However, MATLAB
doesn’t have a built-in SIFT function because it is patented. Instead, you can try
another local feature SURF. In this part, you need to find corresponding points
between two images using SURF features. You are not asked to implement
SURF by yourself, so feel free to call any relevant built-in functions. Run your
code on the provided images img/building 1.jpg and img/building 2.jpg
and include the visualization results in your write-up.
4 Filters in Convolutional Neural Networks
The file you will work on in this section is cnn.m.
In this part, you will get a chance to see what CNN learns on a simple dataset.
You are provided with MNIST dataset and a simple pretrained CNN on it.
5
MNIST is a handwritten digit dataset which contains 60,000 training examples
and 10,000 testing examples. All of the examples are 28 × 28 gray images with
category labels 0 to 9. The file you are working on for this section is cnn.m.
Before getting started, make sure that
1. Install MATLAB Deep Learning toolbox by running the following command in MATLAB:
matlab.addons.supportpackage.internal.explorer.showSupportPackages(’ALEXNET’,’tripwire’)
2. Run the first three lines of cnn.m to load images and labels in training set
and test set, load network model and print out the details of each layer.
If everything works well, you should see the same results as in Figure 3.
Figure 3: Layer details of the pretrained CNN on MNIST
4.1 Filter Visualization [7 points]
Complete the provided code to visualize the 16 filters in the conv1 layer and
report your visualization results in your write-up. Select two filters and explain
in your report what properties do each of the filter functions pick up.
4.2 Network Activation Visualization [3 points]
First, select 5 images you like from test set. Then, using the two filters you
selected in section 4.1 to extract activations on each image, and show the visualization results in your write-up.
4.3 Image Retrieval [14 points]
First, select 3 images you like from test set. Then for each testing image, retrieve
5 most similar images from the first 1000 images in the training set. In your
implementation, use Euclidean distance on the activations of conv3 layer as the
similarity metric.
6
5 Questionnaire [1 point]
Approximately how many hours do you spend on this homework? Please answer
it in your write-up.
7
