Description
2 Background
The purpose of this assignment is to solve a problem involving images with two different types of noise.
As part of this exercise, you will implement a vector median filter since this filter is not available in the
skimage library.
You have been provided with an images folder which contains some sub-folders:
noisy: These are the noisy images corrupted with two different kinds of noise — additive noise and
impulse noise. The severity of the additive noise differs from image to image. The impulse noise
probability is the same for all images.
noisy-test: These are the same as the noisy folder except that only three of the images are included
here. This is because your vector median filter implementation will likely be quite slow, so it is best
to develop your whole assignment solution using only the images in this folder. Then, if you have
time you can run it on all of the images in the noisy folder, but only if you have time! It’s acceptable
to submit a solution that is tested only on these three images.
noiseless: The original images free of corruption from noise. These images are only to be used to
compute PSNR and SSIM denoising performance metrics.
denoised: This folder serves as a place for you to save your denoised images to (if you want to) so that
you can inspect them for correctness. This is provided only to help you stay organized, and it is up
to you whether you choose to use it or not. It’s use is not required.
noisify.py: This is the script that was used to generate the noisy images from the noiseless images. It’s
not necessary to use this file at all in your solution but if you want to know exactly how the noisy
image are generated, you can look in here.
2.1 Assignment Synposis
First you’ll implement the vector median filter. Then you’ll use it to filter the noisy images, and obtain the
PSNR and the SSIM of the noisy and median filtered images by comparing them to the noiseless images.
The filtered images should have better PSNR and SSIM than the noisy images. Then you’ll develop a
customized denoising algorithm for this dataset that can outperform the vector median filter alone and
compare the PSNR and SSIM of the new algorithm to your previous results.
2.2 Implementing the Vector Median Filter (VMF)
Even though VMF isn’t the fastest operation, you can make it incredibly and unnecessarily slow in Python.
I suggest an approach where you only use loops to iterate over the rows and columns of the input image.
Computing the vector distances can be done without use of loops if we consider three powerful numpy
functions: tile, transpose, and reshape.
Example 1
The function numpy.tile() makes a specified number of copies of a matrix along a specified matrix dimension.
u = np . array ([ 1 , 2 , 3 ])
# make 5 copies of u along dimension 0 ( rows ) , and 1 copy along
# dimension 1 ( columns ).
# Copies can be made into the third dimension by adding another element to
# the tuple in the second argument .
v = np . tile (u , (5 , 1))
>>> v
array ([[1 , 2 , 3] ,
[1 , 2 , 3] ,
[1 , 2 , 3] ,
[1 , 2 , 3] ,
[1 , 2 , 3]])
Example 2
The function numpy.transpose() swaps the dimensions of a matrix. A common example is the swapping of the first
and second dimensions (rows and columns) of a matrix which is equivalent to a matrix transpose. The transpose
function generalizes this to any combination of dimensions. As a further example, we can “rotate” v from the
previous example, into the third dimension (imagine rotating the matrix into the page about an axis along the top
matrix row) by swapping the first and third dimensions (dimensions in python are numbered starting at 0):
# The second argument specifies which dimensions should become
# the new dimensions 0, 1 and 2. In this case , dimension 2 ( planes ) becomes
# dimension 0 ( rows ), dimension 1 ( columns ) remains dimension 1 , and dimension 0
# ( rows ) becomes dimension 2 ( planes ) causing the matrix to ” rotate ” about a
# horizontal axis into the page . Now we get a copy of the vector
# [1 ,2 ,3] at each index in the third dimension . (We need to use
# np. expand_dims () on v first because v is a 2D array ; it needs
# to be a 3D array to allow us to transpose into the 3rd dimension
# so expand_dims (v, axis =2) returns an array of shape (v. shape [0] , v. shape [1] , 1)).
w = np . transpose ( np . expand_dims (v , axis =2) , (2 , 1 , 0))
>>> w [: ,: ,0]
array ([[1 , 2 , 3]])
>>> w [: ,: ,1]
array ([[1 , 2 , 3]])
…
>>> w [: ,: ,4]
array ([[1 , 2 , 3]])
Page 2
Example 3
The function numpy.reshape() takes an array and changes its width, height, and depth to new values provided
that the product of these new values is equal to the product of the width, height, and depth of the input array (it can’t
change the total number of matrix elements, just rearrange them). For example, if we take a sub-matrix of a larger
image matrix (that might represent a neighbourhood!) that has dimensions 5 × 5 × 3, we can convert it into a matrix
with 25 rows and 3 columns, where each row consists of the R G B values of one of the 25 pixels:
# get the pixels of the colour image I in a 5×5 window centred on (r,c).
Iwindow = I [r -2: r +3 , c -2: c +3 , :]
# Reshape the 5*5*3 elements of Iwindow into a matrix of size 25*3 so
# that each row of ’colours ’ is the RGB values of a pixel in Iwindow .
colours = np . reshape ( Iwindow , [ IWindow . shape [0] * Iwindow . shape [1] , 3])
VMF Implementation Approach
To compute the median colour of a neighbourhood about (x, y), you will use tile, transpose, and
reshape, to do the following. You will construct from the colour vectors in the neighbourhood about
(x, y) two 3D matrices Y and X as described below. Assume there are n colour vectors in the neighbourhood, e.g. for a 5 × 5 neighbourhood n = 25.
Construct the first matrix, Y, so that it has a depth of n, where each plane is equal to the n × 3 matrix
of colour vectors in the neighbourhood of (x, y). This can be done with code similar to that given above,
and a call to numpy.tile(). Thus each of the n planes of the resulting n × 3 × n array should look like
this:
Y[:, :, i] =
v0
v1
.
.
.
vn−1
=
r1 g1 b1
r2 g2 b2
.
.
.
rn−1 gn−1 bn−1
for 0 = 1 . . . n − 1 (n such planes extending into the third dimension) where v0, v1, . . . , vn−1 are the colour
vectors in the neighbourhood. That is, row i of each plane of Y contains the RGB values of the one pixel
colour in the neighbourhood and each plane is identical to the others.
Construct the second matrix X so that it also has n planes of size n × 3, but where the i-th plane
contains n copies of just one of the colours in the neighbourhood, that is, ri
, gi
, bi
:
X[:, :, i] =
vi
vi
.
.
.
vi
=
ri gi bi
ri gi bi
.
.
.
ri gi bi
for i = 0, . . . , n − 1.
Now, the difference between each plane of the two matrices can be used to compute the sum of the
differences between the i-th colour and every other colour all at once! If you compute abs(X − Y) and
sum the result over dimensions first two dimensions (rows and columns), you are left with a 1 × 1 × n
matrix D where each entry D[1 : 1 : i] is the sum of the Manhattan distances (L1 norms) between the
i-th colour and every other colour, which is just what we need for the vector median filter algorithm! The
output colour for the neighbourhood is then the colour with the smallest sum of manhattan distance —
find the index k of the minimum value in D, and the output colour for the neighbourhood should be
Y[k, :, 0] = vk
.
Page 3
3 Problems
Question 1 (28 points):
Detailed instructions are provided asn1-q1.ipynb. Sample outputs are given below. These are the
outputs from my solution. Depending on the de-noising algorithm you design, your value for the
right-most bar in the bottom two graphs may differ, but the other bars should have the same height
as shown here.
PSNR Noisy PSNR Median
0
5
10
15
20
25
SSIM Noisy SSIM Median
0.0
0.2
0.4
0.6
0.8
1.0
Sample Output for Step 2
PSNR Noisy PSNR Median PSNR Mine
0
5
10
15
20
25
30
SSIM Noisy SSIM Median SSIM Mine
0.0
0.2
0.4
0.6
0.8
1.0
Sample Output for Step 3
The grading rubric is available on Moodle.
Page 4
4 Files Provided
asn1-qX.ipynb: These are iPython notebooks, one for each question, which includes instructions and in
which you will do your assignment.
Various Images in images.zip: As described in Section 2.
5 What to Hand In
Hand in your completed iPython notebook asn1-q1.ipynb.
Page 5
