## Description

Problem 1: Image Demosaicing and Histogram Manipulation (50%)

(a) Bilinear Demosaicing (10%)

To capture color images, digital camera sensors are usually arranged in form of a color filter array (CFA),

called the Bayer array, as shown in Figure 1. Since each sensor at a pixel location only captures one of the

three primary colors (R, G, B), the other two colors have to be re-constructed based on their neighbor

pixel values to obtain the full color. Demosaicing is the process of translating this Bayer array of primary

colors into a color image that contains the R, G, B values at each pixel.

Figure 1: (a) Single CCD sensor covered by a CFA and (b) Bayer pattern [1].

Implement the simplest demosaicing method based on bilinear interpolation. Exemplary demosaicing

results are given in Figure 2. With this method, the missing color value at each pixel is approximated by

bilinear interpolation using the average of its two or four adjacent pixels of the same color. To give an

example, the missing blue and green values at pixel 𝑅3,4 are estimated as:

As for pixel 𝐺3,3, the blue and red values are calculated as:

EE 569 Digital Image Processing: Homework #1

Professor C.-C. Jay Kuo Page 2 of 8

(a) (b)

Figure 2: (a) The original image and (b) the demosaiced image by bilinear interpolation.

Figure 3: The dog image with the CFA sensor input.

(1) Apply the bilinear demosaicing to the Dog image in Figure 3 and show your results.

(2) Compare your demosaiced image with the ground truth color image Dog_ori. Do you observe any

artifacts? If yes, explain the cause of the artifacts and provide your ideas to improve the

demosaicing performance.

(b) Malvar-He-Cutler (MHC) Demosaicing (20%)

Malvar et al. [2] proposed an improved linear interpolation demosaicing algorithm. It yields a higher

quality demosaicing result by adding a 2nd-order cross-channel correction term to the basic bilinear

demosaicing result. Both the bilinear and the MHC demosaicing results of the Fruit_Shop image are

shown in Figure 4.

EE 569 Digital Image Processing: Homework #1

Professor C.-C. Jay Kuo Page 3 of 8

Figure 4: Demosacing results of Fruit_Shop image: the CFA input (left), the bilinear

demosaicing result (middle) and the MHC demosaicing result (right).

The MHC algorithm is stated below.

To estimate a green component at a red pixel location, we have

where 𝐺̂𝑏𝑙 is the bilinear interpolation result and the 2

nd term is a correction term. For the 2

nd term, 𝛼 is a

weight factor, and Δ𝑅 is the discrete 5-point Laplacian of the red channel:

To estimate a red component at a green pixel location, we have

where 𝛥𝐺 is a discrete 9-point Laplacian of the green channel.

To estimate a red component at a blue pixel location,

where 𝛥𝐵 is a discrete 9-point Laplacian of the blue channel. The weights control how much correction

is applied, and their default values are:

The above formulas can be generalized to missing color components at each sensor location.

Consequently, the MHC demosaicing can be implemented by convolution with a set of linear filters. There

are eight different filters for interpolating the different color components at different locations as

illustrated in Figure 5 [2].

EE 569 Digital Image Processing: Homework #1

Professor C.-C. Jay Kuo Page 4 of 8

Figure 5: Filter coefficients.

(1) Implement the MHC linear demosaicing algorithm and apply it to the Dog image in Figure 3.

Show your results.

(2) Compare the MHC and the bilinear demosaicing results and explain the performance differences

between these two algorithms in your own words.

(c) Histogram Manipulation (20%)

Implement two histogram equalization techniques:

• Method A: the transfer-function-based histogram equalization method,

• Method B: the cumulative-probability-based histogram equalization method

to enhance the contrast of the Toy image in Figure 6 below.

(1) Plot the histograms of the red, green and blue channels of the original image. The figure should have

the intensity value as the x-axis and the number of pixels as the y-axis.

(2) Apply Method A to the original image and show the enhanced image. Plot the transfer function for

each channel.

(3) Apply Method B to the original image and show the enhanced image. Plot the cumulative histogram

for each channel.

(4) Discuss your observations on these two enhancement results. Do you have any idea to improve the

current result?

EE 569 Digital Image Processing: Homework #1

Professor C.-C. Jay Kuo Page 5 of 8

Note that MATLAB users CANNOT use functions from the Image Processing Toolbox except displaying

function like imshow().

Figure 6: Toy image

Problem 2: Image Denoising (50 %)

In this problem, you will implement a set of denoising algorithms to improve image quality. You can use

the PSNR (peak-signal-to-noise-ratio) quality metric to assess the performance of your denoising

algorithm. The PSNR value for R, G, B channels can be, respectively, calculated as follows:

Remove noise in the image in Figure 7(b), compare it with the original image in Figure 7(a), and answer

the following questions:

(a) Basic denoising methods (10%)

(1) What is the type of embedded noise in Figure 7(b)?

(2) Apply a linear filter of size N by N to the noisy image. Compare the performance of two choices

of the filter parameters – the uniform weight function and the Gaussian weight function.

EE 569 Digital Image Processing: Homework #1

Professor C.-C. Jay Kuo Page 6 of 8

Figure 7: The original and noisy corn images.

(b) Bilateral Filtering (10%)

In most low-pass linear filters, we often see degradation of edges. However, using some nonlinear filters,

we can preserve the edges. Bilateral filters are one such kind of filters. A discrete bilateral filter is given

by:

where (𝑘, 𝑙) is the neighboring pixel location within the window centered around (𝑖,𝑗), 𝐼 is the image with

noise, 𝑌 is the filtered image. 𝜎𝑐 and 𝜎𝑠 are the spread parameters.

(1) Implement the bilateral denoising filter and apply it to the noisy image.

(2) Explain the roles of 𝜎𝑐 and 𝜎𝑠

. Discuss the change in filter’s performance with respect to the values

of 𝜎𝑐 and 𝜎𝑠

.

(3) Does this filter perform better than linear filters you implemented in Problem 3(a)? Justify your

answer in words.

(c) Non-Local Means (NLM) Filtering (10%)

The non-local mean filter utilizes the pixel value from a larger region rather the mean of a local window

centered around the target pixel. A discrete non-local mean filter with Gaussian weighting function is as

follows:

where 𝐼, 𝑌 are the noisy and filtered images respectively, 𝑁𝑥,𝑦 is the window centered around location

(𝑥, 𝑦), and ℎ is the filtering parameter, 𝑁

′ ≤ 𝑁 and 𝑀′ < 𝑀 denote the window size of your choice.
The Gaussian weighted Euclidian distance between window 𝐼(𝑁𝑖,𝑗)and 𝐼(𝑁𝑘,𝑙) is defined as:
‖𝐼(𝑁𝑖,𝑗) − 𝐼(𝑁𝑘,𝑙)‖
2,𝑎
2
= ∑ 𝐺𝑎
(𝑛1, 𝑛2
)(𝐼(𝑖 − 𝑛1,𝑗 − 𝑛2
) − 𝐼(𝑘 − 𝑛1, 𝑙 − 𝑛2
))
𝑛1,𝑛2∈ℵ
2
EE 569 Digital Image Processing: Homework #1
Professor C.-C. Jay Kuo Page 7 of 8
where ℵ denotes the local neighborhood centered at the origin, 𝑛1
, 𝑛2 ∈ ℵ denotes the relative position in the
neighborhood window. 𝑎 > 0 is the standard deviation of the Gaussian kernel.

(1) Implement the NLM filter and apply it to the noisy image. Try several filter parameters and

discuss their effect on filtering process. Clearly state your final choice of parameters in your

report.

(2) Compare the performance of NLM with filters used in Problem 3(a) and Problem 3(b).

(d) Block matching and 3-D (BM3D) transform filter (10%)

In this part, you will get familiar with another state-of-the-art denoising algorithm proposed in [3].

(1) Please explain the BM3D algorithm in your own words.

(2) Implement the BM3D filter (Write your own code or use any available online source code but

include the source in your reference) to denoise the noisy image House in Figure 7. Show the final

noise-removed image, and compare the PSNR value and visual quality with those in (a).

Note: It is recommended that you use the code provided by the authors on their website [4]. Their code is

written in MATLAB.

(e) Mixed noises in color image (10%)

Figure 8 (b) is a noisy color image corrupted with mixed types of noises. Please identify noise types in the

image and answer the following questions:

(1) What types of noises are there?

(2) Should you perform filtering on individual channels separately for both noise types?

(3) What filters would you like use to remove mixed noise? Can you cascade these filters in any order?

Justify your answer.

(a) (b)

Figure 8: (a) the original pepper image (b) the pepper image with mixed noises.

EE 569 Digital Image Processing: Homework #1

Professor C.-C. Jay Kuo Page 8 of 8

Appendix:

Problem 1: Image Demosaicing and Histogram Manipulation

Dog.raw 600×532 8-bit gray

Dog_ori.raw 600×532 24-bit color(RGB)

Toy.raw 560×400 24-bit color(RGB)

Problem 2: Image Denoising

Corn_gray.raw 320×320 8-bit gray

Corn_noisy.raw 320×320 8-bit gray

Reference Images

All images in this homework are from Google images [5] or the USC-SIPI image database [6].

References

[1] M. E. Celebi et al. (eds.), Color Image and Video Enhancement.

[2] Malvar, Henrique S., Li-wei He, and Ross Cutler. “High-quality linear interpolation for demosaicing

of Bayer-patterned color images.” 2004 IEEE International Conference on Acoustics, Speech, and Signal

Processing. Vol. 3. IEEE, 2004.

[3] Dabov, Kostadin, et al. “Image denoising with block-matching and 3D filtering.” Image Processing:

Algorithms and Systems, Neural Networks, and Machine Learning. Vol. 6064. International Society for

Optics and Photonics, 2006.

[4] [Online] Available: http://www.cs.tut.fi/~foi/GCF-BM3D/

[5] [Online] http://images.google.com/

[6] [Online] http://sipi.usc.edu/database/