## Description

Problem 1: Understanding Successive Subspace Learning (SSL) (50%)

(a) Feedforward-designed Convolutional Neural Networks (FF-CNNs) (20%)

When two CNN layers are cascaded, a non-linear activation function is used in between. As an alternative

to the non-linear activation, Kuo et al. proposed the Saak (subspace approximation via augmented kernels)

transform [1] and the Saab (subspace approximation via adjusted bias) transform [2]. Specifically, Kuo et

al. [2] proposed the first example of a feedforward-designed CNN (FF-CNN), where all model parameters

are determined in a feedforward manner without backpropagation. It has two main cascaded modules:

1) Convolutional layers via multi-stage Saab transforms;

2) Fully-connected (FC) layers via multi-stage linear least squared regression (LLSR).

Although the term “successive subspace learning” (SSL) was not used in [2] explicitly, it does provide

the first SSL design example.

Read paper [2] carefully and answer the following questions:

(1) Summarize the Saab transform with a flow diagram. Explain it in your own words. The codes for

the Saab transform can be found at https://github.com/USC-MCL/EE569_2020Spring.

Please read the codes with the paper to understand the Saab transform better.

(2) Explain similarities and differences between FF-CNN and backpropagation-designed CNN (BPCNNs).

Do not copy any sentence from any paper directly, which is plagiarism. Your scores will depend on the

degree of your understanding.

(b) Successive Subspace Learning (SSL) (30%)

Two interpretable models adopting the SSL principle for the image classification task were proposed by

Chen et al. They are known as PixelHop [3] and PixelHop++ [4]. Read the two papers carefully and

answer the questions below. You can use various tools in your explanation such as flow charts, figures,

formulas etc. You should demonstrate your understanding through your answer.

(1) Explain the SSL methodology in your own words. Compare Deep Learning (DL) and SSL.

(2) What are the functions of Modules 1, 2 and 3, respectively, in the SSL framework?

EE 569 Digital Image Processing: Homework #6

Professor C.-C. Jay Kuo Page 2 of 4

(3) Explain the neighborhood construction and subspace approximation steps in the PixelHop unit and

the PixelHop++ unit and make a comparison. Specifically, explain the differences between the

basic Saab transform and the channel-wise (c/w) Saab transform.

(4) Both PixelHop and PixelHop++ use the Label-Assisted reGression (LAG) unit for supervised

feature dimension reduction. Explain the role and the procedure of LAG in your own words. What

is its advantage?

Problem 2: CIFAR-10 Classification using SSL (50%)

(a) Building a PixelHop++ Model (35%)

An example block diagram of the PixelHop++ framework containing three PixelHop++ Units is shown in

Figure 1. The codes for the c/w Saab transform module, feature selection (cross-entropy computing)

module and the LAG unit are provided in the GitHub link. You can import them in your program to build

your model in Python based on the diagram. Note that in the first PixelHop++ unit, RGB three channels

are processed together instead of channel-wise processing because they are not decorrelated. For your

experiments in this Part (a), follow the parameters indicated in Table 1.

Figure 1 Block diagram of the PixelHop++ model [4]

Table 1 Choice of hyper-parameters of PixelHop++ model for this section

Spatial Neighborhood size in all PixelHop++ units 5×5

Stride 1

Max-pooling (2×2) -to- (1×1)

Energy threshold (T) 0.001

Number of selected features (NS) 1000

� in LAG units 10

Number of centroids per class in LAG units 5

Classifier Random Forest (recommended)

(1) Train Module 1 (PixelHop++ units) using the whole set or a subset of the training images.

Remember to keep balance among different classes (i.e. randomly select 1000 images per class if

you use 10,000 training images). Then train Module 2 and Module 3 on all the 50,000 training

EE 569 Digital Image Processing: Homework #6

Professor C.-C. Jay Kuo Page 3 of 4

images. Report training time and train accuracy. What is your model size in terms of the total

parameter numbers?

(2) Apply your model to 10,000 testing images and report test accuracy.

(3) Keeping the trained Module 1 unchanged, reduce the number of labeled training samples in

Module 2&3 to 1/4, 1/8, 1/16 and 1/32 of the original training number (i.e. 50,000). Apply each

model to 10,000 testing images and draw the curve of test accuracy for each setting with respect

to the number of training images. Show and discuss your results.

(b) Error analysis (15%)

Most often, a dataset contains both easy and difficult classes. Conduct the following error analysis based

on your trained model using 50,000 training images (in Module 2&3):

(1) Compute the confusion matrix and show it as a heat map in your report. Which object class yields

the lowest error rate? Which object class is the most difficult one?

(2) Find out the confusing class groups and discuss why they are easily confused with each other. You

can use some exemplary images to support your statement.

(3) Propose ideas to improve the accuracy of the difficult classes for PixelHop++ and justify your

proposal. There is no need to implement your ideas.

Problem 3: EE569 Competition — CIFAR-10 Classification using SSL (50%)

The parameters in Table 1 may not be optimal. You are free to modify PixelHop++ in Problem 2(a) to

improve its performance in terms of classification accuracy, running speed and model size. For example,

you may adjust the hyper-parameters such as the spatial neighborhood size, energy threshold, the reduced

dimension through LAG units, etc. You may also increase the number of cascaded PixelHop++ units,

adjust the choice of pooling layers (max-pooling, mean-pooling, min-pooling, etc.), and/or other color

representations of input images. Besides, you may consider ensembles as done in [5].

Conduct experiments and report your findings in the following aspects.

• Motivation and logics behind your design (20%):

Draw the diagram of your proposed SSL system and explain it in your own words. Discuss how

you choose the hyper-parameters. Discuss the sources of performance improvement as compared

with that in Problem 2(a).

• Classification accuracy with full and weak supervision (10%):

Report the classification accuracy under different training image numbers used in Module 2&3.

Draw the test accuracy curve with training models obtained by different training image numbers.

Compare it with the curve you obtained using BP-CNN in HW5 Problem 2 with discussion. You

can also study the influence of training image numbers used in Module 1 (optional).

• Running time (10%)

Report the training time and inference time.

• Model size (10%):

Compute and report the size (i.e. the number of parameter numbers) in your model. It is related to

energy threshold (T) and the number of selected cross-entropy-guided features (NS). Compare it

with the model size of your BP-CNN in Problem 2 of HW5.

EE 569 Digital Image Processing: Homework #6

Professor C.-C. Jay Kuo Page 4 of 4

Your grading will be based on the accuracy, running time and model size of your proposed SSL system

in comparison with those of other students in this class.

References

[1] C.-C. Jay Kuo and Yueru Chen, “On data-driven Saak transform,” Journal of Visual Communication

and Image Representation, vol. 50, pp. 237–246, 2018.

[2] C-C Jay Kuo, Min Zhang, Siyang Li, Jiali Duan, and Yueru Chen, “Interpretable convolutional neural

networks via feedforward design,” Journal of Visual Communication and Image Representation, vol.

60, pp. 346–359, 2019.

[3] Yueru Chen and C-C Jay Kuo, “Pixelhop: A successive subspace learning (ssl) method for object

recognition,” Journal of Visual Communication and Image Representation, p. 102749, 2020.

[4] Yueru Chen, Mozhdeh Rouhsedaghat, Suya You, Raghuveer Rao, C.-C. Jay Kuo, “PixelHop++: A

Small Successive-Subspace-Learning-Based (SSL-based) Model for Image Classification,”

https://arxiv.org/abs/2002.03141, 2020

[5] Yueru Chen, Yijing Yang, Wei Wang, C.-C. Jay Kuo, “Ensembles of Feedforward-designed

Convolutional Neural Networks”, in International Conference on Image Processing, 2019