## Description

1. In this problem, consider a Bayes minimum-error classifier. Three classes are described by

normal densities as follows:

in which each is a variable denoting the prior for class (to save on writing). Assume

the are given.

In this problem, the algebra should be done by hand. The plots may be done using computer

or by hand.

(a) Write an expression for the discriminant functions , in two forms:

(i) Expressed in terms of Mahalanobis distance and other given terms;

(ii) Expressed only in terms of , grouped as follows:

and give expressions for and in terms of the same quantities.

(b) State the decision rule in terms of the discriminant functions . Is the classifier

linear, quadratic, or neither?

(c) For this part you are also given:

(i) Give expressions for the discriminant functions and in

terms of the given numbers, in simplest forms. Hint: before plugging in

numbers, you can simplify the by dropping some terms. Is the classifier

linear, quadratic, or neither?

(ii) Let Give equations for the 3 decision boundaries, in simplest form.

(iii) In (non-augmented) feature space, plot the class means, the 3 curves for

, and the decision boundaries. Show clearly the decision regions

and final boundaries.

p x S ( i) = N x,mi,Σi ( ), i = 1, 2, 3

P S( 1) = π1, P S( 2 ) = π 2, P S( 3) = π 3

π i Si

π i

gi( x)

dM ( x,mi)

x, mi , Σi

, and Σi

−1

gi( x) = x

T

W i x + wi x + w0

(i)

W , w, w0

gi( x)

m1 = 1

2

⎡

⎣

⎢ ⎤

⎦

⎥, m2 = 1

−1

⎡

⎣

⎢ ⎤

⎦

⎥, m3 = −2

2

⎡

⎣

⎢ ⎤

⎦

⎥

Σ1 = Σ2 = Σ3 = 1 −1

−1 2

⎡

⎣

⎢ ⎤

⎦

⎥

g1 ( x), g2 ( x) g3 ( x)

gi( x)

π i = 1

3 ∀i.

dM

2

( x,mi) = 1

p. 2 of 4

2. A Naïve Bayes classifier is a Bayes classifier in which the features, conditioned on class, are

assumed independent; thus:

.

Most of the below is similar to your work for Problem 1.

For a Naïve Bayes (minimum error) classifier, let the given class-conditional densities and

priors be:

(a) Write an expression for the discriminant functions , in the form:

and give expressions for all the weights in terms of given scalar quantities, in simplest

form. Let the mean vector of class be denoted .

(b) Is the classifier quadratic, linear, or neither?

(c) For this part you are also given:

(i) Give expressions for the discriminant functions and in

terms of the given numbers, in simplest forms. Hint: before plugging in

numbers, you can simplify the by dropping some terms. Is the classifier

linear, quadratic, or neither?

(ii) Let Give equations for the 3 decision boundaries, in simplest form.

(iii) In (non-augmented) feature space, plot the class means, the 3 curves for

, and the decision boundaries. Show clearly the

decision regions and final boundaries.

Problem 2 continues on next page…

p x S ( i) = p x j S ( i)

j=1

D

∏ ∀i = 1,!,C

p( x Si) = N x,mi,Σi ( ), i = 1, 2, 3

Σi =

σ1

(i) ( )

2

0

0 σ 2

(i) ( )

2

⎡

⎣

⎢

⎢

⎢

⎢

⎤

⎦

⎥

⎥

⎥

⎥

P(S1) = π1, P(S2 ) = π 2, P(S3 ) = π 3

gi( x)

gi( x) = w11

(i)

x1

2 + w12

(i)

x1x2 + w22

(i)

x2

2 + w1

(i)

x1 + w2

(i)

x2 + w0

(i)

Si mi = m1

(i) m2

⎡ (i)

⎣⎢ ⎤

⎦⎥

T

m1 = 1

2

⎡

⎣

⎢ ⎤

⎦

⎥, m2 = 1

−1

⎡

⎣

⎢ ⎤

⎦

⎥, m3 = −2

2

⎡

⎣

⎢ ⎤

⎦

⎥

σ 1

(i) ( )

2

= σ 1

2 = 1, σ 2

(i) ( )

2

= σ 2

2 = 2, ∀i

g1 ( x), g2 ( x) g3 ( x)

gi( x)

π i = 1

3 ∀i.

dM

2

( x,mi) = 1, i = 1,2,3

p. 3 of 4

(d) Compare your plot of 2(c)(iii) with your plot of 1(c)(iii). Does the dependence vs.

independence of features make a substantial difference in the decision boundaries and

regions?

3. This problem uses the minimum risk criterion (instead of minimum error criterion) for

classification (introduced in Lecture 21, covered in Discussion 12 and DHS 2.2). In this

problem, please use our notation ( instead of for class i ).

This problem is based on our 2-class tumor classification example. Suppose the classconditional densities are known (or assumed) to be Gaussian:

For a Bayes minimum risk classifier, answer the parts below.

(a) Write the decision rule for a 2-class Bayes minimum risk classifier, in terms of

conditional risks and . Then, write the decision rule in terms of

, , , , and .

For parts (b)-(d) below, suppose also that on average, 80% of the tumors that are to be

classified (based on their MRI images) are benign (class ), and the other 20% are

cancerous (class ).

(b) Give estimates of the class priors and .

For parts (c)-(e) below, suppose also that misclassifying a cancerous tumor as benign, is

considered much worse than misclassifying a benign tumor as cancerous. Thus we will use

for our cost coefficients:

Also, you are given that the mean vectors and covariance matrices for each class are known

(or estimated) as:

(c) Solve for the decision boundary and regions; that is, give an expression (in simplest

form, based on the given numbers) for the decision rule.

Problem 3 continues on next page…

Si ωi

p x S ( i) = N x,mi,Σi ( ), i = 1, 2

R α1 ( x) R α 2 ( x)

dM

2

( x,mi) Σ1

, Σ2 λ11, λ12 , λ21, λ22 P S1 ( ) P S2 ( )

S1

S2

P S1 ( ) P S2 ( )

λ11 λ12

λ21 λ22

⎡

⎣

⎢

⎢

⎤

⎦

⎥

⎥

= 0 10

1 0

⎡

⎣

⎢ ⎤

⎦

⎥

m1 = 1

4

⎡

⎣

⎢ ⎤

⎦

⎥, m2 = 4

2

⎡

⎣

⎢ ⎤

⎦

⎥

Σ1 = Σ2 = 0.5 −0.5

−0.5 2

⎡

⎣

⎢ ⎤

⎦

⎥

p. 4 of 4

(d) Plot in 2D feature space, the class means, the 2 curves for , the

decision boundary, and show the decision regions (by a small arrow at the boundary

pointing into , or by labelling ).

(e) If the guidelines for ordering an MRI change, and it is estimated that the data points

coming in will be, on average, 50% benign and 50% cancerous, repeat part (d) (or add

it to your part (d) plot, clearly labeled as (e)). How have the boundary and regions

changed?

dM

2

( x,mi) = 1, i = 1,2

Γ1 Γ1 and Γ2