Description
Problem 1
(Logistic regression, 40pts) Implement a perceptron for logistric regression. For your training data, generate 2000 training instances in two sets of random data points (1000 in each) from multi-variate normal
distribution with
µ1 = [1, 0], µ2 = [0, 1.5], Σ1 =
1 0.75
0.75 1
, Σ2 =
1 0.75
0.75 1
(1)
and label them 0 and 1. Generate testing data in the same manner but include 500 instances for each class,
i.e., 1000 in total. Use sigmoid function for your activation function and cross entropy for your objective
function. You will implement a logistic regression for the following questions. Initialize the starting weight
as w = [1, 1, 1]. During training, stop your loop when the objective function (i.e., cross entropy) does not
decrease any more (below certain threshold) or when the gradient is close to 0 or the iteration reaches 10000.
Set your thresholds properly so that the iteration doesn’t reach 10000 for all the learning rate that you will
be using.
1. Perform batch training using gradient descent. Divide the derivative with the total number of training
dataset as you go through iteration (it is very likely that you will get NaN if you don’t do this.).
Set your learning rate to be η = {1, 0.1, 0.01}. How many iterations did you go through the training
dataset? What is the accuracy that you have? What are the edge weights that were learned?
2. Perform online training using gradient descent. Set your learning rate to be η = {1, 0.1, 0.01}. Set
your maximum number of iterations to 10000. How many iterations did you go through your training
dataset? What is the accuracy that you have? What are the edge weights that were learned? Compare
the learned parameters and accuracy to the ones that you got from batch training. Are they the same?
Explain in your report.
Instructor: W. H. Kim (won.kim@uta.edu), TA: Priyank Arora (priyank.arora@mavs.uta.edu) Page 1 of 2
CSE4334/5334 Data Mining Assignment 1
3. Write your own code to draw ROC curve and computing area under the curve(AUC). Evaluate the
performance of your LR classifier with batch training with η = {1, 0.1, 0.01}.
Instructor: W. H. Kim (won.kim@uta.edu), TA: Priyank Arora (priyank.arora@mavs.uta.edu) Page 2 of 2
