Description
Exercises:
1. Perform ridge regression directly using the closed form solution. Use k-fold cross
validate (k=5) to select the optimal 𝜆 parameter. Compute the RMSE value on the test
data.
a. You can begin by running the solver with λ = 400. Then, cut λ down by a factor of 2
and run again. Continue the process of cutting λ by a factor of 2 until you have
models for 10 values of λ in total.
b. Report the λ you chose, why you chose it, and the associated training error.
2. Perform linear regression using the gradient descent algorithm. Compute the RMSE value
on the training data and test data, respectively. Compare the results with your Lab 5 and
see if you can obtain the same results. For the initial weights, you can just use Gaussian
N(0, 1) random variables. Define “converging” as the change in any coefficient between
one iteration and the next is no larger than a small value (e.g., 10−5
).
a. Report training and testing errors.
b. Define a function called problem2(samples): that, given an NxK array, returns
predictions in the form of a Nx1 array. The dummy feature will be included in the
last column of the NxK array.
3. Perform ridge regression with 5-fold cross valuation using the gradient descent algorithm.
Compute the RMSE value on the test data and see if you can obtain the same results with
those from the closed form solution in Exercise 1.
a. Report training and testing errors.
b. Define a function called problem3(samples): that, given an NxK array, returns
predictions in the form of a Nx1 array. The dummy feature will be included in the
last column of the NxK array.
