Description
PyTorch
• We will use pytorch
• It is a library for machine learning with built-in support for gradient
descent
• No need to derive gradients on paper, you get them for free!
• https://pytorch.org/get-started/locally/
Hints about PyTorch
• See: pytorch_minimizeF.py
• Performs gradient descent on a simple function
• The function has global minimum at w1=-0.25, w2=2
• The code does the following
• Defines f(w) using pytorch functions for arithmetic
• Not numpy functions – so that we automatically get
gradient formulas
• Defines pytorch variable to store the evolving values of w
• Defines pytorch variable z=f(w) – since it’s a pytorch variable, it
has gradients w.r.t. to w
• Creates an optimizer (gradient descent), tells it to minimize f(w)
• In a loop:
• Run a single step of the optimizer – a gradient step on f(w)
PyTorch – HW Part A
• Part A
• Redo HW1 in pytorch
PyTorch – HW Part B
• Part B:
• You task is to perform experiments with a linear
classifier on MNIST dataset
• https://pytorch.org/docs/stable/torchvision/datasets.html#mnist
• This is 10-class classification problem:
hand-written digits
• It has 5,000 samples per class
• During development, you can randomly select
a subset of samples, to speed up the
calculations
PyTorch – HW Part B
• MNIST: a 10-class classification problem
• Convert it into 2-class problem by: taking last digit of your V#
(class A), taking last different digit of your V# (class B)
• E.g. V# V00078965: 6-vs-5, V00078966: 9-vs-6
• During training, training on the training set, and also apply the
model to test set, to see how training error and testing error
evolve during training
• Error: % of predictions that are not classified correctly
• The model should be:
• A linear model (wTx+b)
• Classes encoded as +1 / -1
• Use logistic loss
Returning the Assignment
n Solution code should be written by you and you
only (no web/book/friend/etc. code)
n Exception: you can freely use the code provided on BB as
your starting point
n Upload through Blackboard
n A report in PDF
n Plots from part A, like in HW1
n Plot of training risk value vs. iterations of training for part B
n Plot of test set risk value vs. iterations of training for part B
n Plot of training set error and test set error vs. iterations of training for part B
n Code for part A
n Code for part B
