Description
Homework journal:
For this homework, you need to document the progress of the solution in a
Homework Journal. This is a series of screenshots on the progress of the homework, labeled as nnntext.jpg. For instance:
001-Original code.jpg
002-After creating the learning code.jpg
…
012-First try for running.jpg
Screenshots should document evolution of the code, the tries, challenges, the breakthroughs etc. of the
process. Feel free to also document the bugs and how you fixed them.
The easiest way (on Windows) to do this is by hitting Windows Key + Prnt Screen when a step is
achieved. On Ubuntu, you simply hit PrntScreen. You can select, crop and rename the pictures later.
Please crop away any private information from your screen.
Note about academic honesty: The projects on the Berkeley class have been around for a while. A
number of people had posted solutions of various quality to github, bitbucket etc. For instance, you can
google:
“berkeley ai project :site github”
to find at least 10 different projects. Here is a list of some of these projects: advaypakhale,
angelahnicole, dgggit, druyvchadha2212, jasonwu0731, karlapalem, Matiatus, molson194, OctaviPascual
raymondngiam, rsk2327, shiro876, Shourov1. We will run a soft comparison checker between your
implementations and a growing collection of other projects (as well as between the projects submitted
by students on this class). Projects with plagiarized code will receive a 0 grade.
Step 1: Problems Q1.. Q4 from Berkeley AI reinforcement learning
project (10 pts)
Solve problems Q1 to Q4 from the Berkeley AI reinforcement learning project:
http://ai.berkeley.edu/reinforcement.html
Submit:
• the changed files: analysis.py, qlearningAgents.py, valueIterationAgents.py
• the homework journal for this step.
Step 2: Visualize the evolution of the V-value (4 pts)
Re-run problem Q1 from Step 1. Plot the evolution of the V value in the upper-left corner. You can try to
do this in Excel or other spreadsheets. A better solution is to learn and use pyplot.
Submit:
• the homework journal for this step, with the last screenshot showing the plot
Step 3: Visualize the evolution of the Q-value (4 pts)
Re-run problem Q4 from Step 1. Plot the evolution of the four state action (Q) values for the second
state from left in the top row. Put all values in a single plot. Again, Excel is acceptable, but pyplot is
better.
Submit:
• the homework journal for this step, with the last screenshot showing the plot
