Description
One aspect of research in reinforcement learning (or any scientific field) is the replication of
previously published results. There are a few benefits you might reap from replicating papers.
One benefit of replication is that it augments your understanding of the material. Another benefit
is that it puts you in a good position both to extend existing literature and consider new
contributions to your field. Replication is also often challenging. You may find that values of key
parameters are missing, that described methods are ambiguous, or even that there are subtle
errors. Sometimes obtaining the same pattern of results is not possible.
For this project, you will read Richard Sutton’s 1988 paper
. Then you will create an implementation and replication of the results
found in figures 3, 4, and 5. (It might also be informative to compare these results with those in
Chapter 7 of Sutton’s textbook: “
You will present your work in a written report of a maximum of 5 pages. The report should
include a description of the experiment replicated, how the experiment was implemented (the
environment, algorithms, etc), and the outcome of the experiment. You should describe how well
your results match the results given in the paper as well as any significant differences. Describe
any pitfalls you ran into while trying to replicate the experiment from the paper (e.g. unclear
parameters, contradictory descriptions of the procedure to follow, results that differ wildly from
the published results). What steps did you take to overcome those pitfalls? What assumptions did
you make? And, why were these assumptions justified? Add anything else that you think is
relevant to discuss.