This lecture (9 ECTS) will lay the foundations of reinforcement learning (RL). The lecture is devided into three parts: Multiarmed bandits, tabular RL, non-tabular RL.
We will prove everything that we think is needed for a proper understanding of the algorithms but also go into the coding (Python). At many instances of RL convergence proofs are still open (even worse, sometimes algorithms are known to diverge). We will cover theoretical results around RL which sometimes leads to good educated guesses for RL algorithms even though the theoretical assumptions of techniques cannot be checked (or are violated).
Reinforcement learning is a type of machine learning that involves training an agent to make a sequence of decisions in an environment in order to maximize a reward. It is often used to control complex, dynamic systems or to optimize performance. Some applications of reinforcement learning include:
Overall, reinforcement learning offers a way to optimize complex systems by learning how to act in certain situations in order to maximize rewards.
Attention: This text was written by chatGPT, an AI tool based on reinforcement learning (RL) itself (and transformer networks). I do not quite agree with chatGPT, financial markets seem not be very well suited to ML methods. Anyways, as we will cross RL in our future lives in manifold occasions it will be useful to know how RL works.
Students from the study programs Mathematics, WiMa, WiFo, MMDS. We will cover the mathematical background of reinforcement learning, coding (in python) will be part of the exercises.
Prof. Dr. Leif Döring, André Ferdinand, Till Freihaut, Sara Klein, Marc Pritsch, Almut Röder, Leo Vela
Tuesday, B2, garden house (in the garden of B6, 26)
Thursday, B2, garden house (in the garden of B6, 26)
Thursday, B1, garden house (in the garden of B6, 26)
Exams will be oral, here are some hints.
Dates: 1st-5th of June, 26th, 27th, 28th of July, 26th of August-1st of September
|Link to Introduction Repository||https://github.com/aferdina/IntroductionRL/|
Link to the Videos
|1. Exercise Sheet,||1. Solution Sheet|
|2. Exercise Sheet||2. Solution Sheet|
|3. Exercise Sheet||3. Solution Sheet|
|4. Exercise Sheet||4. Solution Sheet|
|5. ExerciseSheet |
(new numeration in
the lecture notes)
|5. Solution Sheet|
|6. Exercise Sheet||6. Solution Sheet|
|7. Exercise Sheet|
|7. Solution Sheet|
|8. Exercise Sheet||8. Solution Sheet|
|9. Exercise Sheet (updated)||9. Solution Sheet|
|10. Exercise Sheet||10. Solution Sheet|
|11. Exercise Sheet||11. Solution Sheet|
|12. Exercise Sheet|
Repository to collaborate at the programming tasks: https://github.com/aferdina/RLFiniteGames
Solutions for the programming tasks in https://github.com/aferdina/Solution_Exercise
Sutton & Barto: “Reinforcement Learning – an Introduction” is available online. This covers all major ideas but skipps essentially all details. In essence, this lecture course follows the core ideas of Sutton & Barto but tries to include as much of the missing mathematics as possible.