Reinforcement Learning (English): Master The Art Of Rl

Reinforcement Learning

What you'll learn

Define what is Reinforcement Learning?

Apply all what is learned using state-of-the art libraries like OpenAI Gym, StabeBaselines, Keras-RL and TensorFlow Agents

Define what are the applications domains and success stories of RL?

Define what are the difference between Reinforcement and Supervised Learning?

Define the main components of an RL problem setup?

Define what are the main ingredients of an RL agent and their taxonomy?

Define what is Markov Reward Process (MRP) and Markov Decision Process (MDP)?

Define the solution space of RL using MDP framework

Solve the RL problems using planning with Dynamic Programming algorithms, like Policy Evaluation, Policy Iteration and Value Iteration

Solve RL problems using model free algorithms like Monte-Carlo, TD learning, Q-learning and SARSA

Differentiate On-policy and Off-policy algorithms

Master Deep Reinforcement Learning algorithms like Deep Q-Networks (DQN), and apply them to Large Scale RL

Master Policy Gradients algorithms and Actor-Critic (AC, A2C, A3C)

Master advanced DRL algorithms like DDPG, TRPO and PPO

Define what is model-based RL, and differentiate it from planning, and what are their main algorithms and applications?

Requirements

Machine Learning basics

Deep Learning basics

Probability

Programming and Problem solving basics

Python programming

Description

Hello and welcome to our course; Reinforcement Learning. Reinforcement Learning is a very exciting and important field of Machine Learning and AI. Some call it the crown jewel of AI.In this course, we will cover all the aspects related to Reinforcement Learning or RL. We will start by defining the RL problem, and compare it to the Supervised Learning problem, and discover the areas of applications where RL can excel. This includes the problem formulation, starting from the very basics to the advanced usage of Deep Learning, leading to the era of Deep Reinforcement Learning.In our journey, we will cover, as usual, both the theoretical and practical aspects, where we will learn how to implement the RL algorithms and apply them to the famous problems using libraries like OpenAI Gym, Keras-RL, TensorFlow Agents or TF-Agents and Stable Baselines.The course is divided into 6 main sections:1- We start with an introduction to the RL problem definition, mainly comparing it to the Supervised learning problem, and discovering the application domains and the main constituents of an RL problem. We describe here the famous OpenAI Gym environments, which will be our playground when it comes to practical implementation of the algorithms that we learn about.2- In the second part we discuss the main formulation of an RL problem as a Markov Decision Process or MDP, with simple solution to the most basic problems using Dynamic Programming.3- After being armed with an understanding of MDP, we move on to explore the solution space of the MDP problem, and what the different solutions beyond DP, which includes model-based and model-free solutions. We will focus in this part on model-free solutions, and defer model-based solutions to the last part. In this part, we describe the Monte-Carlo and Temporal-Difference sampling based methods, including the famous and important Q-learning algorithm, and SARSA. We will describe the practical usage and implementation of Q-learning and SARSA on control tabular maze problems from OpenAI Gym environments.4- To move beyond simple tabular problems, we will need to learn about function approximation in RL, which leads to the mainstream RL methods today using Deep Learning, or Deep Reinforcement Learning (DRL). We will describe here the breakthrough algorithm of DeepMind that solved the Atari games and AlphaGO, which is Deep Q-Networks or DQN. We also discuss how we can solve Atari games problems using DQN in practice using Keras-RL and TF-Agents.5- In the fifth part, we move to Advanced DRL algorithms, mainly under a family called Policy based methods. We discuss here Policy Gradients, DDPG, Actor-Critic, A2C, A3C, TRPO and PPO methods. We also discuss the important Stable Baseline library to implement all those algorithms on different environments in OpenAI Gym, like Atari and others.6- Finally, we explore the model-based family of RL methods, and importantly, differentiating model-based RL from planning, and exploring the whole spectrum of RL methods.Hopefully, you enjoy this course, and find it useful.

Overview

Section 1: Introduction

Lecture 1 Course introduction

Lecture 2 Course overview

Section 2: Introduction to Reinforcement Learning

Lecture 3 Module intro and roadmap

Lecture 4 What is RL?

Lecture 5 What RL can do?

Lecture 6 The RL problem setup (AREA)

Lecture 7 Reward

Lecture 8 RL vs. Supervised Learning

Lecture 9 State

Lecture 10 AREA examples and quizes

Lecture 11 Gym Environments

Lecture 12 Inside RL agent - RL agent ingredients

Lecture 13 Policy

Lecture 14 Value

Lecture 15 Model

Lecture 16 RL agents taxonomy

Lecture 17 Prediction vs Control

Section 3: Markov Decision Process (MDP)

Lecture 18 Module intro and roadmap

Lecture 19 Markov Chain and Markov Process (MP)

Lecture 20 Markov Reward Process (MRP)

Lecture 21 Markov Decision Process (MDP)

Lecture 22 Prediction

Lecture 23 Bellman Equations with action-value function Q

Lecture 24 Control

Section 4: MDP solutions spaces

Lecture 25 Module intro and roadmap

Lecture 26 Planning with Dynamic Programming (DP)

Lecture 27 Prediction with DP - Policy Evaluation

Lecture 28 Control with DP - Policy Iteration and Value Iteration

Lecture 29 Value Iteration example

Lecture 30 Prediction with Monte-Carlo - MC Policy Evaluation

Lecture 31 Prediction with Temporal-Difference (TD)

Lecture 32 TD Lambda

Lecture 33 Control with Monte-Carlo - MC Policy Iteration

Lecture 34 Control with TD - SARSA

Lecture 35 On-policy vs. Off-policy

Lecture 36 Q-learning

Lecture 37 MDP solutions summary

Section 5: Deep Reinforcement Learning (DRL)

Lecture 38 Module intro and roadmap

Lecture 39 Large Scale Reinforcement Learning

Lecture 40 DNN as function approximator

Lecture 41 Value Function Approximation

Lecture 42 DNN policies

Lecture 43 Value function approximation with DL encoder-decoder pattern

Lecture 44 Deep Q-Networks (DQN)

Lecture 45 DQN Atari Example with Keras-RL and TF-Agents

Section 6: Advanced DRL

Lecture 46 Module intro and roadmap

Lecture 47 Value-based vs Policy based vs Actor-Critic

Lecture 48 Policy Gradients (PG)

Lecture 49 REINFORCE - Monte-Carlo PG

Lecture 50 AC - Actor-Critic

Lecture 51 A2C - Advantage Actor-Critic

Lecture 52 A3C - Asynchronous Advantage Actor-Critic

Lecture 53 TRPO - Trusted Region Policy Optimization

Lecture 54 PPO - Proximal Policy Optimization

Lecture 55 DDPG - Deep Determinstic Policy Gradients

Lecture 56 StableBaselines library overview

Lecture 57 Atari example with stable-baselines

Lecture 58 Mario example with stable-baselines

Lecture 59 StreetFighter example with stable-baselines

Section 7: Model-based Reinforcement Learning

Lecture 60 Module intro and roadmap

Lecture 61 Model learning methods

Lecture 62 Model learning with Supervised Learning and Function Approximation

Lecture 63 Sample based planning

Lecture 64 Dyna - Intergation planning and Learning

Section 8: Conclusion

Lecture 65 Conclusion

Section 9: Material

Lecture 66 Slides

Machine Learning Researchers,Machine Learning Engineers,Data Scientists