Introduction To Reinforcement Learning Motion Control

The brain of futuristic autonomous systems

What you'll learn

Build motivation to take part in autonomous system product development

Recognize schools that teach RL motion control and companies that use it

Understand the development process of RL motion controlled product

Understand the analogy between PID and RL motion controllers

Practice PID tunning (using the Course online tool)

Practice RL NN optimization (using MATLAB)

Practice RL NN controller deployment into a real system (using AWS DeepRacer)

Innovate an autonomous system and plan its product and business development

Requirements

No skills are needed to practice PID tuning. You will learn everything you need to know.

MATLAB will be needed to practice RL controller development

AWS Deep Racer is needed to practice RL, NN controller deployment into an autonomous car

Description

The course objective is to introduce upcoming AI/ML technology, as used for intelligent control of complex autonomous mechatronics systems. The course is intended for students, technicians, engineers, and managers of mechanical, electrical, computer science and information technology as well as business managers, marketing and entrepreneurs, who are interested to be active, as business developers and team members, in an innovative, product development project of complex autonomous systems.  The course will be structured in four parts. The first part will discuss the AI pyramid and the different type of ML technology which are used in automation motion control. The second part will focus on the analogy between common PID control technology which most mechatronics engineers are familiar with and RL motion control with decision making ability, including the differences in product development. The third part will focus on the internal structure of RL motion control including simulation, animation and the learning process of the autonomous brain to make optimal motion decisions in real time. The last part will discuss various options in getting started with RL motion control product development in startups, small and medium size companies. The course does not require any tools or programming capabilities and is not intended to make the students experts in AI/ML or control technology. It is designed as a first step for the students to followup in learning and participation in real product development with a professional team.

Overview

Section 1: Introduction

Lecture 1 Motivation

Lecture 2 Who is Working on RL Motion Controllers?

Lecture 3 The Pyramid of AI Values

Lecture 4 Types of AI/ ML in Motion Control

Section 2: Bridging the Gap between PID and RL

Lecture 5 Lecture 5: Common PID system blocks

Lecture 6 Lecture 6: Futuristic RL system blocks

Lecture 7 Lecture 7: Difference in PID and RL product development

Section 3: RL Under the Microscope

Lecture 8 Lecture 8: RL simulation and animation of “States and Actions”

Lecture 9 Lecture 9: “Reward” function of desired system performance

Lecture 10 Lecture 10: RL “Agents”, “Critics” and “Actors”

Lecture 11 Lecture 11: “NN” “Optimization” using “Bellman” and “Linear programming”

Lecture 12 Lecture 12: NN controller “Deployment" into the system hardware

Section 4: The Next Steps

Lecture 13 Lecture 13: Recommended startup options

Anyone who is interested in autonomous systems (mobile, robots, exoskeletons, humanoids),Preferably engineering students and practitioners (ME, EE, CS, IT),Entrepreneurs and managers (general, engineering and marketing) in the automation business,Interdisciplinary mechatronics team members of an automation product development project