'Y'

Reinforcement Learning - from Fundamentals to Deep RL

Oriented towards python programmers or ML practitioners who want to understand the RL framework in detail.
Code: EAS-027
Duration: 30 hours

Description

Our training provides an overview of Reinforcement Learning. We start from the mathematics needed, through basic RL algorithms, and get into Deep Reinforcement Learning and the latest state of the art methods used today. We go through some applications in detail and also cover some significant achievements in the field so far.

The course focuses on some of the main issues which arise when dealing with RL in the real world, and goes through some of the main algorithms which are the backbone of newer RL systems.

Significant theoretical knowledge is gained, not only for RL but for ML in general, with practical applications throughout. From the basics of Linear Algebra, Calculus, and Probabilities, we go through Dynamic Programming and Markov Processes, to finally reach the ubiquitous Q-learning and its deep variants, as well as some policy gradient methods.

We strive to give a comprehensive overview, at least with respect to different fundamental techniques employed in the literature. The second half of the course is only about RL with neural networks, with research papers discussed in detail and various applications explained.

Practical tools are also discussed and used in exercises (from Pytorch to Ray).

Roadmap

Linear algebra
  • Overview of concepts

Calculus
  • Integration
  • Derivation
  • Examples

Probabilities
  • Random variables
  • Density functions
  • Expectation
  • Conditional, joint and marginal probabilities
  • Examples
  • Practice

RL Introduction
  • Markov Decision Processes
  • Dynamic programming with example
  • Bellman equation
  • Policy evaluation
  • Policy Iteration
  • Value Iteration
  • Examples
  • Practice

Model-based vs model-free
  • Learning and planning
  • Deterministic
  • Stochastic
  • Linear value-function approximation
  • Comparison and practice

Algorithms
  • Q-learning
  • Sarsa
  • Actor-critic
  • Policy gradient
  • Monte-Carlo tree methods
  • Exploration vs exploitation
  • Examples
  • Practice

Deep Reinforcement Learning
  • Nonlinear function approximation
  • The DeepMind breakthrough
  • Alpha-Star Explained

Latest technologies in DRL
  • Memory, Attention, Recurrence
  • Inverse RL
  • Multi-agent
  • Hierarchical
  • Evolved rewards – AutoRL
  • Policy optimization

Applications and use
  • Trading
  • Speech understanding and question answering (optional)
  • Load balancing (optional)
  • Other uses (optional)

Pytorch / Tensorflow
  • Tensor basics
  • Implementing a RL algorithm from scratch
  • Testing and visualizing
  • Practice

Ray + RLlib
  • Main concepts: actors, futures, memory sharing, etc.
  • Worked example
  • Different algorithms
  • Grid search and visualization
  • Practice

Visualization and explainability
  • SMDP, AMDP, SAMDP
  • Projection to 3D-space with TSNE
  • Examples

Objectives

  • Understand the main modelling difficulties in developing RL algorithms
  • Learn how to use existent algorithms and understand issues which arise in state representation or reward shaping
  • Visualize and adapt the algorithm or reward mechanism, so that the agent learns one thing or another
  • Be able to apply or develop RL algorithms for real-life problems

Target Audience

  • Software Developers who are knowledgeable about Python/Machine Learning, yet lack experience in Reinforcement Learning
  • Machine Learning Engineers
  • Data Scientists

Prerequisites

  • Graduate level of calculus, probability theory, discrete mathematics
  • Basic knowledge of machine learning
  • Ability to understand Python code


Schedule in Online Prices
Register for the next course
+
Registering in advance ensures you have priority. We’ll notify you when we schedule the next course on this topic
Online €500
Bucharest €500
Krakow €500
Wroclaw €500

Invoices for the course will be issued in local currency. All fees above can change according to training location and delivery mode and are subject to change while scheduling. Price does not include VAT.

Courses you may be interested in:

Sign up

First Name*
Last Name*
Company
Email*
Preferred Location
Phone
Your Comment
Still have questions?
Connect with us
Thank you.
Your request has been received.
Thank you!
The form has been submitted successfully.