About Advanced AI and Deep Reinforcement Learning
Reinforcement learning (RL) is an area of machine learning and considered as advanced AI. Moreover, concerned with how software agents ought to take actions in an environment in order to maximize the notion of cumulative reward. Reinforcement learning is one of three basic machine learning paradigms, alongside supervised learning and unsupervised learning.
Reinforcement learning differs from supervised learning in not needing labeled input/output pairs to be presented, and in not needing sub-optimal actions to be explicitly corrected. Instead, the focus is on finding a balance between exploration of uncharted territory and exploitation of current knowledge.
About this Course
This course is all about the application of deep learning and neural networks to reinforcement learning. Specifically, the combination of deep learning with reinforcement learning has led to AlphaGo beating a world champion in the strategy game Go, it has led to self-driving cars, and it has led to machines that can play video games at a superhuman level.
The world is changing at a very fast pace. The state of California is changing its regulations so that self-driving car companies can test their cars without a human in the car to supervise. Reinforcement learning is an entirely different kind of machine learning than supervised and unsupervised learning.
Supervised and unsupervised machine learning algorithms are for analyzing and making predictions about data, whereas reinforcement learning is about training an agent to interact with an environment and maximize its reward. Unlike supervised and unsupervised learning algorithms, reinforcement learning agents have an impetus they want to reach a goal.
This is such a fascinating perspective, it can even make supervised / unsupervised machine learning, and “data science” seems boring in hindsight. Why train a neural network to learn about the data in a database, when you can train a neural network to interact with the real-world?
While deep reinforcement learning and AI has a lot of potentials, it also carries with it a huge risk. Bill Gates and Elon Musk have made public statements about some of the risks that AI poses to economic stability and even our existence. Also don’t think like humans, and so they come up with novel and non-intuitive solutions to reach their goals, often in ways that surprise domain experts – humans who are the best at what they do.
Open AI is a non-profit founded by Elon Musk, Sam Altman (Y Combinator), and others, in order to ensure that AI progresses in a way that is beneficial, rather than harmful. Part of the motivation behind Open AI is the existential risk that AI poses to humans. They believe that open collaboration is one of the keys to mitigating that risk.
One of the great things about Open AI is that they have a platform called the Open AI Gym, which will make heavy use of this course. It allows anyone, anywhere in the world, to train their reinforcement learning agents in standard environments. In this course, you will build upon what you did in the last course by working with more complex environments, specifically, those provided by the Open AI Gym:
- Mountain Car
- Atari games
To train effective learning agents, you will need new techniques of advanced AI. You will also extend your knowledge of temporal difference learning by looking at the TD Lambda algorithm, look at a special type of neural network called the RBF network, look at the policy gradient method, and you will end the course by looking at Deep Q-Learning (DQN) and A3C (Asynchronous Advantage Actor-Critic).
In this course on Advanced AI, you will be going through:
1. Introduction and Logistics
- Where to get the Code.
- How to Succeed in this Course.
2. The Basic of Reinforcement Learning
- Elements of a Reinforcement Learning Problem, States, Actions, Rewards, and Policies.
- Markov Decision Processes (MDPs) Value Functions and the Bellman Equation.
- What does it mean to “learn”?
- Solving the Bellman Equation with Reinforcement Learning Part 1 and Part2.
3. Advanced AI: Open AI Gym and Basic Reinforcement Learning Technique
- Open AI Gym Tutorial, Random Search and Saving a Video.
- CartPole with Bins (Theory and code).
- RBF Neural Networks.
- RBF Networks with Mountain Car (Code) and CartPole (Theory).
- Theano and Tensor flow Warm up.
- Plugging in a Neural Network.
4. TD Lambda
- N-Step Methods and codes.
- TD Lambda and Lambda in Code.
5. Policy Gradients
- Policy Gradient Methods and Policy Gradient in TensorFlow for CartPole.
- Policy Gradient in Theano for CartPole.
- Continuous Action Spaces.
- Mountain Car Continuous Specifics,Theano,Tensorflow,Tensorflow(v2) and Theano(v2).
6. Deep Q Learning
- Deep Q-Learning Intro and Techniques.
- Deep Q-Learning in Tensorflow for CartPole and Theano for CartPole.
- Additional Implementation Details for Atari and Pseudo code and Replay Memory.
- Deep Q-Learning in Tensorflow for Breakout and Theano for Breakout.
- Partially Observable MDPs.
- A3C – Theory and Outline.
- A3C – Code point 1, Code point 2, Code point 3 and Code point 4 (Warm up).
8. Theano and Tensorflow Basic Review
- Theano Basics and Neural Network in Code.
- Tensor flow Basics and Neural Network in Code.
What you will learn from this Advanced AI course:
- Build various deep learning agents (including DQN and A3C).
- Apply a variety of advanced reinforcement learning algorithms to any problem.
- Q-Learning with Deep Neural Networks.
- Policy Gradient Methods with Neural Networks.
- Reinforcement Learning with RBF Networks.
- Use Convolutional Neural Networks with Deep Q-Learning.
Who this Course is for?
- Professionals and students with strong technical backgrounds who wish to learn state-of-the-art AI techniques.
- College-level math is helpful (calculus, probability)
- Object-oriented programming
- Python coding: if/else, loops, lists, dicts, sets
- Numpy coding: matrix and vector operations
- Linear regression
- Gradient descent
- Know how to build ANNs and CNNs in Theano or Tensor Flow
- Markov Decision Processes (MDPs)
- Know how to implement Dynamic Programming, Monte Carlo, and Temporal Difference Learning to solve MDPs
Note: Your review matters
If you have already done this course, kindly drop your review in our reviews section. It would help others to get useful information and better insight into the course offered.
- About our policies and review criteria.
- How can you choose and compare online courses?
- How to add Courses to your Wishlist?
- You can suggest courses to add to our website.
- Covers a wide-range of advanced reinforcement learning topics.
- Excellent and advanced topics.
- Good details and manual coding.
- Concise details about each algorithms.
- Should have more explanation on the theory.
- Need to improve coding lectures.
Specification: Advanced AI: Deep Reinforcement Learning in Python