Hands-on intelligent agents with OpenAI Gym : a step-by-step guide to develop AI agents using deep reinforcement learning / Praveen Palanisamy.

Format:

Book

Author/Creator:

Palanisamy, Praveen, author.

Language:

English

Subjects (All):

Artificial intelligence--Data processing.

Artificial intelligence.

Application software--Development.

Application software.

Physical Description:

1 online resource (246 pages) : illustrations

Edition:

1st edition

Place of Publication:

Birmingham ; Mumbai : Packt, 2018.

System Details:

text file

Summary:

Implement intelligent agents using PyTorch to solve classic AI problems, play console games like Atari, and perform tasks such as autonomous driving using the CARLA driving simulator Key Features Explore the OpenAI Gym toolkit and interface to use over 700 learning tasks Implement agents to solve simple to complex AI problems Study learning environments and discover how to create your own Book Description Many real-world problems can be broken down into tasks that require a series of decisions to be made or actions to be taken. The ability to solve such tasks without a machine being programmed requires a machine to be artificially intelligent and capable of learning to adapt. This book is an easy-to-follow guide to implementing learning algorithms for machine software agents in order to solve discrete or continuous sequential decision making and control tasks. Hands-On Intelligent Agents with OpenAI Gym takes you through the process of building intelligent agent algorithms using deep reinforcement learning starting from the implementation of the building blocks for configuring, training, logging, visualizing, testing, and monitoring the agent. You will walk through the process of building intelligent agents from scratch to perform a variety of tasks. In the closing chapters, the book provides an overview of the latest learning environments and learning algorithms, along with pointers to more resources that will help you take your deep reinforcement learning skills to the next level. What you will learn Explore intelligent agents and learning environments Understand the basics of RL and deep RL Get started with OpenAI Gym and PyTorch for deep reinforcement learning Discover deep Q learning agents to solve discrete optimal control tasks Create custom learning environments for real-world problems Apply a deep actor-critic agent to drive a car autonomously in CARLA Use the latest learning environments and algorithms to upgrade your intelligent agent development skills Who this book is for If you're a student, game/machine learning developer, or AI enthusiast looking to get started with building intelligent agents and algorithms to solve a variety of problems with the OpenAI Gym interface, this book is for you. You will also find this book useful if you want to learn how to build deep reinforcement learning-based agents to solve problems in your domain of interest. Though the book covers all the basic concepts that you need to know, some working kn...

Contents:

Cover

Title Page

Copyright and Credits

Dedication

Packt Upsell

Contributors

Table of Contents

Preface

Chapter 1: Introduction to Intelligent Agents and Learning Environments

What is an intelligent agent?

Learning environments

What is OpenAI Gym?

Understanding the features of OpenAI Gym

Simple environment interface

Comparability and reproducibility

Ability to monitor progress

What can you do with the OpenAI Gym toolkit?

Creating your first OpenAI Gym environment

Creating and visualizing a new Gym environment

Summary

Chapter 2: Reinforcement Learning and Deep Reinforcement Learning

What is reinforcement learning?

Understanding what AI means and what's in it in an intuitive way

Supervised learning

Unsupervised learning

Reinforcement learning

Practical reinforcement learning

Agent

Rewards

Environment

State

Model

Value function

State-value function

Action-value function

Policy

Markov Decision Process

Planning with dynamic programming

Monte Carlo learning and temporal difference learning

SARSA and Q-learning

Deep reinforcement learning

Practical applications of reinforcement and deep reinforcement learning algorithms

Chapter 3: Getting Started with OpenAI Gym and Deep Reinforcement Learning

Code repository, setup, and configuration

Prerequisites

Creating the conda environment

Minimal install - the quick and easy way

Complete install of OpenAI Gym learning environments

Instructions for Ubuntu

Instructions for macOS

MuJoCo installation

Completing the OpenAI Gym setup

Installing tools and libraries needed for deep reinforcement learning

Installing prerequisite system packages

Installing Compute Unified Device Architecture (CUDA)

Installing PyTorch

Summary.

Chapter 4: Exploring the Gym and its Features

Exploring the list of environments and nomenclature

Nomenclature

Exploring the Gym environments

Understanding the Gym interface

Spaces in the Gym

Chapter 5: Implementing your First Learning Agent - Solving the Mountain Car problem

Understanding the Mountain Car problem

The Mountain Car problem and environment

Implementing a Q-learning agent from scratch

Revisiting Q-learning

Implementing a Q-learning agent using Python and NumPy

Defining the hyperparameters

Implementing the Q_Learner class's __init__ method

Implementing the Q_Learner class's discretize method

Implementing the Q_Learner's get_action method

Implementing the Q_learner class's learn method

Full Q_Learner class implementation

Training the reinforcement learning agent at the Gym

Testing and recording the performance of the agent

A simple and complete Q-Learner implementation for solving the Mountain Car problem

Chapter 6: Implementing an Intelligent Agent for Optimal Control using Deep Q-Learning

Improving the Q-learning agent

Using neural networks to approximate Q-functions

Implementing a shallow Q-network using PyTorch

Implementing the Shallow_Q_Learner

Solving the Cart Pole problem using a Shallow Q-Network

Experience replay

Implementing the experience memory

Implementing the replay experience method for the Q-learner class

Revisiting the epsilon-greedy action policy

Implementing an epsilon decay schedule

Implementing a deep Q-learning agent

Implementing a deep convolutional Q-network in PyTorch

Using the target Q-network to stabilize an agent's learning

Logging and visualizing an agent's learning process

Using TensorBoard for logging and visualizing a PyTorch RL agent's progress.

Managing hyperparameters and configuration parameters

Using a JSON file to easily configure parameters

The parameters manager

A complete deep Q-learner to solve complex problems with raw pixel input

The Atari Gym environment

Customizing the Atari Gym environment

Implementing custom Gym environment wrappers

Reward clipping

Preprocessing Atari screen image frames

Normalizing observations

Random no-ops on reset

Fire on reset

Episodic life

Max and skip-frame

Wrapping the Gym environment

Training the deep Q-learner to play Atari games

Putting together a comprehensive deep Q-learner

Hyperparameters

Launching the training process

Testing performance of your deep Q-learner in Atari games

Chapter 7: Creating Custom OpenAI Gym Environments - CARLA Driving Simulator

Understanding the anatomy of Gym environments

Creating a template for custom Gym environment implementations

Registering custom environments with OpenAI Gym

Creating an OpenAI Gym-compatible CARLA driving simulator environment

Configuration and initialization

Configuration

Initialization

Implementing the reset method

Customizing the CARLA simulation using the CarlaSettings object

Adding cameras and sensors to a vehicle in CARLA

Implementing the step function for the CARLA environment

Accessing camera or sensor data

Sending actions to control agents in CARLA

Continuous action space in CARLA

Discrete action space in CARLA

Sending actions to the CARLA simulation server

Determining the end of episodes in the CARLA environment

Testing the CARLA Gym environment

Chapter 8: Implementing an Intelligent - Autonomous Car Driving Agent using Deep Actor-Critic Algorithm

The deep n-step advantage actor-critic algorithm

Policy gradients

The likelihood ratio trick.

The policy gradient theorem

Actor-critic algorithm

Advantage actor-critic algorithm

n-step advantage actor-critic algorithm

n-step returns

Implementing the n-step return calculation

Deep n-step advantage actor-critic algorithm

Implementing a deep n-step advantage actor critic agent

Initializing the actor and critic networks

Gathering n-step experiences using the current policy

Calculating the actor's and critic's losses

Updating the actor-critic model

Tools to save/load, log, visualize, and monitor

An extension - asynchronous deep n-step advantage actor-critic

Training an intelligent and autonomous driving agent

Training and testing the deep n-step advantage actor-critic agent

Training the agent to drive a car in the CARLA driving simulator

Chapter 9: Exploring the Learning Environment Landscape - Roboschool, Gym-Retro, StarCraft-II, DeepMindLab

Gym interface-compatible environments

Roboschool

Quickstart guide to setting up and running Roboschool environments

Gym retro

Quickstart guide to setup and run Gym Retro

Other open source Python-based learning environments

StarCraft II - PySC2

Quick start guide to setup and run StarCraft II PySC2 environment

Downloading the StarCraft II Linux packages

Downloading the SC2 maps

Installing PySC2

Playing StarCraftII yourself or running sample agents

DeepMind lab

DeepMind Lab learning environment interface

reset(episode=-1, seed=None)

step(action, num_steps=1)

observations()

is_running()

observation_spec()

action_spec()

num_steps()

fps()

events()

close()

Quick start guide to setup and run DeepMind Lab

Setting up and installing DeepMind Lab and its dependencies

Playing the game, testing a randomly acting agent, or training your own!

Chapter 10: Exploring the Learning Algorithm Landscape - DDPG (Actor-Critic), PPO (Policy-Gradient), Rainbow (Value-Based)

Deep Deterministic Policy Gradients

Core concepts

Proximal Policy Optimization

Core concept

Off-policy learning

On-policy

Rainbow

DQN

Double Q-Learning

Prioritized experience replay

Dueling networks

Multi-step learning/n-step learning

Distributional RL

Noisy nets

Quick summary of advantages and applications

Other Books You May Enjoy

Index.

Notes:

Includes index.

Description based on print version record.

ISBN:

9781788835138

1788835131

OCLC:

1048817965