Exploiting Environment Configurability in Reinforcement Learning / A. M. Metelli.

Format:

Book

Author/Creator:

Metelli, A. M., author.

Series:

Frontiers in artificial intelligence and applications ; Volume 361.

Frontiers in Artificial Intelligence and Applications Series ; Volume 361

Language:

English

Subjects (All):

Reinforcement learning.

Physical Description:

1 online resource (377 pages)

Edition:

First edition.

Place of Publication:

Amsterdam, Netherlands : IOS Press BV, [2022]

Summary:

In recent decades, Reinforcement Learning (RL) has emerged as an effective approach to address complex control tasks.In a Markov Decision Process (MDP), the framework typically used, the environment is assumed to be a fixed entity that cannot be altered externally.

Contents:

Intro

Title page

Abstract

Contents

List of Figures

List of Tables

List of Algorithms

List of Symbols and Notation

Acknowledgments

Introduction

What is Reinforcement Learning?

Why Environment Configurability?

Original Contributions

Overview

Foundations of Sequential Decision-Making

Markov Decision Processes

Markov Reward Processes

Markov Chains

Performance Indexes

Value Functions

Optimality Criteria

Exact Solution Methods

Reinforcement Learning Algorithms

Temporal Difference Methods

Function Approximation

Policy Search

Modeling Environment Configurability

Configurable Markov Decision Processes

Motivations and Examples

Definition

Bellman Equations and Operators

Taxonomy

Related Literature

Solution Concepts for Conf-MDPs

Cooperative Setting

Non-Cooperative Setting

Learning in Cooperative Configurable Markov Decision Processes

Learning in Finite Cooperative Conf-MDPs

Relative Advantage Functions

Performance Improvement Bound

Safe Policy Model Iteration

Theoretical Analysis

Experimental Evaluation

Examples of Conf-MDPs

Learning in Continuous Conf-MDPs

Solving Parametric Conf-MDPs

Relative Entropy Model Policy Search

Approximation of the Transition Model

Experiments

Applications of Configurable Markov Decision Processes

Policy Space Identification

Generalized Likelihood Ratio Test

Policy Space Identification in a Fixed Env

Analysis for the Exponential Family

Policy Space Identification in a Configurable Env

Connections with Existing Work

Experimental Results

Control Frequency Adaptation

Persisting Actions in MDPs

Bounding the Performance Loss.

Persistent Fitted Q-Iteration

Persistence Selection

Related Works

Open Questions

Discussion and Conclusions

Learning in Conf-MDPs

Applications of Conf-MDPs

Appendices

Additional Results and Proofs

Additional Results and Proofs of Chapter 6

Additional Results and Proofs of Chapter 7

Additional Results and Proofs of Chapter 8

Additional Results and Proofs of Chapter 9

Exponential Family Policies

Gaussian and Boltzmann Linear Policies as Exponential Family distributions

Fisher Information Matrix

Subgaussianity Assumption

Bibliography.

Notes:

Includes bibliographical references.

Description based on publisher supplied metadata and other sources.

Description based on print version record.

Other Format:

Print version: Metelli, A. M. Exploiting Environment Configurability in Reinforcement Learning

ISBN:

9781643683638

OCLC:

1373388321