Numerical methods of simulation and optimization of piecewise deterministic Markov processes : application to reliability / Benoîte de Saporta, François Dufour, Huilong Zhang.

Format:

Book

Author/Creator:

Saporta, Benoîte de, author.

Contributor:

Dufor, François.

Zhang, Huilong (Lecturer in mathematics)

Language:

English

Subjects (All):

Markov processes--Numerical solutions.

Markov processes.

Physical Description:

1 online resource (299 p.)

Edition:

1st ed.

Place of Publication:

London, England ; Hoboken, New Jersey : ISTE : Wiley, 2016.

Summary:

Mark H.A. Davis introduced the Piecewise-Deterministic Markov Process (PDMP) class of stochastic hybrid models in an article in 1984. Today it is used to model a variety of complex systems in the fields of engineering, economics, management sciences, biology, Internet traffic, networks and many more. Yet, despite this, there is very little in the way of literature devoted to the development of numerical methods for PDMDs to solve problems of practical importance, or the computational control of PDMPs. This book therefore presents a collection of mathematical tools that have been recently developed to tackle such problems. It begins by doing so through examples in several application domains such as reliability. The second part is devoted to the study and simulation of expectations of functionals of PDMPs. Finally, the third part introduces the development of numerical techniques for optimal control problems such as stopping and impulse control problems.

Contents:

Cover

Title Page

Copyright

Contents

Preface

Introduction

I.1. Preliminaries

I.2. Overview of the chapters

PART 1: Piecewise Deterministic Markov Processes and Quantization

Chapter 1: Piecewise Deterministic Markov Processes

1.1. Introduction

1.2. Notation

1.3. Definition of a PDMP

1.4. Regularity assumptions

1.4.1. Lipschitz continuity along the flow

1.4.2. Regularity assumptions on the local characteristics

1.5. Time-augmented process

1.6. Embedded Markov chain

1.7. Stopping times

1.8. Examples of PDMPs

1.8.1. Poisson process with trend

1.8.2. TCP

1.8.3. Air conditioning unit

1.8.4. Crack propagation model

1.8.5. Repair workshop model

Chapter 2: Examples in Reliability

2.1. Introduction

2.2. Structure subject to corrosion

2.2.1. PDMP model

2.2.2. Deterministic time to reach the boundary

2.3. The heated hold-up tank

2.3.1. Tank dynamics

2.3.2. PDMP model

Chapter 3: Quantization Technique

3.1. Introduction

3.2. Optimal quantization

3.2.1. Optimal quantization of a random variable

3.2.2. Optimal quantization of a Markov chain

3.3. Simulation of PDMPs

3.3.1. Simulation of time-dependent intensity

3.3.2. Simulation of trajectories

3.4. Quantization of PDMPs

3.4.1. Scale of coordinates of the state variable

3.4.2. Cardinality of the mode variable

PART 2: Simulation of Functionals

Chapter 4: Expectation of Functionals

4.1. Introduction

4.2. Recursive formulation

4.2.1. Lipschitz continuity

4.2.2. Iterated operator

4.2.3. Approximation scheme

4.3. Lipschitz regularity

4.4. Rate of convergence

4.5. Time-dependent functionals

4.6. Deterministic time horizon

4.6.1. Direct estimation of the running cost term

4.6.2. Bounds of the boundary jump cost term

4.6.3. Bounds in the general case.

4.7. Example

4.8. Conclusion

Chapter 5: Exit Time

5.1. Introduction

5.2. Problem setting

5.2.1. Distribution

5.2.2. Moments

5.2.3. Computation horizon

5.3. Approximation schemes

5.4. Convergence

5.4.1. Distribution

5.4.2. Moments

5.5. Example

5.6. Conclusion

Chapter 6: Example in Reliability: Service Time

6.1. Mean thickness loss

6.2. Service time

6.2.1. Mean service time

6.2.2. Distribution of the service time

6.3. Conclusion

PART 3: Optimization

Chapter 7: Optimal Stopping

7.1. Introduction

7.2. Dynamic programming equation

7.3. Approximation of the value function

7.4. Lipschitz continuity properties

7.4.1. Lipschitz properties of J and K

7.4.2. Lipschitz properties of the value functions

7.5. Error estimation for the value function

7.5.1. Second term

7.5.2. Third term

7.5.3. Fourth term

7.5.4. Proof of theorem 7.1

7.6. Numerical construction of an ε-optimal stopping time

7.7. Example

Chapter 8: Partially Observed Optimal Stopping Problem

8.1. Introduction

8.2. Problem formulation and assumptions

8.3. Optimal filtering

8.4. Dynamic programming

8.4.1. Preliminary results

8.4.2. Optimal stopping problem under complete observation

8.4.3. Dynamic programming equation

8.5. Numerical approximation by quantization

8.5.1. Lipschitz properties

8.5.2. Discretization scheme

8.5.3. Numerical construction of an ε-optimal stopping time

8.6. Numerical example

Chapter 9: Example in Reliability: Maintenance Optimization

9.1. Introduction

9.2. Corrosion process

9.3. Air conditioning unit

9.4. The heated hold-up tank

9.4.1. Problem setting and simulation

9.4.2. Numerical results and validation

9.5. Conclusion

Chapter 10: Optimal Impulse Control

10.1. Introduction

10.2. Impulse control problem.

10.3. Lipschitz-continuity properties

10.3.1. Lipschitz properties of the operators

10.3.2. Lipschitz properties of the operator L

10.4. Approximation of the value function

10.4.1. Time discretization

10.4.2. Approximation of the value functions on the control grid U

10.4.3. Approximation of the value function

10.4.4. Step-by-step description of the algorithm

10.4.5. Practical implementation

10.5. Example

10.6. Conclusion

Bibliography

Index.

Notes:

Description based upon print version of record.

Includes bibliographical references and index.

Description based on online resource; title from PDF title page (ebrary, viewed February 17, 2016).

ISBN:

9781119145141

1119145147

9781119145097

1119145090

OCLC:

933265716