Markov processes for stochastic modeling / Oliver C. Ibe, University of Massachusetts, Lowell, MA, USA.

Format:

Book

Author/Creator:

Ibe, Oliver C.

Series:

Elsevier insights.

Elsevier insights Markov processes for stochastic modeling

Language:

English

Subjects (All):

Markov processes.

Stochastic processes.

Physical Description:

1 online resource (xviii, 494 pages) : illustrations.

Edition:

2nd ed.

Place of Publication:

Amsterdam, Netherlands : Elsevier, c2013.

London : Elsevier, 2013.

Language Note:

English

System Details:

text file

Summary:

Markov processes are processes that have limited memory. In particular, their dependence on the past is only through the previous state. They are used to model the behavior of many systems including communications systems, transportation networks, image segmentation and analysis, biological systems and DNA sequence analysis, random atomic motion and diffusion in physics, social mobility, population studies, epidemiology, animal and insect migration, queueing systems, resource management, dams, financial engineering, actuarial science, and decision systems. Covering a wide range of

Contents:

Front Cover; Markov Processes for Stochastic Modeling; Copyright page; Contents; Acknowledgments; Preface to the Second Edition; Preface to the First Edition; 1 Basic Concepts in Probability; 1.1 Introduction; 1.1.1 Conditional Probability; 1.1.2 Independence; 1.1.3 Total Probability and the Bayes' Theorem; 1.2 Random Variables; 1.2.1 Distribution Functions; 1.2.2 Discrete Random Variables; 1.2.3 Continuous Random Variables; 1.2.4 Expectations; 1.2.5 Expectation of Nonnegative Random Variables; 1.2.6 Moments of Random Variables and the Variance; 1.3 Transform Methods; 1.3.1 The s-Transform

1.3.2 The z-Transform1.4 Bivariate Random Variables; 1.4.1 Discrete Bivariate Random Variables; 1.4.2 Continuous Bivariate Random Variables; 1.4.3 Covariance and Correlation Coefficient; 1.5 Many Random Variables; 1.6 Fubini's Theorem; 1.7 Sums of Independent Random Variables; 1.8 Some Probability Distributions; 1.8.1 The Bernoulli Distribution; 1.8.2 The Binomial Distribution; 1.8.3 The Geometric Distribution; 1.8.4 The Pascal Distribution; 1.8.5 The Poisson Distribution; 1.8.6 The Exponential Distribution; 1.8.7 The Erlang Distribution; 1.8.8 Normal Distribution; 1.9 Limit Theorems

1.9.1 Markov Inequality1.9.2 Chebyshev Inequality; 1.9.3 Laws of Large Numbers; 1.9.4 The Central Limit Theorem; 1.10 Problems; 2 Basic Concepts in Stochastic Processes; 2.1 Introduction; 2.2 Classification of Stochastic Processes; 2.3 Characterizing a Stochastic Process; 2.4 Mean and Autocorrelation Function of a Stochastic Process; 2.5 Stationary Stochastic Processes; 2.5.1 Strict-Sense Stationary Processes; 2.5.2 Wide-Sense Stationary Processes; 2.6 Ergodic Stochastic Processes; 2.7 Some Models of Stochastic Processes; 2.7.1 Martingales; Stopping Times; 2.7.2 Counting Processes

2.7.3 Independent Increment Processes2.7.4 Stationary Increment Process; 2.7.5 Poisson Processes; Interarrival Times for the Poisson Process; Compound Poisson Process; Combinations of Independent Poisson Processes; Competing Independent Poisson Processes; Subdivision of a Poisson Process; 2.8 Problems; 3 Introduction to Markov Processes; 3.1 Introduction; 3.2 Structure of Markov Processes; 3.3 Strong Markov Property; 3.4 Applications of Discrete-Time Markov Processes; 3.4.1 Branching Processes; 3.4.2 Social Mobility; 3.4.3 Markov Decision Processes

3.5 Applications of Continuous-Time Markov Processes3.5.1 Queueing Systems; 3.5.2 Continuous-Time Markov Decision Processes; 3.5.3 Stochastic Storage Systems; 3.6 Applications of Continuous-State Markov Processes; 3.6.1 Application of Diffusion Processes to Financial Options; 3.6.2 Applications of Brownian Motion; 3.7 Summary; 4 Discrete-Time Markov Chains; 4.1 Introduction; 4.2 State-Transition Probability Matrix; 4.2.1 The n-Step State-Transition Probability; 4.3 State-Transition Diagrams; 4.4 Classification of States; 4.5 Limiting-State Probabilities; 4.5.1 Doubly Stochastic Matrix

4.6 Sojourn Time

Notes:

Includes bibliographical references.

Description based on online resource; title from title page (ebrary, viewed June 05, 2013).

Previous edition: Amsterdam; London: Academic, 2009.

ISBN:

9780124078390

0124078397

OCLC:

848902271