Bayesian field theory / Jörg C. Lemm.

Format:

Book

Author/Creator:

Lemm, Jörg C.

Language:

English

Subjects (All):

Bayesian field theory.

Physical Description:

xix, 411 pages : illustrations ; 24 cm

Place of Publication:

Baltimore, Md. : Johns Hopkins University Press, 2003.

Summary:

There is a considerable amount of interesting discussion on inference generally and, in particular, on Bayesian inference. While a statistician might find the language and point-of-view somewhat different, this is a useful resource for those curious about the use of statistics in modern physics." -- "Mathematical Reviews

Contents:

2 Bayesian framework 9

2.1 Bayesian models 9

2.1.1 Independent, dependent, and hidden variables 9

2.1.2 Energies, free energies, and errors 20

2.1.3 Bayes' theorem: Posterior, prior, and likelihood 27

2.1.4 Predictive density and learning 30

2.1.5 Mutual information and learning 33

2.1.6 Maximum A Posteriori Approximation (MAP) 36

2.1.7 Normalization, non-negativity, and specific priors 41

2.1.8 Numerical case study: A fair coin? 46

2.2 Bayesian decision theory 59

2.2.1 Loss and risk 59

2.2.2 Loss functions for approximation 59

2.2.3 General loss functions and unsupervised learning 62

2.2.4 Log-loss and Maximum A Posteriori Approximation 63

2.2.5 Empirical risk minimization 64

2.2.6 Interpretations of Occam's razor 67

2.2.7 Approaches to empirical learning 70

2.3 A priori information 72

2.3.1 Controlled, measured, and structural priors 72

2.3.2 Noise induced priors 75

3 Gaussian prior factors 85

3.1 Gaussian prior factor for log-likelihoods 85

3.1.1 Lagrange multipliers: Error functional E(L) 85

3.1.2 Normalization by parameterization: Error functional E(g) 89

3.1.3 The Hessians H[subscript L], H[subscript g] 91

3.2 Gaussian prior factor for likelihoods 92

3.2.1 Lagrange multipliers: Error functional E(P) 92

3.2.2 Normalization by parameterization: Error functional E(z) 94

3.2.3 The Hessians H[subscript P], H[subscript z] 95

3.3 Quadratic density estimation and empirical risk minimization 96

3.4 Numerical case study: Density estimation with Gaussian specific priors 99

3.5 Gaussian prior factors for general fields 112

3.5.1 The general case 112

3.5.2 Square root of P 114

3.5.3 Distribution functions 115

3.6 Covariances and invariances 116

3.6.1 Approximate invariance 116

3.6.2 Infinitesimal translations 118

3.6.3 Approximate periodicity 120

3.6.4 Approximate fractals 122

3.7 Non-zero means 123

3.8 Regression 126

3.8.1 Gaussian regression 126

3.8.2 Exact predictive density 138

3.8.3 Gaussian mixture regression (cluster regression) 141

3.8.4 Support vector machines and regression 142

3.8.5 Numerical case study: Approximately invariant regression (AIR) 143

3.9 Classification 161

4 Parameterizing likelihoods: Variational methods 167

4.1 General likelihood parameterizations 167

4.2 Gaussian priors for likelihood parameters 170

4.3 Linear trial spaces 171

4.4 Linear regression 174

4.5 Mixture models 179

4.6 Additive models 179

4.7 Product ansatz 181

4.8 Decision trees 182

4.9 Projection pursuit 183

4.10 Neural networks 183

5 Parameterizing priors: Hyperparameters 187

5.1 Quenched and annealed prior normalization 187

5.2 Saddle point approximations and hyperparameters 191

5.2.1 Joint MAP 191

5.2.2 Stepwise MAP 192

5.2.3 Pointwise approximation 192

5.2.4 Marginal posterior and empirical Bayes 193

5.2.5 Some variants of stationarity equations 195

5.3 Adapting prior means 197

5.3.1 General considerations 197

5.3.2 Density estimation and nonparametric boosting 198

5.3.3 Unrestricted variation 199

5.3.4 Regression 199

5.4 Adapting prior covariances 202

5.4.2 Automatic relevance detection 202

5.4.3 Local masses and gauge theories 203

5.4.4 Invariant determinants 204

5.4.5 Regularization parameters 207

5.4.6 Cross-validation 210

5.5 Integer hyperparameters 215

5.6 Hyperfields 216

5.7 Auxiliary fields 221

5.8 Non-quadratic potentials 224

6 Mixtures of Gaussian prior factors 229

6.1 Multimodal energy surfaces 229

6.2 Prior mixtures for density estimation 231

6.3 Numerical case study: Prior mixtures for density estimation 231

6.4 Prior mixtures for regression 239

6.4.1 General framework 239

6.4.2 High and low temperature limits 240

6.4.3 Equal covariances and the ferromagnet 242

6.4.4 Some analytical calculations for mixture models 244

6.5 Local mixtures 248

6.6 Numerical case study: Image completion 249

7 Bayesian inverse quantum theory (BIQT) 257

7.1 Bayesian inverse quantum statistics (BIQS) 258

7.1.1 The likelihood model of quantum theory 258

7.1.2 Prior models for potentials 263

7.1.3 Numerical case study: Inverse quantum statistics 269

7.1.4 Classical approximation 273

7.1.5 Numerical case study: Reconstruction of perturbed periodic potentials using hyperfields and auxiliary fields 276

7.2 Bayesian inverse time-dependent quantum theory (BITDQ) 288

7.2.1 Quantum time series 288

7.2.2 Numerical case study: Time-dependent systems 291

7.3 Bayesian inverse many-body theory 299

7.3.1 Systems of fermions 299

7.3.2 Bayesian inverse Hartree-Fock theory (BIHF) 301

7.3.3 Numerical case study: Inverse Hartree-Fock theory 304

A A priori information and a posteriori control 313

B Probability, free energy, energy, information, entropy, and temperature 323

B.1 Statistics and statistical mechanics 323

B.2 Probability 325

B.2.1 Normalization factors or partition sums 325

B.2.2 Log-probabilities, bit numbers, and free energies 326

B.3 Random variables 326

B.3.1 Averages 327

B.3.2 Information 327

B.3.3 Energy 328

B.4 Temperature and external fields 330

B.4.1 Maximum entropy and Boltzmann-Gibbs distributions 331

B.4.2 Annealing methods 332

B.4.3 Generating functions 332

B.5 Conditional probabilities and disordered systems 339

C Iteration procedures: Learning 345

C.1 Numerical solution of stationarity equations 345

C.2 Learning matrices 348

C.2.1 Learning algorithms for density estimation 348

C.2.2 Linearization and Newton algorithm 349

C.2.3 Massive relaxation 350

C.2.4 Gaussian relaxation 354

C.2.5 Inverting in subspaces 354

C.2.6 Boundary conditions 355

C.3 Initial configurations and kernel methods 357

C.3.1 Truncated equations 357

C.3.2 Kernels for L 359

C.3.3 Kernels for P 361.

Notes:

Includes bibliographical references (pages 365-402) and index.

ISBN:

0801872200

OCLC:

50184931