Mixtures : estimation and applications / edited by Kerrie L. Mengersen, Christian P. Robert, D. Michael Titterington.

Format:

Book

Contributor:

Mengersen, Kerrie L.

Robert, Christian P., 1961-

Titterington, D. M.

Patrick B. and Anne Gaydosh Hughes Memorial Book Fund.

Series:

Wiley series in probability and statistics

Language:

English

Subjects (All):

Mixture distributions (Probability theory).

Physical Description:

xviii, 311 pages : illustrations ; 24 cm.

Place of Publication:

Chichester, West Sussex : Wiley, 2011.

Summary:

Mengersen (Queensland U. of Technology, Australia), Robert (U. Paris-Dauphine, France), and Titterington (U. of Glasgow, UK) present, alongside two of their own contributions, 12 papers from a March 2010 workshop held at the International Centre for Mathematical Sciences in Edinburgh, Scotland on the use of statistical mixture distributions for modeling scenarios in which certain variables are measured but a categorical variable is missing (such as when clinical data on a patient is available but their disease category is not), as well as variations such as when the missing variable follows a Markov chain model and latent structure models in which the missing variable or variables represent model-enriching devices rather than real physical entities. The papers explore methodological and applied issues of statistical mixture distributions. Both Bayesian and non-Bayesina methods are addressed and prominent application areas include biology and economics. Annotation ©2011 Book News, Inc., Portland, OR (booknews.com)

Contents:

1 The Em algorithm, variational approximations and expectation propagation for mixtures / D. Michael Titterington Titterington, D. Michael 1

1.1 Preamble 1

1.2 The Em algorithm 2

1.2.1 Introduction to the algorithm 2

1.2.2 The E-step and the M-step for the mixing weights 3

1.2.3 The M-step for mixtures of univariate Gaussian distributions 4

1.2.4 M-step for mixtures of regular exponential family distributions formulated in terms of the natural parameters 6

1.2.5 Application to other mixtures 7

1.2.6 Em as a double expectation 8

1.3 Variational approximations 8

1.3.1 Preamble 8

1.3.2 Introduction to variational approximations 9

1.3.3 Application of variational Bayes to mixture problems 11

1.3.4 Application to other mixture problems 16

1.3.5 Recursive variational approximations 18

1.3.6 Asymptotic results 19

1.4 Expectation-propagation 20

1.4.1 Introduction 20

1.4.2 Overview of the recursive approach to be adopted 22

1.4.3 Finite Gaussian mixtures with an unknown mean parameter 23

1.4.4 Mixture of two known distributions 24

1.4.5 Discussion 26

Acknowledgements 27

References 27

2 Online expectation maximisation / Olivier Cappé Cappé, Olivier 31

2.1 Introduction 31

2.2 Model and assumptions 33

2.3 The EM algorithm and the limiting Em recursion 36

2.3.1 The batch Em algorithm 36

2.3.2 The limiting Em recursion 37

2.3.3 Limitations of batch Em for long data records 38

2.4 Online expectation maximisation 40

2.4.1 The algorithm 40

2.4.2 Convergence properties 41

2.4.3 Application to finite mixtures 45

2.4.4 Use for batch maximum-likelihood estimation 47

2.5 Discussion 50

References 52

3 The limiting distribution of the Em test of the order of a finite mixture / Jiahua Chen Chen, Jiahua, Pengfei Li Li, Pengfei 55

3.1 Introduction 55

3.2 The method and theory of the Em test 56

3.2.1 The definition of the Em test statistic 57

3.2.2 The limiting distribution of the EM test statistic 58

3.3 Proofs 60

3.4 Discussion 74

References 74

4 Comparing Wald and likelihood regions applied to locally identifiable mixture models / Daeyoung Kim Kim, Daeyoung, Bruce G. Lindsay Lindsay, Bruce G. 77

4.1 Introduction 77

4.2 Background on likelihood confidence regions 79

4.2.1 Likelihood regions 79

4.2.2 Profile likelihood regions 80

4.2.3 Alternative methods 81

4.3 Background on simulation and visualisation of the likelihood regions 82

4.3.1 Modal simulation method 82

4.3.2 Illustrative example 84

4.4 Comparison between the likelihood regions and the Wald regions 85

4.4.1 Volume/volume error of the confidence regions 86

4.4.2 Differences in univariate intervals via worst case analysis 87

4.4.3 Illustrative example (revisited) 88

4.5 Application to a finite mixture model 89

4.5.1 Nonidentifiabilities and likelihood regions for the mixture parameters 90

4.5.2 Mixture likelihood region simulation and visualisation 93

4.5.3 Adequacy of using the Wald confidence region 94

4.6 Data analysis 95

4.7 Discussion 98

References 99

5 Mixture of experts modelling with social science applications / Isobel Claire Gormley Gormley, Isobel Claire, Thomas Brendan Murphy Murphy, Thomas Brendan 101

5.1 Introduction 101

5.2 Motivating examples 102

5.2.1 Voting blocs 102

5.2.2 Social and organisational structure 103

5.3 Mixture models 103

5.4 Mixture of experts models 104

5.5 A mixture of experts model for ranked preference data 107

5.5.1 Examining the clustering structure 111

5.6 A mixture of experts latent position cluster model 112

5.7 Discussion 118

Acknowledgements 118

References 118

6 Modelling conditional densities using finite smooth mixtures / Feng Li Li, Feng, Mattias Villani Villani, Mattias, Robert Kohn Kohn, Robert 123

6.1 Introduction 123

6.2 The model and prior 125

6.2.1 Smooth mixtures 125

6.2.2 The component models 126

6.2.3 The prior 128

6.3 Inference methodology 129

6.3.1 The general MCMC scheme 129

6.3.2 Updating β and I using variable-dimension finite-step Newton proposals 130

6.3.3 Model comparison 132

6.4 Applications 133

6.4.1 A small simulation study 133

6.4.2 Lidar data 135

6.4.3 Electricity expenditure data 137

6.5 Conclusions 141

Acknowledgements 142

Appendix: Implementation details for the gamma and log-normal models 142

References 143

7 Nonparametric mixed membership modelling using the IBP compound Dirichlet process / Sinead Williamson Williamson, Sinead, Chong Wang Wang, Chong, Katherine A. Heller Heller, Katherine A., David M. Blei Blei, David M. 145

7.1 Introduction 145

7.2 Mixed membership models 146

7.2.1 Latent Dirichlet allocation 147

7.2.2 Nonparametric mixed membership models 147

7.3 Motivation 148

7.4 Decorrelating prevalence and proportion 150

7.4.1 Indian buffet process 150

7.4.2 The IBP compound Dirichlet process 150

7.4.3 An application of the ICD: focused topic models 153

7.4.4 Inference 154

7.5 Related models 155

7.6 Empirical studies 156

7.7 Discussion 158

References 159

8 Discovering nonbinary hierarchical structures with Bayesian rose trees / Charles Blundell Blundell, Charles, Yee Whye Teh Teh, Yee Whye, Katherine A. Heller Heller, Katherine A. 161

8.1 Introduction 161

8.2 Prior work 163

8.3 Rose trees, partitions and mixtures 165

8.4 Avoiding needless cascades 169

8.4.1 Cluster models 171

8.5 Greedy construction of Bayesian rose tree mixtures 172

8.5.1 Prediction 175

8.5.2 Hyperparameter optimisation 176

8.6 Bayesian hierarchical clustering, Dirichlet process models and product partition models 176

8.6.1 Mixture models and product partition models 177

8.6.2 PCluster and Bayesian hierarchical clustering 178

8.7 Results 179

8.7.1 Optimality of tree structure 179

8.7.2 Hierarchy likelihoods 180

8.7.3 Partially observed data 182

8.7.4 Psychological hierarchies 182

8.7.5 Hierarchies of Gaussian process experts 182

8.8 Discussion 183

References 185

9 Mixtures of factor analysers for the analysis of high-dimensional data / Geoffrey J. McLachlan McLachlan, Geoffrey J., Jangsun Baek Baek, Jangsun, Suren I. Rathnayake Rathnayake, Suren I. 189

9.1 Introduction 189

9.2 Single-factor analysis model 191

9.3 Mixtures of factor analysers 192

9.4 "Mixtures of common factor analysers (MCFA) 193

9.5 Some related approaches 196

9.6 Fitting of factor-analytic models 197

9.7 Choice of the number of factors q 199

9.8 Example 199

9.9 Low-dimensional plots via MCFA approach 200

9.10 Multivariate t-factor analysers 202

9.11 Discussion 205

10 Dealing with label switching under model uncertainty / Sylvia Fruhwirth-Schnatter Fruhwirth-Schnatter, Sylvia 213

10.1 Introduction 213

10.2 Labelling through clustering in the point-process representation 214

10.2.1 The point-process representation of a finite mixture model 214

10.2.2 Identification through clustering in the point-process representation 217

10.3 Identifying mixtures when the number of components is unknown 220

10.3.1 The role of Dirichlet priors in overfitting mixtures 221

10.3.2 The meaning of K for overfitting mixtures 222

10.3.3 The point-process representation of overfitting mixtures 224

10.3.4 Examples 227

10.4 Overfitting heterogeneity of component-specific parameters 231

10.4.1 Overfitting heterogeneity 231

10.4.2 Using shrinkage priors on the component-specific location parameters 233

10.5 Concluding remarks 237

References 237

11 Exact Bayesian analysis of mixtures / Christian P. Robert Robert, Christian P., Kerrie L.

Mengersen Mengersen, Kerrie L. 241

11.1 Introduction 241

11.2 Formal derivation of the posterior distribution 242

11.2.1 Locally conjugate priors 242

11.2.2 True posterior distributions 244

11.2.3 Poisson mixture 246

11.2.4 Multinomial mixtures 250

11.2.5 Normal mixtures 252

References 254

12 ManifoId MCMC for mixtures / Vassilios Stathopoulos Stathopoulos, Vassilios, Mark Girolami Girolami, Mark 255

12.1 Introduction 255

12.2 Markov chain Monte Carlo Methods 257

12.2.1 Metropolis-Hastings 257

12.2.2 Gibbs sampling 257

12.2.3 Manifold Metropolis adjusted Langevin algorithm 258

12.2.4 Manifold Hamiltonian Monte Carlo 259

12.3 Finite Gaussian mixture models 259

12.3.1 Gibbs sampler for mixtures of univariate Gaussians 261

12.3.2 Manifold MCMC for mixtures of univariate Gaussians 261

12.3.3 Metric tensor 262

12.3.4 An illustrative example 263

12.4 Experiments 266

12.5 Discussion 272

Acknowledgements 273

Appendix 273

References 274

13 How many components in a finite mixture? / Murray Aitkin Aitkin, Murray 277

13.1 Introduction 277

13.2 The galaxy data 278

13.3 The normal mixture model 279

13.4 Bayesian analyses 280

13.4.1 Escobar and West 281

13.4.2 Phillips and Smith 281

13.4.3 Roeder and Wasserman 282

13.4.4 Richardson and Green 282

13.4.5 Stephens 283

13.5 Posterior distributions for K (for flat prior) 283

13.6 Conclusions from the Bayesian analyses 285

13.7 Posterior distributions of the model deviances 286

13.8 Asymptotic distributions 286

13.9 Posterior deviances for the galaxy data 287

13.10 Conclusions 291

References 291

14 Bayesian mixture models: a blood-free dissection of a sheep / Clair L. Alston Alston, Clair L., Kerrie L. Mengersen Mengersen, Kerrie L., Graham E. Gardner Gardner, Graham E. 293

14.1 Introduction 293

14.2 Mixture models 294

14.2.1 Hierarchical normal mixture 294

14.3 Altering dimensions of the mixture model 296

14.4 Bayesian mixture model incorporating spatial information 298

14.4.1 Results 301

14.5 Volume calculation 302

14.6 Discussion 307

References 307.

Notes:

Includes bibliographical references and index.

Local Notes:

Acquired for the Penn Libraries with assistance from the Patrick B. and Anne Gaydosh Hughes Memorial Book Fund.

ISBN:

9781119993896

111999389X

OCLC:

698450396