Ensemble Methods : Foundations and Algorithms / Zhi-Hua Zhou.

Format:

Book

Author/Creator:

Zhou, Zhi-Hua (Computer scientist), author.

Series:

Chapman & Hall/CRC machine learning & pattern recognition series.

Chapman and Hall/CRC Machine Learning and Pattern Recognition Series

Language:

English

Subjects (All):

Multiple comparisons (Statistics).

Set theory.

Mathematical analysis.

Physical Description:

1 online resource (364 pages)

Edition:

Second edition.

Place of Publication:

Boca Raton, Florida : CRC Press, [2025]

Summary:

Ensemble methods that train multiple learners and then combine them to use, with \textit{Boosting} and \textit{Bagging} as representatives, are well-known machine learning approaches. An ensemble is significantly more accurate than a single learner, and ensemble methods have already achieved great success in various real-world tasks.

Contents:

Cover

Half Title

Series Page

Title Page

Copyright Page

Contents

Preface to the Second Edition

Preface to the First Edition

Notations

1. Introduction

1.1. Terminology

1.2. Popular Learning Algorithms

1.2.1. Decision Trees

1.2.2. Neural Networks

1.2.3. Naïve Bayes Classifier

1.2.4. k-Nearest Neighbor

1.2.5. Linear Discriminant Analysis

1.2.6. Logistic Regression

1.2.7. Support Vector Machines and Kernel Methods

1.3. Evaluation and Comparison

1.4. Ensemble Methods

1.5. Applications of Ensemble Methods

1.6. Further Readings

2. Boosting

2.1. A General Boosting Procedure

2.2. AdaBoost

2.3. Illustrative Examples

2.4. Theoretical Understanding

2.4.1. Training and Generalization

2.4.2. Surrogate Loss and Optimization

2.4.3. Why AdaBoost Resistant to Overfitting

2.5. Gradient Boosting and Implementations

2.5.1. Gradient Boosting

2.5.2. XGBoost and LightGBM

2.6. Extensions

2.6.1. Multi-Class Learning

2.6.2. Multi-Label Learning

2.6.3. Noise Tolerance

2.7. Further Readings

3. Bagging

3.1. Two Ensemble Paradigms

3.2. Bagging

3.3. Illustrative Examples

3.4. Random Subspace and Random Forest

3.5. Theoretical Understanding

3.5.1. About Bagging

3.5.2. About Random Subspace

3.5.3. About Random Forest

3.6. Spectrum of Randomization

3.7. Further Readings

4. Combination Methods

4.1. Benefits of Combination

4.2. Averaging and Voting

4.2.1. Averaging

4.2.2. Voting

4.2.3. Theoretical Explanations

4.3. Combining by Learning

4.3.1. Stacking

4.3.2. Infinite Ensemble

4.4. Dynamic Classifier Selection

4.5. Mixture of Experts

4.6. Other Combination Methods

4.6.1. Algebraic Methods

4.6.2. Behavior Knowledge Space Method

4.6.3. Decision Template Method.

4.6.4. Error-Correcting Output Codes

4.7. Further Readings

5. Diversity

5.1. Ensemble Diversity

5.2. Error Decomposition

5.2.1. Error-Ambiguity Decomposition

5.2.2. Bias-Variance-Covariance Decomposition

5.3. Diversity Measures

5.3.1. Pairwise Measures

5.3.2. Non-Pairwise Measures

5.3.3. Summary and Visualization

5.3.4. Limitation of Diversity Measures

5.4. Theoretical Exploration

5.4.1. PAC Understanding

5.4.2. Information Theoretic Diversity

5.5. Diversity Enhancement

5.6. Structural Diversity

5.7. Further Readings

6. Ensemble Pruning

6.1. What is Ensemble Pruning

6.2. Many Could Be Better Than All

6.3. Categorization of Pruning Methods

6.4. Ordering-Based Pruning

6.5. Clustering-Based Pruning

6.6. Optimization-Based Pruning

6.6.1. Heuristic Optimization Pruning

6.6.2. Mathematical Programming Pruning

6.6.3. Probabilistic Pruning

6.6.4. Pareto Optimization Pruning

6.7. Further Readings

7. Clustering Ensemble

7.1. Clustering

7.1.1. Clustering Methods

7.1.2. Clustering Evaluation

7.1.3. Why Clustering Ensemble

7.2. Categorization of Clustering Ensemble Methods

7.3. Similarity-Based Methods

7.4. Graph-Based Methods

7.5. Relabeling-Based Methods

7.6. Transformation-Based Methods

7.7. Further Readings

8. Anomaly Detection and Isolation Forest

8.1. Anomaly Detection

8.2. Sequential Ensemble Methods

8.3. Parallel Ensemble Methods

8.4. Isolation Forest

8.5. Isolation Forest Extensions

8.6. Learning with Emerging New Class

8.7. Further Readings

9. Semi-Supervised Ensemble

9.1. Semi-Supervised Learning

9.2. Semi-Supervised and Ensemble Mutually Helpful

9.2.1. Learner Combination Helpful to Semi-Supervised Learning

9.2.2. Unlabeled Data Helpful to Ensemble Learning.

9.3. Semi-Supervised Sequential Ensemble Methods

9.4. Semi-Supervised Parallel Ensemble Methods

9.5. Semi-Supervised Clustering Ensemble

9.6. Semi-Supervised Diversity Enhancement

9.7. Further Readings

10. Class-Imbalance and Cost-Sensitive Ensemble

10.1. Class-Imbalance and Cost-Sensitive Learning

10.2. Performance Evaluation

10.2.1. G-Mean, F-Measure, and P-R Curve

10.2.2. ROC Curve and AUC

10.2.3. Total Cost and Cost Curve

10.3. The General Rescaling Approach

10.4. Theoretical Understanding

10.5. Cost-Sensitive Ensemble

10.5.1. Bagging-Based Methods

10.5.2. Boosting-Based Methods

10.6. Class-Imbalance Ensemble

10.6.1. Bagging-Based Methods

10.6.2. Boosting-Based Methods

10.6.3. Hybrid Ensemble Methods

10.7. Further Readings

11. Deep Learning and Deep Forest

11.1. Deep Neural Networks and Ensemble

11.2. Deep Forest

11.3. Forest and Auto-Encoder

11.4. Forest and Hierarchical Distributed Representation

11.5. Deep Forest Extensions

11.5.1. Acceleration

11.5.2. Metric Learning

11.5.3. Multi-Label Learning

11.6. Theoretical Exploration

11.6.1. Generalization

11.6.2. Tree-Layer Structure

11.7. Further Readings

12. Advanced Topics

12.1. Weakly Supervised Learning

12.1.1. Incomplete Supervision

12.1.2. Inexact Supervision

12.1.3. Inaccurate Supervision

12.2. Open-Environment Learning

12.2.1. Changing Data Distributions

12.2.2. Decremental/Incremental Features

12.2.3. Varied Learning Objects

12.3. Reinforcement Learning

12.3.1. Learning to Maximize Long-Term Rewards

12.3.2. Critic Estimation

12.3.3. Uncertainty Estimation

12.4. Online Learning

12.4.1. Learning with Non-Stationary Data Streams

12.4.2. Drifting Ensembles

12.4.3. Online Ensemble

12.5. Improving Understandability.

12.5.1. Ensemble Reduction to Single Model

12.5.2. Rule Extraction from Ensemble

12.5.3. Visualization of Ensemble

12.6. Future Directions of Ensembles

12.7. Further Readings

References

Index.

Notes:

Includes bibliographical references and index.

Description based on publisher supplied metadata and other sources.

Description based on print version record.

ISBN:

9781040307663

1040307663

9781040307632

1040307639

OCLC:

1484556630