Optimizing hyperparameters for machine learning algorithms in production / Jonathan Krauss.

Format:

Book

Author/Creator:

Krauss, Jonathan, author.

Language:

English

Subjects (All):

Engineering.

Physical Description:

1 online resource (260 pages)

Edition:

1. Auflage.

Place of Publication:

Aachen : Apprimus Verlag, [2022]

Summary:

Machine learning (ML) offers the potential to train data-based models and therefore to extract knowledge from data. Due to an increase in networking and digitalization, data and consequently the application of ML are growing in production. The creation of ML models includes several tasks that need to be conducted within data integration, data preparation, modeling, and deployment. One key design decision in this context is the selection of the hyperparameters of an ML algorithm - regardless of whether this task is conducted manually by a data scientist or automatically by an AutoML system. Therefore, data scientists and AutoML systems rely on hyperparameter optimization (HPO) techniques: algorithms that automatically identify good hyperparameters for ML algorithms. The selection of the HPO technique is of great relevance, since it can improve the final performance of an ML model by up to 62 % and reduce its errors by up to 95 %, compared to computing with default values. As the selection of the HPO technique depends on different domain-specific influences, it becomes more and more popular to use decision support systems to facilitate this selection. Since no approach exists, which covers the requirements from the production domain, the main research question of this thesis was: Can a decision support system be developed that supports in the selecting of HPO techniques in the production domain?

Contents:

Intro

I Table of Contents

II Formula Symbols and Abbreviations

III List of Figures

IV List of Tables

1 Introduction and Motivation

1.1 Motivation

1.2 Problem Statement, Thesis Goal, and Research Questions

1.3 Research Approach

2 Fundamentals in the Selection of Hyperparameter OptimizationTechniques

2.1 Application Areas of Machine Learning in Production

2.1.1 Predictive Maintenance

2.1.2 Predictive Quality

2.1.3 Process Parameter Prediction

2.2 Machine Learning Pipeline in Production

2.2.1 Phases and Tasks within the Machine Learning Pipeline

2.2.2 Automated Machine Learning (AutoML)

2.3 Machine Learning and Hyperparameters

2.3.1 Types of Machine Learning

2.3.2 Supervised Learning: Categories and Mathematical Formulation

2.3.3 Hyperparameters

2.3.4 Comparing Machine Learning Algorithms and Inherent Hyperparameters

2.4 Hyperparameter Optimization

2.4.1 Importance of Hyperparameter Optimization

2.4.2 Mathematical Problem of Hyperparameter Optimization

2.4.3 Solving the HPO Problem

2.4.4 Hyperparameter Optimization Techniques

2.4.5 Selecting a Hyperparameter Optimization Technique

2.5 Decision Support Systems and Expert Systems

2.6 Derivation of Requirements of the Decision Support System

2.6.1 Setting Up the Decision Support System

2.6.2 Inner Logic of the Decision Support System

2.6.3 Usage of the Decision Support System

2.7 Interim Conclusion and Action Fields

3 Existing Approaches and Required Action

3.1 Existing Approach

3.1.1 Non Computer-based Approaches Targeted at the HPO Technique Selection

3.1.2 Computer-based Approaches Targeted at HPO Technique SelectionProprietary AutoML systems

3.1.3 Approaches from other Algorithm Selection Problems

3.2 Critical Evaluation of Existing Approaches and Required Action.

3.3 Interim Conclusion

4 Development of an Expert System for the Selection ofHPO Techniques in Production

4.1 Design of System Architecture and General Description of itsComponents

4.1.1 System Architecture, Component Interaction, and Embedding into theEnvironment

4.1.2 Knowledge Base

4.1.3 Data Base

4.1.4 Inference Engine

4.2 Setting Up the Knowledge Base

4.2.1 Knowledge Sources

4.2.2 Generation of Rules

4.2.3 Selection of Antecedents and Consequents

4.3 Detailing and Modifying the Expert System

4.3.1 Conversion of Referential Values

4.3.2 Rule Expansion versus Arithmetic Approach

4.3.3 Handling Biases by Adjusting Antecedent Weights

4.3.4 Knock-out Rules

4.3.5 Detailed and Modified Knowledge Base

4.4 Implementation of the Expert System

4.5 Interim Conclusion

5 Verification and Validation

5.1 Verification

5.1.1 Dynamic Input Testing

5.1.2 Evaluation of Compliance with System Requirements

5.2 Case Study for Validation

5.2.1 General set-up of the case study

5.2.2 Data Set 1: Prediction of Process Parameters in Blisk Milling

5.2.3 Data Set 2: Predictive Quality in Sensor System Production

5.2.4 Data Set 3: Predictive Quality in Surface Crack Detection

5.2.5 Data Set 4: Predictive Maintenance for APS Failure at Scania Trucks

5.2.6 Data Set 5: Predictive Maintenance for Turbofan Engine Degradation

5.3 Analyzing the Results for Validation

5.3.1 Data Set 1: Prediction of Process Parameters in Blisk Milling

5.3.2 Data Set 2: Predictive Quality in Sensor System Production

5.3.3 Data Set 3: Predictive Quality in Surface Crack Detection

5.3.4 Data Set 4: Predictive Maintenance for APS Failure at Scania Trucks

5.3.5 Data Set 5: Predictive Maintenance for Turbofan Engine Degradation

5.3.6 Summary of the Validation Results.

5.4 Conclusion and Critical Discussion

6 Outlook and Summary

6.1 Outlook and Need for Further Research

6.2 Summary

V Bibliography

VI Annex

A.1 Steps and Example of CMA-ES

A.2 Steps and Examples of GPBO

A.3 Steps and Examples of SMAC

A.4 Steps and Examples of TPE

A.5 Hyperparameters and their Configuration Spaces of theML Algorithms and the Default Hyperparameters

A.6 Example for Predictive Quality Cheat Sheet

A.7 Example for Predictive Maintenance Cheat Sheet

A.8 Example for Prediction of Process Parameters HarveyBalls

A.9 AutoML Systems and their Functionalities

A.10 Machine Learning Algorithms and Data Sets in Production

A.11 Techniques for Verification and Validation

A.12 Available Data Sets for ML in Production

A.13 Knowledge Acquisition from Experts

A.14 Knowledge Acquisition from Scientific Papers

A.15 Antecedents in the HPO Knowledge Base that Build onML Algorithm Selection Criteria

A.16 Importance of the Antecedents of the HPO-BRBES

A.17 List of Data Preparation Techniques

B.1 Source Code of BRBES, Including the Rules in theKnowledge Base

B.2 Source Code and Results of Validation.

Notes:

Description based on print version record.

Other Format:

Print version: Krauß, Jonathan Optimizing Hyperparameters for Machine Learning Algorithms in Production

ISBN:

9783985550746