Modern optimization methods / Qingna LI.

Format:

Book

Author/Creator:

LI, Qingna, author.

Series:

Current Natural Sciences Series

Language:

English

Subjects (All):

Applied mathematics.

Physical Description:

1 online resource (168 pages)

Edition:

First edition.

Place of Publication:

France : EDP Sciences, [2023]

Summary:

With the fast development of big data and artificial intelligence, a natural question is how do we analyze data more efficiently? One of the efficient ways is to use optimization. What is optimization? Optimization exists everywhere. People optimize. As long as you have choices, you do optimization. Optimization is the key of operations research. This book introduces the basic definitions and theory about numerical optimization, including optimality conditions for unconstrained and constrained optimization, as well as algorithms for unconstrained and constrained problems. Moreover, it also includes the nonsmooth Newton's method, which plays an important role in large-scale numerical optimization. Finally, based on the author's research experiences, several latest applications about optimization are introduced, including optimization algorithms for hypergraph matching, support vector machine and bilevel optimization approach for hyperparameter selection in machine learning. With these optimization tools, one can deal with data more efficiently.

Contents:

Intro

Modern Optimization Methods

Preface

Contents

Introduction

About Optimization

Classification of Optimization

Preliminaries in Convex Analysis

Exercises

Fundamentals of Optimization

Unconstrained Optimization Problem

What is a Solution?

Definitions of Different Solutions

Recognizing a Local Minimum

Nonsmooth Problems

Overview of Algorithms

Line Search Strategy

Trust Region Strategy

Convergence

Scaling

Line Search Methods

Step Length

The Wolfe Conditions

The Goldstein Conditions

Sufficient Decrease and Backtracking

Convergence of Line Search Methods

Rate of Convergence

Steepest Descent Method

Newton's Method

Quasi-Newton Methods

Trust Region Methods

Outline of the Trust Region Approach

Algorithms Based on the Cauchy Point

The Cauchy Point

The Dogleg Method

Two-Dimensional Subspace Minimization

Global Convergence

Reduction Obtained by the Cauchy Point

Convergence to Stationary Points

Local Convergence

Other Enhancements

Conjugate Gradient Methods

Linear Conjugate Gradient Method

Conjugate Direction Method

Conjugate Gradient Method

A Practical Form of the Conjugate Gradient Method

Preconditioning

Nonlinear Conjugate Gradient Methods

The Polak-Ribiere Method and Variants

Semismooth Newton's Method

Semismoothness

Nonsmooth Version of Newton's Method

Support Vector Machine

Semismooth Newton's Method for SVM

Theory of Constrained Optimization

Local and Global Solutions

Smoothness

Examples

Tangent Cone and Constraint Qualifications

First-Order Optimality Conditions

Second-Order Conditions

Duality

KKT Condition

Dual Problem

Exercises.

Penalty and Augmented Lagrangian Methods

The Quadratic Penalty Method

Exact Penalty Method

Augmented Lagrangian Method

Quadratic Penalty Method for Hypergraph Matching

Hypergraph Matching

Mathematical Formulation

Relaxation Problem

Quadratic Penalty Method for (8.21)

Numerical Results

Augmented Lagrangian Method for SVM

Support Vecotr Machine

Augmented Lagrangian Method (ALM)

Semismooth Newton's Method for the Subproblem

Reducing the Computational Cost

Convergence Result of ALM

Numerical Results on LIBLINEAR

Bilevel Optimization and Its Applications

Bilevel Model for a Case of Hyperparameter Selection in SVC

An MPEC Formulation

The Global Relaxation Method (GRM)

MPEC-MFCQ: A Hidden Property

Bibliography.

Notes:

Description based on publisher supplied metadata and other sources.

Description based on print version record.

Includes bibliographical references.

ISBN:

9782759831753

2759831752