Sample efficient bayesian optimization from local search to preference learning Kaiwen Wu

Format:

Book

Thesis/Dissertation

Author/Creator:

Wu, Kaiwen, author.

Contributor:

University of Pennsylvania. Computer and Information Science., degree granting institution.

Language:

English

Subjects (All):

Computer science.

Computer engineering.

Applied mathematics.

0984.

0464.

0364.

Local Subjects:

Computer science.

Computer engineering.

Applied mathematics.

0984.

0464.

0364.

Genre:

Academic theses

Physical Description:

1 online resource (117 pages)

Contained In:

Dissertations Abstracts International 87-12B

Place of Publication:

Ann Arbor : ProQuest Dissertations and Theses, 2026

Language Note:

English

Summary:

Bayesian optimization (BO) is a powerful framework for sample-efficient optimization of expensive black-box functions. It has been successfully applied to a wide range of domains including machine learning hyperparameter tuning and engineering designs. However, Bayesian optimization is conventionally restricted to problems of moderate dimensionality (e.g., d ≤ 20) where direct, albeit noisy, function evaluations are available. This thesis advances the applicability of Bayesian optimization to broader settings: high-dimensional search spaces and preference-based optimization where no direct function evaluations are available.The first part of this thesis focuses on high-dimensional Bayesian optimization, where standard BO methods often fall short due to the curse of dimensionality. A promising recent development is the use of local search strategies, which deliver strong empirical performance compared to conventional global approaches. We provide a convergence analysis of local BO algorithms that navigate the objective using gradient information inferred from Gaussian processes. We derive convergence rates to stationary points and show that sample complexity scales polynomially with the input dimension. This contrasts sharply with the exponential dependence typical of global BO methods. Then, we propose a new local BO algorithm that follows the most-probable descent direction. This approach explicitly accounts for posterior uncertainty in the surrogate model and outperforms methods that simply follow the model-predicted gradient direction.The second part of this thesis focuses on Bayesian optimization applied to preference learning where no direct function evaluation is available. In this setting, we wish to optimize an objective through pairwise comparisons provided by a user. In particular, under the standard probit observation model, the knowledge gradient acquisition function has generally been considered intractable, which has been a major barrier to practical use in preferential Bayesian optimization. Contrary to this prevailing view, we derive a closed-form knowledge gradient acquisition function for preferential Bayesian optimization under this model. We then show that this acquisition function yields strong empirical performance in this setting

Notes:

Source: Dissertations Abstracts International, Volume: 87-12, Section: B.

Advisors: Gardner, Jacob R. Committee members: Bastani, Osbert; Perdikaris, Paris; Wong, Eric; Garnett, Roman

Ph.D. University of Pennsylvania 2026

Vendor supplied data

Local Notes:

School code: 0175

ISBN:

9798247973225

Access Restriction:

Restricted for use by site license