The public defence of the thesis takes place on Monday March 17th, 2024 at 13:15

Thesis title: Bayesian Optimization when Knowing Little and A Lot

Author: Carl Hvarfner, Department of Computer Science, Lund University

Faculty opponent: Associate Professor Luigi Acerbi, University of Helsinki, Finland

Examination Committee:

• Associate Professor Carola Doerr, Sorbonne University, France
• Associate Professor Fredrik Lindsten, Linköping University
• Doctor Aaron Klein, ScaDS.AI, Germany
• Deputy: Professor Bo Bernhardsson, Lund University

Session chair: Head of department Elin A. Topp, Lund University

Supervisors:

• Professor Jacek Malec, Lund University
• Associate Professor Luigi Nardi, Lund University

Location: M:D, M-building, Ole Römers väg 1F, Lund, Sweden

For download: Follow this link to download the pdf: Link to be added later

Abstract

Bayesian Optimization has emerged as a crucial technique for optimizing costly, black-box functions where each evaluation comes at a high cost, such as in scientific experiments, and machine learning hyperparameter optimization. By combining probabilistic modeling with sequential decision-making, Bayesian Optimization achieves efficient exploration, guiding the search toward optimal parameters with minimal data. However, real-world applications present three main challenges: leveraging expert knowledge, ensuring accurate model assumptions, and managing high-dimensional search spaces.

This thesis addresses these challenges by advancing Bayesian Optimization in three key areas. First, it develops methods to incorporate practitioner insights directly into the optimization process, using domain expert knowledge to guide the search more efficiently and reduce the need for extensive evaluations. Second, it proposes techniques for dynamically validating and adapting model assumptions, enabling the Gaussian Process surrogates commonly used in Bayesian Optimization to align more closely with the complexities of real-world objective functions. Finally, this work introduces adaptive strategies for high-dimensional optimization, allowing Bayesian Optimization to focus on relevant subspaces and improve sample efficiency in vast parameter spaces, thereby mitigating the "Curse of Dimensionality."

These contributions collectively enhance Bayesian Optimization’s robustness, adaptability, and efficiency, positioning it as a more powerful tool for sample-efficient optimization in complex, resource-intensive scenarios. By demonstrating these improvements through theoretical insights and empirical evaluations, this thesis establishes a pathway for more effective Bayesian Optimization in diverse, real-world applications where data is sparse and costly to obtain.