Indexado en

Licencia y uso

Citaciones

Altmetrics

Análisis de autorías institucional

Compartir

Alpha entropy search for new information-based Bayesian optimization

Publicado en:KNOWLEDGE-BASED SYSTEMS 322 (): 113612- - 2025-07-08 322(), DOI: 10.1016/j.knosys.2025.113612

Autores: Fernandez-Sanchez, Daniel; Garrido-Merchan, Eduardo C; Hernandez-Lobato, Daniel

Afiliaciones

Resumen

Bayesian optimization (BO) methods based on information theory have obtained state-of-the-art results in several tasks. These techniques rely on the Kullback-Leibler (KL) divergence to compute the acquisition function. We introduce a novel information-based class of acquisition functions for BO called Alpha Entropy Search (AES). AES is based on the alpha-divergence, which generalizes the KL-divergence. Iteratively, AES selects the next evaluation point as the one whose associated target value has the highest level of dependency with respect to the location and associated value of the global maximum of the optimization problem. Dependency is measured in terms of the alpha-divergence, as an alternative to the KL-divergence. Intuitively, this favors evaluating the objective function at the most informative points about the global maximum. The alpha-divergence has a free parameter alpha, which determines the behavior of the divergence, balancing local and global differences. Therefore, different values of alpha result in different acquisition functions. AES acquisition lacks a closed-form expression. However, we propose an efficient and accurate approximation using a truncated Gaussian distribution. In practice, the value of alpha can be chosen by the practitioner, but here we suggest using combination of acquisition functions obtained by simultaneously considering a range of alpha values. We provide an implementation of AES in BOTorch and we evaluate its performance in synthetic, benchmark, and real-world experiments involving the tuning of the hyper-parameters of a deep neural network. These experiments show that AES performance is competitive with other information-based acquisition functions such as JES, MES, or PES.

Palabras clave