{rfName}

Indexed in

License and use

Citations

Altmetrics

Analysis of institutional authors

Fernandez-Sanchez, DanielCorresponding AuthorHernandez-Lobato, DanielAuthor

Share

June 9, 2025
Publications
>
Article
No

Alpha entropy search for new information-based Bayesian optimization

Publicated to:KNOWLEDGE-BASED SYSTEMS 322 (): 113612- - 2025-07-08 322(), DOI: 10.1016/j.knosys.2025.113612

Authors: Fernandez-Sanchez, Daniel; Garrido-Merchan, Eduardo C; Hernandez-Lobato, Daniel

Affiliations

Univ Autonoma Madrid, Escuela Politecn Super, Comp Sci Dept, Machine Learning Grp, Francisco Tomas & Valiente 11, Madrid 28049, Spain - Author
Univ Pontificia Comillas, Inst Res Technol IIT, Alberto Aguilera 23, Madrid 28015, Spain - Author

Abstract

Bayesian optimization (BO) methods based on information theory have obtained state-of-the-art results in several tasks. These techniques rely on the Kullback-Leibler (KL) divergence to compute the acquisition function. We introduce a novel information-based class of acquisition functions for BO called Alpha Entropy Search (AES). AES is based on the alpha-divergence, which generalizes the KL-divergence. Iteratively, AES selects the next evaluation point as the one whose associated target value has the highest level of dependency with respect to the location and associated value of the global maximum of the optimization problem. Dependency is measured in terms of the alpha-divergence, as an alternative to the KL-divergence. Intuitively, this favors evaluating the objective function at the most informative points about the global maximum. The alpha-divergence has a free parameter alpha, which determines the behavior of the divergence, balancing local and global differences. Therefore, different values of alpha result in different acquisition functions. AES acquisition lacks a closed-form expression. However, we propose an efficient and accurate approximation using a truncated Gaussian distribution. In practice, the value of alpha can be chosen by the practitioner, but here we suggest using combination of acquisition functions obtained by simultaneously considering a range of alpha values. We provide an implementation of AES in BOTorch and we evaluate its performance in synthetic, benchmark, and real-world experiments involving the tuning of the hyper-parameters of a deep neural network. These experiments show that AES performance is competitive with other information-based acquisition functions such as JES, MES, or PES.

Keywords

Quality index

Impact and social visibility

From the perspective of influence or social adoption, and based on metrics associated with mentions and interactions provided by agencies specializing in calculating the so-called "Alternative or Social Metrics," we can highlight as of 2025-08-02:

  • The use, from an academic perspective evidenced by the Altmetric agency indicator referring to aggregations made by the personal bibliographic manager Mendeley, gives us a total of: 2.
  • The use of this contribution in bookmarks, code forks, additions to favorite lists for recurrent reading, as well as general views, indicates that someone is using the publication as a basis for their current work. This may be a notable indicator of future more formal and academic citations. This claim is supported by the result of the "Capture" indicator, which yields a total of: 1 (PlumX).

With a more dissemination-oriented intent and targeting more general audiences, we can observe other more global scores such as:

  • The Total Score from Altmetric: 1.
  • The number of mentions on the social network X (formerly Twitter): 1 (Altmetric).

It is essential to present evidence supporting full alignment with institutional principles and guidelines on Open Science and the Conservation and Dissemination of Intellectual Heritage. A clear example of this is:

Leadership analysis of institutional authors

There is a significant leadership presence as some of the institution’s authors appear as the first or last signer, detailed as follows: First Author (FERNANDEZ SANCHEZ, DANIEL) and Last Author (HERNANDEZ LOBATO, DANIEL).

the author responsible for correspondence tasks has been FERNANDEZ SANCHEZ, DANIEL.