CERN Accelerating science

Article
Report number	arXiv:2311.17508
Title	Model Performance Prediction for Hyperparameter Optimization of Deep Learning Models Using High Performance Computing and Quantum Annealing
Author(s)	Amboage, Juan Pablo García (CERN ; U. Santiago de Compostela (main)) ; Wulff, Eric (CERN) ; Girone, Maria (CERN) ; Pena, Tomás F. (U. Santiago de Compostela (main))
Publication	2024
Imprint	2023-11-29
Number of pages	7
In:	EPJ Web Conf. 295 (2024) 12005
In:	26th International Conference on Computing in High Energy & Nuclear Physics, Norfolk, Virginia, Us, 8 - 12 May 2023, pp.12005
DOI	10.1051/epjconf/202429512005
Subject category	physics.data-an ; Other Fields of Physics ; cs.LG ; Computing and Computers
Abstract	Hyperparameter Optimization (HPO) of Deep Learning-based models tends to be a compute resource intensive process as it usually requires to train the target model with many different hyperparameter configurations. We show that integrating model performance prediction with early stopping methods holds great potential to speed up the HPO process of deep learning models. Moreover, we propose a novel algorithm called Swift-Hyperband that can use either classical or quantum support vector regression for performance prediction and benefit from distributed High Performance Computing environments. This algorithm is tested not only for the Machine-Learned Particle Flow model used in High Energy Physics, but also for a wider range of target models from domains such as computer vision and natural language processing. Swift-Hyperband is shown to find comparable (or better) hyperparameters as well as using less computational resources in all test cases.
Copyright/License	publication: © 2024 The authors preprint: (License: CC BY 4.0) CC-BY-4.0

Corresponding record in: Inspire