Hem > CMS Collection > CMS Preprints > Autoencoder-based Anomaly Detection System for Online Data Quality Monitoring of the CMS Electromagnetic Calorimeter |
Article | |
Report number | arXiv:2309.10157 ; CMS-NOTE-2023-009 |
Title | Autoencoder-based Anomaly Detection System for Online Data Quality Monitoring of the CMS Electromagnetic Calorimeter |
Author(s) |
Abadjiev, D. (Northeastern U.) ; Adams, T. (Yerevan Phys. Inst. ; Florida State U.) ; Adzic, P. (VINCA Inst. Nucl. Sci., Belgrade ; Belgrade U.) ; Ahmad, M. (Texas A-M) ; Amendola, C. (IRFU, Saclay) ; Andrews, M.B. (Carnegie Mellon U.) ; Arcidiacono, R. (INFN, Turin ; Piemonte Orientale U., Alessandria) ; Argiro, S. (INFN, Turin ; Turin U.) ; Askew, A. (Yerevan Phys. Inst. ; Florida State U.) ; Auffray, E. (CERN) ; Azzolini, V. (MIT) ; Bailleux, D. (Notre Dame U.) ; Band, R. (Notre Dame U.) ; Barney, D. (CERN) ; Barria, P. (INFN, Rome) ; Bartosik, N. (INFN, Turin) ; Basile, C. (INFN, Rome ; Rome U.) ; Bastos, D. (LIP, Lisbon) ; Bell, K.W. (Rutherford) ; Besancon, M. (IRFU, Saclay) ; Bianco, R. (INFN, Rome) ; Biino, C. (INFN, Turin) ; Blinov, V. (Unlisted) ; Borca, C. (INFN, Turin ; Turin U.) ; Bornheim, A. (Caltech) ; Brown, R.M. (Rutherford) ; Campana, M. (INFN, Rome ; Rome U.) ; Castells, S. (Notre Dame U.) ; Cavallari, F. (INFN, Rome) ; Cetorelli, F. (INFN, Milan Bicocca ; Milan Bicocca U.) ; Chatterjee, R.M. (Minnesota U.) ; Chatterjee, S. (Indian Inst. Tech., Madras) ; Chaudhary, G. (Panjab U., Chandigarh) ; Chen, J.A. (Taiwan, Natl. Taiwan U.) ; Chernyavskaya, N. (CERN) ; Chung, H. (Virginia U.) ; Cipriani, M. (CERN) ; Cokic, L. (VINCA Inst. Nucl. Sci., Belgrade) ; Cooke, C. (Rutherford) ; Cossio, F. (INFN, Turin) ; Couderc, F. (IRFU, Saclay) ; Cristoforetti, D. (INFN, Turin ; Turin U.) ; Cucciati, G. (Notre Dame U.) ; Cunqueiro Mendez, L. (INFN, Rome ; Rome U.) ; Da Silva Di Calafiori, D. (ETH, Zurich (main)) ; Dafinei, I. (INFN, Rome) ; Cockerill, D.J. A. (Rutherford) ; Dejardin, M. (IRFU, Saclay) ; Del Re, D. (INFN, Rome ; Rome U.) ; Della Ricca, G. (INFN, Trieste) ; Depasse, P. (IP2I, Lyon) ; Dervan, J. (Northeastern U.) ; Di Marco, E. (INFN, Rome) ; Diemoz, M. (INFN, Rome) ; Dimova, T. (Unlisted) ; Dissertori, G. (ETH, Zurich (main)) ; Dittmar, M. (ETH, Zurich (main)) ; Dolgopolov, A. (Notre Dame U.) ; Donegà, M. (ETH, Zurich (main)) ; Dordevic, M. (VINCA Inst. Nucl. Sci., Belgrade) ; El Mamouni, H. (IP2I, Lyon) ; Errico, F. (INFN, Bari ; Bari U.) ; Espinosa, F. (MIT) ; Faure, J.L. (IRFU, Saclay) ; Fay, J. (IP2I, Lyon) ; Fernandez Menendez, J. (ICTEA, Oviedo) ; Ferri, F. (IRFU, Saclay) ; Finco, L. (U. Nebraska, Lincoln) ; Fiori, F. (INFN, Pisa ; Pisa, Scuola Normale Superiore) ; Frahm, E. (Minnesota U.) ; Funk, W. (CERN) ; Gadek, T. (ETH, Zurich (main)) ; Gajownik, J. (Rutherford) ; Galli, M. (ETH, Zurich (main)) ; Ganjour, S. (IRFU, Saclay) ; Gascon, S. (IP2I, Lyon) ; Ghezzi, A. (INFN, Milan Bicocca ; Milan Bicocca U.) ; Ghose, P. (Yerevan Phys. Inst. ; Florida State U.) ; Gninenko, S. (Unlisted) ; Goadhouse, S. (Virginia U.) ; Godinovic, N. (Split Tech. U.) ; Golubev, N. (Unlisted) ; Govoni, P. (INFN, Milan Bicocca ; Milan Bicocca U.) ; Gras, P. (IRFU, Saclay) ; Hakala, J. (Virginia U.) ; Hamel de Monchenault, G. (IRFU, Saclay) ; Harilal, A. (Northeastern U. ; Carnegie Mellon U.) ; Härringer, N. (ETH, Zurich (main)) ; Hashmi, R. (Yerevan Phys. Inst. ; Florida State U.) ; Heath, H.F. (U. Bristol (main)) ; Hirosky, R. (Virginia U.) ; Ho, K.W. (Notre Dame U.) ; Hou, X. (Beijing, Inst. High Energy Phys. ; BSRF, Beijing) ; Ingram, Q. (PSI, Villigen) ; Jain, Sh. (Rutherford ; Tata Inst.) ; Javaid, T. (Beihang U. ; BSRF, Beijing) ; Jessop, C. (Notre Dame U.) ; Jimènez, R. (ETH, Zurich (main)) ; Joshi, B.M. (Minnesota U.) ; Jourd'hui, E. (IP2I, Lyon) ; Kaadze, K. (Kansas State U.) ; Kao, Y.-W. (Taiwan, Natl. Taiwan U.) ; Kardapoltsev, L. (Unlisted) ; Khurana, R. (Taiwan, Natl. Taiwan U.) ; King, J. (Kansas U.) ; Kirilovas, A. (Vilnius U.) ; Konstantinov, D. (Unlisted) ; Kovac, M. (Split Tech. U.) ; Krishna, A. (Northeastern U.) ; Kuo, C.M. (Taiwan, Natl. Central U.) ; Lambrecht, L. (Gent U.) ; Lavizzari, G. (INFN, Milan Bicocca ; Milan Bicocca U.) ; Lecoq, P. (CERN) ; Ledovskoy, A. (Virginia U.) ; Legger, F. (INFN, Turin ; Turin U.) ; Lelas, D. (Split Tech. U.) ; Li, Y. y. (Taiwan, Natl. Taiwan U.) ; Liang, Z. (Tsinghua U., Beijing) ; Lin, W. (Taiwan, Natl. Central U.) ; Longo, E. (INFN, Rome ; Rome U.) ; Loukas, N. (Notre Dame U.) ; Lu, R. -S. (Taiwan, Natl. Taiwan U.) ; Lustermann, W. (ETH, Zurich (main)) ; Lutton, L. (Notre Dame U.) ; Lyon, A. -M. (ETH, Zurich (main)) ; Maeshima, K. (Fermilab) ; Malcles, J. (IRFU, Saclay) ; Mandrik, P. (Unlisted) ; Manzoni, R.A. (ETH, Zurich (main)) ; Marchese, L. (ETH, Zurich (main)) ; Marinelli, N. (Notre Dame U.) ; Marini, A.C. (CERN) ; Martin, L. (Northeastern U.) ; Marzocchi, B. (Northeastern U.) ; Mascellani, A. (ETH, Zurich (main) ; Ecole Polytechnique, Lausanne) ; Massironi, A. (INFN, Milan Bicocca) ; Matveev, V. (Unlisted) ; Mazza, G. (INFN, Turin) ; Meridiani, P. (INFN, Rome) ; Mijic, M. (VINCA Inst. Nucl. Sci., Belgrade) ; Mijuskovic, J. (Montenegro U. ; IRFU, Saclay) ; Milenovic, P. (VINCA Inst. Nucl. Sci., Belgrade) ; Milosevic, J. (VINCA Inst. Nucl. Sci., Belgrade) ; Monteno, M. (INFN, Turin) ; Monti, F. (Beijing, Inst. High Energy Phys.) ; Moortgat, F. (CERN) ; Mousa, J. (Cyprus U.) ; Mudholkar, T. (Carnegie Mellon U.) ; Nessi-Tedaldi, F. (ETH, Zurich (main)) ; Nicolaou, C. (Cyprus U.) ; Nigamova, A. (Hamburg U.) ; Obertino, M.M. (INFN, Turin ; Turin U.) ; Organtini, G. (INFN, Rome ; Rome U.) ; Orimoto, T. (Northeastern U.) ; Orlandi, F. (INFN, Turin ; Turin U.) ; Ovtin, I. (Unlisted) ; Paganis, E. (Taiwan, Natl. Taiwan U.) ; Papagiannis, D. (CERN) ; Pandolfi, F. (INFN, Rome) ; Paramatti, R. (INFN, Rome ; Rome U.) ; Park, K. (Carnegie Mellon U.) ; Pastrone, N. (INFN, Turin) ; Paulini, M. (Carnegie Mellon U.) ; Pauss, F. (ETH, Zurich (main)) ; Petkovic, , A. (Split Tech. U.) ; Petraityte, E. (Vilnius U.) ; Pettinacci, V. (INFN, Rome) ; Petyt, D. (Rutherford) ; Pigazzini, S. (ETH, Zurich (main)) ; Pinolini, B.S. (INFN, Milan Bicocca) ; Prova, P.R. (Yerevan Phys. Inst. ; Florida State U.) ; Quaranta, C. (INFN, Rome ; Rome U.) ; Ragazzi, S. (INFN, Milan Bicocca ; Milan Bicocca U.) ; Rahatlou, S. (INFN, Rome ; Rome U.) ; Rasteiro Da Silva, J.C. (LIP, Lisbon) ; Razis, P.A. (Cyprus U.) ; Rebello Teles, P. (Rio de Janeiro, CBPF) ; Reis, T. (Rutherford) ; Riti, F. (ETH, Zurich (main)) ; Rogan, C. (Kansas U.) ; Romanteau, T. (LLR, Palaiseau) ; Rosowsky, A. (IRFU, Saclay) ; Rovelli, C. (INFN, Rome) ; Rovere, M. (CERN) ; Rusack, R. (Minnesota U.) ; Salvi, G. (Rutherford) ; Sancar, O. (Minnesota U.) ; Sanchez, A. (Carnegie Mellon U.) ; Sandever, C. (Rutherford) ; Santanastasio, F. (INFN, Rome ; Rome U.) ; Saradhy, R. (Minnesota U.) ; Sarkar, U. (LLR, Palaiseau) ; Schneider, M. (CERN) ; Schroeder, N. (Minnesota U.) ; Sculac, A. (Split Tech. U.) ; Sculac, T. (Split Tech. U.) ; Shahzad, M.A. (Beijing, Inst. High Energy Phys. ; BSRF, Beijing) ; Shepherd-Themistocleous, C.H. (Rutherford) ; Simkina, P. (IRFU, Saclay) ; Singla, A. (Panjab U., Chandigarh) ; Singovsky, A. (Notre Dame U.) ; Skovpen, Y. (Unlisted) ; Smith, V.J. (U. Bristol (main)) ; Soffi, L. (INFN, Rome ; Rome U.) ; Stachon, K. (ETH, Zurich (main)) ; Steen, A. (CERN) ; Steggemann, J. (ETH, Zurich (main) ; Ecole Polytechnique, Lausanne) ; Succar, M. (Andes U., Bogota) ; Tao, J. (Beijing, Inst. High Energy Phys.) ; Tishelman-Charny, A. (Northeastern U.) ; Tiwari, P.C. (Indian Inst. Tech., Madras ; Bangalore, Indian Inst. Sci.) ; Tornago, M. (INFN, Turin ; Turin U.) ; Tramontano, R. (INFN, Rome ; Rome U.) ; Tsai, L. -S. (Taiwan, Natl. Taiwan U.) ; Usai, E. (Alabama U.) ; Valsecchi, D. (ETH, Zurich (main) ; CERN) ; Vagnerini, A. (U. Nebraska, Lincoln) ; Varela, J. (LIP, Lisbon) ; Venditti, R. (INFN, Bari) ; Verma, P.V. (Indian Inst. Tech., Madras) ; Vlassov, E. (INFN, Turin) ; Wachirapusitanand, V. (Chulalongkorn U.) ; Wamorkar, T. (Northeastern U.) ; Wang, C. (Beijing, Inst. High Energy Phys. ; BSRF, Beijing) ; Wang, J. (Beijing, Inst. High Energy Phys.) ; Wadud, M.A. (Minnesota U.) ; Yu, S.S. (Taiwan, Natl. Central U.) ; Zabi, A. (LLR, Palaiseau) ; Zghiche, A. (LLR, Palaiseau) ; Zhang, L. (Caltech) ; Zhu, R.Y. (Caltech) ; Zygala, L. (Notre Dame U.) |
Publication | 2024-06-24 |
Imprint | 2023-09-18 |
Number of pages | 19 |
Note | Replaced with the published version. Added the journal reference and the DOI |
In: | Comput. Softw. Big Sci. 8 (2024) 11 |
DOI | 10.1007/s41781-024-00118-z |
Subject category | physics.data-an ; Other Fields of Physics ; hep-ex ; Particle Physics - Experiment ; cs.LG ; Computing and Computers ; physics.ins-det ; Detectors and Experimental Techniques |
Accelerator/Facility, Experiment | CERN LHC ; CMS |
Abstract | The CMS detector is a general-purpose apparatus that detects high-energy collisions produced at the LHC. Online Data Quality Monitoring of the CMS electromagnetic calorimeter is a vital operational tool that allows detector experts to quickly identify, localize, and diagnose a broad range of detector issues that could affect the quality of physics data. A real-time autoencoder-based anomaly detection system using semi-supervised machine learning is presented enabling the detection of anomalies in the CMS electromagnetic calorimeter data. A novel method is introduced which maximizes the anomaly detection performance by exploiting the time-dependent evolution of anomalies as well as spatial variations in the detector response. The autoencoder-based system is able to efficiently detect anomalies, while maintaining a very low false discovery rate. The performance of the system is validated with anomalies found in 2018 and 2022 LHC collision data. Additionally, the first results from deploying the autoencoder-based system in the CMS online Data Quality Monitoring workflow during the beginning of Run 3 of the LHC are presented, showing its ability to detect issues missed by the existing system. |
Copyright/License | publication: © 2024 The Author(s) (License: CC-BY-4.0) preprint: (License: CC BY 4.0) |