CERN Accelerating science

ATLAS Slides
Report number	ATL-DAQ-SLIDE-2018-559
Title	Experience with Shifter Assistant
Author(s)	Kazarov, Andrei (Petersburg Nuclear Physics Institute) ; Avolio, Giuseppe (European Laboratory for Particle Physics, CERN) ; Corso-Radu, Alina (University of California, Irvine)
Corporate author(s)	The ATLAS collaboration
Collaboration	ATLAS Collaboration
Submitted to	23rd International Conference on Computing in High Energy and Nuclear Physics, CHEP 2018, Sofia, Bulgaria, 9 - 13 Jul 2018
Submitted by	[email protected] on 30 Jul 2018
Subject category	Particle Physics - Experiment
Accelerator/Facility, Experiment	CERN LHC ; ATLAS
Free keywords	complex event processing ; CEP ; operations ; intelligence
Abstract	The Trigger and DAQ (TDAQ) system of the ATLAS experiment is a complex distributed computing system, composed of O(60000) of applications running on more than 2500 computers. The system is operated by a crew of operators on shift. An important aspect of operations is to minimize the downtime of the system caused by runtime failures, such as human errors, unawareness, miscommunication, etc. The paper describes recent developments in one of “intelligent” TDAQ frameworks, the Shifter Assistant (SA) and summarizes the experience of it’s use in operations of ATLAS in the course of LHC Run 2. SA is a framework whose main aim is to automatize routine system checks, error detection and diagnosis, events correlation etc. in order to help the operators to react on runtime problems promptly and effectively. The tool is based on CEP (Complex Event Processing) technology. It constantly processes this stream of operational events (O(100kHz)) over a set of “directives” (or rules) in the knowledge base, producing human-oriented alerts and making shifters aware about the operational issues. More then 200 directives were developed by TDAQ and ATLAS detector experts for different domains. In this paper we also describe different types of directives which were developed in course of Run 2, and present few examples of most interesting ones, demonstrating the power of CEP for this type of applications.
Related document	Conference Paper ATL-DAQ-PROC-2018-037