CERN Accelerating science

ATLAS Slides
Report number	ATL-SOFT-SLIDE-2015-143
Title	Fine grained event processing on HPCs with the ATLAS Yoda system
Author(s)	Calafiura, Paolo (Lawrence Berkeley National Laboratory and University of California, Berkeley) ; De, Kaushik (The University of Texas at Arlington) ; Guan, Wen (Department of Physics, University of Wisconsin) ; Maeno, Tadashi (Brookhaven National Laboratory (BNL)) ; Nilsson, Paul (Brookhaven National Laboratory (BNL)) ; Oleynik, Danila (Joint Institute for Nuclear Research) ; Panitkin, Sergey (Brookhaven National Laboratory (BNL)) ; Tsulaia, Vakhtang (Lawrence Berkeley National Laboratory and University of California, Berkeley) ; van Gemmeren, Peter (Argonne National Laboratory) ; Wenaus, Torre (Brookhaven National Laboratory (BNL))
Corporate author(s)	The ATLAS collaboration
Submitted to	21st International Conference on Computing in High Energy and Nuclear Physics, Okinawa, Japan, 13 - 17 Apr 2015
Submitted by	[email protected] on 06 Apr 2015
Subject category	Particle Physics - Experiment
Accelerator/Facility, Experiment	CERN LHC ; ATLAS
Abstract	High performance computing facilities present unique challenges and opportunities for HENP event processing. The massive scale of many HPC systems means that fractionally small utilizations can yield large returns in processing throughput. Parallel applications which can dynamically and efficiently fill any scheduling opportunities the resource presents benefit both the facility (maximal utilization) and the (compute-limited) science. The ATLAS Yoda system provides this capability to HENP-like event processing applications by implementing event-level processing in an MPI-based master-client model that integrates seamlessly with the more broadly scoped ATLAS Event Service. Fine grained, event level work assignments are intelligently dispatched to parallel workers to sustain full utilization on all cores, with outputs streamed off to destination object stores in near real time with similarly fine granularity, such that processing can proceed until termination with full utilization. The system offers the efficiency and scheduling flexibility of preemption without requiring the application actually support or employ checkpointing. We will present the new Yoda system, its motivations, architecture, implementation, and applications in ATLAS data processing at several US HPC centers.