1.
|
Grid production with the ATLAS Event Service
/ ATLAS Collaboration
ATLAS has developed and previously presented a new computing architecture, the Event Service, that allows real time delivery of fine grained workloads which process dispatched events (or event ranges) and immediately streams outputs. The principal aim was to profit from opportunistic resources such as commercial cloud, supercomputing, and volunteer computing, and otherwise unused cycles on clusters and grids. [...]
ATL-SOFT-SLIDE-2018-385.-
Geneva : CERN, 2018 - 1 p.
Fulltext: PDF; External link: Original Communication (restricted to ATLAS)
In : 23rd International Conference on Computing in High Energy and Nuclear Physics, CHEP 2018, Sofia, Bulgaria, 9 - 13 Jul 2018
|
|
2.
|
Production Experience with the ATLAS Event Service
/ Benjamin, Douglas (Duke University, Department of Physics) ; Calafiura, Paolo (Lawrence Berkeley National Laboratory and University of California, Berkeley) ; Childers, John Taylor (Argonne National Laboratory) ; De, Kaushik (The University of Texas at Arlington) ; Guan, Wen (Department of Physics, University of Wisconsin) ; Maeno, Tadashi (Brookhaven National Laboratory (BNL)) ; Nilsson, Paul (Brookhaven National Laboratory (BNL)) ; Tsulaia, Vakhtang (Lawrence Berkeley National Laboratory and University of California, Berkeley) ; van Gemmeren, Peter (Argonne National Laboratory) ; Wenaus, Torre (Brookhaven National Laboratory (BNL))
/ATLAS Collaboration
The ATLAS Event Service (ES) has been designed and implemented for efficient running of ATLAS production workflows on a variety of computing platforms, ranging from conventional Grid sites to opportunistic, often short-lived resources, such as spot market commercial clouds, supercomputers and volunteer computing. The Event Service architecture allows real time delivery of fine grained workloads to running payload applications which process dispatched events or event ranges and immediately stream the outputs to highly scalable Object Stores. [...]
ATL-SOFT-SLIDE-2016-678.-
Geneva : CERN, 2016 - 1 p.
Fulltext: PDF; External link: Original Communication (restricted to ATLAS)
In : 22nd International Conference on Computing in High Energy and Nuclear Physics, CHEP 2016, San Francisco, Usa, 10 - 14 Oct 2016
|
|
3.
|
|
|
Production experience with the ATLAS Event Service
/ Benjamin, Douglas (Duke University, Department of Physics) ; Calafiura, Paolo (Lawrence Berkeley National Laboratory and University of California, Berkeley) ; Childers, John Taylor (Argonne National Laboratory) ; De, Kaushik (The University of Texas at Arlington) ; Guan, Wen (Department of Physics, University of Wisconsin) ; Maeno, Tadashi (Brookhaven National Laboratory (BNL)) ; Nilsson, Paul (Brookhaven National Laboratory (BNL)) ; Tsulaia, Vakhtang (Lawrence Berkeley National Laboratory and University of California, Berkeley) ; van Gemmeren, Peter (Argonne National Laboratory) ; Wenaus, Torre (Brookhaven National Laboratory (BNL))
The ATLAS Event Service (AES) has been designed and implemented for efficient running of ATLAS production workflows on a variety of computing platforms, ranging from conventional Grid sites to opportunistic, often short-lived resources, such as spot market commercial clouds, supercomputers and volunteer computing. [...]
ATL-SOFT-PROC-2017-006.
-
2017. - 8 p.
Original Communication (restricted to ATLAS) - IOP Open Access Article - Full text
|
|
4.
|
Scaling up ATLAS Event Service to production levels on opportunistic computing platforms
/ Benjamin, Douglas (Duke University, Department of Physics) ; Caballero, Jose (Brookhaven National Laboratory (BNL)) ; Ernst, Michael (Brookhaven National Laboratory (BNL)) ; Guan, Wen (Department of Physics, University of Wisconsin) ; Hover, John (Brookhaven National Laboratory (BNL)) ; Lesny, David (University of Illinois at Urbana-Champaign) ; Maeno, Tadashi (Brookhaven National Laboratory (BNL)) ; Nilsson, Paul (Brookhaven National Laboratory (BNL)) ; Tsulaia, Vakhtang (Lawrence Berkeley National Laboratory and University of California, Berkeley) ; van Gemmeren, Peter (Argonne National Laboratory) et al.
Continued growth in public cloud and HPC resources is on track to overcome the dedicated resources available for ATLAS on the WLCG. Example of such platforms are Amazon AWS EC2 Spot Instances, Edison Cray XC30 supercomputer, backfill at the Tier-2 and Tier-3 sites, opportunistic resources at the Open Science Grid, and ATLAS High Level Trigger farm between the data taking periods. [...]
ATL-SOFT-SLIDE-2016-012.-
Geneva : CERN, 2016 - 1 p.
Fulltext: PDF; External link: Original Communication (restricted to ATLAS)
In : 17th INTERNATIONAL WORKSHOP ON ADVANCED COMPUTING AND ANALYSIS TECHNIQUES IN PHYSICS RESEARCH, Valparaíso, Valparaíso, Chile, 18 - 22 Jan 2016
|
|
5.
|
|
|
Scaling up ATLAS Event Service to production levels on opportunistic computing platforms
/ Benjamin, Douglas (Duke University, Department of Physics) ; Caballero, Jose (Brookhaven National Laboratory (BNL)) ; Ernst, Michael (Brookhaven National Laboratory (BNL)) ; Guan, Wen (Department of Physics, University of Wisconsin) ; Hover, John (Brookhaven National Laboratory (BNL)) ; Lesny, David (University of Illinois at Urbana-Champaign) ; Maeno, Tadashi (Brookhaven National Laboratory (BNL)) ; Nilsson, Paul (Brookhaven National Laboratory (BNL)) ; Tsulaia, Vakhtang (Lawrence Berkeley National Laboratory and University of California, Berkeley) ; van Gemmeren, Peter (Argonne National Laboratory) et al.
Continued growth in public cloud and HPC resources is on track to exceed the dedicated resources available for ATLAS on the WLCG. [...]
ATL-SOFT-PROC-2016-001.
-
2016. - 5 p.
Original Communication (restricted to ATLAS) - Full text - Fulltext
|
|
6.
|
The ATLAS Event Service: A New Approach to Event Processing
/ Calafiura, Paolo (Lawrence Berkeley National Laboratory and University of California, Berkeley) ; De, Kaushik (The University of Texas at Arlington) ; Guan, Wen (Department of Physics, University of Wisconsin) ; Maeno, Tadashi (Brookhaven National Laboratory (BNL)) ; Nilsson, Paul (Brookhaven National Laboratory (BNL)) ; Oleynik, Danila (Joint Institute for Nuclear Research) ; Panitkin, Sergey (Brookhaven National Laboratory (BNL)) ; Tsulaia, Vakhtang (Lawrence Berkeley National Laboratory and University of California, Berkeley) ; van Gemmeren, Peter (Argonne National Laboratory) ; Wenaus, Torre (Brookhaven National Laboratory (BNL))
The ATLAS Event Service (ES) implements a new fine grained approach to HEP event processing, designed to be agile and efficient in exploiting transient, short-lived resources such as HPC hole-filling, spot market commercial clouds, and volunteer computing. Input and output control and data flows, bookkeeping, monitoring, and data storage are all managed at the event level in an implementation capable of supporting ATLAS-scale distributed processing throughputs (about 4M CPU-hours/day). [...]
ATL-SOFT-SLIDE-2015-150.-
Geneva : CERN, 2015 - 17 p.
Fulltext: PDF; External link: Original Communication (restricted to ATLAS)
|
|
7.
|
|
8.
|
Fine grained event processing on HPCs with the ATLAS Yoda system
/ Calafiura, Paolo (Lawrence Berkeley National Laboratory and University of California, Berkeley) ; De, Kaushik (The University of Texas at Arlington) ; Guan, Wen (Department of Physics, University of Wisconsin) ; Maeno, Tadashi (Brookhaven National Laboratory (BNL)) ; Nilsson, Paul (Brookhaven National Laboratory (BNL)) ; Oleynik, Danila (Joint Institute for Nuclear Research) ; Panitkin, Sergey (Brookhaven National Laboratory (BNL)) ; Tsulaia, Vakhtang (Lawrence Berkeley National Laboratory and University of California, Berkeley) ; van Gemmeren, Peter (Argonne National Laboratory) ; Wenaus, Torre (Brookhaven National Laboratory (BNL))
High performance computing facilities present unique challenges and opportunities for HENP event processing. The massive scale of many HPC systems means that fractionally small utilizations can yield large returns in processing throughput. [...]
ATL-SOFT-SLIDE-2015-143.-
Geneva : CERN, 2015 - 12 p.
Fulltext: PDF; External link: Original Communication (restricted to ATLAS)
In : 21st International Conference on Computing in High Energy and Nuclear Physics, Okinawa, Japan, 13 - 17 Apr 2015
|
|
9.
|
|
|
Fine grained event processing on HPCs with the ATLAS Yoda system
/ Calafiura, Paolo (Lawrence Berkeley National Laboratory and University of California, Berkeley) ; De, Kaushik (The University of Texas at Arlington) ; Guan, Wen (Department of Physics, University of Wisconsin) ; Maeno, Tadashi (Brookhaven National Laboratory (BNL)) ; Nilsson, Paul (Brookhaven National Laboratory (BNL)) ; Oleynik, Danila (Joint Institute for Nuclear Research) ; Panitkin, Sergey (Brookhaven National Laboratory (BNL)) ; Tsulaia, Vakhtang (Lawrence Berkeley National Laboratory and University of California, Berkeley) ; van Gemmeren, Peter (Argonne National Laboratory) ; Wenaus, Torre (Brookhaven National Laboratory (BNL))
High performance computing facilities present unique challenges and opportunities for HENP event processing. [...]
ATL-SOFT-PROC-2015-004.
-
2015. - 5 p.
Original Communication (restricted to ATLAS) - Full text - IOP Open Access article
|
|
10.
|
|
|
Building and Using Containers on HPCs for the ATLAS Experiment
/ Yang, Wei (SLAC National Accelerator Laboratory) ; Benjamin, Douglas (Argonne National Laboratory) ; Childers, John Taylor (Argonne National Laboratory) ; Lesny, David (University of Illinois at Urbana-Champaign) ; Oleynik, Danila (Joint Institute for Nuclear Research) ; Panitkin, Sergey (Brookhaven National Laboratory (BNL)) ; Tsulaia, Vakhtang (Lawrence Berkeley National Laboratory and University of California, Berkeley) ; Zhao, Xin (Brookhaven National Laboratory (BNL))
The HPC environment presents several challenges to the ATLAS experiment in running their automated computational workflow smoothly and efficiently, in particular on issues such as software distribution and IO load. [...]
ATL-SOFT-PROC-2018-047.
-
2018.
Original Communication (restricted to ATLAS) - Full text
|
|