| ATLAS Note | |
| Report number | ATL-SOFT-PROC-2023-007 |
| Title | Updates to the ATLAS Data Carousel Project |
| Author(s) | Borodin, Misha (University of Iowa (US)) ; Cameron, David (University of Oslo (NO)) ; Klimentov, Alexei (Brookhaven National Laboratory (US)) ; Korchuganova, Tatiana (University of Pittsburgh (US)) ; Lassnig, Mario (CERN) ; Maeno, Tadashi (Brookhaven National Laboratory (US)) ; Musheghyan, Haykuhi (Georg August Universitaet Goettingen (DE)) ; South, David (Deutsches Elektronen-Synchrotron (DE)) ; Zhao, Xin (Brookhaven National Laboratory (US)) |
| Corporate Author(s) | The ATLAS collaboration |
| Collaboration | ATLAS Collaboration |
| Publication | 2024 |
| Imprint | 15 Aug 2023 |
| Number of pages | 6 |
| In: | EPJ Web Conf. 295 (2024) 01054 |
| In: | 26th International Conference on Computing in High Energy & Nuclear Physics, Norfolk, Virginia, Us, 8 - 12 May 2023 |
| DOI | 10.1051/epjconf/202429501054 |
| Subject category | Particle Physics - Experiment |
| Accelerator/Facility, Experiment | CERN LHC ; ATLAS |
| Free keywords | Data Carousel ; Tape |
| Abstract | The High Luminosity upgrade to the LHC (HL-LHC) is expected to deliver scientific data at the multi-exabyte scale. In order to address this unprecedented data storage challenge, the ATLAS experiment launched the Data Carousel project in 2018. Data Carousel is a tape-driven workflow whereby bulk production campaigns with input data resident on tape are executed by staging and promptly processing a sliding window to disk buffer such that only a small fraction of inputs are pinned on disk at any one time. Data Carousel is now in production for ATLAS in Run3. In this paper, we will provide updates on recent Data Carousel R&D projects, including data-on-demand and tape smart writing. Data-on-demand removes from disk data that has not been accessed for a predefined period, when users request them, they will be either staged from tape or recreated by following the original production steps. Tape smart writing employs intelligent algorithms for file placement on tape in order to retrieve data back more efficiently, which is our long term strategy to achieve optimal tape usage in Data Carousel. |
Corresponding record in: Inspire
Element opprettet 2023-08-15, sist endret 2024-10-30
PDF