CERN Accelerating science

ATLAS Slides
Report number ATL-SOFT-SLIDE-2022-001
Title intelligent Data Delivery Service (iDDS)
Author(s) Guan, Wen (University of Wisconsin Madison (US)) ; Maeno, Tadashi (Brookhaven National Laboratory (US)) ; Padolski, Siarhei (Brookhaven National Laboratory (US)) ; Bockelman, Brian Paul (University of Wisconsin Madison (US)) ; Wenaus, Torre (Brookhaven National Laboratory (US)) ; Barreiro Megino, Fernando Harald (University of Texas at Arlington (US)) ; Lin, Fa-Hui (University of Texas at Arlington (US)) ; Zhang, Rui (University of Wisconsin Madison (US))
Corporate author(s) The ATLAS collaboration
Collaboration ATLAS Collaboration
Submitted to 30th International Symposium on Lepton Photon Interactions at High Energies, Manchester, Gb, 10 - 14 Jan 2022
Submitted by wen.guan@cern.ch on 07 Jan 2022
Subject category Particle Physics - Experiment
Accelerator/Facility, Experiment CERN LHC ; ATLAS
Abstract The intelligent Data Delivery Service (iDDS) has been developed to cope with the huge increase of computing and storage resource usage in the coming LHC data taking. It has been designed to intelligently orchestrate workflow and data management systems, decoupling data pre-processing, delivery, and main processing in various workflows. It is an experiment-agnostic service that has been deployed to serve data carousel, hyperparameter optimization, multiple-steps DAG (Directed Acyclic Graphs) workflows and so on. Here we will present the motivation for iDDS, the architecture, use cases and current status for ATLAS and Rubin Observatory exercise, and plans for the future.



 Записът е създаден на 2022-01-07, последна промяна на 2022-01-07


Пълен текст:
Сваляне на пълен текстPDF
External link:
Сваляне на пълен текстOriginal Communication (restricted to ATLAS)
  • Send to ScienceWise.info