| Article | |
| Title | Hybrid analysis pipelines in the REANA reproducible analysis platform |
| Author(s) | Rodríguez, Diego (CERN) ; Mačiulaitis, Rokas (CERN) ; Okraska, Jan (CERN) ; Šimko, Tibor |
| Publication | 2020 |
| Number of pages | 7 |
| In: | EPJ Web Conf. 245 (2020) 06041 |
| In: | 24th International Conference on Computing in High Energy and Nuclear Physics, Adelaide, Australia, 4 - 8 Nov 2019, pp.06041 |
| DOI | 10.1051/epjconf/202024506041 |
| Subject category | Computing and Computers |
| Abstract | We introduce the feasibility of running hybrid analysis pipelines in the REANA reproducible analysis platform. The REANA platform allows researchers to specify declarative computational workflow steps describing the analysis process and to execute analysis workload on remote containerised compute clouds. We have designed an abstract job controller component permitting to execute different parts of the analysis workflow on different compute backends, such as HTCondor, Kubernetes and SLURM. We have prototyped the designed solution including the job execution, job monitoring, and input/output file staging mechanism between the various compute backends. We have tested the prototype using several particle physics model analyses. The present work introduces support for hybrid analysis workflows in the REANA reproducible analysis platform and paves the way towards studying underlying performance advantages and challenges associated with hybrid analysis patterns in complex particle physics data analyses. |
| Copyright/License | © 2020-2025 The Authors (License: CC-BY-4.0) |
Corresponding record in: Inspire
Record created 2021-02-16, last modified 2021-02-18
