CERN Accelerating science

Article
Title	Multicore job scheduling in the Worldwide LHC Computing Grid
Author(s)	Forti, A (Manchester U.) ; Yzquierdo, A P (PIC, Bellaterra ; Madrid, CIEMAT) ; Hartmann, T (KIT, Karlsruhe, SCC) ; Alef, M (KIT, Karlsruhe, SCC) ; Lahiff, A (Rutherford) ; Templon, J (NIKHEF, Amsterdam) ; Pra, S Dal (INFN, CNAF) ; Gila, M (Zurich, ETH-CSCS/SCSC) ; Skipsey, S (Glasgow U.) ; Acosta-Silva, C (PIC, Bellaterra ; Barcelona, IFAE) ; Filipcic, A (Stefan Inst., Ljubljana) ; Walker, R (Munich U.) ; Walker, C J (Queen Mary, U. of London) ; Traynor, D (Queen Mary, U. of London) ; Gadrat, S (CC, Villeurbanne)
Publication	2015
Number of pages	8
In:	J. Phys.: Conf. Ser. 664 (2015) 062016
In:	21st International Conference on Computing in High Energy and Nuclear Physics, Okinawa, Japan, 13 - 17 Apr 2015, pp.062016
DOI	10.1088/1742-6596/664/6/062016
Subject category	Computing and Computers
Abstract	After the successful first run of the LHC, data taking is scheduled to restart in Summer 2015 with experimental conditions leading to increased data volumes and event complexity. In order to process the data generated in such scenario and exploit the multicore architectures of current CPUs, the LHC experiments have developed parallelized software for data reconstruction and simulation. However, a good fraction of their computing effort is still expected to be executed as single-core tasks. Therefore, jobs with diverse resources requirements will be distributed across the Worldwide LHC Computing Grid (WLCG), making workload scheduling a complex problem in itself. In response to this challenge, the WLCG Multicore Deployment Task Force has been created in order to coordinate the joint effort from experiments and WLCG sites. The main objective is to ensure the convergence of approaches from the different LHC Virtual Organizations (VOs) to make the best use of the shared resources in order to satisfy their new computing needs, minimizing any inefficiency originated from the scheduling mechanisms, and without imposing unnecessary complexities in the way sites manage their resources. This paper describes the activities and progress of the Task Force related to the aforementioned topics, including experiences from key sites on how to best use different batch system technologies, the evolution of workload submission tools by the experiments and the knowledge gained from scale tests of the different proposed job submission strategies.
Copyright/License	publication: © 2015-2024 The Author(s) (License: CC-BY-3.0)

Corresponding record in: Inspire