CERN Accelerating science

Article
Report number	FERMILAB-CONF-24-0684-CSAID
Title	Boosting RDataFrame performance with transparent bulk event processing
Author(s)	Guiraud, Enrico (CERN) ; Blomer, Jakob (CERN) ; Canal, Philippe (Fermilab) ; Naumann, Axel (CERN)
Publication	2024
Number of pages	8
In:	EPJ Web Conf. 295 (2024) 06006
In:	26th International Conference on Computing in High Energy & Nuclear Physics, Norfolk, Virginia, Us, 8 - 12 May 2023, pp.06006
DOI	10.1051/epjconf/202429506006
Abstract	RDataFrame is ROOT’s high-level interface for Python and C++ data analysis. Since it first became available, RDataFrame adoption has grown steadily and it is now poised to be a major component of analysis software pipelines for LHC Run 3 and beyond. Thanks to its design inspired by declarative programming principles, RDataFrame enables the development of highperformance, highly parallel analyses without requiring expert knowledge of multi-threading and I/O: user logic is expressed in terms of self-contained, small computation kernels tied together by a high-level API. This design completely decouples analysis logic from its actual execution, and opens several interesting avenues for workflow optimization. In particular, in this work we explore the benefits of moving internal data processing from an event-by-event to a bulkby-bulk loop. This refactoring dramatically reduces the framework’s runtime overheads; in collaboration with the I/O layer it improves data access patterns; it exposes information that optimizing compilers might use to auto-vectorize the invocation of user-defined computations; finally, while existing user-facing interfaces remain unaffected, it becomes possible to additionally offer interfaces that explicitly expose bulks of events, useful e.g. for the injection of GPU kernels into the analysis workflow. In order to inform similar future R&D;, design challenges will be presented, as well as an investigation of the relevant timememory trade-off backed by novel performance benchmarks.
Copyright/License	publication: © 2024-2025 The authors CC-BY-4.0

Corresponding record in: Inspire