Table of contents

Empowering the Revolution: Bringing Machine Learning to High Performance Computing.

In affectionate memory of Denis Perret-Gallix (1949-2018)

ACAT 2019 was held from March 11 to 15, 2019 at the Steinmatte Conference Centre in Saas-Fee, the famous Swiss ski resort. This 19^th edition brought together experts to explore and confront the cutting-edge of computing, data analysis, and theoretical calculation techniques in fundamental physics research and beyond.

This workshop can be considered a landmark in the series, both because it was held at a dramatic moment in the history of computing and physics research, and for the quality and diversity of the conference contributions.

Most of the credit for this success goes to the vision and inspiration provided by the founder of the workshop series and chair of the International Advisory Committee from 1990 to 2018, Denis Perret-Gallix. Tragically, Denis unexpectedly passed away in June 2018, leaving us the great challenge of realising his vision for this workshop without his wise and informed guidance and warm friendship. These proceedings are dedicated to his memory.

The ACAT workshop series also wishes to acknowledge the generous contribution from Edmond Offermann, both as a sponsor and as an active and interested participant in all the sessions. His contributions to this workshop were instrumental to its overall success.

During the workshop many fundamental Machine Learning (ML) issues were addressed, such as: how can optimise ML techniques on high performance computing hardware (including GPUs, TPUs, FPGAs and, perhaps, Quantum Computing) to improve efficiency and accuracy? How does one extract scientific meaning from a ML analysis? How do we extract new scientific information from the internal weights of a neural net? Other issues treated included the estimation of systematic errors or biases introduced by the training stage; the risk of overfitting; as well as the difficult question of hyperparameter optimization.

The workshop also addressed the rapid development of Quantum Computing (QC), which not only could provide fast algorithms for HEP workflows, but also opens a new dimension in theoretical physics, the simulation of quantum systems. Proposed by R. Feynman in 1981, this is a solution to an intractable problem for classical computers.

The presentations addressed some of the questions about QC faced by physicists: Are today's QC suitable for solving lattice QCD calculations to a level that classical supercomputers cannot undertake? Can we use QC to solve real HEP problems?

Focusing on ML and QC does not mean that the "bread and butter" topics of the previous ACAT workshop were ignored. On the contrary, the boost coming from

these more recent topics, sparked new interest and novel ideas in the more traditional topics of the ACAT workshop series.

All papers published in this volume of Journal of Physics: Conference Series have been peer reviewed through processes administered by the proceedings Editors. Reviews were conducted by expert referees to the professional and scientific standards expected of a proceedings journal published by IOP Publishing.

The extremely low flux of ultra-high energy cosmic rays (UHECR) makes their direct observation by orbital experiments practically impossible. For this reason all current and planned UHECR experiments detect cosmic rays indirectly by observing the extensive air showers (EAS) initiated by cosmic ray particles in the atmosphere. Various types of shower observables are analyzed in the modern UHECR experiments including a secondary radio signal and fluorescent light from the excited nitrogen molecules. Most of the data is collected by the network of surface stations which allows to measure the lateral EAS profile. The raw observables in this case are the time-resolved signals for the set of adjacent triggered stations. The Monte Carlo shower simulation is performed in order to recover the primary particle properties. In traditional techniques the MC simulation is used to fit some synthetic observables such as the shower rise time, the shower front curvature and the particle density normalized to a given distance from the core. In this talk we'll consider an alternative approach based on the deep convolutional neural network trained on a large Monte-Carlo dataset, using the detector signal time series as an input. The above approach has proven its efficiency with the Monte-Carlo simulations of the Telescope Array Observatory surface detector. We'll discuss in detail how the network architecture is optimized for this particular task.

We report on multi-loop integral computations executed on a PEZY/Exascaler large-scale (immersion cooling) computing system. The programming model requires a host program written in C++ with a PZCL (OpenCL-like) kernel. However the kernel can be generated by the Goose compiler interface, which allows parallelizing program loops according to compiler directives. As an advantage, the executable derived from a program instrumented with Goose pragmas can be run on multiple devices and multiple nodes without changes to the program. We use lattice rules and lattice copy (composite) rules on PEZY to approximate integrals for multi-loop self-energy diagrams with and without masses. GPU results are also given and the performnce on the different architectures is compared.

Efficient phase space integration is important for some calculations in collider physics. Using the Altarelli-Parisi splitting functions as the underlying probability for a splitting, we developed a phase space calculation for initial state radiation, that relies on Monte Carlo integration.

The increase of computational resources with the generalization of massively parallel supercomputers benefits to various fields of physics among which turbulence and fluid mechanics, making it possible to increase time and space accuracy and gain further knowledge in fundamental mechanisms. Parametric studies, high fidelity statistics, high resolutions, can be realized. However, this access poses many problems in terms of data management, analysis and visualization. Traditional workflow, consisting of writing raw data on disks and performing post-processing to extract physical quantities of interest, considerably slows down the analysis, if not becomes impossible, because of data transfer, storage and re-accessibility issues. This is particularly difficult when it comes to visualization. Usage has to be revisited to maintain consistency with the accuracy of the computation step and in this context, in situ processing is a promising approach. We developed an in situ analysis and visualization strategy with an hybrid method for transitional and turbulent flow analysis with a pseudo-spectral solver. It is shown to have a low impact on computational time with a reasonable increase of resource usage, while enriching data exploration. Large time sequences have been analyzed. This could not have been achieved with the traditional workflow. Moreover, computational steering has been performed with real-time adjustment of the simulations, thereby getting closer to a numerical experiment process.

The probability density function for the visible sector of a Riemann-Theta Boltzmann machine can be taken conditional on a subset of the visible units. We derive that the corresponding conditional density function is given by a reparameterization of the Riemann-Theta Boltzmann machine modelling the original probability density function. Therefore the conditional densities can be directly inferred from the Riemann-Theta Boltzmann machine.

Since modern-day multi-loop Feynman diagram computations often require manipulating billions of terms, taking up terabytes of memory, a powerful symbolic manipulation toolkit (SMT) is essential. The de facto solution is Form, but it has several shortcomings. In this work we present reForm, a new SMT in early development, that will handle the same workload as Form but does not have its shortcomings. We showcase some features of reForm, including Python and C APIs. Finally, we provide benchmarks for polynomial GCD computations, which show that reForm often outperforms its competitors. A link to the source code of the technical preview version is provided.

Preliminary numerical results of calculating the 5-loop QED contributions to the electron (and muon) anomalous magnetic moment are presented. The results include the total contribution of the 5-loop Feynman diagrams without lepton loops and the contributions of nine gauge-invariant classes that form that set. A discrepancy with known results is revealed. The contributions of the gauge-invariant classes are presented for the first time. The method of the computation is briefly described (with the corresponding references). The calculation is based on the following elements:

(i)a subtraction procedure for removing infrared (IR) and ultraviolet (UV) divergences point by point in Feynman parametric representation before integration;

(ii)a nonadaptive Monte Carlo integration method that is founded on probability density functions (PDF) that are constructed for each Feynman diagram individually using its combinatorial structure;

(iii)a GPU-based numerical integration with the help of the supercomputer "Govorun" (JINR, Dubna, Russia).

This note gives an update on recent developments in FeynArts, FormCalc, and LoopTools, and shows how the new features were used in making the latest version of FeynHiggs.

The couplings of the Higgs boson to other particles are increasingly well measured by the ATLAS and CMS experiments. The Higgs boson trilinear self-coupling however is still largely unconstrained, mainly due to the low cross-section for Higgs boson pair production. We present inclusive and differential results for the NLO QCD corrections to Higgs boson pair production with the full top-quark mass dependence, where the Higgs trilinear coupling is varied to non-SM values. The fixed-order calculation is supplemented by parton showering within the Powheg-BOX-V2 event generator, and both Pythia8 and Herwig7 parton-shower algorithms are implemented in a preliminary study of shower effects.

In this talk we discuss some of the computational aspects of some recent computations of double Higgs production in gluon fusion. We consider the challenges encountered in computing the high-energy limit of the NLO virtual corrections and the large top quark mass limit of the NNLO virtual corrections.

Complete one-loop electroweak radiative corrections to polarized Bhabha scattering are presented. Numerical results are shown for the conditions of future circular and linear electron-positron colliders with polarized beams. A new Monte Carlo event generator for simulation of Bhabha scattering is created. Higher order QED collinear radiation factors are evaluated in the next-to-leading logarithmic approximation.

A new Monte Carlo event generator MCSANCee for simulations of processes at future e⁺e⁻ colliders is presented. Complete one-loop electroweak radiative corrections and polarization of the initial beams are taken into account. The present generator includes the following processes: ${e}^{+}{e}^{-}\to {e}^{+}{e}^{-},{\mu }^{+}{\mu }^{-},{\tau }^{+}{\tau }^{-},ZH$. Numerical results for the ${e}^{+}{e}^{-}\to ZH$ process are shown. Plans for the further extension of the MCSANCee generator are discussed.

Form is a symbolic manipulation system, which is especially advantageous for handling gigantic expressions with many small terms, as often occurs in real problems in perturbative quantum field theory. In this work we describe some main features of FORM, such as the preprocessor and $-variables with emphasizing on benefit of metaprogramming, and introduce a new feature: a topology generator.

In this contribution I will discuss the practicalities of storing events from a NNLO calculation on disk with the view of "replaying" the simulation for a diferent analysis and under diferent conditions, such as a diferent PDF ft or a diferent scale setting. I also present a way to store a compact representation of the matrix elements for low multiplicity processes.

We present STR (Star-Triangle Relations), a Mathematica package designed to solve Feynman integrals by means of the method of uniqueness in any Euclidean spacetime dimension. We provide a set of tools to draw Feynman diagrams and interact with them only by the use of the mouse. Throughout the use of a graphic interface, the package should be easily accessible to users with little or no previous experience on diagrams computation.

We report on an ongoing work initiated by Prof. Shimizu, proposing a method to numerically compute two-loop scalar integrals as sums of two-dimensional integrals of generalised one-loop N-point functions analytically computed and integrated over some simple weight functions. The analytic computation of the generalised one-loop N-point functions in a systematic way motivates a novel approach sketched in this talk.

We present the HepMC3 library designed to perform manipulations with event records of High Energy Physics Monte Carlo Event Generators (MCEGs). The library is a natural successor of HepMC and HepMC2 libraries used in the present and in the past. HepMC3 supports all functionality of previous versions and significantly extends them. In comparison to the previous versions, the default event record has been simplified, while an option to add arbitrary information to the event record has been implemented. Particles and vertices are stored separately in an ordered graph structure, reflecting the evolution of a physics event and enabling usage of sophisticated algorithms for event record analysis. The I/O functionality of the library has been extended to support common input and output formats of HEP MCEGs, including formats used in Fortran HEP MCEGs, formats used in HepMC2 library and ROOT. The functionality of the library allows the user to implement a customised input or output format. The library is already supported by popular modern MCEGs (e.g. Sherpa and Pythia8) and can replace the older HepMC versions in many others.

Higher order calculations in perturbative Quantum Field Theories often produce coupled linear systems of differential equations which factorize to first order. Here we present an algorithm to solve such systems in terms of iterated integrals over an alphabet the structure of which is implied by the coefficient matrix of the given system. We apply this method to calculate the master integrals in the color–planar and complete light quark contributions to the three-loop massive form factors.

A Monte Carlo generator to simulate events of single-photon annihilation to hadrons at center-of-mass energies below 2 GeV is described. The generator is based on existing data on cross sections of various exclusive channels of e⁺e⁻ annihilation obtained in scan and ISR experiments. It is extensively used in the software packages for analysis of experiments at the Novosibirsk e⁺e⁻ colliders VEPP-2000 and VEPP-4M aimed at high-precision measurements of hadronic cross sections with different applications, e.g. to calculations of the hadronic vacuum polarization for the muon anomalous magnetic moment.

Implementation of modern algorithms in computer algebra requires the use of generic and high-performance instruments. Rings is an open-source library, written in Java and Scala programming languages, which implements basic concepts and algorithms from computational commutative algebra while demonstrating quite a high performance among existing software. It rigorously uses generic programming approach, providing a well-designed generic API with a fully typed hierarchy of algebraic structures and algorithms for commutative algebra. Polynomial arithmetic, GCDs, factorization, and Gröbner bases are implemented with the use of modern asymptotically fast algorithms. The use of the Scala language brings a quite novel powerful, strongly typed functional programming model allowing to write short, expressive, and fast code for applications in high-energy physics and other research areas.

Decoding the nature of Dark Matter (DM) as a crucial part of Beyond-the-Standard-Model (BSM) theory is one of the most important problems of modern particle physics. DM potentially provides unique signatures at collider and non-collider experiments. These signatures are quite generic, however their details could allow us to delineate various BSM models and the properties of DM. While there are many comprehensive studies of the phenomenology of various appealing BSM models, exhibiting "top-bottom" approach, there is no clear strategy for the reverse task of identifying the underlying theory from the new signatures. To solve this problem one should consider the comprehensive set of signatures, database of models and use modern methods, including machine learning and artificial intelligence, to decode the underlying theory from potential signals of new physics we are expecting from the coming experimental data. One of the tools which could be helpful to solve the problem is High Energy Physics Model Database (HEPMDB) which was created to make a step forward towards solving this problem. It is aimed to facilitate connection between HEP theory and experiment, to store, validate and explore BSM models and to collect their signatures. DM decoding is based on the unique complementarity of Large Hadron Collider (LHC) potential as well as on the potential DM direct and indirect detection experiments to probe DM. The combination of our knowledge on this complementarity, modern analysis methods, comprehensive database of BSM models and their signatures is the key point of decoding the nature of DM and the whole underlying theory of Nature.

One major challenge for the legacy measurements at the LHC is that the likelihood function is not tractable when the collected data is high-dimensional and the detector response has to be modeled. We review how different analysis strategies solve this issue, including the traditional histogram approach used in most particle physics analyses, the Matrix Element Method, Optimal Observables, and modern techniques based on neural density estimation. We then discuss powerful new inference methods that use a combination of matrix element information and machine learning to accurately estimate the likelihood function. The MadMiner package automates all necessary data-processing steps. In first studies we find that these new techniques have the potential to substantially improve the sensitivity of the LHC legacy measurements.

The sophistication of fully exclusive MC event generation has grown at an extraordinary rate since the start of the LHC era, but has been mirrored by a similarly extraordinary rise in the CPU cost of state-of-the-art MC calculations. The reliance of experimental analyses on these calculations raises the disturbing spectre of MC computations being a leading limitation on the physics impact of the HL-LHC, with MC trends showing more signs of further cost-increases rather than the desired speed-ups. I review the methods and bottlenecks in MC computation, and areas where new computing architectures, machine-learning methods, and social structures may help to avert calamity.

The modern scientific data processing is not only a collection of powerful algorithms but also a whole infrastructure of facilities for data reading, processing, and results output. As the amount of data grows, so grows a need for automation of the process. The proper automation requires not only improvement of existing frameworks, but also a search of new ways to organize data processing in a way it could be automated and parallelized. DataForge experimental framework solves some problems by making the analysis configuration declarative instead of imperative. In this article, we present a limited demonstration of the idea applied to "Troitsk nu-mass" experiment in the search for sterile neutrino.

We present CutLang, an analysis description language and runtime interpreter for high energy collider physics data analyses. An analysis description language is a declerative domain specific language that can express all elements of a data analysis in an easy and unambiguous way. A full-fledged human readable analysis description language, incorporating logical and mathematical expressions, would eliminate many programming difficulties and errors, consequently allowing the scientist to focus on the goal, but not on the tool. In this paper, we discuss the guiding principles and scope of the CutLang language, implementation of the CutLang runtime interpreter and the CutLang framework, and demonstrate an example of top pair reconstruction.

ATLAS is one of the general purpose experiments observing hadron collisions at the LHC at CERN. Its trigger and data acquisition system (TDAQ) is responsible for selecting and transporting interesting physics events from the detector to permanent storage where the data are used for further processing. The transient storage of ATLAS TDAQ is the last component of the online system in the data flow. It records selected events at several GB/s to non-volatile storage before transfer to offline permanent storage. The transient storage is a distributed system consisting of high-performance direct-attached storage servers accounting for 480 hard drives. A distributed multi-threaded C++ application operates the hardware. The transient storage is also responsible for computing a checksum for the data, which is used to ensure data integrity of the transferred data. Reliability and efficiency of this system are critical for the operations of TDAQ as well. This paper presents the existing multi-threading strategy of the software and how the available hardware resources are used. We then introduce how multi-threaded checksum computation was introduced to increase significantly the maximum throughput of the system. We discuss the key concepts of the implementation with a focus on the importance of overhead minimization. Finally the paper reports on the tests done on the production system to demonstrate the validity of the implementation and measurements of the performance improvement in the view of future LHC and ATLAS upgrades.

The Trigger and Data Acquisition system of the ATLAS experiment at the Large Hadron Collider at CERN is composed of a large number of distributed hardware and software components which provide the data-taking functionality of the overall system. During data-taking, huge amounts of operational data are created in order to constantly monitor the system. The Persistent Back-End for the ATLAS Information System of TDAQ (P-BEAST) is a system based on a custom-built timeseries database. It is used to archive and retrieve any operational monitoring data for the applications requesting it. P-BEAST stores about 18 TB of highly compacted and compressed raw monitoring data per year. Since P-BEAST's creation, several promising database technologies for fast access to time-series have become available. InfluxDB and ClickHouse were the most promising candidates for improving the performance and functionality of the current implementation of P-BEAST. This paper presents a short description of main features of both technologies and a description of the tests ran on both database systems. Then, the results of the performance testing performed using a subset of archived ATLAS operational monitoring data are presented and compared.

Over the next few years, the LHC will prepare for the upcoming High-Luminosity upgrade in which it is expected to deliver ten times more pp collisions. This will create a harsher radiation environment and higher detector occupancy. In this context, the ATLAS experiment, one of the general purpose experiments at the LHC, plans substantial upgrades to the detectors and to the trigger system in order to efficiently select events. Similarly, the Data Acquisition System (DAQ) will have to redesign the data-flow architecture to accommodate for the large increase in event and data rates. The Phase-II DAQ design involves a large distributed storage system that buffers data read out from the detector, while a computing farm (Event Filter) analyzes and selects the most interesting events. This system will have to handle 5.2 TB/s of input data for an event rate of 1 MHz and provide access to 3 TB/s of these data to the filtering farm. A possible implementation for such a design is based on distributed file systems (DFS) which are becoming ubiquitous among the big data industry. Features of DFS such as replication strategies and smart placement policies match the distributed nature and the requirements of the new data-flow system. This paper presents an up-to-date performance evaluation of some of the DFS currently available: GlusterFS, HadoopFS and CephFS. After characterization of the future data-flow systems workload, we report on small-scale raw performance and scalability studies. Finally, we conclude on the suitability of such systems to the tight constraints expected for the ATLAS experiment in phase-II and, in general, what benefits the HEP community can take from these storage technologies.

Information concerning the operation, configuration and behavior of the ATLAS experiment needs to be reported, gathered and shared reliably with the whole ATLAS community which comprises over three thousand scientists geographically distributed all over the world. To provide such functionality, a logbook facility, Electronic Logbook for the information storage of ATLAS (ELisA), has been developed and actively used since the beginning of the LHC Run 2 period. The facility includes a user-friendly web interface to browse activity logs and to report on system operations with a configurable email notification system; a RESTful API used programmatically by other tools and services of the data acquisition infrastructure, and a set of client API libraries and utilities to help user's interaction with the REST API. Given its generic configuration capabilities, the ELisA facility has been recently deployed as a stand-alone logbook for other projects such as the commissioning of different sub-detectors and the offline assessment of data-quality. To ease this operation and to potentially extend ELisA usage to other projects, an extension of the database backend support is being implemented to reduce the dependency on the ORACLE database for the logbook deployment. Also, the deployment process of the logbook is being improved. This contribution will present the status of the logbook facility as well as the extensions and improvements implemented to ease the logbook portability to other projects.^*

We evaluate key patterns and estimate throughput bounds of simulated transformation of conventional high energy physics (HEP) data processing workflows to heterogeneous equivalents. The simulation parameter space includes the number of offloaded tasks, CPU/accelerator ratios of intra-task computations, offload latencies, and run time efficiency of offloaded computations. The simulation is performed for a diverse set of state-of-the-art event reconstruction scenarios from ATLAS, LHCb and CMS - the frontier HEP experiments of the Large Hadron Collider project.

Athena is the software framework used in the ATLAS experiment throughout the data processing path, from the software trigger system through offline event reconstruction to physics analysis. The shift from high-power single-core CPUs to multi-core systems in the computing market means that the throughput capabilities of the framework have become limited by the available memory per process. For Run 2 of the Large Hadron Collider (LHC), ATLAS has exploited a multi-process forking approach with the copy-on-write mechanism to reduce the memory use. To better match the increasing CPU core count and, therefore, the decreasing available memory per core, a multi-threaded framework, AthenaMT, has been designed and is now being implemented. The ATLAS High Level Trigger (HLT) system has been remodelled to fit the new framework and to rely on common solutions between online and offline software to a greater extent than in Run 2. We present the implementation of the new HLT system within the AthenaMT framework, which is going to be used in ATLAS data-taking during Run 3 (2021 onwards) of the LHC. We also report on interfacing the new framework to the current ATLAS Trigger and Data Acquisition (TDAQ) system, which aims to bring increased flexibility whilst needing minimal modifications to the current system. In addition, we show some details of architectural choices which were made to run the HLT selection inside the ATLAS online dataflow, such as the handling of the event loop, returning of the trigger decision and handling of errors.

JANA2 is a multi-threaded event reconstruction framework being developed for Experimental Nuclear Physics. It is an LDRD¹ funded project that will be the successor of the original JANA framework. JANA2 is a near complete rewrite emphasizing C++ language features that have only become available since the introduction of the C++11 standard. Features such as shared pointers, language native threading, and atomics are employed. This paper outlines the status of the project.

The ATLAS experiment at the LHC at CERN will move to use the Front-End Link eXchange (FELIX) system in a staged approach for LHC Run 3 (2021) and LHC Run 4 (2026). FELIX will act as the interface between the data acquisition; detector control and TTC (Timing, Trigger and Control) systems; and new or updated trigger and detector front-end electronics. FELIX functions as a router between custom serial links from front end ASICs and FPGAs to data collection and processing components via a commodity switched network. Links may aggregate many slower links or be a single high bandwidth link. FELIX also forwards the LHC bunch-crossing clock, fixed latency trigger accepts and resets received from the TTC system to front-end electronics. The FELIX system uses commodity server technology in combination with FPGA-based PCIe I/O cards. The FELIX servers run a software routing platform serving data to network clients. Commodity servers connected to FELIX systems via the same network run innovative multi-threaded software for event fragment building, processing, buffering and forwarding. This proceeding will describe the design and status of the FELIX based readout for the Run 3 upgrade, during which a subset of the detector will be migrated. It will also show how the same concept has been successfully introduced into the demonstrator test bench of the ATLAS Pixel Inner Tracker, acting as a proof of concept towards the longer term Run 4 upgrade in which all detectors will adopt a FELIX based readout.

The design and performance of the ATLAS Inner Detector (ID) trigger algorithms running online on the high level trigger (HLT) processor farm for 13 TeV LHC collision data with high pile-up are discussed. The HLT ID tracking is a vital component in all physics signatures in the ATLAS trigger for the precise selection of the rare or interesting events necessary for physics analysis without overwhelming the offline data storage in terms of both size and rate. To cope with the high expected interaction rates in the 13 TeV LHC collisions the ID trigger was redesigned during the 2013-15 long shutdown. The performance of the ID trigger in Run 2 from 13 TeV LHC collisions exceeded expectations as the pile-up increased throughout the run periods. The detailed efficiencies and resolutions of the trigger in a wide range of physics signatures spanning the entire Run 2 production luminosity data-taking are presented, to demonstrating that the trigger responded well under the extreme pile-up conditions. The performance of the ID trigger algorithms in ever higher pile-up collisions illustrates how the ID trigger continued to enable the ATLAS physics program and will continue to do so in the future.

In particle physics, workflow management systems are primarily used as tailored solutions in dedicated areas such as Monte Carlo event generation. However, physicists performing data analyses are usually required to steer their individual workflows manually, which is time-consuming and often leads to undocumented relations between particular workloads. We present the Luigi Analysis Workflows (Law) Python package, which is based on the open-source pipelining tool Luigi, originally developed by Spotify. It establishes a generic design pattern for analyses of arbitrary scale and complexity, and shifts the focus from executing to defining the analysis logic. Law provides the building blocks to seamlessly integrate interchangeable remote resources without, however, limiting itself to a specific choice of infrastructure. In particular, it encourages and enables the separation of analysis algorithms on the one hand, and run locations, storage locations, and software environments on the other hand. To cope with the sophisticated demands of end-to-end HEP analyses, Law supports job execution on WLCG infrastructure (ARC, gLite) as well as on local computing clusters (HTCondor, LSF), remote file access via most common protocols through the GFAL2 library, and an environment sandboxing mechanism with support for Docker and Singularity containers. Moreover, the novel approach ultimately aims for analysis preservation out-of-the-box. Law is entirely experiment independent and developed open-source.

The ATLAS experiment at the Large Hadron Collider (LHC) operated very successfully in the years 2008 to 2013, a period identified as Run 1. ATLAS achieved an overall data-taking efficiency of 94%, largely constrained by the irreducible dead-time introduced to accommodate the limitations of the detector read-out electronics. Out of the 6% dead-time only about 15% could be attributed to the central trigger and DAQ system, and out of these, a negligible fraction was due to the Control and Configuration sub-system. Despite these achievements, and in order to improve the efficiency of the whole DAQ system in Run 2 (2015-2018), the first long LHC shutdown (2013-2014) was used to carry out a complete revision of the control and configuration software. The goals were three-fold: properly accommodate additional requirements that could not be seamlessly included during steady operation of the system; re-factor software that had been repeatedly modified to include new features, thus becoming less maintainable; and seize the opportunity of modernizing software written even before Run 1, thus profiting from the rapid evolution in IT technologies. This upgrade was carried out retaining the important constraint of minimally impacting the mode of operation of the system and public APIs, in order to maximize the acceptance of the changes by the large user community. This paper presents, using a few selected examples, how the work was approached and which new technologies were introduced into the ATLAS DAQ system, and how they were performing in course of Run 2. Despite these being specific to this system, many solutions can be considered and adapted to different distributed DAQ systems.

The next LHC Runs, nominally Run III and Run IV, pose problems to the offline and computing systems in CMS. Run IV in particular will need completely different solutions, given the current estimates of LHC conditions and Trigger estimates. We report on the R&D process CMS has established, in order to gain insight on the needs and the possible solutions for the 2020+ CMS computing.

NanoAOD is an event data format that has recently been commissioned by the CMS Collaboration. It only includes high level physics object information and is about 20 times more compact than the MiniAOD format. NanoAOD can be easily customised for development activities and supports automated data analysis workflows. The current status and perspectives of NanoAOD design and implementation are reviewed.

During the last years we have carried out a renewal of the Building Management System (BMS) software of our data center with the aim of improving the data collection capability. Considering the complex physical distribution of the technical plants and the limits of the actual building hosting our center, a system that simply monitors and collects all the necessary information and provides alarms only in case of major failures has proven to be unsatisfactory. In 2017 we suffered a major flood due to one main water pipeline failure in the public street. After this disastrous event, clearly far beyond our control, we were however forced to reconsider completely the physical site robustness of our building in addition to the current monitoring and alarm system capabilities. It was clear that in some specific cases, alerts should be triggered hours or days before the actual main problem arises in order to allow efficient human intervention and proper escalation process. This paradigm could be easily applied to almost all the infrastructure components in our site, mainly the electric power distribution and continuity systems as well as the whole cooling devices. For this reason, in parallel to a consistent increase in the sensor widespread distribution of our BMS data collector system, a study of a predictive maintenance approach applicability to our site has been started. Predictive maintenance techniques aims at prevent unexpected infrastructure components failures or major events with the study of the whole monitoring data collection and the creation of appropriate statistical models with the help of big data analysis and machine learning techniques. An improvement in the Power Distribution Units (PDUs) monitoring in our site and the introduction of a dedicated network of water leak sensors were the first steps for increasing the data collection information at our disposal. With sufficient monitoring statistical information stored in our BMS system a preliminary and exploratory predictive data analysis proof of concept could be constructed. This could lead to the model building phase and the creation of a prototype with the aim of forecasting future infrastructure main failure events and forthcoming error conditions. The general idea is, conceivably, an approach to the predictive maintenance model where it would be possible to introduce scheduled corrective actions for the purpose of preventing potential failures in the next future and increasing the site overall reliability.

The increasing LHC luminosity in Run III and, consequently, the increased number of simultaneous proton-proton collisions (pile-up) pose significant challenges for the CMS experiment. These challenges will affect not only the data taking conditions, but also the data processing environment of CMS, which requires an improvement in the online triggering system to match the required detector performance. In order to mitigate the increasing collision rates and complexity of a single event, various approaches are being investigated. Heterogenous computing resources, recently becoming prominent and abundant, may be significantly better performing for certain types of workflows. In this work, we investigate implementations of common algorithms targeting heterogenous platforms, such as GPUs and FPGAs. The local reconstruction algorithms of the CMS calorimeters, given their granularity and intrinsic parallelizability, are among the first candidates considered for implementation in such heterogenous platforms. We will present the current development status and preliminary performance results. Challenges and various obstacles related to each platform, together with the integration into CMS experiments framework, will be further discussed.

RooFit [1,2] is the main statistical modeling and fitting package used to extract physical parameters from reduced particle collision data, e.g. the Higgs boson experiments at the LHC [3,4]. RooFit aims to separate particle physics model building and fitting (the users' goals) from their technical implementation and optimization in the back-end. In this paper, we outline our efforts to further optimize this back-end by automatically running parts of user models in parallel on multi-core machines. A major challenge is that RooFit allows users to define many different types of models, with different types of computational bottlenecks. Our automatic parallelization framework must then be flexible, while still reducing run-time by at least an order of magnitude, preferably more. We have performed extensive benchmarks and identified at least three bottlenecks that will benefit from parallelization. We designed a parallelization layer that allows us to parallelize existing classes with minimal effort, but with high performance and retaining as much of the existing class's interface as possible. The high-level parallelization model is a task-stealing approach. Preliminary results show speed-ups of factor 2 to 20, depending on the exact model and parallelization strategy.

As a data-intensive computing application, high-energy physics requires storage and computing for large amounts of data at the PB level. IHEP computing center is beginning to use tiered storage architectures, such as tape, disk or solid state drives to reduce hardware purchase costs and power consumption. At present, automatic data migration strategies are mainly used to resolve data migration between memory and disk. So the rules are relatively simple. This paper attempted to use the deep learning algorithm model to predict the evolution trend of data access heat as the basis for data migration. The implementation of some initial parts of the system were discussed, as well as the file trace collector and the LSTM model. At last some preliminary experiments are conducted with these parts.

The traditional partial wave analysis (PWA) algorithm is designed to process data serially which requires a large amount of memory that may exceed the memory capacity of one single node to store runtime data. It is quite necessary to parallelize this algorithm in a distributed data computing framework to improve its performance. Within an existing production-level Hadoop cluster, we implement PWA algorithm on top of Spark to process data storing on low-level storage system HDFS. But in this case, sharing data through HDFS or internal data communication mechanism of Spark is extremely inefficient. In order to solve this problem, this paper presents an in-memory parallel computing method for PWA algorithm. With this system, we can easily share runtime data in parallel algorithms. We can ensure complete data locality to keep compatibility with the traditional data input/output way and cache most repeatedly used data in memory to improve the performance, owe to the data management mechanism of Alluxio.

In the Run-3 of LHCb, the High Level Trigger will have to process events at full LHC collision rate (30 MHz). This is a very challenging goal, and delegating some low-level tasks to FPGA accelerators can be very helpful by saving precious computing time. In particular, the 2D pixel geometry of the new LHCb VELO detector makes the cluster-finding process particularly CPU-time demanding. We realized and tested a highly parallel FPGA-based clustering algorithm, capable of performing this reconstruction in real time at 30 MHz event rate using a modest amount of hardware resources, that can be a viable alternative solution.

Certifying the data recorded by the Compact Muon Solenoid (CMS) experiment at CERN is a crucial and demanding task as the data is used for publication of physics results. Anomalies caused by detector malfunctioning or sub-optimal data processing are difficult to enumerate a priori and occur rarely, making it difficult to use classical supervised classification. We base out prototype towards the automation of such procedure on a semi-supervised approach using deep autoencoders. We demonstrate the ability of the model to detect anomalies with high accuracy, when compared against the outcome of the fully supervised methods. We show that the model has great interpretability of the results, ascribing the origin of the problems in the data to a specific sub-detector or physics object. Finally, we address the issue of feature dependency on the LHC beam intensity.

Modern high-end FPGAs, as they are often used for hardware-level trigger applications, offer enough arithmetic performance to include artificial neural networks of considerable size into such systems. Yet, there are only very few examples of the inclusion of ANNs into high-performance hardware triggers, which is especially due to the complex and time-consuming development for FPGAs, and the need for an optimized design in order to make efficient use of the FPGA capabilities. We developed a library that provides three types of layers: Fully-connected dense layers, as well as 2D multi-channeled convolution and maximum pooling layers. For maximum design control, these were designed with VHDL and optimized for the specific data flow and control requirements of each layer type. By that, it was possible to obtain multiple hundred MHz processing frequency and have only little resource overhead beyond what is required for the actual computation for the individual layers. Furthermore, we created a Python-based toolkit that builds on these layer implementations to make it possible to take a trained network from the Keras framework and create the FPGA firmware and initialization data without requirement of in-depth understanding by the user. The resulting (deep) network designs can process data coming in at multiple ten MHz at multiple hundred MHz processing frequency and latencies ranging from tens to few hundreds of nanoseconds, depending on the network size.

FITS (Flexible Image Transport System) is a common format for astronomical data storage. [1]. Even though astronomical data is now processed mostly using software, visual data inspection by a human is still important during equipment or software commissioning and while observing. We present Fips [2, 3], a cross-platform FITS file viewer released as open source software¹. To the best of our knowledge, it is for the first time that the image rendering algorithms are implemented mostly on GPU (graphics processing unit). We show that it is possible to implement a fully-capable FITS viewer using OpenGL [4] interface, including movie support for representing 3D data. We also emphasise the advantages of using GPUs for efficient image handling.

Distinct HEP workflows have distinct I/O needs; while ROOT I/O excels at serializing complex C++ objects common to reconstruction, analysis workflows typically have simpler objects and can sustain higher event rates. To meet these workflows, we have developed a "bulk I/O" interface, allowing multiple events' data to be returned per library call. This reduces ROOT-related overheads and increases event rates – orders-of-magnitude improvements are shown in microbenchmarks.

Unfortunately, this bulk interface is difficult to use as it requires users to identify when it is applicable and they still "think" in terms of events, not arrays of data. We have integrated the bulk I/O interface into the new RDataFrame analysis framework inside ROOT. As RDataFrame's interface can provide improved type information, the framework itself can determine what data is readable via the bulk IO and automatically switch between interfaces. We demonstrate how this can improve event rates when reading analysis data formats, such as CMS's NanoAOD.

The LHC's Run3 will push the envelope on data-intensive workflows and, since at the lowest level this data is managed using the ROOT software framework, preparations for managing this data are starting already. At the beginning of LHC Run 1, all ROOT data was compressed with the ZLIB algorithm; since then, ROOT has added support for additional algorithms such as LZMA and LZ4, each with unique strengths. This work must continue as industry introduces new techniques - ROOT can benefit saving disk space or reducing the I/O and bandwidth for online and offline needs of experiments by introducing better compression algorithms. In addition to alternate algorithms, we have been exploring alternate techniques to improve parallelism and apply pre-conditioners to the serialized data.

We have performed a survey of the performance of the new compression techniques. Our survey includes various use cases of data compression of ROOT files provided by different LHC experiments. We also provide insight into solutions applied to resolve bottlenecks in compression algorithms, resulting in improved ROOT performance.

ROOT is a large code base with a complex set of build-time dependencies; there is a significant difference in compilation time between the "core" of ROOT and the full-fledged deployment. We present results on a "delayed build" for internal ROOT packages and external packages. This gives the ability to offer a "lightweight" core of ROOT, later extended by building additional modules to extend the functionality of ROOT. As a part of this work, we have improved the separation of ROOT code into distinct modules and packages with minimal dependencies. This approach gives users better flexibility and the possibility to combine various build features without rebuilding from scratch.

Dependency hell is a common problem found in software and particularly in HEP software ecosystem. We would like to discuss an improvement of artifact management ("lazy-install") system as a solution to the "dependency hell" problem.

HEP software stack usually consists of multiple sub-projects with dependencies. The development model is often distributed, independent and non-coherent among the sub-projects. We believe that software should be designed to take advantage of other software components that are already available, or have already been designed and implemented for use elsewhere rather than "reinventing the wheel".

The main idea is to build the ROOT project and all of its dependencies recursively and incrementally, making it fundamentally different than just adding one external project and rebuilding from scratch. In addition, this allows to keep a list of dependencies to be able to resolve possible incompatibility of transitive dependencies caused by the versions conict.

In our contribution, we will present our approach to artifact management system of ROOT together with a set of examples and use cases.

ROOT has several features which interact with libraries and require implicit header inclusion. This can be triggered by reading or writing data on disk, or user actions at the prompt. Often, the headers are immutable, and reparsing is redundant. C++ Modules are designed to minimize the reparsing of the same header content by providing an efficient on-disk representation of C++ Code. ROOT has released a C++ Modules-aware technology preview, which intends to become the default for the ROOT 6.20 release.

In this paper, we will summarize our experience with migrating C++ Modules to LHC experiment's software codebases, particularly for CMS software (CMSSW). We outline the challenges with adopting C++ Modules for CMSSW, including the integration of C++ Modules support in the CMS build system and we will evaluate the CMSSW performance benefits.

During Run3 of the LHC the LHCb detector will process a 30 MHz event rate with a full detector readout followed by a software trigger. To deal with the increased computational requirements, the software framework is reviewed and optimized on a large scale. One challenge is the efficient scheduling of O(10³)-O(10⁴) algorithms in the High Level Trigger (HLT) application. This document describes the design of a new algorithm scheduler which allows for static-order intra-event scheduling with minimum complexity while still providing the required flexibility.

The need for nested data structures and combinatorial operations on arbitrary length lists has prevented particle physicists from adopting array-based data analysis frameworks, such as R, MATLAB, Numpy, and Pandas. These array frameworks work well for purely rectangular tables and hypercubes, but arrays of variable length arrays, called "jagged arrays," are out of their scope. However, jagged arrays are a fundamental feature of particle physics data, as well as combining them to search for particle decays. To bridge this gap, we developed the awkward-array library, and in this paper we present feedback from some of the first physics groups using it for their analyses. They report similar computational performance to analysis code written in C++, but are split on the ease-of-use of array syntax. In a series of four phone interviews, all users noted how different array programming is from imperative programming, but whereas some found it easier in all aspects, others said it was more difficult to write, yet easier to read.

Efficient random number generation with high quality statistical properties and exact reproducibility of Monte Carlo simulations are important requirements in many areas of computational science. VecRNG is a package providing pseudo-random number generation (pRNG) in the context of a new library VecMath. This library bundles up several general-purpose mathematical utilities, data structures, and algorithms having both SIMD and SIMT (GPUs) support based on VecCore. Several state-of-the-art RNG algorithms are implemented as kernels supporting parallel generation of random numbers in scalar, vector, and Cuda workflows. In this report, we will present design considerations, implementation details, and computing performance of parallel pRNG engines on both CPU and GPU. Reproducibility of propagating multiple particles in parallel for HEP event simulation is demonstrated, using GeantV-based examples, for both sequential and fine-grain track-level concurrent simulation workflows. Strategies for efficient uses of vectorized pRNG and non-overlapping streams of random number sequences in concurrent computing environments is discussed as well.

The German CMS community (DCMS) as a whole can benefit from the various compute resources, available to its different institutes. While Grid-enabled and National Analysis Facility resources are usually shared within the community, local and recently enabled opportunistic resources like HPC centers and cloud resources are not. Furthermore, there is no shared submission infrastructure available.

Via HTCondor's [1] mechanisms to connect resource pools, several remote pools can be connected transparently to the users and therefore used more efficiently by a multitude of user groups. In addition to the statically provisioned resources, also dynamically allocated resources from external cloud providers as well as HPC centers can be integrated. However, the usage of such dynamically allocated resources gives rise to additional complexity. Constraints on access policies of the resources, as well as workflow necessities have to be taken care of. To maintain a well-defined and reliable runtime environment on each resource, virtualization and containerization technologies such as virtual machines, Docker, and Singularity, are used.

The ATLAS experiment has produced hundreds of petabytes of data and expects to have one order of magnitude more in the future. This data are spread among hundreds of computing Grid sites around the world. The EventIndex is the complete catalogue of all ATLAS events, real and simulated, keeping the references to all permanent files that contain a given event in any processing stage. It provides the means to select and access event data in the ATLAS distributed storage system, and provides support for completeness and consistency checks and trigger and offline selection overlap studies. The EventIndex employs various data handling technologies like Hadoop and Oracle databases, and it is integrated with other parts of the ATLAS distributed computing infrastructure, including systems for data, metadata, and production management. The project has been in operation since the start of LHC Run 2 in 2015, and it is in permanent development in order to satisfy the production and analysis demands and follow technology evolution. The main data store in Hadoop, based on MapFiles and HBase, has worked well during Run 2 but new solutions are being explored for the future. This paper reports on the current system performance and on the studies of a new data storage prototype that can carry the EventIndex through Run 3.

DODAS stands for Dynamic On Demand Analysis Service and is a Platform as a Service toolkit built around several EOSC-hub services designed to instantiate and configure on-demand container-based clusters over public or private Cloud resources. It automates the whole workflow from service provisioning to the configuration and setup of software applications. Therefore, such a solution allows using "any cloud provider", with almost zero effort. In this paper, we demonstrate how DODAS can be adopted as a deployment manager to set up and manage the compute resources and services required to develop an AI solution for smart data caching. The smart caching layer may reduce the operational cost and increase flexibility with respect to regular centrally managed storage of the current CMS computing model. The cache space should be dynamically populated with the most requested data. In addition, clustering such caching systems will allow to operate them as a Content Delivery System between data providers and end-users. Moreover, a geographically distributed caching layer will be functional also to a data-lake based model, where many satellite computing centers might appear and disappear dynamically. In this context, our strategy is to develop a flexible and automated AI environment for smart management of the content of such clustered cache system. In this contribution, we will describe the identified computational phases required for the AI environment implementation, as well as the related DODAS integration. Therefore we will start with the overview of the architecture for the pre-processing step, based on Spark, which has the role to prepare data for a Machine Learning technique. A focus will be given on the automation implemented through DODAS. Then, we will show how to train an AI-based smart cache and how we implemented a training facility managed through DODAS. Finally, we provide an overview of the inference system, based on the CMS-TensorFlow as a Service and also deployed as a DODAS service.

Linux containers have gained widespread use in high energy physics, be it for services using container engines such as containerd/kubernetes, for production jobs using container engines such as Singularity or Shifter, or for development workflows using Docker as a local container engine. Thus the efficient distribution of the container images, whose size usually ranges from a few hundred megabytes to a few tens of gigabytes, is becoming a pressing concern. Because container images show similar characteristics than scientific application stacks, unpacking the images in CernVM-FS can remedy the distribution issues provided that the container engine at hand is able to use such unpacked images from CernVM-FS. In this contribution, we willl report on recent advances in the integration of Singularity, Docker, and containerd with CernVM-FS. We show improvements in the publishing of container images from a Docker registry that rely on new means of directly ingesting image tarballs. Well also show a repository file system structure for storing container images that are optimized for storing both container engines using flat root file systems (Singularity) as well as container engines using layers (containerd, Docker). To evaluate the benefits of our approach, we show concrete use cases and figures for production and development images from LHC experiments stored in the recently created unpacked.cern.ch repository.

In recent years the usage of machine learning techniques within data-intensive sciences in general and high-energy physics in particular has rapidly increased, in part due to the availability of large datasets on which such algorithms can be trained, as well as suitable hardware, such as graphic or tensor processing units, which greatly accelerate the training and execution of such algorithms. Within the HEP domain, the development of these techniques has so far relied on resources external to the primary computing infrastructure of the WLCG (Worldwide LHC Computing Grid). In this paper we present an integration of hardware-accelerated workloads into the Grid through the declaration of dedicated queues with access to hardware accelerators and the use of Linux container images holding a modern data science software stack. A frequent use-case in the development of machine learning algorithms is the optimization of neural networks through the tuning of their Hyper Parameters (HP). For this often a large range of network variations must be trained and compared, which for some optimization schemes can be performed in parallel – a workload well suited for Grid computing. An example of such a hyper-parameter scan on Grid resources for the case of flavor tagging within ATLAS is presented.

A Tier-3g Facility within the computing resources of Istanbul Aydin University has been planned and installed with the TR-ULAKBIM national Tier-2 center. The facility is intended to provide an upgraded data analysis infrastructure to CERN researchers who are the members in the recent nation-wide projects and international projects such as ATLAS and CMS experiments. The fundamental design of Tier-3g has been detailed in this work with an emphasis on technical implementations of the following parts: Virtualization of all nodes, VOMS usage for reaching fast experimental data in the WLCG network, batch cluster / multicore computing with HTCONDOR and PROOF systems, usage of grid proxies to access code libraries in AFS and CVMFS, dynamic disk space allocation and remote system mounting of EOS. We also present the interpretation of test results that was obtained by the simulation of typical analysis codes.

In the context of ROOT7, the graphics system is completely redefined. Based on client server architecture and with the use of modern C++ and JavaScript, ROOT7 provides a new web based graphics system. The new concepts of ROOT7 can be displayed directly in the browsers using the new classes for opening a new web window, communicate with the server and exchange data between front and back end and JavaScript ROOT (JSROOT).

This paper describes our experience using and extending JupyterLab. We started with a copy of CERN SWAN environment, but now our project evolves independently. A major difference is that we switched from classic Jupyter Notebook to JupyterLab, because our users are more insterested in text editor plus terminal workflow rather than in Notebook workflow. However, like in SWAN, we are still using CVMFS to load Jupyter kernels and other software packages, and EOS to store user home directories.

Data Acquisition (DAQ) and Data Quality Monitoring (DQM) are key parts in the HEP data chain, where the data are processed and analyzed to obtain accurate monitoring quality indicators. Such stages are complex, including an intense processing work-flow and requiring a high degree of interoperability between software and hardware facilities. Data recorded by DAQ sensors and devices are sampled to perform live (and offline) DQM of the status of the detector during data collection providing to the system and scientists the ability to identify problems with extremely low latency, minimizing the amount of data that would otherwise be unsuitable for physical analysis. DQM stage performs a large set of operations (Fast Fourier Transform (FFT), clustering, classification algorithms, Region of Interest, particles tracking, etc.) involving the use of computing resources and time, depending on the number of events of the experiment, sampling data, complexity of the tasks or the quality performance. The objective of our work is to show a proposal with aim of developing a general optimization of the DQM stage considering all these elements. Techniques based on computational intelligence like EA can help improve the performance and therefore achieve an optimization of task scheduling in DQM.

Deep Learning techniques are being studied for different applications by the HEP community: in this talk, we discuss the case of detector simulation. The need for simulated events, expected in the future for LHC experiments and their High Luminosity upgrades, is increasing dramatically and requires new fast simulation solutions. Here we present updated results on the development of 3DGAN, one of the first examples using three-dimensional convolutional Generative Adversarial Networks to simulate high granularity electromagnetic calorimeters. In particular, we report on two main aspects: results on the simulation of a more general, realistic physics use case and on data parallel strategies to distribute the training process across multiple nodes on public cloud resources.

Data-intensive end-user analyses in high energy physics require high data throughput to reach short turnaround cycles. This leads to enormous challenges for storage and network infrastructure, especially when facing the tremendously increasing amount of data to be processed during High-Luminosity LHC runs. Including opportunistic resources with volatile storage systems into the traditional HEP computing facilities makes this situation more complex.

Bringing data close to the computing units is a promising approach to solve throughput limitations and improve the overall performance. We focus on coordinated distributed caching by coordinating workows to the most suitable hosts in terms of cached files. This allows optimizing overall processing efficiency of data-intensive workows and efficiently use limited cache volume by reducing replication of data on distributed caches.

We developed a NaviX coordination service at KIT that realizes coordinated distributed caching using XRootD cache proxy server infrastructure and HTCondor batch system. In this paper, we present the experience gained in operating coordinated distributed caches on cloud and HPC resources. Furthermore, we show benchmarks of a dedicated high throughput cluster, the Throughput-Optimized Analysis-System (TOpAS), which is based on the above-mentioned concept.

Continuous Integration (CI) and Continuous Development (CD) are common techniques in software development. Continuous Integration is the practice of bringing together code from multiple developers into a single repository, while Continuous Development is the process by which new releases are automatically created and tested. CI/CD pipelines are available in popular automation tools such as GitLab, and act to enhance and accelerate the software development process. Continuous Deployment, in which automation is employed to push new software releases into the production environment, follows naturally from CI/CD, but is not as well established due to business and legal requirements. Such requirements do not exist in the Worldwide LHC Compute Gird (WLCG), making the use of continuous deployment to simplify the management of grid resources an attractive proposition. We have developed work presented previously on containerised worker node environments by introducing continuous deployment techniques and tooling, and show how these, in conjunction with CI/CD, can reduce the management burden at a WLCG Tier-2 resource. In particular, benefits include reduced downtime as a result of code changes and middleware updates.

The current experiments in high energy physics (HEP) have a huge data rate. To convert the measured data, an enormous number of computing resources is needed and will further increase with upgraded and newer experiments. To fulfill the ever-growing demand the allocation of additional, potentially only temporary available non-HEP dedicated resources is important. These so-called opportunistic resources cannot only be used for analyses in general but are also well-suited to cover the typical unpredictable peak demands for computing resources. For both use cases, the temporary availability of the opportunistic resources requires a dynamic allocation, integration, and management, while their heterogeneity requires optimization to maintain high resource utilization by allocating best matching resources. To find the best matching resources which should be allocated is challenging due to the unpredictable submission behavior as well as an ever-changing mixture of workflows with different requirements.

Instead of predicting the best matching resource, we base our decisions on the utilization of resources. For this reason, we are developing the resource manager TARDIS (Transparent Adaptive Resource Dynamic Integration System) which manages and dynamically requests or releases resources. The decision of how many resources TARDIS has to request is implemented in COBalD (COBald - The Opportunistic Balancing Daemon) to ensure further allocation of well-used resources while reducing the amount of insufficiently used ones. TARDIS allocates and manages resources from various resource providers such as HPC centers or commercial and public clouds while ensuring a dynamic allocation and efficient utilization of these heterogeneous opportunistic resources.

Furthermore, TARDIS integrates the allocated opportunistic resources into one overlay batch system which provides a single point of entry for all users. In order to provide the dedicated HEP software environment, we use virtualization and container technologies.

In this contribution, we give an overview of the dynamic integration of opportunistic resources via TARDIS/COBalD in our HEP institute as well as how user analyses benefit from these additional resources.

The Solenoidal Tracker at RHIC (STAR) is a multi-national supported experiment located at Brookhaven National Lab. The raw physics data captured from the detector is on the order of tens of PBytes per data acquisition campaign, which makes STAR fit well within the definition of a big data science experiment. The production of the data has typically run on standard nodes or on standard Grid computing environments. All embedding simulations (complex workflow mixing real and simulated events) have been run on standard Linux resources at the National Energy Research Scientific Computing Center (NERSC) aka PDSF. However, HPC resources such as Cori have become available for STAR's data production as well as embedding, and STAR has been the very first experiment to show feasibility of running a sustainable data production campaign on this computing resource. The use of Docker containers with Shifter is required to run on HPC @ NERSC – this approach encapsulates the environment in which a standard STAR workflow runs. From the deployment of a tailored Scientific Linux environment (requiring many of its own libraries and special configurations required to run) to the deployment of third-party software and the STAR specific software stack, it has become impractical to rely on a set of containers containing each specific software release. To this extent, solutions based on the CERN VM File System (CVMFS) for the deployment of software and services have been employed in HENP, but one needs to make careful scalability considerations when using a resource like Cori, such as not allowing all software to be deployed in containers or bare node. Additionally, CVMFS clients are not compatible on Cori nodes and one needs to rely on an indirect NFS/DVS mount scheme. In our contribution, we will discuss our strategies from the past and our current solution based on CVMFS. Furthermore, running on HPC is not a simple task as each aspect of the workflow must be enabled to scale, run efficiently, and the workflow needs to fit within the boundaries of the provided queue system (SLURM in this case). Lastly, we will also discuss what we have learned so far about what is the best method for grouping jobs to maximize a single 48 core HPC node within a specific time frame and maximize our workflow efficiency.

The SND detector has been operating at the VEPP-2000 collider (BINP, Russia) for several years. We present its new DQM system.

The system deals with multiple parameters being numbers, histograms or user opinions to form a data quality decision (good, bad, etc). It calculates SND subsystem and overall statuses and produces summaries available via web-interface. The system supports multiple parameter sets for different usages like daily checks, precise subsystem control, preparing data for processing, etc. The parameter values are analyzed using special ROOT macros yielding quality checks that could be supplemented or corrected by users.

The system is a part of a new SND information system implemented as a web application using Node.js and MySQL. The ROOT macros are written by SND subsystems experts using a simple framework. Histograms are displayed using JSROOT.

The system is put into production. It is consistent, however there are some improvements to implement.

The German-Russian Astroparticle Data Life Cycle Initiative (GRADLCI) aims to develop a data life cycle (DLC), namely a clearly defined and maximally automated data processing pipeline for a combined analysis of data from the experiment KASCADE-Grande (Karlsruhe, Germany) and experiments installed at the Tunka Valley in Russia (TAIGA). The important features of such an astroparticle DLC include scalability for handling large amounts of data, heterogeneous data integration, and exploiting parallel and distributed computing at every possible stage of the data processing. In this work we provide an overview of the technical challenges and solutions worked out so far by the GRADLCI group in the framework of a far-reaching analysis and data center. We will touch the peculiarities of data management in astroparticle physics and employing distributed computing for simulations and physics analyses in this field.

Storage has been identified as the main challenge for the future distributed computing infrastructures: Particle Physics (HL-LHC, DUNE, Belle-II), Astrophysics and Cosmology (SKA, LSST). In particular, the High Luminosity LHC (HL-LHC) will begin operations in the year of 2026 with expected data volumes to increase by at least an order of magnitude as compared with the present systems. Extrapolating from existing trends in disk and tape pricing, and assuming flat infrastructure budgets, the implications for data handling for end-user analysis are significant. HENP experiments need to manage data across a variety of mediums based on the types of data and its uses: from tapes (cold storage) to disks and solid state drives (hot storage) to caches (including world wide access data in clouds and "data lakes"). The DataLake R&D project aims at exploring an evolution of distributed storage while bearing in mind very high demands of the HL-LHC era. Its primary objective is to optimize hardware usage and operational costs of a storage system deployed across distributed centers connected by fat networks and operated as a single service. Such storage would host a large fraction of the data and optimize the cost, eliminating inefficiencies due to fragmentation. In this talk we will highlight current status of the project, its achievements, interconnection with other research activities in this field like WLCG-DOMA and ATLAS-Google DataOcean, and future plans.

Maintaining the huge grid computing facilities for LHC experiments and replacing their hardware every few years has been very expensive. The California State University (CSU) ATLAS group recently received $250,000 AWS cloud credit from the CSU Chancellor's Office to build the first virtual US ATLAS Tier 3 to explore cloud solutions for ATLAS. We will use this award to set up full ATLAS computing environments on the cloud for ATLAS physics analysis frame works, MC generation, simulation and production. We will also develop policies for ATLAS members to submit jobs to the cloud and develop an economic model focused especially on the cost effectiveness of cloud solutions for ATLAS through extensive real user experience. The results will help ATLAS computing and physics communities decide future directions with incoming LHC upgrades.

HEPSPEC-06(HS06) is a decade old suite used to benchmark CPU resources for WLCG. Its adoption spans from hardware vendors, to site managers, funding agencies and software experts. Although it is stable, reproducible and accurate, it is reaching the end of its life. Initial hints of lack of correlations with HEP applications have been collected. Looking for suitable alternatives the HEPiX Benchmarking Working Group has evaluated SPEC CPU 2017 with a number of fast benchmarks. The studies that have been done so far do not show any major advantage in adopting SPEC CPU 2017 with respect to HS06.

A suite based on workloads that HEP experiments run can be an alternative to industrial standard benchmarks. The adoption by LHC experiments of modern software development techniques simplifies the ability to package, distribute and maintain a field-specific benchmark suite. The HEPiX Benchmarking Working Group is actively working to make this possible.

This report summarises the progress of the HEPiX Benchmarking Working Group in building a benchmarking suite based on HEP workloads. Comparisons of results with SPEC CPU 2017 and HS06 will be discussed.

Denis Perret-Gallix was the founder of the AIHENP-ACAT workshop series in 1990 and has chaired its international advisory committee until June 2018. He passed away on June 28 during one of the mountain bike climbs he used to enjoy. He was a high energy experimental physicist affiliated to IN2P3-CNRS (research director). He worked at Rutherford Lab., SLAC (DELCO experiment), CERN (CHARM and L3 experiments) and KEK developing low temperature dark matter detectors and event generators for the collider (LEP, LHC, ILC) physics simulations. He was the director of the CNRS Tokyo office from 2000 to 2004 and played a leading role in the France-Japan Particle Physics Laboratory which was established in 2006.

Any attempt to predict any future subject includes in general two components: extrapolations of the current trends and discussions about possible surprises. The extrapolation of the trends based on the latest function derivatives seems to be the easiest one. However, one has to be careful with fashionable trends and associated hype peaks. When one looks at the HEP software in the past 40 years, one sees regions of stability, often coinciding with the development and exploitation of a large accelerator, but also big changes when the hardware or/and software technology permits large steps. These large steps are often associated with surprises unthinkable a few years before. These two observations will likely continue to be true for the coming few decades, if not more. This paper is an attempt to analyze the current function derivatives based on similar analysis many years ago, and also propose some directions and developments that may become key components for our future software.

The ATLAS experiment implemented an ensemble of neural networks (Neural-Ringer algorithm) dedicated to improving the performance of event filters selecting electrons in the high-input-rate online environment of the Large Hadron Collider (LHC) at CERN. This algorithm has been used online since 2017 to select electrons with transverse energies (E_T) above 15 GeV. By taking advantage of calorimetry knowledge, the ensemble employs ring energy sums concentric to the electron candidate energy barycenter. The training procedure and final structure of the ensemble are designed to keep detector response flat with respect to particle energy and position. A detailed study was carried out to assess possible profile distortions in crucial offline quantities through the usage of statistical tests and residual analysis. NeuralRinger operation maintained high electron efficiency while improving fake rejection by a factor of 2 to 3, with negligible residuals in the offline quantities.

The need for large scale and high fidelity simulated samples for the ATLAS experiment motivates the development of new simulation techniques. Building on the recent success of deep learning algorithms at interpolation as well as image generation, Variational Auto-Encoders and Generative Adversarial Networks are investigated for modeling the response of the electromagnetic calorimeter for photons in a central calorimeter region over a range of energies. The synthesized showers are compared to showers from a full detector simulation using Geant4. This study demonstrates the potential of using such algorithms for fast calorimeter simulation for the ATLAS experiment in the future.

In the High–Luminosity Large Hadron Collider (HL–LHC), one of the most challenging computational problems is expected to be finding and fitting charged-particle tracks during event reconstruction. The methods currently in use at the LHC are based on the Kalman filter. Such methods have shown to be robust and to provide good physics performance, both in the trigger and offline. In order to improve computational performance, we explored Kalman-filter-based methods for track finding and fitting, adapted for many-core SIMD (single instruction, multiple data) and SIMT (single instruction, multiple thread) architectures. Our adapted Kalman-filter-based software has obtained significant parallel speedups using such processors, e.g., Intel Xeon Phi, Intel Xeon SP (Scalable Processors) and (to a limited degree) NVIDIA GPUs. Recently, an effort has started towards the integration of our software into the CMS software framework, in view of its exploitation for the Run III of the LHC. Prior reports have shown that our software allows in fact for some significant improvements over the existing framework in terms of computational performance with comparable physics performance, even when applied to realistic detector configurations and event complexity. Here, we demonstrate that in such conditions physics performance can be further improved with respect to our prior reports, while retaining the improvements in computational performance, by making use of the knowledge of the detector and its geometry.

During the LHC Run 3, the instantaneous luminosity received by LHCb will be increased going from 1.1 to 5.6 expected visible Primary Vertices (PVs) per event. To face this challenge, the LHCb detector will be upgraded and will adopt a purely software trigger. This has fueled increased interest in alternative highly-parallel and GPU friendly algorithms for tracking and reconstruction. We will present a novel prototype algorithm for vertexing in the LHCb upgrade conditions. We use a custom kernel to transform the sparse 3D space of hits and tracks into a dense 1D dataset, and then apply Deep Learning techniques to find PV locations. By training networks on our kernels using several Convolutional Neural Network layers, we have achieved better than 90% efficiency with no more than 0.2 False Positives (FPs) per event. Beyond its physics performance, this algorithm also provides a rich collection of possibilities for visualization and study of 1D convolutional networks. We will discuss the design, performance, and future potential areas of improvement and study, such as possible ways to recover the full 3D vertex information.

Track finding and fitting are among the most complex parts of event reconstruction in high-energy physics, and usually dominate the computing time in a high luminosity environment. A central part of track reconstruction is the transport of a given track parametrisation (i.e. the parameter estimation and associated covariance matrices) through the detector, respecting the magnetic field setup and the traversed detector material. While track propagation in a sparse environment (e.g. tracking detector with layers) can be sufficiently well approximated by considering discrete interactions at several positions, the propagation in a material dense environment (e.g. calorimeters) is better served by a continuous application of material effects. Recently, a common tracking software project (Acts), originally from the Common Tracking code of the ATLAS experiment, has been developed in order to preserve the algorithmic concepts from the LHC start-up era and prepare them for the high luminosity era of the LHC and beyond. The software is designed in an abstract, detector independent way and prepared to allow highly parallelised execution of all involved software modules, including magnetic field access and alignment conditions. Therefore the propagation algorithm needs to be both flexible and adjustable. The implemented solution using a fourth order Runge-Kutta-Nyström integration and its extension with continuous material integration and eventual time propagation is presented and the navigation through different geometry setups involving different environments is demonstrated.

We investigate how a Generative Adversarial Network could be used to generate a list of particle four-momenta from LHC proton collisions, allowing one to define a generative model that could abstract from the irregularities of typical detector geometries. As an example of application, we show how such an architecture could be used as a generator of LHC parasitic collisions (pileup). We present two approaches to generate the events: unconditional generator and generator conditioned on missing transverse energy. We assess generation performances in a realistic LHC data-analysis environment, with a pileup mitigation algorithm applied.

Due to the large size of datasets accumulated at the LHC, analysis results are often limited by systematic effects. The application of multivariate analysis techniques such as Boosted Decision Trees (BDTs) or artificial neural nets typically maximises the statistical significance of the results while ignoring systematic effects. There is a known strategy to mitigate systematic effects for neural nets but no firmly established procedure for BDTs. We present a method to incorporate systematic uncertainties into a BDT, the systematics-aware BDT (saBDT). We evaluate our method on open data of the ATLAS Higgs machine learning challenge and compare our results to neural nets trained with an adversary.

As more and more data is accumulated from BESIII detector, the reduction of the CPU time of data processing procedures is significant for the experiment to get physics results efficiently with limited hardware resources. In this study, we designed a tag-based analysis method in BESIII offline software system. Typical physics analysis results show the tag-based analysis with reformed DST file can reduce the jobs running time to about 1/10, with high CPU efficiency and low read throughput, without inducing additional disk space occupation.

We compared convolutional neural networks to the classical boosted decision trees for the separation of atmospheric particle showers generated by gamma rays from the particle-induced background. We conduct the comparison of the two techniques applied to simulated observation data from the Cherenkov Telescope Array. We then looked at the Receiver Operating Characteristics (ROC) curves produced by the two approaches and discuss the similarities and differences between both. We found that neural networks overperformed classical techniques under specific conditions.

Astrophysical observations have demonstrated the existence of dark matter over the past decades. Experimental efforts in the search for dark matter are largely focused on the well-motivated weakly interacting massive particles (WIMPs) as a dark matter candidate. Current experiments in direct detection are producing increasingly competitive limits on the cross section of WIMP-nucleon scattering. The main experimental challenge for all direct detection experiments is the presence of background signals. These backgrounds need to be either eliminated by providing sufficient shielding or discriminated from WIMP signals. In this work, semi-supervised learning techniques are developed to discriminate alpha recoils from nuclear recoils induced by WIMPs in the PICO-60 detector. The two semi-supervised learning techniques, gravitational differentiation and iterative cluster nucleation, maximize the effect of the most confidently predicted data samples on subsequent training iterations. Classifications using both techniques can reproduce the traditional acoustic parameter with accuracies over 98%. The best model yields an accuracy of 99.2% and a class-wise standard deviation value of 0.11. These techniques can reliably serve as an intermediate verification tool before the acoustic parameter is constructed in future detectors.

The ATLAS experiment at the Large Hadron Collider has a complex heterogeneous distributed computing infrastructure, which is used to process and analyse exabytes of data. Metadata are collected and stored at all stages of data processing and physics analysis. All metadata could be divided into operational metadata to be used for the quasi on-line monitoring, and archival to study the behaviour of corresponding systems over a given period of time (i.e. long-term data analysis). Ensuring the stability and efficiency of complex and large-scale systems, such as those in the ATLAS Computing, requires sophisticated monitoring tools, and the long-term monitoring data analysis becomes as important as the monitoring itself. Archival metadata, which contains a lot of metrics (hardware and software environment descriptions, network states, application parameters, errors) accumulated for more than a decade, can be successfully processed by various machine learning (ML) algorithms for classification, clustering and dimensionality reduction. However, the ML data analysis, despite the massive use, is not without shortcomings: the underlying algorithms are usually treated as "black boxes", as there are no effective techniques for understanding their internal mechanisms. As a result, the data analysis suffers from the lack of human supervision. Moreover, sometimes the conclusions made by algorithms may not be making sense with regard to the real data model. In this work we will demonstrate how the interactive data visualization can be applied to extend the routine ML data analysis methods. Visualization allows an active use of human spatial thinking to identify new tendencies and patterns found in the collected data, avoiding the necessity of struggling with the instrumental analytics tools. The architecture and the corresponding prototype of Interactive Visual Explorer (InVEx) - visual analytics toolkit for the multidimensional data analysis of ATLAS computing metadata will be presented. The web-application part of the prototype provides an interactive visual clusterization of ATLAS computing jobs, search for computing jobs non-trivial behaviour and its possible reasons.

We introduce a novel approach for the reconstruction of particle properties for the SHiP detector. The SHiP experiment significantly focuses on finding effects of dark matter particle interaction. A characteristic trace of such an interaction is an electromagnetic shower. Our algorithm aims to reconstruct the energy and origin of such showers using online Target Tracker subdetectors that do not suffer from pile-up. Thus, the online observation of the excess of events with proper energy can be a signal for a dark matter. Two different approaches were applied: classical, using Gaussian Mixtures and machine learning based on a convolutional neural network. We've refined the output of the previous step by clusterization techniques to improve transverse coordinate estimation. The obtained results are 25% for energy resolution, 0.8 cm for position resolution in the longitudinal direction and 1 mm in the transverse direction, without any usage of the emulsion.

Data analysis in high energy physics has to deal with data samples produced from different sources. One of the most widely used ways to unfold their contributions is the sPlot technique. It uses the results of a maximum likelihood fit to assign weights to events. Some weights produced by sPlot are by design negative. Negative weights make it difficult to apply machine learning methods. The loss function becomes unbounded. This leads to divergent neural network training. In this paper we propose a mathematically rigorous way to transform the weights obtained by sPlot into class probabilities conditioned on observables, thus enabling to apply any machine learning algorithm out-of-the-box.

The Belle II experiment, beginning data taking with the full detector in early 2019, is expected to produce a volume of data ffty times that of its predecessor. This dramatic increase in data comes the opportunity for studies of rare previously inaccessible processes. The investigation of such rare processes in a high data-volume environment requires a correspondingly high volume of Monte Carlo simulations to prepare analyses and gain a deep understanding of the contributing physics processes to each individual study. This presents a signifcant challenge in terms of computing resource requirements and calls for more intelligent methods of simulation, in particular background processes with very high rejection rates. This work presents a method of predicting in the early stages of the simulation process the likelihood of relevancy of an individual event to the target study using convolutional neural networks. The results show a robust training that is integrated natively into the existing Belle II analysis software framework.

Automated performance tuning is a tricky task for a large scale storage system. Traditional methods highly reply on experience of system administrators and cannot adapt to changes of working load and system configurations. Reinforcement learning is a promising machine learning paradigm which learns an optimized strategy from the trials and errors between agents and environments. Combining with the strong feature learning capability of deep learning, deep reinforcement learning has showed its success in many fields. We implemented a performance parameter tuning engine based on deep reinforcement learning for Lustre file system, a distributed file system widely used in HEP data centres. Three reinforcement learning algorithms: Deep Q-learning, A2C, and PPO are enabled in the tuning engine. Experiments show that, in a small test bed, with IOzone workload, this method can increase the random read throughput by about 30% compared to default settings of Lustre. In the future, it is possible to apply this method to other parameter tuning use cases of data centre operations.

The BESIII inner drift chamber is suffering from aging effect. Cylindrical Gas Electron Multiplier Inner Tracker (CGEM-IT) is considered as one of the upgrade candidates. A simulation study with Garfield++ program has been performed to understand the drift behaviour and to build a full digitization model. Parameters related to Lorentz angle, diffusion effect, drift time, multiplication and induction are obtained from the Garfield++ simulation. A preliminary digitization model is implemented in the BESIII Offline Software System (BOSS) based on those parameters.

The ATLAS experiment records data from the proton-proton collisions produced by the Large Hadron Collider (LHC). The Tile Calorimeter is the hadronic sampling calorimeter of ATLAS in the region $|\eta |\lt 1.7$. It uses iron absorbers and scintillators as active material. Jointly with the other calorimeters it is designed for reconstruction of hadrons, jets, $\tau$-lepton and missing transverse energy. It also assists in muon identification. The energy deposited by the particles in the Tile Calorimeter is read out by approximately 10,000 channels. The signal provided by the readout electronics for each channel is digitized at 40 MHz and its amplitude is estimated by an optimal filtering algorithm. The increase of LHC luminosity leads to signal pile-up that deforms the signal of interest and compromises the amplitude estimation performance. This work presents the proposed algorithm, based on the Wiener filter theory, for energy estimation in the Tile Calorimeter under high pile-up conditions during LHC Run 3. The performance of the proposed method is studied under various pile-up conditions and compared with the current optimal filtering method using proton-proton collision data.

Drift chamber is the main tracking detector for high energy physics experiment BESIII. Due to the high luminosity and high beam intensity, the BESIII drift chamber is suffered from the beam background and electronics noise which represent a computing challenge to the reconstruction software. Deep learning developments in the last few years have shown tremendous improvements in the analysis of data especially for object classification. Here we present a first study of deep learning architectures applied to the real data of BESIII drift chamber to accomplish the hit classification of the background and signal.

Variable-dependent scale factors are commonly used in HEP to improve shape agreement of data and simulation. The choice of the underlying model is of great importance, but often requires a lot of manual tuning e.g. of bin sizes or fitted functions. This can be alleviated through the use of neural networks and their inherent powerful data modeling capabilities. We present a novel and generalized method for producing scale factors using an adversarial neural network. This method is investigated in the context of the bottom-quark jet-tagging algorithms within the CMS experiment. The primary network uses the jet variables as inputs to derive the scale factor for a single jet. It is trained through the use of a second network, the adversary, which aims to differentiate between the data and rescaled simulation.

The LHCb experiment is dedicated to the study of the c- and b-hadron decays, including long-lived particles such as K_s and strange baryons $({\Lambda }^{0},{\Xi }^{-}, {etc}\ldots )$. These kind of particles are difficult to reconstruct by the LHCb tracking system since they escape detection in the first tracker. A new method to evaluate the performance of the different tracking algorithms for long-lived particles using real data samples has been developed. Special emphasis is laid on particles hitting only part of the tracking system of the new LHCb upgrade detector.

We present a new approach to identifcation of boosted neutral particles using Electromagnetic Calorimeter (ECAL) of the LHCb detector. The identifcation of photons and neutral pions is currently based on the geometric parameters which characterise the expected shape of energy deposition in the calorimeter. This allows to distinguish single photons in the electromagnetic calorimeter from overlapping photons produced from high momentum π⁰ decays. The novel approach proposed here is based on applying machine learning techniques to primary calorimeter information, that are energies collected in individual cells around the energy cluster. This method allows to improve separation performance of photons and neutral pions and has no signifcant energy dependence.

The increasing luminosities of future Large Hadron Collider runs and next generation of collider experiments will require an unprecedented amount of simulated events to be produced. Such large scale productions are extremely demanding in terms of computing resources. Thus new approaches to event generation and simulation of detector responses are needed. In LHCb, the accurate simulation of Cherenkov detectors takes a sizeable fraction of CPU time. An alternative approach is described here, when one generates high-level reconstructed observables using a generative neural network to bypass low level details. This network is trained to reproduce the particle species likelihood function values based on the track kinematic parameters and detector occupancy. The fast simulation is trained using real data samples collected by LHCb during run 2. We demonstrate that this approach provides high-fidelity results.

Deep learning architectures in particle physics are often strongly dependent on the order of their input variables. We present a two-stage deep learning architecture consisting of a network for sorting input objects and a subsequent network for data analysis. The sorting network (agent) is trained through reinforcement learning using feedback from the analysis network (environment). The optimal order depends on the environment and is learned by the agent in an unsupervised approach. Thus, the two-stage system can choose an optimal solution which is not known to the physicist in advance. We present the new approach and its application to the signal and background separation in top-quark pair associated Higgs boson production.

Accurate particle identification (PID) is one of the most important aspects of the LHCb experiment. Modern machine learning techniques such as neural networks (NNs) are efficiently applied to this problem and are integrated into the LHCb software. In this research, we discuss novel applications of neural network speed-up techniques to achieve faster PID in LHC upgrade conditions. We show that the best results are obtained using variational dropout sparsification, which provides a prediction (feedforward pass) speed increase of up to a factor of sixteen even when compared to a model with shallow networks.

Particle identification is a key ingredient of most of LHCb results. Muon identification in particular is used at every stage of the LHCb trigger. The objective of the muon identification is to distinguish muons from charged hadrons under strict timing constraints. For this task, we use a state-of-the-art gradient boosting algorithm trained with real background-subtracted data. In this proceedings we present the algorithm along with the evaluation of its performance on signal and background rejection.

Finding tracks downstream of the magnet at the earliest LHCb trigger level is not part of the baseline plan of the upgrade trigger, on account of the significant CPU time required to execute the search. Many long-lived particles, such as K_S⁰ and strange baryons, decay after the vertex track detector, so that their reconstruction efficiency is limited. We present a study of the performance of a future innovative real-time tracking system based on FPGAs, developed within a R&D effort in the context of the LHCb Upgrade Ib (LHC Run 4), dedicated to the reconstruction of the particles downstream of the magnet in the forward tracking detector (Scintillating Fibre Tracker), that is capable of processing events at the full LHC collision rate of 30 MHz.

Machine learning methods are integrated into the pipelined first level (L1) track trigger of the upgraded flavor physics experiment Belle II at KEK in Tsukuba, Japan. The novel triggering techniques cope with the severe background from events outside the small collision region provided by the new SuperKEKB asymmetric-energy electron-positron collider. Using the precise drift-time information of the central drift chamber which provides axial and stereo wire layers, a neural network L1 trigger estimates the 3D track parameters of tracks, based on input from the axial wire planes provided by a 2D track finder. An extension of this 2D Hough track finder to a 3D finder is proposed, where the single hit representations in the Hough plane are trained using Monte Carlo. This 3D finder improves the track finding efficiency by including the stereo sense wires as input. The estimated polar track angle allows a specialization of the subsequent neural networks to sectors in the polar angle.

The Compact Muon Solenoid (CMS) is one of the general-purpose detectors at the CERN Large Hadron Collider (LHC) which collects enormous amounts of physics data. Before the final physics analysis can proceed, data has to be checked for quality (certified) by passing a number of automatic (like physics objects reconstruction, histogram preparation) and manual (checking, comparison and decision making) steps. Last manual step of decision making is very important, error-prone and demands a lot of manpower. Decision making (certification) is currently under active research in computer science for automation by applying recent advancements from computer science, specifically, machine learning (ML).

Ultimately, CMS data certification is a binary classification task where various ML techniques are being investigated for applicability. Just like in any other ML task the hyperparameter tuning is a difficult problem, there is no golden rule and each use case is different. This study explored meta-learning applicability, it is a hyper-parameters finding technique where algorithm learns hyper-parameters from previous training experiments. An Evolutionary genetic algorithm has been used to tune hyper-parameters of a neural network, like number of hidden layers, number of neurons per layer, activation functions, dropouts, training batch size and optimizer. Initially, the genetic algorithm takes manually specified set of hyper-parameters and then evolves towards the near-optimal solution. Genetic stochastic operators, crossover and mutation, were applied to avoid local optimal solutions.

This study shows that by carefully seeding the initial solution the optimal is likely to be found. The proposed solution has improved AUC score of neural network used for CERN CMS data certification. Similar algorithm can be applied for other machine learning models for hyper-parameter optimization.

Measurements in Liquid Argon Time Projection Chamber (LArTPC) neutrino detectors, such as the MicroBooNE detector at Fermilab [1], feature large, high fidelity event images. Deep learning techniques have been extremely successful in classification tasks of photographs, but their application to LArTPC event images is challenging, due to the large size of the events. Events in these detectors are typically two orders of magnitude larger than images found in classical challenges, like recognition of handwritten digits contained in the MNIST database or object recognition in the ImageNet database. Ideally, training would occur on many instances of the entire event data, instead of many instances of cropped regions of interest from the event data. However, such efforts lead to extremely long training cycles, which slow down the exploration of new network architectures and hyperparameter scans to improve the classification performance. We present studies of scaling a LArTPC classification problem on multiple architectures, spanning multiple nodes. The studies are carried out on simulated events in the MicroBooNE detector. We emphasize that it is beyond the scope of this study to optimize networks or extract the physics from any results here. Institutional computing at Pacific Northwest National Laboratory and the SummitDev machine at Oak Ridge National Laboratory's Leadership Computing Facility have been used. To our knowledge, this is the first use of state-of-the-art Convolutional Neural Networks for particle physics and their attendant compute techniques onto the DOE Leadership Class Facilities. We expect benefits to accrue particularly to the Deep Underground Neutrino Experiment (DUNE) LArTPC program, the flagship US High Energy Physics (HEP) program for the coming decades.

In this work, we propose an approach for electromagnetic shower generation on a track level. Currently, Monte Carlo simulation occupies 50-70% of total computing resources that are used by physicists experiments worldwide. Thus, speedup of the simulation step allows to reduce simulation cost and accelerate synthetic experiments. In this paper, we suggest dividing the problem of shower generation into two separate issues: graph generation and tracks features generation. Both these problems can be efficiently solved with a cascade of deep autoregressive generative network and graph convolution network. The novelty of the proposed approach lies in the application of graph neural networks to the generation of a complex recursive physical process.

Probability distribution functions (PDFs) are very used in modeling random processes and physics simulations. Improving the performance of algorithms that generate many random numbers under complex PDFs is often a very challenging task when methods as direct functions are not available. In this work we present general strategies on how to vectorize some PDFs using VecCore library. We show the results for the Exponential, Gaussian, discrete Poisson and Gamma probability distributions.

We present a neural network architecture designed to autonomously create characteristic features of high energy physics collision events from basic four-vector information. It consists of two stages, the first of which we call the Lorentz Boost Network (LBN). The LBN creates composite particles and rest frames from the combination of final state particles, and then boosts said particles into their corresponding rest frames. From these boosted particles, characteristic features are created and used by the second network stage to solve a given physics problem. We apply our model to the task of separating top-quark pair associated Higgs boson events from a $t\bar{t}$ background, and observe improved performance compared to using domain unspecific deep neural networks. We also investigate the learned combinations and boosts to gain insights into what the network is learning.

We provide an algorithm for detection of possible dark matter particle interactions recorded within NEWSdm detector. The NEWSdm (Nuclear Emulsions for WIMP Search directional measure) is an underground Direct detection Dark Matter search experiment. The usage of recent developments in the nuclear emulsions allows probing new regions in the WIMP parameter space. The directional approach, which is the key feature of the NEWSdm experiment, gives the unique chance of overcoming the neutrino floor. Deep Neural Networks were used for separation between potential DM signal and various classes of background. In this paper, we present the usage of deep 3D Convolutional Neural Networks to take into account the physical peculiarities of the datasets and report the achievement of the required 10⁴ background rejection power.

In High Energy Physics, tests of homogeneity are used primarily in two cases: for verification that data sample does not differ significantly from numerically produced Monte Carlo sample and for verifying separation of signal from background. Since Monte Carlo samples are usually weighted, it is necessary to modify classical homogeneity tests in order to apply them to weighted samples. In ROOT, the only homogeneity tests that allow testing weighted samples are implemented for binned data. However, after the data are binned the full information is lost. Therefore we compare these ordinary tests with modified versions of the Kolmogorov-Smirnov, Anderson-Darling, and Cramér-von Mises tests that use full sample information. The proposed tests are compared by estimating a probability of type-I error which is crucial for a test's reliability.

We introduce two new loss functions designed to directly optimize the statistical significance of the expected number of signal events when training neural networks and decision trees to classify events as signal or background. The loss functions are designed to directly maximize commonly used estimates of the statistical significance, $s/\sqrt{s+b}$, and the so-called Asimov estimate, Z_a. We consider their use in a toy search for Supersymmetric particles with 30 fb⁻¹ of 14 TeV data collected at the LHC. In the case that the search for this model is dominated by systematic uncertainties, it is found that the loss function based on Z_a can outperform the binary cross entropy in defining an optimal search region. The same approach is applied to a boosted decision tree by modifying the objective function used in gradient tree boosting.

We introduce a novel implementation of a reinforcement learning (RL) algorithm which is designed to find an optimal jet grooming strategy, a critical tool for collider experiments. The RL agent is trained with a reward function constructed to optimize the resulting jet properties, using both signal and background samples in a simultaneous multi-level training. We show that the grooming algorithm derived from the deep RL agent can match state-of-the-art techniques used at the Large Hadron Collider, resulting in improved mass resolution for boosted objects. Given a suitable reward function, the agent learns how to train a policy which optimally removes soft wide-angle radiation, allowing for a modular grooming technique that can be applied in a wide range of contexts. These results are accessible through the corresponding GroomRL framework.

In recent years, great progress has been made in the fields of machine translation, image classification and speech recognition by using deep neural networks and associated techniques (deep learning). Recently, the astroparticle physics community successfully adapted supervised learning algorithms for a wide range of tasks including background rejection, object reconstruction, track segmentation and the denoising of signals. Additionally, the first approaches towards fast simulations and simulation refinement indicate the huge potential of unsupervised learning for astroparticle physics. We summarize the latest results, discuss the algorithms and challenges and further illustrate the opportunities for the astrophysics community offered by deep learning based algorithms.

Triple-GEM detectors are a well known technology in high energy physics. In order to have a complete understanding of their behavior, in parallel with on beam testing, a Monte Carlo code has to be developed to simulate their response to the passage of particles. The software must take into account all the physical processes involved from the primary ionization up to the signal formation, e.g. the avalanche multiplication and the effect of the diffusion on the electrons. In the case of gas detectors, existing software such as Garfield already perform a very detailed simulation but are CPU time consuming. A description of a reliable but faster simulation is presented here: it uses a parametric description of the variables of interest obtained by suitable preliminary Garfield simulations and tuned to the test beam data. It can reproduce the real values of the charge measured by the strip, needed to reconstruct the position with the Charge Centroid method. In addition, particular attention was put to the simulation of the timing information, which permits to apply also the micro-Time Projection Chamber position reconstruction, for the first time on a triple-GEM. A comparison between simulation and experimental values of some sentinel variables in different conditions of magnetic field, high voltage settings and incident angle will be shown.

RooFit and RooStats, the toolkits for statistical modelling in ROOT, are used in most searches and measurements at the Large Hadron Collider. The data to be collected in Run 3 will enable measurements with higher precision and models with larger complexity, but also require faster data processing.

In this work, first results on modernising RooFit's collections, restructuring data flow and vectorising likelihood fits in RooFit will be discussed. These improvements will enable the LHC experiments to process larger datasets without having to compromise with respect to model complexity, as fitting times would increase significantly with the large datasets to be expected in Run 3.

During the second phase upgrade program developed for LHC and its experiments, the main hadronic calorimeter of ATLAS (TileCal) will replace completely its readout electronics, but the optical signal pathway and detector will be kept unchanged. During the R&D studies for the upgrade, initial analyses for improving the calorimeter granularity were made. A granularity improvement could be achieved through the introduction of Multi-Anode Photomultiplier Tubes (MA-PMTs) into the calorimeter readout chain, together with applications of image processing algorithms for identifying sub-regions on calorimeter cells. This paper presents the latest results from using a Generative Adversarial Network (GAN) to generate synthetic images, which simulate real images formed in the MA-PMT. After the classification of cell sub-regions, preliminary results show a classification accuracy of more than 98% on the experimental test set.

Micro Pattern Gas Detectors (MPGD) are the new frontier in gas trackers. Among this kind of devices, the Gas Electron Multiplier (GEM) chambers are widely used. The experimental signals acquired with the detector must obviously be reconstructed and analysed. In this contribution, a new offline software to perform reconstruction, alignment and analysis on the data collected with APV-25 and TIGER ASICs will be presented. GRAAL (Gem Reconstruction And Analysis Library) is able to measure the performance of a MPGD detector with a strip segmented anode (presently). The code is divided in three parts: reconstruction, where the hits are digitized and clusterized; tracking, where a procedure fits the points from the tracking system and uses that information to align the chamber with rotations and shifts; analysis, where the performance is evaluated (e.g. efficiency, spatial resolution,etc.). The user must set the geometry of the setup and then the program returns automatically the analysis results, taking care of different conditions of gas mixture, electric field, magnetic field, geometries, strip orientation, dead strip, misalignment and many others.

A large number of physics processes as seen by the ATLAS experiment manifest as collimated, hadronic sprays of particles known as 'jets.' Jets originating from the hadronic decay of massive particles are commonly used in searches for new physics. ATLAS has employed multivariate discriminants for the challenging task of identifying the origin of a given jet. However, such classifiers exhibit strong non-linear correlations with the invariant mass of the jet, complicating analyses which make use of the mass spectrum. A comprehensive study of different mass-decorrelation techniques is performed with ATLAS simulated datasets, comparing designed decorrelated taggers (DDT), fixed-efficiency k-NN regression, convolved substructure (CSS), adversarial neural networks (ANNs), and adaptive boosting for uniform efficiency (uBoost). Performance is evaluated using suitable metrics for classification and mass-decorrelation.

Hadronic decays of vector bosons and top quarks are increasingly important to the ATLAS physics program, both in measurements of the Standard Model and searches for new physics. At high energies, these decays are collimated into a single overlapping region of energy deposits in the detector, referred to as a jet. However, vector bosons and top quarks are hidden under an enormous background of other processes producing jets. The ATLAS experiment has employed boosted decision trees and deep neural networks to the challenging task of identifying hadronically-decaying vector bosons and top quarks and rejecting other jet backgrounds. These discriminants are becoming increasingly complex and using more advanced machine learning techniques. The methods currently used to tag these objects are described. In order to improve the tagger performance on the signal efficiency and background rejection, new in-situ techniques are applied, thus directly evaluating the agreement between data and simulation after applying an arbitrarily complex classifier. The precision obtained by applying the in-situ techniques is presented.

In radio-based physics experiments, sensitive analysis techniques are often required to extract signals at or below the level of noise. For a recent experiment at the SLAC National Accelerator Laboratory to test a radar-based detection scheme for high energy neutrino cascades, such a sensitive analysis was employed to dig down into a spurious background and extract a putative signal. In this technique, the backgrounds are decomposed into an orthonormal basis, into which individual data vectors (signal + background) can be expanded. This expansion is a filter that can extract signals with amplitudes ∼1 % of the background. This analysis technique is particularly useful for applications when the exact signal characteristics (spectral content, duration) are not known. In this proceeding we briefly present the results of this analysis in the context of test-beam experiment 576 (T576) at SLAC.