| Report number | LHCb-TALK-2015-074 |
| Title | docker & HEP : containerization of applications for development, distribution and preservation |
| Category | Talk |
| Author(s) | Binet, Sebastien (LAL, Université Paris-Sud, CNRS/IN2P3, Orsay, France) ; COUTURIER, Ben (CERN) |
| Imprint | 2015-04-13. - 16. |
| Presented at | 21st International Conference on Computing in High Energy and Nuclear Physics, Okinawa, Japan, 13 - 17 Apr 2015 |
| Keywords | virtualization ; containerization ; data preservation ; software distribution ; Go programming ; golang ; docker |
| Abstract | HEP software stacks are not shallow. Indeed, HEP experiments' software are usually many applications in one (reconstruction, simulation, analysis, ...) and thus require many libraries - developed in-house or by third parties - to be properly compiled and installed. Moreover, because of resource constraints, experiments' software is usually installed, tested, validated and deployed on a very narrow set of platforms, architectures, toolchains and operating systems. As a consequence, bootstrapping a software environment on a developer machine or deploying the software on production or user machines is usually perceived as tedious and iterative work, especially when one wants the native performances of bare metal. `Docker` containers provide an interesting avenue for packaging applications and development environment, relying on the Linux kernel capabilities for process isolation, adding "git"-like capabilities to the filesystem layer and providing (close to) native CPU, memory and I/O performances. This paper will introduce in more details the modus operandi of `Docker` containers and then focus on the `hepsw/docks` containers which provide containerized software stacks for -among others- `LHCb`. The paper will then discuss various strategies experimented with to package software (e.g. `CVMFS`, `RPM`s, from-source) and applied to optimize provisioning speed and disk usage, leveraging the caching system of `Docker`. Finally, the paper will report on benchmarks comparing workloads on native, VM and `Docker` containers setups. |
Email contact: [email protected]
Rekord stworzony 2015-04-24, ostatnia modyfikacja 2016-06-29
