CERN Accelerating science

Talk
Title	Evolution of the HEPS Jupyter-based remote data analysis System
Video	If you experience any problem watching the video, click the download button below Download Embed Show n. of views Mp4: 480p (presenter) 720p (presenter) 1080p (presenter) 240p (presenter) 360p (presenter) Subtitles: English FrançAis Copy-paste this code into your page: <iframe width="640" height="360" frameborder="0" src="http://cds.cern.ch/video/2767292?" allowfullscreen></iframe>
Author(s)	Liu, Zhibin (speaker) (Institute of High Energy Physis, CAS; University of Chinese Academy of Sciences)
Corporate author(s)	CERN. Geneva
Imprint	2021-05-20. - 776.
Series	(Conferences) (25th International Conference on Computing in High Energy & Nuclear Physics)
Lecture note	on 2021-05-20T11:55:00
Subject category	Conferences
Abstract	High Energy Photon Source (HEPS) has the characteristic of large amount of data, high timeliness, and diverse requirements for scientific data analysis. Generally, researchers need to spend a lot of time in the configuration of the experimental environment. In response to the above problems, we introduce a remote data analysis system for HEPS. The platform provides users a web-based interactive interface with Jupyter, which makes scientists are able to process data analysis anytime and anywhere. Particularly, we discuss the system architecture as well as the key points of this system. A solution of managing and scheduling heterogeneous computing resources (CPU and GPU) is proposed, which adopts Kubernetes to achieve centralized heterogeneous resources management and resource expansion on demand. An improved Kubernetes resource scheduler is discussed. The schedular dispatches resources to upper applications in combination with the cluster status, which can transparently and quickly deployment the data analysis environment for users in seconds and reach the maximum resource utilization. We also introduce an automated deployment solution to improve the work efficiency of developers and help deploy multidisciplinary applications faster and better in the production environment. A unified certification is illustrated to make sure the security of remote data access and data analysis. Finally, we will show the running status of the system.
Copyright/License	© 2021-2024 CERN
Submitted by	[email protected]