| Preprint | |
| Report number | arXiv:2112.03074 |
| Title | The Service Analysis and Network Diagnosis DataPipeline |
| Author(s) | Weitzel, Derek (Nebraska U.) ; McKee, Shawn (Michigan U.) ; Bockelman, Brian Paul (Unlisted, US, WI) ; Thiltges, John (Nebraska U.) ; Babik, Marian (CERN) ; Vukotic, Ilija (Chicago U.) |
| Imprint | 2021-12-06 |
| Number of pages | 10 |
| Note | 10 pages, to be published in 2021 IEEE Workshop on Innovating the Network for Data-Intensive Science |
| Subject category | hep-ex ; Particle Physics - Experiment ; cs.NI ; Computing and Computers |
| Abstract | Modern network performance monitoring toolkits, such as perfSONAR, take a remarkable number of measurements about the local network environment. To gain a complete picture of network performance, however, one needs to aggregate data across a large number of endpoints. The Service Analysis and Network Diagnosis (SAND) data pipeline collects data from diverse sources and ingests these measurements into a message bus. The message bus allows the project to send the data to multiple consumers, including a tape archive, an Elasticsearch database, and a peer infrastructure at CERN. In this paper, we explain the architecture and evolution of the SAND data pipeline, the scale of the resulting dataset, and how it supports a wide variety of network analysis applications. |
| Other source | Inspire |
| Copyright/License | preprint: (License: arXiv nonexclusive-distrib 1.0) |
Record created 2021-12-07, last modified 2021-12-17
