002792359 001__ 2792359
002792359 005__ 20211217100156.0
002792359 0248_ $$aoai:cds.cern.ch:2792359$$pcerncds:FULLTEXT$$pcerncds:CERN:FULLTEXT$$pcerncds:CERN
002792359 037__ $$9arXiv$$aarXiv:2112.03074$$ccs.NI
002792359 035__ $$9arXiv$$aoai:arXiv.org:2112.03074
002792359 035__ $$9Inspire$$aoai:inspirehep.net:1984823$$d2021-12-08T02:49:35Z$$h2021-12-08T08:20:11Z$$mmarcxml$$ttrue$$uhttps://inspirehep.net/api/oai2d
002792359 035__ $$9Inspire$$a1984823
002792359 041__ $$aeng
002792359 100__ $$aWeitzel, [email protected]$$uNebraska U.$$vUniversity of Nebraska -Lincoln Lincoln, NE
002792359 245__ $$9arXiv$$aThe Service Analysis and Network Diagnosis DataPipeline
002792359 269__ $$c2021-12-06
002792359 300__ $$a10 p
002792359 500__ $$9arXiv$$a10 pages, to be published in 2021 IEEE Workshop on Innovating the
Network for Data-Intensive Science
002792359 520__ $$9arXiv$$aModern network performance monitoring toolkits, such as perfSONAR, take a remarkable number of measurements about the local network environment. To gain a complete picture of network performance, however, one needs to aggregate data across a large number of endpoints. The Service Analysis and Network Diagnosis (SAND) data pipeline collects data from diverse sources and ingests these measurements into a message bus. The message bus allows the project to send the data to multiple consumers, including a tape archive, an Elasticsearch database, and a peer infrastructure at CERN. In this paper, we explain the architecture and evolution of the SAND data pipeline, the scale of the resulting dataset, and how it supports a wide variety of network analysis applications.
002792359 540__ $$3preprint$$aarXiv nonexclusive-distrib 1.0$$uhttp://arxiv.org/licenses/nonexclusive-distrib/1.0/
002792359 65017 $$2arXiv$$ahep-ex
002792359 65017 $$2SzGeCERN$$aParticle Physics - Experiment
002792359 65017 $$2arXiv$$acs.NI
002792359 65017 $$2SzGeCERN$$aComputing and Computers
002792359 690C_ $$aCERN
002792359 690C_ $$aPREPRINT
002792359 700__ $$aMcKee, [email protected]$$uMichigan U.$$vUniversity of Michigan Ann Arbor, MI
002792359 700__ $$aBockelman, Brian [email protected]$$uUnlisted, US, WI$$vMorgridge Institute of Research Madison, WI
002792359 700__ $$aThiltges, [email protected]$$uNebraska U.$$vUniversity of Nebraska -Lincoln Lincoln, NE
002792359 700__ $$aBabik, [email protected]$$uCERN$$vEuropean Organisation for Nuclear Research Geneva, Switzerland
002792359 700__ $$aVukotic, [email protected]$$uChicago U.$$vUniversity of Chicago Chicago, IL
002792359 8564_ $$82340845$$s50249$$uhttp://cds.cern.ch/record/2792359/files/CollectorMonitoring.png$$y00000 : Heat map of query times per toolkit
002792359 8564_ $$82340846$$s62118$$uhttp://cds.cern.ch/record/2792359/files/sand_pipe.png$$y00007 OSG Network Monitoring Platform - distributed deployment that collects, stores, visualises and provides APIs for the measurements collected by the WLCG perfSONAR infrastructure
002792359 8564_ $$82340847$$s2962637$$uhttp://cds.cern.ch/record/2792359/files/2112.03074.pdf$$yFulltext
002792359 8564_ $$82340848$$s90646$$uhttp://cds.cern.ch/record/2792359/files/MessageBus.png$$y00002 Message Bus configuration for the perfSONAR data pipeline
002792359 8564_ $$82340849$$s118113$$uhttp://cds.cern.ch/record/2792359/files/alert-subscribe.png$$y00005 The draft web interface to allow users to self-subscribe to alerts.
002792359 8564_ $$82340850$$s99042$$uhttp://cds.cern.ch/record/2792359/files/HTCondorTCPEmail.png$$y00006 HTCondor TCP metrics email
002792359 8564_ $$82340851$$s63316$$uhttp://cds.cern.ch/record/2792359/files/TapeReplay.png$$y00004 Message Bus configuration for tape replay
002792359 8564_ $$82340852$$s64506$$uhttp://cds.cern.ch/record/2792359/files/RunningPending.png$$y00001 : Number of running or pending queries to toolkits
002792359 8564_ $$82340853$$s80931$$uhttp://cds.cern.ch/record/2792359/files/MessageBusPublishedGraph.png$$y00003 Message Bus Grafana monitoring
002792359 960__ $$a11
002792359 980__ $$aConferencePaper
002792359 980__ $$aPREPRINT