Virgo detector characterization and data quality: tools

GWTC-3 [1], the most recent edition of the GW transient catalog edited by the LIGO [2], Virgo [3], and now KAGRA [4] scientific collaborations includes 90 GW signals recorded between 2015 and 2020 during three successive data-taking campaigns called observation runs (in short O_n with $n = 1,2,3$). These discoveries are the result of the joint work of hundreds of scientists worldwide, bringing together a wide range of expertise, ranging from the instrumental side to data analyses. Important components of this global effort are the DetChar activities [5, 6] that focus on studying the detector noises in all their variety: detector characterization on the one hand and data quality studies on the other.

Indeed, the Virgo GW strain channel h(t), reconstructed from its raw data, is dominated by noises of various origins (fundamental, technical or environmental [7]), with different and time-varying characteristics (amplitude and frequency contents). Detector characterization targets the smooth and usually stationary noise floor, which makes the envelope of the sensitivity curve. Beyond that, two main categories of noise artifacts are studied in detail as they can impact the performances of the instrument in detecting genuine GW signals. The first one includes all noise transients, also called glitches, while the second one gathers all long-lasting noise excesses, the spectral noises (i.e. lines or bumps depending on their bandwidth type, narrow or wide, in the frequency domain).

To help understand the source of some of the noises that affect the Virgo sensitivity, hundreds of auxiliary channels monitor continuously the detector control systems as well as its local environment [8]. They are also useful to vet the GW candidates by assessing whether or not they seem to be of terrestrial origin.

The third LIGO-Virgo Observing Run (O3) lasted about 11 months in total. It was divided into two parts: O3a, from 1 April 2019 to 1 October 2019; O3b, from 1 November 2019 to 27 March 2020, separated by a one-month commissioning break in October 2019. O3b should have lasted one more month but the worldwide Covid-19 pandemic forced the LIGO Scientific and Virgo Collaborations to end the data taking prematurely. A few days later, all three detectors were shutdown to cope with the various lock-down constraints.

The Virgo detector takes data in a configuration called Science mode. It corresponds to periods during which the instrument is controlled at its nominal working point, with that control stable and accurate enough to assume that the recorded data are of good quality and suitable for physics analysis. This assumption is checked in real time against online data quality checks and the corresponding dataset is further refined by offline studies to make the final Virgo dataset.

All DetChar analyses rely on dedicated software frameworks, generically called tools in the following. Some have been designed and set up within Virgo to meet the goals of the DetChar group, while others have been developed partly or totally by LIGO colleagues. In fact, any DetChar tool can potentially be used by the three groups (Virgo, LIGO, and KAGRA) thanks to the long-lasting collaborations among them.

More than 100 computing servers have been used in real-time during O3 to control and monitor the Virgo detector, run various data quality checks and perform specific DetChar tasks. Data are processed by the tools described in the following sections. Their outputs are included in the live data streams if they are available with a latency low-enough (about 15 s), or stored on disk otherwise. The end products of these analyses are converted into information for the control room and live summary plots that are updated with a latency of a few minutes at most, and regularly archived for reuse during offline analyses.

All this software framework is controlled using the VPM software interface, that allows to configure, start/stop and monitor processes running on Virgo online servers. These include detector control, data transfer to and from Virgo, as well as the analysis of the reconstructed h(t) stream by the online GW search algorithms running in the EGO computing center. All actions performed using the VPM interface are logged and recorded, in order to reconstruct as accurately as possible the running conditions at any given time, should this need arise.

The most important DetChar tools used by the Virgo group during the O3 run are classified in a few main categories depending on their usage or target: monitoring, generic data analysis, glitches and spectral noise investigations, or database management. Yet, they are not independent: they are often combined to characterize specific features of the detector, or to provide a complete overview of the quality of the Virgo data around a GPS time of interest. Such GPS ranges are called segments.

The flowchart in figure 1 presents an overview of the main analyses carried out by the DetChar group and shows the corresponding tools described in the following. The arrows follow the dataflow which starts from the detector raw data (top left corner) and goes all the way down to the final consumers of DetChar products: the on-duty crew in the Virgo control room, the broad community of DetChar users and the data analysts.

Zoom In Zoom Out Reset image size

This article categorizes the different DetChar tools used during the O3 run. It aims at becoming the main reference for those which had not been documented yet, while it puts all of them into perspective, by showing how they interact and complement each other.

First, section 2 presents the main monitoring tools commonly used to make quick and basic analysis of the Virgo data, or to get a digest of the detector status and of its performance at any given time. Then, section 3 highlights a wide set of DetChar-generic tools: a software layer to ease access to the Virgo data for the users; band-limited RMS-based estimates as quantities to identify noise contributions and follow their evolution over time; different methods to test the noise stationarity and Gaussianity, two hypothesis which are often explicitly or implicitly assumed by many data analyses; finally, some tools detecting and studying peculiarities during data taking phases, that is sudden drops of the sensitivity and losses of the detector global working point.

Section 4 focuses on transient noise bursts, or glitches. It includes well-established tools to identify them in time-frequency representations and to search for their origins. In addition, more recent scattered light monitors are described as well, as this kind of noise appears to be a real nuisance for all ground-based GW detectors. The following section 5 is dedicated to the tools used to identify spectral noise features and to search for their origin. For the latter part of the investigations, coherence in the frequency domain is the key quantity used to connect spectral noise in the GW strain h(t) with auxiliary channels.

Then, the short section 6 introduces two databases jointly used by LIGO and Virgo, one to gather data quality inputs for the analyses, and the other to store information about all GW candidates found by those searches. Finally, section 7 presents the DQR, the new DetChar framework developed specifically for the O3 run in order to vet the GW candidates, in particular those found in low latency by the online data analyses.

Section 8 concludes this article by providing the outlines of the DetChar technical work to prepare the O4 run, ¹¹⁰ using all the experience accumulated during O3, and later on to analyze its dataset.

The main results of the Virgo DetChar analyses performed on data from O3 are presented in the companion article [9].

The abbreviations and acronyms used in this article are defined in a dedicated section at the end.

The dataDisplay software [10] allows the user to read (online or offline) Virgo data and to visualize various types of plots for all the channels available from the DAQ. For instance, it helps to investigate quickly the time evolution of a noise artifact, the coherence between two control signals or the time-frequency characteristics of a transient noise, etc. It has been used extensively during the O2 ¹¹¹ and O3 runs and all over the AdV detector commissioning in between. Figure 2 shows an example of the dataDisplay interface and output.

Zoom In Zoom Out Reset image size

The DMS [11, 12] provides a detailed live status of all the components that make the Virgo detector operate, from the hardware parts to the online software used to control the instrument, acquire the data and process it. It also includes the monitoring of environmental sensors from around the experimental areas. Each of the many DMS monitors uses a set of DAQ channels, combines them by performing mathematical and logical operations on their outputs and produces a flag whose value can take four severity levels, each associated with a color for visual display. A web interface is used to display and browse the DMS monitor flags with a few second-latency, both in the Virgo control room and remotely.

For instance, figure 3 shows the Virgo detector status about 4 s after the detection of the GW event GW190412 [13]. The DMS web interface looks like a checkerboard. Each row, labeled in the most-left column, corresponds to a different part of the instrument (mirror suspensions, vacuum system, etc). Moving to the right of the screenshot, that part is broken down in smaller sets that are each associated with a cell on the web interface. Each cell can contain many DMS flags and its color reflects the highest severity among all these flags (green $\leftrightarrow$ no alarm; yellow $\leftrightarrow$ warning; red $\leftrightarrow$ alarm (not present on that particular snapshot); grey $\leftrightarrow$ some information is missing). Clicking on a cell gives access to the flag individual information: their values and associated severities.

Zoom In Zoom Out Reset image size

In addition to the live global detector status, a new DMS archival system has been set up for the O3 run: complete DMS snapshots are taken every ${\sim} 10$ s and archived. They can be retrieved later at any time, by running a playback application that uses the same interface as the live DMS. This functionality is particularly convenient to check the status of the detector a posteriori, when a GW candidate or a particular feature in the data have been identified.

The VIM [14, 15] manages a collection of automated scripts that update every few minutes a wide set of plots and tables; all these monitoring products are permanently archived on a daily basis. A web interface allows users to browse that database, both for live monitoring of the experiment and for offline investigations. VIM is an essential tool that provides a direct access to a detailed status of the various Virgo detector components and of related frameworks, such as calibration and online data processing, data transfer or online data analyses. A snapshot of the VIM web interface is shown in figure 4.

Zoom In Zoom Out Reset image size

In-depth studies of a particular feature observed in the data or analyses scanning a significant fraction of the dataset require the use of dedicated software. Common and key building blocks of these codes have access to the DAQ channels and to the detector component configurations. Thus, dedicated packages have been developed over the years to provide simplified and generic interfaces to these data: they rely on low-level core packages like the FrameLib software library [16] but calls to these functions are hidden to the users. These packages interact with the software, hardware and data of the Virgo interferometer: they are widely used within the collaboration, from daily use in the control room to DetChar studies. The two main collections of such functions are PythonVirgoTools [17] and MatlabVirgoTools, targeting Python and Matlab developers, respectively.

BRMSs of DAQ channels computed in specific frequency ranges after Fourier-transforming the time series are useful indicators for transient disturbances or new features in the data – see equation (1) in section 3.3.1 below. For instance, low-frequency BRMS of seismometer data allow to separate different contributions to the seismic noise at EGO [7]. Going from low to high frequencies, one can isolate successively: distant and potentially strong earthquakes; sea activity on the Tuscany coastline; anthropogenic contributions with day/night and weekly periodicities; finally, on-site activities. In addition, BRMS is used to monitor the excitation of the violin modes, i.e. the resonances of the mirror suspensions.

In Virgo, various software frameworks can compute BRMS. One worth-mentioning is BRMSMon, a dedicated software that is widely used by the environmental monitoring team and in data quality studies. In addition to generating BRMS, BRMSMon can compare their values to thresholds (either fixed or adaptive) and logically combine the outputs of these comparisons into binary channels called flags. For instance, assuming a collection of nine sensors installed in different EGO buildings, one can create a flag that is active (value equal to 1) if at least five of these nine sensors exceed their own threshold and inactive (value 0) otherwise. The BRMSMon output channels, sampled at 1 Hz, are included in the DAQ.

Several tools have been implemented to perform statistical tests to verify the stationarity and Gaussianity of the data. These properties are indeed the typical assumptions at the base of most of the statistical analyses of GW search pipelines, such as MBTA [18], PyCBC [19] and GstLAL [20] that are based on the matched filter technique [21, 22]. Moreover, the onset of a non-stationary behavior of the detector can be the symptom of some hardware malfunction or some contamination from environmental noises. In any case, it requires prompt investigations of its causes and, possibly, the actuation of adequate mitigation strategies.

3.3.1. BRiSTOL.

The detector output records can be described as the realization of a stochastic process. This process is said to be (strict-sense) stationary if its statistical properties are left unchanged by shifts in time, which, in most practical situations, allows us to estimate them from a sufficiently long realization of the process (ergodic hypothesis), and this estimate does not depend on when, in time, it has been performed. For most of the analysis, it is sufficient to consider a weaker form of stationarity that involves only the first distribution moments, namely the mean and the covariance function. Moreover, if the process is stationary, we can define the PSD of the process as the Fourier transform of its covariance [22].

The BRiSTOL statistical test [23] verifies the hypothesis of weak-sense stationarity by verifying that subsequent PSD estimates, in predefined frequency bands, are compatible with the same probability distribution. The corresponding test statistics are based on a set of BRMS time series, estimated on an equal number of bands:

$$\begin{equation} \mathit{BRMS}_t(b) = \sqrt{\int_{f\in b} \hat{S}_t(f)\,\mathrm df}, \qquad \mathrm{for} \quad b\in\left\{[f_1 ^{\,\max}, f_1 ^{\,\min}],\ldots,[f_K ^{\,\max}, f_K ^{\,\min}] \right\} \end{equation} \tag{ 1 }$$

for data $x_{t_n}$ recorded at Nyquist rate f_S , $t_n = t+n/f_S$, where $\hat{S}_t(f)$ is a PSD estimate referred to time t, and obtained with the periodogram method [24]:

$$\begin{equation} \hat{S}_t(f_k) = \frac{1}{N}\bigg|\sum_{n = 0} ^{N-1} x_{t_n} e^{-2\pi i\, nf_k/f_S}\bigg|^2,\qquad f_k = \frac{k\,f_S}{N}, \quad\text{for}\quad k = 0,\ldots,N-1. \end{equation} \tag{ 2 }$$

Two modifications have been implemented to make equation (1) more suitable for the study of transient noise, in particular to identify 'slow non-stationarities', namely changes in the statistical properties of the data over time scales longer than a second. First, frequencies corresponding to spectral lines (refer to section 5.2 for more details) have been removed from the integral in (1). These lines are narrow features in the PSD of the data, originating from resonances in various parts of the interferometer and their harmonics. Their intensities can be orders of magnitude larger than the neighboring noise floor. Hence, if a line is present in a band where we are about to compute the BRMS, it is likely to dominate the final estimate, and also the corresponding fluctuations, preventing us from probing the features of the underlying noise floor. To remove these lines, we identify them with an algorithm similar to the one developed for the NoEMi pipeline [25], and based on the prominence of their PSD [26].

Second, also glitches are typically removed from the BRMS time series. As these glitches can manifest at a rate of about 10 per minute (see O3 glitch rates in [9]), every data segment longer than a few seconds is likely to contain one of them, hence leading us to reject stationarity over such time scales. Dedicated algorithms, based on excess noise identification, are typically used to find them, and will be described in section 4. However, these algorithms are in general not sensitive to slower non-stationarities. To make BRiSTOL specifically targeted at the latter, we have then proceeded to identify glitches on the BRMS time series by means of an algorithm based on a rolling median absolute deviation (defined as the median absolute difference from the median), and to remove the corresponding data points from the analysis.

For each frequency band, the resulting modified BRMS time series are divided into 'chunks' of equal duration, which are used to test stationarity. This hypothesis is tested by means of a two-sample Kolmogorov–Smirnov test [27] for each pair of consecutive chunks, and the corresponding p-values are compared to a test significance α (to be decided in advance). The (null) hypothesis of stationarity is rejected when a p-value is less than or equal to α, meaning that the estimates in the two chunks are drawn from different distributions and the underlying process has changed.

There are two advantages in using BRMS-based quantities. First, averaging over the frequencies of each band has a similar variance reduction effect than the means in Welch's PSD estimation method [28]. This in turn allows a finer time resolution while maintaining a moderate variance for the test statistics, that is, the empirical distribution of the BRMS. Second, the various non-stationarities typically manifest in specific frequency bands, closely related to the noise source that generated them. For example, the main harmonic of scattered light is usually visible below $30\,\mathrm{Hz}$; non-linear and non-stationary couplings of the angular controls with the $150\,\mathrm{Hz}$ harmonic line are characteristic of a tight region around it, etc. So, without losing much of resolution, we can perform the noise characterization directly on these bands instead of on each frequency bin comprising the spectrum of the signal.

The output of BRiSTOL can be visualized as a time–frequency map where, for each bin, the value of the test statistic p-value is reported. An example of such a map is shown in figure 5, where different color palettes are used to highlight those regions where stationarity should be rejected (red) and where not (blue). The time resolution of the test is given by the duration of each chunk and that of the PSD estimates, typically one minute and one second respectively; that in frequency is determined by the band division of the spectrum for computing the BRMS, which is conveniently done choosing exponentially spaced frequency intervals.

Zoom In Zoom Out Reset image size

This tool has been developed in the commissioning phase preceding O3, and has been used during the run as well, to assess the quality of the data as part of the event validation procedure (refer to [9] for more details).

3.3.2. rayleighSpectro—Gaussianity test.

Similarly to what was discussed for the stationarity hypothesis, Gaussianity has to be tested separately in the different regions of the spectrum where noise sources can show up. rayleighSpectro [29], based on the Rayleigh test [30], does this by means of a consistency check on the PSD estimated from the data with what is expected for stationary Gaussian noise. Indeed, if the data is compatible with the hypothesis of Gaussianity, the periodogram estimator in equation (2) is asymptotically (with N large) described by an exponential distribution of parameter $S(f_k)^{-1}$ [31], where $S(f_k)$ is the processed PSD. The corresponding ASD estimator, obtained as the square root of equation (2), is described by a Rayleigh distribution with parameter $\sqrt{S(f_k)/2}$. The scaling property of this distribution can be used to construct consistency tests. For example, the standard deviation of the ASD estimates obtained on non-overlapping segments provides an estimator of the standard deviation of this variable, which equals to $\sqrt{(4-\pi)\,S(f_k)/2}$. Similarly, the mean of these estimates provides an estimator of the mean: $\sqrt{\pi S(f_k)/2}$. The ratio of these two quantities gives a test statistic that, at each frequency f_k , is asymptotically equal to

$$\begin{equation} \frac{\sqrt{4-\pi}}{\sqrt{\pi}}\simeq 0.52. \end{equation} \tag{ 3 }$$

The actual value of the previous quantity for a finite number of averages and the corresponding critical values for performing statistical tests have been computed in [29, 32].

If the noise is not Gaussian, or its properties has changed while estimating the standard deviation and mean of its ASD, the resulting test statistic will take values different than what reported in (3). This means that the Rayleigh test is sensitive to both non-Gaussianities and non-stationarities in the data. Smaller values of the statistic are associated with data having smaller fluctuations than those expected for a Gaussian process; spectral lines usually behave in this way. Larger values are instead typical of non-stationary noises, such as glitches, that produce a larger variance of the ASD estimates.

By dividing the data into chunks of duration $\Delta t$, one can obtain a time–frequency map, similar to a spectrogram, showing with time resolution $\Delta t$ the frequencies and times where the data significantly depart from the expected value of equation (3).

The output of rayleighSpectro are included in VIM and also used in the DQR for event validation (see [9] for more details).

The interplay between this tool and BRiSTOL for the assessment of the stationarity and Gaussianity of the data is the following. BRiSTOL assesses where the data is compatible with the hypothesis of wide sense stationarity, that is, the second order moments (i.e. the covariance or the RMS) are left unchanged by shifts in time. This corresponds also to a strong sense stationarity (the invariance of the probability distribution of the noise) if the data is also Gaussian, that is, completely characterized by its mean and covariance functions, as tested by rayleighSpectro. Conversely, deviations from these assumptions can be tested independently. This is useful for example with regions corresponding to spectral lines that are typically stationary in very good approximation but not Gaussian.

Figures 5 and 6 show examples of the application of these two tools to 1 h of data at the beginning of O3a. In the first plot, BRiSTOL highlights many slow non-stationarities at frequencies up to about 20 Hz, most likely due to high microseismic activity, as well as a loud glitch at about 15:45 UTC. This one is clearly identified by the Rayleigh test with values of the test statistic larger than what is expected for stationary and Gaussian noise. Moreover, in the color map of figure 6, spectral lines, in particular those associated with the 100, 150 and 200 Hz harmonics of the mains (the European power grid frequency is 50 Hz), are highlighted in blue, corresponding to values of the test statistic smaller than the asymptotical limit in the Gaussian case, given in equation (3). In the left-hand side panel of the same image, the 450 Hz frequency of the main test masses violin modes, and its first harmonic at about 900 Hz, are highlighted as well.

Zoom In Zoom Out Reset image size

Figure 7 shows another example of a Rayleigh spectrum from O3, corresponding to a period during which a transient noise was present for several minutes between 10 Hz and 20 Hz.

Zoom In Zoom Out Reset image size

The BNS range is the average distance up to which the merger of a BNS system can be detected. The average is taken over the source location in the sky and the BNS system orientation, while a detection is defined by convention as a SNR of 8 or above. This quantity is related to the live detector sensitivity and can be used as a summary statistic to monitor the overall performance of the instrument in relation to transient searches.

Two useful high-level data quality monitors are based on the BNS range downwards excursions. One tags BNS range drops, that are significant and sudden decreases in this value, usually flagging data quality problems. The other automatically generates (logical) gates that are applied on the GW strain channel to smooth out to zero the data that are affected by a strong noise transient. BNS range drops and gates are related but not equivalent depending on the frequency contents of the noise burst.

3.4.1. BNS range drops.

A BNS range drop means that the live sensitivity of the detector is degrading significantly, at least in a given frequency band, possibly in the entire bandwidth of the instrument. Therefore, it is important to identify transient sensitivity worsenings and investigate their causes. BNS range drops are very diverse: the decrease goes from a few percents to almost the full range, while the drops can last from a few seconds to minutes.

During O3, BNS range drops were detected using an absolute threshold on the live value of that quantity. After the end of the run, adaptive methods able to follow the natural evolution of the BNS range and to locate all significant drops have been developed. Figure 8 shows examples of the output of the adaptive BNS range drop locator running on O3 data.

Zoom In Zoom Out Reset image size

3.4.2. Gates.

Losses of the global working point of the Virgo interferometer (the mandatory configuration sensitive to passing GWs) do not just interrupt the data taking: they decrease the overall duty cycle as few tens of minutes are needed after such events to restore the conditions for taking good-quality data sensitive to the passing of GWs. Therefore, categorizing control losses is important to understand their main causes and to get alerted when a new class of control losses appears, or when an already known category becomes more frequent.

An extensive offline study of the global control losses in Science data-taking mode during the O3 run has lead to the identification of the root cause of the control losses in most cases [7]. The experience gained with this work will be useful for the pre-O4 commissioning phase (noise hunting) and the subsequent data taking periods in two ways. First, the categories identified during O3 will be reused as a starting point to investigate new control losses. Then, an online monitor will analyze these global control losses within minutes of their occurrence; it will provide a set of automated plots for further human diagnosis and possibly point out their probable cause. This framework is currently under development and will reuse the approach (if not the proper software infrastructure) of the DQR (see section 7).

The Omicron [33] search algorithm is used to detect and characterize transient noises. The data is processed using a Q transform [34] which consists in decomposing a time series x(t) onto a generic basis of complex-valued sinusoidal Gaussian functions centered on time τ and frequency f:

$$\begin{equation} X(\tau, f, \sigma_t) = \int_{-\infty}^{+\infty}{x(t) \frac{W}{\sigma_t\sqrt{2\pi}}\exp{\left[-\frac{(t-\tau)^2}{2\sigma_t^2}\right]} e^{-2i \pi f t}\mathrm dt}. \end{equation} \tag{ 4 }$$

This transformation is a modification of the standard short Fourier transform in which the analysis window size σ_t varies inversely with the frequency and is characterized by a quality factor Q: $\sigma_t = Q/(\sqrt{8}\pi f\,)$. The parameter space $(\tau, f, Q)$ is tiled to guarantee both a high detection efficiency and an optimized processing speed. The noise of the input signal x(t) is whitened prior to the Q transform such that all noise frequencies have the same weight. This is done through the normalization factor W which includes an estimate of the local stationary noise, such that the Q transform coefficient X directly measures the SNR associated to each individual tile $(\tau, f, Q)$. A glitch in the data is detected by Omicron as a collection of tiles with high-SNR values. An Omicron glitch is characterized by a set of parameters $(\tau, f, Q)$ given by the tile with the highest SNR value. Omicron offers a two-dimensional representation of glitches where the SNR distribution of tiles is plotted in one or several Q planes. GW events can also be visualized with Omicron: see figure 9.

Zoom In Zoom Out Reset image size

The UPV algorithm [35] has been developed to detect and characterize noise correlations between two glitch data samples; one derived from the GW strain channel h(t) and the other derived from an auxiliary channel. The algorithm tunes, considering Omicron triggers of a given auxiliary channel, a signal-to-noise ratio threshold such that, when a trigger is above threshold, there is a high probability to find a coincident glitch in the h(t) data. In O3, the Virgo data were processed with the UPV algorithm on a daily basis to support the noise characterization effort; some auxiliary channels were identified by UPV as exhibiting glitches correlated with h(t) glitches, providing hints about the noise coupling in the detector.

The VetoPerf analysis tool measures the performance of a data quality flag. A data quality flag is defined as a list of time segments targeting transient noise events. VetoPerf counts the number of the h(t) triggers detected by Omicron which are coincident with the data quality flag time segments. From this, it derives performance numbers and produces diagnostic plots characterizing that data quality flag.

Scattered light is a non-linear, non-stationary noise affecting the sensitivity of the interferometer in the GW detection frequency band. As adaptive algorithms such as EMD [36–38] are suitable for the analysis of non-linear, non-stationary data, they can be used to quickly identify optical components which are sources, i.e. culprits, of scattered light [39]. As part of the detector characterization effort, a tool was developed and applied to Virgo O3 data with the aim of identifying culprits of scattered light in the DARM DOF of the detector [40]. The tool employs the recently developed time varying filter EMD algorithm (tvf-EMD) [41] which gives more accurate results compared to EMD [40]. When scattered light is affecting the detector, arches show up in DARM spectrograms. The arches frequency and their time of occurrence is given by the so called predictor (measured in Hz):

$$\begin{equation} f_{\textrm{arch}}(t) = 2\frac{|v(t)|}{\lambda}, \end{equation} \tag{ 5 }$$

where v(t) is the velocity at which the optical component is moving and λ is the laser wavelength. Equation (5) is computed using the position data of several optics of the detector, such as for example the SWEB. Having obtained predictors for several optical components, the tool computes the instantaneous amplitudes IA(t), i.e. the envelope of DARM oscillatory modes which are extracted by tvf-EMD. IA(t) can then be correlated with the list of predictors. The optical component with the highest correlation among its predictor and the IA(t) of DARM is considered to be the culprit of the scattered light noise witnessed in DARM. Visual counterproof can be performed (see figure 6 from [40]) overlapping the culprit predictor on the DARM spectrogram [42]. The methodology of [39, 40] was extended and integrated in the gwadaptive-scattering pipeline, an automated Python code which allowed to characterize the origin of scattered light glitches in LIGO during the O3 run [43]. Furthermore, adaptive analysis can be used to monitor daily the onset and time evolution of scattered light noise in connection with microseismic noise variability [44]. These daily analyses have been integrated in the gwadaptive-scattering pipeline as well.

Within the VIM (see section 2.3), spectrograms spanning periods from one day to a week are regularly updated using the custom Spectro software [29]. This framework is based on a set of ROOT [45, 46] scripts that provide various indicators (BRMS, Rayleigh spectra, etc), useful to help investigating non-stationary spectral lines or intermittent noises. The Spectro tool has also been used during O3 to probe the time-frequency pattern of the glitches associated with BNS range drops.

The NoEMi tool [25, 47] tracks on a daily basis spectral lines, both stationary and wandering ones, and searches for coincidences between the lines found in a main channel—typically the GW channel h(t) — and in a list of auxiliary channels. The NoEMi configuration defines several parameters and thresholds, like for instance: the threshold on the critical ratio ¹¹² for peak selection in the spectra, the frequency resolution (linked to the time length of the data segments over which the FFT is computed), the name of the main channel, the list of auxiliary channels to search for coincidences. During O2, the NoEMi software produced daily results and looked for peaks in the spectra using a frequency resolution of 1 mHz. With this configuration, NoEMi looked for coincident spectral peaks between the DARM channel and approximately 40 auxiliary channels.

During the break between the O2 and O3 runs, the NoEMi software has been intensively modified, resolving the main issues identified in the old version. The original code was not well-structured (and hence difficult to modify) and also not fully-efficient CPU-wise. Furthermore, the original version produced several static files which were unessential for the final output. As a further improvement, the MySQL database which stores all parameters of each spectral line found during the run has been normalized, meaning that useless or redundant data have been removed and that the data storage is now more coherent. The database scalability has been improved as well, in order to allow storing more data and handling a higher load of requests. Additionally, a more dynamic interaction with the web interface used to browse the results has been introduced. The new version of the code has been used for the first time in O3.

During O3, NoEMi used the same set of ∼40 auxiliary channels as in O2, plus an additional set of ∼140 environmental channels, e.g. seismic, magnetic, and acoustic probes. The coincidence between a line in the GW strain channel h(t) and the signal of one of the environmental monitor, suggests that the noise line originates from a physical source such as a vacuum pump, a cooling fan, an electronic device, etc. This information helps to identify the instrumental origin of detected lines in h(t), and it has been included in the official Virgo-O3 line list publicly released by the GWOSC [48]. Figure 10 illustrates the lines identified in the Virgo GW strain channel during O3.

Zoom In Zoom Out Reset image size

Internally, lines that have been identified are stored in a dedicated database that includes detailed information about them: most notably their times of appearance, and links pointing to the associated documentation (logbook entries, studies, mitigation actions, etc). The contents of the database can be compared with a new NoEMi processing, to find out quickly which lines identified by NoEMi are already known and which ones are not.

The BruCo [49] is a python-based tool designed to search for correlated noise among channels. The code (version 2017-01-23) is publicly available from the git repository [50], which also provides a description of the argument list. An instance of the repository is kept with Virgo-specific data access features.

BruCo computes the magnitude-squared coherence between a main channel (typically, but not necessarily, the detector strain channel h(t)) and all auxiliary channels that, at the time of interest, are recorded by the DAQ system. Optionally, a set of redundant channels which are known a priori to be correlated with the main one, can be excluded. In Virgo, during the O3 run, there were approximately 3000 non-redundant channels with a sampling frequency $\geqslant$1 kHz. To deal with the high computational load required by this analysis, BruCo implements the option of multi-core parallel processing in up to 10 threads.

In the BruCo implementation adopted for Virgo during O3, a continuous Science data segment of length T = 800 s is selected for the main channel h(t) and, in turn, for each auxiliary channel n(t). Each data segment is resampled to a targeted output frequency of 2 kHz with the Fourier resampler scipy.signal.resample, divided into $N_{\textrm{ave}} = 100$ sub-segments (8 s long) and the averaged magnitude-squared coherence is computed, as:

$$\begin{equation} C_{h,n}(f_i) = \frac{\mid < FFT_n(f_i)^{*} FFT_h(f_i) >\mid ^{2}}{< \mid FFT_h(f_i)\mid ^{2}>< \mid FFT_n(f_i)\mid ^{2}>} \end{equation} \tag{ 6 }$$

where FFT denotes the windowed FFT, '$\lt\gt$' denotes the averaging operation, f_i is the ith frequency bin, and '$^{*}$' the complex conjugate operation. With these parameters, the frequency resolution is $\mathrm df = N_{\textrm{ave}} / T = 0.125$ Hz. Coherence is examined up to 1 kHz, and its value is deemed significant if it exceeds a threshold set to 0.03, a value corresponding to the 95% confidence level of the distribution of averaged coherence between random data [51], given the selected parameters.

BruCo jobs were run regularly and automatically during the whole O3 run. Daily results are HTML-formatted and made accessible in a dedicated VIM web page (see section 2.3). The BruCo VIM summary page allows to quickly spot noise paths contributing to the GW strain channel h(t) in specific frequency bands. In addition, BruCo has often been used as an on-demand analysis tool to examine specific time periods.

BruCo main output is a table that contains, for each frequency bin, the ordered list of the auxiliary channels that are most coherent with the main channel. The cell background is color-coded in shades of red from full red (maximum coherence: 1) to white (no coherence) as shown in figure 11. For each auxiliary channel, a plot (see figure 13) of the projected coherence quantity, $h_{n}(f) = \lt FFT_{h}(f)\gt\sqrt{C_{h,n}(f)}$, is produced and linked to the table. In the hypothesis of linear coupling, this quantity estimates the contribution to the strain channel of the noise witnessed by the nth auxiliary channel [51]. Additionally, the VIM daily summary page contains the list of the top ranked channels in the frequency bins with coherence greater than 0.3 shown in table 1, and a plot of the combined projected coherence greater than 0.5 presented in figure 12.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Figure 13 shows BruCo daily plots illustrating one example of noise contamination spotted during O3, which triggered a more in-depth investigation [8, 53].

The interferometer noise spectrum can sometimes present peculiar structures as a consequence of non-linear couplings between different noise processes. Some of these structures consist of two pairs of sidebands around known spectral lines, which are not explained by means of the previously described linear coherence methods. One example of this kind of noise is the bilinear noise, generated by the coupling of two noise sources that jointly affect a third signal. In GW detectors, the main cause of this bilinear noise is the up-conversion of the low frequency seismic noise, that can affect the mirrors angular controls, which couples with some narrow-band noise processes, like power mains lines ($50\,\mathrm{Hz}$ fundamental frequency or its harmonics) or calibration lines (see for example [5, 23]).

The MONET [54] is a python-coded framework designed to investigate these sidebands. The main hypothesis at the basis of this tool is that the sidebands are due to the coupling of a carrier signal with the low-frequency (with a typical cutoff frequency $f_{\textrm c}$ of a few Hz) part of an auxiliary channel, called the modulator signal. Under this hypothesis, MONET searches for coherence between a main channel (typically, but not necessarily, the detector strain channel or the DARM channel) and a new signal given by the product, in the time domain, of the chosen carrier signal and a modulator signal. That carrier signal can be a real channel or a simulated signal such as, for example, a sinusoidal signal.

Similarly to what is done with BruCo, continuous Science data segments of a specific time length T are selected for all the channels to be investigated with MONET. Then, each data segment is resampled to a targeted output frequency ($f^{\,\,\textrm{out}}$) and, finally, the magnitude-squared coherence is computed using equation (6). For the analysis of the O3 Virgo data, we typically used the following values: $f_{\textrm c} = 5\,\mathrm{Hz}$, $T = 1200\,\mathrm{s}$ and $f^{\,\,\textrm{out}} = 1\,\mathrm{kHz}$.

MONET can be executed on demand, investigating dozens of auxiliary channels and spectral lines in every single processing. The outputs are made available through a hierarchical structure:

• A main directory, whose name is built from the main channel name, the initial GPS time and the time length of the segment of data to be analyzed.
• The main directory contains one sub-directory for each carrier signal used.
• Each sub-directory contains in turn several folders, one for each modulator channel.

The sub-directory contains a table in text format and a summary figure. The table (see table 2 for an excerpt example) is made of three columns and several rows. Each row corresponds to a different frequency bin, indicated in the first column. The second and third columns display, for each modulator channel, the computed above-threshold coherence value and the name of that channel respectively. Figure 14 shows a MONET summary plot for the DARM channel during the pre-O3 commissioning phase. Red points marking the frequencies at which coherence above threshold is found with at least one auxiliary channel are superimposed to the main channel ASD. For instance, the red points at ${\sim} 200$ Hz shown in figure 14 correspond to the frequency bins reported in table 2; at such frequencies, a coherence value above the chosen threshold was found for several modulator channels.

Zoom In Zoom Out Reset image size

In each innermost folder, a table and a plot are generated, in which the coherence values associated with the specific modulator channel for each frequency bin are reported. Several other plots are also produced, in which the ASD of the main channel is reported, together with the noise projection based on the coherence values, around the specific spectral lines to be investigated as seen in figure 15.

Zoom In Zoom Out Reset image size

The results of MONET can help identify the origin of the observed sidebands in the main channel. The modulating channels that, once multiplied by the carrier signal, exhibit the largest coherence with the main channel can witness those parts of the detector that are more sensitive to low frequency noise, usually due to microseismic activity, and its up-conversion through bilinear couplings.

During O3 and the preceding commissioning phase [55–57], MONET has been effectively used to characterize the spectral noise in the strain channel and guide the commissioning activity on how to improve it. This has been the case for the observed sidebands of the $150\,\mathrm{Hz}$ mains frequency harmonic line [58]. The increase in the noise around this frequency during intense microseismic activity has been found coherent with the angular controls of the BS as modulating channels. Moreover, under similar noise conditions, the value of the coherence, hence the strength of the bilinear coupling and the observed overall noise level at $150\,\mathrm{Hz}$, was proven to depend on the way the BS is controlled ('full bandwidth' vs. 'drift control'), convincing the team working on the interferometer alignment to prefer to switch the operational mode of the BS during high microseismic activity with an improvement of up to $2\,\mathrm{Mpc}$ of BNS range [23].

In addition to that, MONET has helped characterize the effect of the beam alignment on the PR mirror on the linear noise subtraction of the frequency noise coupling due to the arm asymmetry used to produce the reconstructed GW strain signal h(t). Large values of coherence are found with the channels controlling the PR alignment at $1111\,\mathrm{Hz}$, the frequency of the injected line for the laser frequency stabilization control loop, when the efficiency of the noise subtraction was smaller [59]. This has been interpreted as a consequence of the PR mirror transverse position modulating the frequency noise coupling, hence producing non-linear effects, not adequately taken into account by means of the linear subtraction method [60]. This effect has been quantified in a reduction of $2\,\mathrm{Mpc}$ of BNS range when the beam is transversely misaligned on the PR.

For each data quality flag, the DQSEGDB [61] stores the segments (integer GPS ranges) during which that particular flag is active, meaning that the set of conditions it is based on is fulfilled. For instance, one such flag tags the GPS segments during which the Virgo detector is taking data in Science mode. There are two ways to fill this database with Virgo flags:

• Online, during the data taking, through the SegOnline server [9] that is compiling information provided by various data streams;
• Offline, by completing or fixing existing segment sets, or adding new data quality flags to monitor additional conditions.

A versioning system is used to keep track of changes in segment lists that can modify a particular flag, i.e. that impact offline analyses, by changing the contents of the dataset they are processing. By convention, the highest version number corresponds to the best (most recent) segment list and is the one queried by default.

During O3, the GraceDB) [62] has been the central place where information about transient GW candidates was uploaded and stored: online search triggers, source localization estimates in the sky, data quality information, other metadata, etc. In particular, GraceDB triggered automatically frameworks like the DQR through the LVAlert [63] when candidate events of interest were identified; and, consequently, DQR results (see section 7) got uploaded back to GraceDB as soon as they became available. GraceDB has a public-faced portal that provides information about the public alerts shared with the astronomer community, while most of its data are private and reserved to the LIGO, Virgo and KAGRA collaborations.

The DQR is a framework developed by LIGO and Virgo for the O3 run, in order to quickly gather enough information to vet the significant triggers found by the online transient GW searches. The goal is to either confirm the associated public alert, or have it retracted at once. All 80 public alerts delivered during O3 [64] (of which 24 have been retracted) have used this input.

A DQR runs on a computing cluster where the h(t) strain channel and the associated raw data auxiliary channels are available in low latency. Therefore, each collaboration (Virgo at EGO and LIGO with its two detectors) was responsible for the implementation, the operation, the monitoring and the upgrade of its own DQR framework. There was however an agreement on a common format for the check outputs, originally developed by LIGO [65].

The DQR framework is triggered by GraceDB through the LVAlert protocol. A JSON payload received from GraceDB allows for the generation and the configuration of a new DQR. The checks are then processed and their results are uploaded back to GraceDB, alongside all the records associated with that particular GW candidate. In order for the DQR to be triggered, a candidate event is required to have a false-alarm rate below 1/day. This is a conservative threshold, much higher than that required to release the candidate as a public alert, but still low enough to keep the computational cost of generating the DQRs under control. Therefore, in average, only a handful of DQRs were automatically processed on a daily basis during the ${\sim}330$ d of the O3 run: not a high CPU load overall, but still about 20 times more DQRs than the number of public alerts that had to be vetted.

Figure 16 summarizes the Virgo DQR architecture used during the O3 run. When a trigger with a low-enough false alarm rate is received, a new DQR is created and configured, using information from GraceDB. Then, the data quality checks are run in parallel on the EGO HTCondor [66] farm. As soon as a given check is complete, its results are uploaded back to GraceDB. In parallel, the DQR progress and results are immediately available for Virgo DetChar experts and on-duty people, through an EGO-internal web server. The DQR format [65], originally developed by LIGO, is lighter and more versatile than the GraceDB user interface: it ensures a direct access to the Virgo DQR outputs. The DQR web page URL is automatically sent to the relevant internal mailing list as soon as the newly-created DQR processing starts.

Zoom In Zoom Out Reset image size

The Virgo DQR framework has evolved quite significantly over the course of O3. Partly to tune and improve the workflow based on the experience accumulated when stressing the system during the actual run, but mainly to extend the scope of the DQR by adding additional data quality checks. These new checks were either tests that had been foreseen but could not have been implemented by the start of O3, or new procedures that brought additional information that was found missing or useful when vetting real O3 triggers.

Therefore, at the end of O3, the Virgo DQR included 34 checks, for a total of 99 jobs. There are roughly three jobs per check: the first, to run the code and process the data; the second, to post-process the check results and convert them to the LIGO-Virgo common DQR format; finally, the third to upload the results back to GraceDB. Some checks included a fourth job as an initial configuration phase while others, developed specifically for the DQR, produced directly check outputs in the required DQR output format, meaning that those checks required one job less.

The Virgo DQR checks have been categorized in the following way:

• Key checksThey bring information mandatory to properly vet a candidate event. This includes: the top-level status of the detector at the time of the trigger; some time-frequency spectrograms of the GW strain data at different timescales around that time; finally, the scan of the main data quality flags available online, in order to look for any obvious problem in the data.
• Characterization of the Virgo detector noise around the time of the triggerThe noise transients (glitches) are inventoried and their potential overlap with the time-frequency extent expected for the candidate is probed, if applicable. In addition, searches for noise correlations in the time domain and noise coherences in the frequency domain are run, such as tests of noise Gaussianity and stationarity.
• Detailed Virgo statusSeveral different analysis contribute to this global picture of the instrument. All data quality flags available are checked. In addition, the DMS database is scanned to extract the snapshots closest in time to the trigger, to see what warnings or alarms were on, if any. Also, the logfiles of all the online servers running in the DAQ are scanned to spot errors that could be coincident with the trigger or impact it. Finally, various live data/reference comparison plots are generated to check the time series and distributions of a subset of the DAQ channels.
• Digest of the environment statusThis includes checking the seismic noise at EGO in various frequency bands corresponding to different sources (microseism related to sea activity on the Tuscany shoreline or local anthropogenic activities: see [7] for details), the sea activity and the weather.