Using deep learning to enhance event geometry reconstruction for the telescope array surface detector

The standard SD event reconstruction [8] is built on fitting of the individual station readings with the predefined empirical functions. The event geometry is reconstructed using the arrival times of the shower front measured by the triggered stations. The shower front is approximated by the empirical functions first proposed by J Linsley and L Scarsi [34], then modified by the AGASA experiment [35] and fine-tuned to fit the TA data in a self-consistent manner. The integral signals of the individual stations are used to estimate the particle density at distance of 800 meters from the core S₈₀₀ which plays a role of the lateral distribution profile normalization [36].

The fit of shower front and lateral distribution function is performed with 7 free parameters [37]: the position of the shower core ($x_{\mathrm{core}}$, $y_{\mathrm{core}}$), the arrival direction of the shower in the horizontal coordinates (θ, φ), the signal amplitude at the distance of 800 meters from the shower core S₈₀₀, the time offset t₀ and Linsley front curvature parameter a[ 34]. The following functions $t(\vec R)$ and S(r) are employed for the shower front and the lateral distribution correspondingly:

$$\begin{align*} t \left(\vec R \right) & = t_0 + t_{\mathrm{plane}}\left(\vec R \right) + a \times \left(1+r/R_L\right)^{1.5} LDF\left(r \right)^{-0.5},\\ S \left(r \right) & = S_{800} \times LDF\left(r \right), \end{align*}$$

where LDF(r) is the empirical lateral distribution profile and $t_{\mathrm{plane}}$ is the arrival time of the shower plane at the station with the location given by $\vec R$ in the pre-defined coordinate system of the array centered at the Central Laser Facility (CLF) [38]:

$$\begin{align*} LDF\left(r \right) & = \left(\frac{r}{R_m}\right)^{-1.2}\left(1+\frac{r}{R_m}\right)^{-(\eta-1.2)}\left(1+\frac{r^2}{R_1^2}\right)^{-0.6},\\ t_{\mathrm{plane}}\left(\vec R \right) & = \frac{1}{c} \vec n \left(\vec R - \vec R_{\mathrm{core}}\right), \end{align*}$$

where $\vec R_{\mathrm{core}}$ is the location of the shower core, $\vec n$ – unit vector towards the direction of the arrival of a primary particle, c is the speed of light and r is a distance from the station to the shower core:

$$\begin{align*} r & = \sqrt{(\vec R - \vec R_{core})^2 - (\vec n (\vec R - \vec R_{core}))^2},\\ R_m & = 90.0 \mathrm{m}, R_1 = 1000 \mathrm{m}, R_L = 30 \mathrm{m},\\ \eta & = 3.97 - 1.79 \left(\sec\left(\theta\right) -1\right). \end{align*}$$

After the fit is performed, the primary particle energy is estimated as a function

$$\begin{equation*} E = E_{SD}(S_{800}, \theta) \end{equation*}$$

of the density S₈₀₀ at the distance of 800 m from the shower core and the zenith angle θ using the lookup table obtained with the Monte Carlo simulation [8].

In this work we use the full Monte Carlo simulation of TA SD events induced by the protons and nuclei for both the standard and the machine learning-based reconstructions. Primary energies are distributed assuming the spectrum of the HiRes experiment [39] in $10^{17.5}-10^{20.5}$ eV energy range. The events are generated by CORSIKA version 7.3500 [40] with the EGS4 [41] model for the electromagnetic interactions, QGSJET II-03 [42] and FLUKA version 2 011.2b [43] for high- and low-energy hadronic interactions. The thinning and dethinnig procedures with parameters described in (Stokes:2011wf) are used to reduce the calculation time. Both proton and nuclei event sets are sampled assuming isotropic primary flux with zenith angles $\theta \lt 60^\circ$. The total of 12642 CORSIKA events were simulated for primary protons, 8549 for helium, 8550 for nitrogen and 8715 for iron. The showers from the proton and the nuclei libraries are processed by the code simulating the real time calibration detector response by means of GEANT4 package (Agostinelli:2002hh). Each CORSIKA event is thrown to the random locations and azimuth angles within the SD area multiple times. The Monte-Carlo simulations are described in details in references (AbuZayyad:2012ru, 2014arXiv1403.0644T, Matthews:2017waf), (AbuZayyad:2012ru, 2014arXiv1403.0644T, Matthews:2017waf), (AbuZayyad:2012ru, 2014arXiv1403.0644T, Matthews:2017waf). We apply the standard anisotropy quality cuts [48] to the simulated events in the same way they are applied to the data. The standard cuts include the constraint on the reconstructed zenith angle $\theta_{rec} \lt 55^\circ$. The total number of the simulated events which pass the quality cuts is about 1.3 million.

In essence the method is using the machine learning technique to construct the inverse function for the full detector Monte Carlo simulation. The latter allows to calculate the raw observables as a function of primary particle properties. In case of the Telescope Array surface detector the raw observables are time-resolved signals for the set of the adjacent triggered detectors. The application of this approach to the ultra-high-energy cosmic-ray experiment was first demonstrated in reference [33] on toy Monte Carlo simulation roughly following the geometry of Auger observatory. In this work we use the full detector Monte Carlo simulation [8] of the TA observatory to produce the time-resolved signals from the two layers of SD stations. The inverse function is constructed by means of a multi-layer feed-forward artificial neural network (NN). This class of parameterized algorithms is known to be able to approximate any continuous function with any finite accuracy [49]. The NN free parameters, weights, are tuned in the so-called training procedure which is in essence the error minimization on a set of training examples.

In practice, the achievable accuracy is often limited by the stochastic nature of the problem introducing unavoidable bias. In case of EAS development the main source of the stochastic uncertainty is perhaps the first particle interaction which occurs at a random depth. In this work we try to enhance the existing event reconstruction procedure. To rate the relative performance of the enhanced reconstruction with respect to the baseline one the explained variance score could be used

$$\begin{equation} EV(y, \hat{y}) = 1 - \frac{Var\{y - \hat{y}\}}{Var\{y\}} \end{equation} \tag{ 1 }$$

Here y is the true value of the quantity being predicted, which can also be interpreted as the error of the baseline model (zero approximation), and $\hat{y}$ is our estimate of y, i.e. the correction to zero approximation calculated in the enhanced model. The variance in 1 is calculated over the entire test data set. The ideal reconstuction would have EV = 1, while the presence of unavoidable bias means that EV is limited from above by some value $EV_{\textrm{max}}\lt 1$. If an enhanced reconstruction performs better than the baseline one, then EV > 0. The systematic uncertainty, e.g. hadronic interaction model at ultra-high energies, does not constrain the NN accuracy nominally, however it should be taken into account when interpreting the NN model output. Other than that, the accuracy of the NN model is limited just by the complexity of the network, i.e. the number of trainable weights, with more complex networks requiring more data to train.

For our purpose we utilize the convolutional NN [50], a special kind of feed-forward neural network which is known to perform well in image or sequence processing. Their superior performance compared to other feed-forward networks, such as multi-layer perceptron (MLP) is explained by their ability to efficiently extract and combine local image features on different scales using less free parameters (trained weights). This is achieved by limiting the amount of the surrounding cells to which the next layer neurons are connected and reusing the same weights for different parts of the image. A convolution layer is described by the kernel of fixed size which is less or equal to the size of image. If the size of kernel is equal to the size of image the convolution layer is equivalent to the fully connected perceptron layer. In this work have tried different kernel sizes including the full size of image (or sequence for waveform features extraction).

In figure 2 we show the optimal NN architecture which we arrived to. The typical UHECR event triggers from 5 to 10 neighbour stations. The readouts from 4 × 4 or 6 × 6 stations around the event core are used as an input with each pixel corresponding to a particular station. There are two time-resolved signals, one per layer, for each station. The typical length of the signal recorded with 20 ns time resolution does not exceed 256 points. An example input event is shown in figure 3. The overall dimensionality of the raw input data is therefore 4 × 4 × 256 × 2 or 6 × 6 × 256 × 2. The position of the event core is estimated using the standard reconstruction.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

The full time-resolved signal from the two scintillator layers of each surface detector station is first converted to a 28-dimensional feature vector using waveform encoder, consisting of six convolutional layers followed by the max-pooling. The number of the waveform features was one of the parameters being optimized with the range from 8 to 64. The waveform encoder kernel size was varied in the range 3-8. In addition, for the waveform feature extraction we tried to use single or double layer perceptron which resulted in worse performance. The scaled exponential linear units are used as the activations to achieve the self-normalizing property [51]. Slightly inferior accuracy was achieved with the ReLU activation function.

The extracted features from the waveforms are treated as a multichannel image using a sequence of 2D convolution layers. We also add several extra station (or station signal) properties S to the set of the extracted signal features before feeding them into the 2D convolution model. These properties include 3 detector coordinates in form of an offset from the perfect rectangular grid, the detector on/off state and saturation flags, integral signal and signal onset time relative to the plane shower front.

In order to compensate the missing or switched off detectors, we have introduced a special Normalize layer, somewhat similar to the Dropout[ 52]. Namely, it drops the pixels corresponding to the missing detectors and multiplies the activations of the present detectors by a factor of $N_{total}/(N_{total}-N_{missing})$. Unlike Dropout the above layer works exactly the same way in training and inference mode. We found that this trick enhances the explained variance (1) on the test data by few percent.

The 2D-convolutional model part consists of two blocks built from the two convolutional layers with 3 × 3 and 2 × 2 kernels followed by max-pooling. Kernel sizes were varied from 2 to 4 and the number of convolutions between successive pooling operations was varied from 1 to 3. The input signal in each convolutional block is concatenated with the output before pooling operation, which facilitates reusing the extracted features at different scales. This trick gives another few percent enhancement in terms of the explained variance. We also tried to replace classic convolution blocks with the separable convolution as it was suggested in [33] and found no positive effect on the accuracy.

The 2D convolutional model outputs a feature vector which is then processed by the two fully connected layers, the latter being the output layer. The optimal number of neurons in the first dense layer was chosen from the range between 16 and 128. Before the feature vector is passed to the first dense layer, it is concatenated with N extra event properties, such as the year, the season, the time of day ¹ and (optionally) the standard reconstruction data (e.g. S₈₀₀). At this step we have found it useful to add a set of 14 composition sensitive synthetic observables calculated in the standard reconstruction [53]. The event observables include Linsley shower front curvature parameter, area-over-peak, S_b parameter of the lateral distribution function, the sum of the signals of all stations of the event, total number of peaks in all the waveforms, number of waveform peaks in the station with the largest signal, asymmetry of the signal in the upper and the lower scintillator layers, see the appendix of [53] for a complete list. The output may contain one or several observables. The mean square error is used as a loss function. In this work we use three components of the arrival direction unit vector X, Y and Z as the output variables. We also tried to use zenith and azimuth angles φ and θ for the output and found it less efficient. We suppose that X, Y and Z is a better choice to use for the angular distance minimization with the mean square error loss, since in this case for small errors the loss is proportional to the average square of the angular distance between the true and predicted directions, while in case of spherical coordinates there is a degeneracy in φ for small zenith angles θ.

The raw waveform data is converted to the log scale before it is passed as an input to the model. The rest of the input and output data are normalized to ensure their zero mean and unit variance. Finally, for the arrival direction reconstruction we have found it useful to predict the correction to the standard reconstruction instead of its absolute value.

The optimal model shown in figure 2 has about 120 thousand adjustable weights, which we optimize using the adaptive learning rate method Adadelta [54]. We split the data into three parts: 80% : 10% : 10% for training, test and validation purposes. The training set consists of roughly 1 million events. Batch gradient decent technique is used with batch size of 1024 samples roughly 1000 batches are passed to training algorithm during single training step (so called epoch). The model is evaluated on the validation data at the end of each epoch. Training stops if the loss function on the validation data is not improving for more than 5 epochs. The optimization is performed for at most 400 epochs, however usually it stops after less than 100 epochs. We tried to apply several regularization techniques, such as L2-regularisation, noise admixture and α-dropout [51] and eventually found the early-stop procedure to be sufficient to avoid overfitting. This is expected result for the model proposed since number of free model parameters is roughly 10 times less than the number of training samples.

To justify the choice of the model design we have also trained and evaluated the models not using event features and/or waveform data. The model performance in terms of the average, $(EV_{X} + EV_{Y} + EV_{Z})/3$, explained variance is shown in table 1. We see that even minimalistic model using only detector features, of which the most important are integral signal and timing, still improves reconstruction accuracy, but utilizing the raw waveform data gives the largest impact on the accuracy. The NN model is implemented in Python using Keras[ 55] library with the Tensorflow backend. We also use hyperopt package [56] to optimize model hyperparameters, such as dimensionality of the waveform encoder output, the shapes of the convolution kernels and the dense layer size.