Chasing the Tail of Cosmic Reionization with Dark Gap Statistics in the Lyα Forest over 5 < z < 6

This study is based on spectra of 55 QSOs at 5.5 ≲ z_em ≲ 6.5 taken with the X-Shooter spectrograph on the Very Large Telescope (VLT; Vernet et al. 2011) and the Echellette Spectrograph and Imager (ESI) on Keck (Sheinis et al. 2002). Of these, 23 X-Shooter spectra are from the XQR-30 VLT Large Programme. The XQR-30 program is targeting 30 bright QSOs at 5.8 ≲ z ≲ 6.6 for the study of reionization and other aspects of the early Universe. The full data set will be described in V. D'Odorico et al. (in preparation). The 23 objects out of the XQR-30 sample selected for this project are those that meet our S/N threshold and do not contain strong BAL features. In addition, we use 30 spectra reduced from archival X-Shooter and ESI data, of which 27 are from the sample of Becker et al. (2019). Recent deep (20 hr) X-Shooter observations (PI: Fuyan Bian) of the lensed z = 6.5 QSO J0439+1634 are also included in the dark gaps statistics. Finally, we acquired a deep (7 hr) ESI spectrum of SDSS J1250+3130. Observations for all objects except SDSS J1250+3130 were taken without any foreknowledge of dark gaps in the Lyα forest. In the case of SDSS J1250+3130, we targeted the QSO based on indications from a shallower (1 hr) ESI spectrum that its spectrum contained a long dark gap in the Lyα forest. We discuss the impact of including this object on our results in Section 3.4.

Details of the data reduction are given in Becker et al. (2019). Briefly, we used a custom pipeline that includes optimal techniques for sky subtraction (Kelson 2003) and one-dimensional spectral extraction (Horne 1986). Telluric absorption corrections were computed for individual exposures using models based on the Cerro Paranal Advanced Sky Model (Noll et al. 2012; Jones et al. 2013). The spectra were extracted using 10 km s⁻¹ pixels for the VIS arm of X-Shooter and 15 km s⁻¹ pixels for ESI. Typical resolutions for the X-Shooter and ESI are FWHM ≈ 25 km s⁻¹ and 45 km s⁻¹, respectively. In addition, for J0439+1634, to reduce the contamination from continuum emission of a foreground galaxy, we fit a power law of the flux zero-point over the Lyα forest and subtract it from the flux. The spectra are plotted in Figure 1.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

View figure set (55 panels)

Tarball of figure set images

We adopt QSO redshifts measured from CO, [C ii] 158 μm, or Mg ii lines if available. Otherwise we use redshifts inferred from the apparent start of the Lyα forest, following Becker et al. (2019). Table 1 summarizes QSO spectra used in this work with the QSO redshifts, instruments, and estimated signal-to-noise ratios, which is calculated as the median ratio of unabsorbed QSO continuum to noise per 30 km s⁻¹ near 1285 Å in the rest frame.

Table 1. QSO Spectra Used in This Work

No.	QSO	${z}_{\mathrm{em}}^{[\mathrm{Ref}.]}$	Source	Instrument	S/N
(1)	(2)	(3)	(4)	(5)	(6)
1	J2207-0416	5.529 b	archival (B19)	X-Shooter	42
2	J0108+0711	5.577 b	archival (B19)	X-Shooter	29
3	J1335-0328	5.693 b	archival (B19)	X-Shooter	30
4	SDSSJ0927+2001	5.7722 c	archival (B19)	X-Shooter	76
5	SDSSJ1044-0125	5.7847 o	other archival	ESI	71
6	PSOJ065+01	5.790 q	XQR-30	X-Shooter	47
7	PSOJ308-27	5.794 q	XQR-30	X-Shooter	58
8	SDSSJ0836+0054	5.810 g	other archival	ESI	152
9	PSOJ004+17	5.8165 e	other archival	X-Shooter	21
10	SDSSJ0002+2550	5.820 b	archival (B19)	ESI	93
11	PSOJ242-12	5.834 q	XQR-30	X-Shooter	28
12	SDSSJ0840+5624	5.8441 n	archival (B19)	ESI	41
13	SDSSJ0005-0006	5.847 b	archival (B19)	ESI	24
14	PSOJ025-11	5.849 q	XQR-30	X-Shooter	53
15	PSOJ183-12	5.857 q	XQR-30	X-Shooter	66
16	SDSSJ1411+1217	5.904 g	archival (B19)	ESI	46
17	PSOJ108+08	5.950 q	XQR-30	X-Shooter	70
18	PSOJ056-16	5.9670 e	archival (B19)	X-Shooter	35
19	PSOJ029-29	5.981 q	XQR-30	X-Shooter	51
20	SDSSJ0818+1722	5.997 b	archival (B19)	X-Shooter	108
21	ULASJ0148+0600	5.998 b	archival (B19)	X-Shooter	126
22	PSOJ340-18	5.999 b	archival (B19)	X-Shooter	32
23	PSOJ007+04	6.0008 d	XQR-30	X-Shooter	53
24	SDSSJ2310+1855	6.0031 o	XQR-30	X-Shooter	81
25	SDSSJ1137+3549	6.007 j	archival (B19)	ESI	28
26	ATLASJ029.9915-36.5658	6.021 b	XQR-30	X-Shooter	48
27	SDSSJ1306+0356	6.0330 k	archival (B19)	X-Shooter	71
28	J0408-5632	6.035 q	XQR-30	X-Shooter	71
29	ULASJ1207+0630	6.0366 d	archival (B19)	X-Shooter	25
30	SDSSJ2054-0005	6.0391 o	archival (B19)	ESI	29
31	PSOJ158-14	6.0681 e	XQR-30	X-Shooter	59
32	SDSSJ0842+1218	6.0763 d	XQR-30	X-Shooter	71
33	SDSSJ1602+4228	6.079 j	archival (B19)	ESI	34
34	PSOJ239-07	6.1098 e	XQR-30	X-Shooter	65
35	CFHQSJ1509-1749	6.1225 d	archival (B19)	X-Shooter	54
36	SDSSJ2315-0023	6.124 b	archival (B19)	ESI	25
37	ULASJ1319+0950	6.1330 o	archival (B19)	X-Shooter	86
38	SDSSJ1250+3130	6.137 j	new observation	ESI	53
39	VIKJ2318-3029	6.1458 d	archival (B19)	X-Shooter	21
40	PSOJ217-16	6.1498 d	XQR-30	X-Shooter	68
41	PSOJ217-07	6.165 q	XQR-30	X-Shooter	42
42	PSOJ359-06	6.1718 e	XQR-30	X-Shooter	67
43	PSOJ060+24	6.177 q	XQR-30	X-Shooter	53
44	PSOJ065-26	6.1877 d	XQR-30	X-Shooter	73
45	PSOJ308-21	6.2341 d	archival (B19)	X-Shooter	26
46	SDSSJ1030+0524	6.309 f	archival (B19)	X-Shooter	35
47	SDSSJ0100+2802	6.3270 l	archival (B19)	X-Shooter	212
48	ATLASJ025.6821-33.4627	6.3373 k	archival (B19)	X-Shooter	61
49	J1535+1943	6.381 q	XQR-30	X-Shooter	30
50	SDSSJ1148+5251	6.4189 h	archival (B19)	ESI	64
51	J1212+0505	6.4386 d	XQR-30	X-Shooter	41
52	J0439+1634	6.5188 p	new observation	X-Shooter	224
53	VDESJ0224-4711	6.5223 m	XQR-30	X-Shooter	29
54	PSOJ036+03	6.541 a	archival (B19)	X-Shooter	38
55	PSOJ323+12	6.5881 i	XQR-30	X-Shooter	30

Notes. Columns: (1) QSO index number; (2) QSO name; (3) QSO redshift with reference; (4) source of the spectrum used for dark gap statistics; (5) instrument used for dark gap statistics; (6) continuum signal-to-noise ratio per 30 km s⁻¹ near rest wavelength 1285 Å. Sources of the spectra are as follows. XQR-30: spectra from the XQR-30 program; new observation: spectra from new observations; archival (B19): archival spectra used and reduced in Becker et al. (2019); other archival: spectra from the public archives but not included in Becker et al. (2019). Redshift lines and their references are described below.

^a [C ii] 158 μm: Bañados et al. (2015). ^b Apparent start of the Lyα forest: Becker et al. (2019). ^c CO: Carilli et al. (2007). ^d [C ii] 158 μm: Decarli et al. (2018). ^e [C ii] 158 μm: Eilers et al. (2020). ^f Mg ii: Jiang et al. (2007). ^g Mg ii: Kurk et al. (2007). ^h [C ii] 158 μm: Maiolino et al. (2005). ⁱ [C ii] 158 μm: Mazzucchelli et al. (2017). ^j Mg ii: Shen et al. (2019). ^k [C ii] 158 μm: Venemans et al. (2020). ^l [C ii] 158 μm: Wang et al. (2019). ^m [C ii] 158 μm: Wang et al. (2021). ⁿ CO: Wang et al. (2010). ^o [C ii] 158 μm: Wang et al. (2013). ^p [C ii] 158 μm: Yang et al. (2019). ^q Apparent start of the Lyα forest: this work.

Download table as:  ASCIITypeset image

The detection of dark gaps relies on the construction of the intrinsic continuum over the Lyα forest. In order to estimate QSO continua blueward of the Lyα emission line, we use Principal Component Analysis (PCA), which is less biased than the conventional power-law fitting (e.g., Bosman et al. 2021b). In this work, we apply the log-PCA method of Davies et al. (2018b) as implemented in the Lyα forest portion of the spectrum by Bosman et al. (2021a), with 15 and 10 components used for the red-side (rest-frame wavelength λ₀ > 1230 Å) and blue-side (λ₀ < 1170 Å) continua, respectively. For each QSO, we fit the red-side continuum with principal components, and map the corresponding red principal component coefficients to the blue side coefficients with a projection matrix. For X-Shooter spectra with observations from the NIR arm, we fit the red continuum over 1230 < λ < 2000 Å in the rest frame.

The ESI spectra are fitted using an optical-only PCA, which is presented in Bosman et al. (2021b). QSOs with strong broad absorption lines (BALs) in their spectra were excluded from our sample. For QSOs with mild absorption features that interfere minimally with the Lyα forest, we mask out the absorption lines when fitting their spectra. In addition, we intentionally leave out the Lyα emission peak and the proximity zone when fitting and predicting the continuum on account of the large object-to-object variations in these regions. The typical 1σ uncertainty of the PCA continuum fitting over the Lyα forest is less than 10%. Continuum fits and blue-side predictions are shown in Figure 1 along with the QSO spectra. We also verify that our dark gap statistics results do not significantly change if we use power-law continua (see Appendix C), which have a typical bias of ∼10% over the Lyα forest (Bosman et al. 2021b).

We define a dark gap to be a continuous spectral region in which all pixels binned to 1h⁻¹ Mpc have an observed normalized flux F = F_obs/F_c < 0.05, where F_obs is the observed flux and F_c is the continuum flux. The minimum length of a dark gap is 1h⁻¹ Mpc. We apply this definition when searching for dark gaps in both the real data and the mock spectra. The bin size and flux threshold were chosen to enable a uniform analysis over our large sample of spectra. A bin size of 1h⁻¹ Mpc (corresponding to a velocity interval of Δv ≃ 150 km s⁻¹ at z = 5.6) provides a convenient scale that preserves most of the structure of the Lyα forest. The choice of the flux threshold F_t is mainly restricted by the quality of the data. Our choice of F_t = 0.05 corresponds to nondetection of transmission lower than approximately twice the binned flux error (2σ) in the spectrum with the lowest S/N in our sample. Using such a threshold, all dark gaps longer than 30h⁻¹ Mpc have τ_eff > 4. ¹⁷ We have tested that using 0.1 or 0.025 for the flux threshold does not change our conclusions fundamentally when applying the same criteria to both the observed and mock spectra. Setting F_t = 0.1 tends to yield dark gaps that are less opaque, while setting F_t = 0.025 would decrease the number of usable QSO sightlines from 55 to 37.

In order to avoid the QSO proximity region, we identify dark gaps in the Lyα forest starting from 7 proper Mpc (pMpc) blueward from the QSO, which is close to the size of the largest proximity zones of bright QSOs at these redshifts (Eilers et al. 2017, 2020). On the blue end, we limit our search to greater than 1041 Å in the rest frame in order to avoid contamination from associated Lyβ or O vi absorption (e.g., Becker et al. 2015). For the purpose of comparing our results to simulations, we wish to avoid dark gaps that may be truncated by transmission peaks within the proximity zone. When quantifying the fraction of lines of sight that intersect gaps of length L ≥ 30h⁻¹ Mpc (Sections 3.4 and 3.5), the highest redshift at which we register an individual sightline that shows a long gap, if any, is therefore 30h⁻¹ Mpc blueward of our proximity zone cut, although the gap may include pixels that extend up to the proximity zone. Nevertheless, we still record the full lengths of gaps extending to this 30h⁻¹ Mpc "buffer zone" when searching for the longest possible dark gaps in both data and simulations. Dark gaps completely located in the QSO proximity zone and/or in this "buffer zone," however, are discarded. This ensures that the pixel at the red end of each sightline may intersect a long (L ≥ 30h⁻¹ Mpc) dark gap. ¹⁸ Finally, we limit our analysis to z < 6 because the mean transmitted flux at z > 6 is so low that most spectra show long dark gaps, making the dark gap statistics less informative.

We note that there is no perfect way to handle the proximity zone effect. It is difficult to precisely define and measure the proximity zone size for each QSO, which partly motivates our choice to use a fixed proximity zone cut. The proximity zone for the brightest QSOs in our sample (e.g., SDSS J0100+2802 and VDES J0224+4711) may be larger than our adopted cut of 7 pMpc. Fortunately, the use of an additional 30h⁻¹ Mpc buffer zone minimizes the potential effect of the larger proximity zone of these objects. In addition, because we limit our statistics over 5 < z < 6, proximity zone transmission at z > 6 toward some extremely bright QSOs does not impact our results. Still, one should treat dark gaps near the QSO proximity zone with caution.

Noisy residuals from skyline subtraction and telluric correction may divide an otherwise continuous region of depressed flux. To deal with this, when searching for dark gaps we mask out ±75 km s⁻¹ intervals of the spectra centered at peaks in the flux error array, which typically correspond to skyline residuals. The exception to this is that we do not mask out any pixels with F > 3σ_F . For consistency, we apply the same masking procedure to the mock spectra. ¹⁹ In Appendix B, we use the mock spectra to show that such masking only produces a minor change in the results. We also test that the impact of masking telluric correction residuals near 7600-7650 Å is neglectable.

As for contamination from damped Lyα systems (DLAs) or other metal-enriched absorbers, we made no correction for their effect on dark gap detection following, e.g., Fan et al. (2006). Even strong DLAs can hardly, on their own, produce dark gaps as long as 30 h⁻¹ Mpc, which are the primary focus of this work. Nevertheless, in the results we label dark gaps with intervening metal systems for reference based on the systems identified by Chen et al. (2017) and Becker et al. (2019), as well as our own inspections. We visually searched all L ≥ 30h⁻¹ Mpc dark gaps for metal absorbers not listed in the literature. The systems were identified via the coincidence of multiple metal lines in redshift. The metal lines we used include C ii λ1334, C iv λ λ1548, 1550, O i λ1302, Mg ii λ λ2797, 2803, Al ii λ1670, Si ii λ1527, and Si iv λ λ1394, 1403. A detection required these metal lines (if available) to have significant absorption features and self-consistent velocity profiles at the same redshift. We have a good wavelength coverage for most metal lines mentioned above in QSO spectra taken with X-Shooter. Even for these objects, however, we caution that the list of metal absorbers may be still incomplete. A full list of metal absorbers in the XQR-30 spectra will be presented by R. Davies et al. (2021, in preparation). We also note that the simulations we used do not include DLAs or other metal-enriched absorbers.

Long dark gaps play an important role in characterizing the IGM in the later stages of reionization. Among 50 dark gaps with L ≥ 30h⁻¹ Mpc detected in our sample, Figure 2 displays some notable examples. They either extend down to or below z ∼ 5.5, are extremely long (L > 80h⁻¹ Mpc), or both.

Zoom In Zoom Out Reset image size

Two long dark gaps entirely at z < 5.5 are identified toward PSO J183-12 and PSOJ340-18. They span z_gap = 5.36 − 5.47 and z_gap = 5.31 − 5.42, corresponding to lengths of L = 37h⁻¹ Mpc and L = 34h⁻¹ Mpc, respectively. Most spikes and sharp dips with negative flux in the unbinned spectra inside the two gaps are skyline subtraction residuals, as indicated by the peaks in the flux error array. Both dark gaps are highly opaque, with τ_eff > 6. The spectra of both QSOs have a good coverage of redshifted common metal lines. We searched their X-Shooter VIS and NIR spectra and found no metal absorption within the redshift ranges of the dark gaps. In addition, a 30h⁻¹ Mpc dark gap extending just above z = 5.5 is identified toward SDSS J1250+3130. Most of the spikes inside this gap are also probably due to sky lines as indicated by peaks in the flux error array.

The fourth through sixth rows in Figure 2 display three examples of long dark gaps extending down to z ∼ 5.5. The long gap extending to z = 5.46 with a length of L = 68h⁻¹ Mpc toward PSOJ025-11 is one of the longest troughs below redshift six discovered in this work. The only weak transmission peaks in the unbinned flux array that seem to be real are the ones at z_abs ≃ 5.47, 5.48, and 5.67. Overall, however, it is extremely dark, with τ_eff ≥ 6.4. We also reproduce the detection of the long trough discovered toward ULAS J0148+0600 by Becker et al. (2015), which extends down to z = 5.5 with a total length L > 110h⁻¹ Mpc. Due to the use of a different definition of dark gap compared to Becker et al. (2015), the trough detected in this work includes an additional small transmission peak that appears in the unbinned spectrum near the blue end. This yields a slightly larger L but a comparable τ_eff value. We also find a gap of L = 81h⁻¹ Mpc extending down to z = 5.64 toward SDSS J1250+3130. Spikes within the trough are skyline subtraction residuals, as shown by peaks in the error array. We do not see any strong metal absorbers that would indicate dense absorption systems such as DLAs or Lyman limit systems (LLSs), in any of these gaps. Finally, we find dark gaps longer than 110h⁻¹ Mpc toward several QSOs with the highest redshifts in our sample. This is not surprising because the IGM is more neutral at higher redshifts and therefore more likely to produce large Lyα opaque regions. For example, Barnett et al. (2017) identified a 240h⁻¹ Mpc gap at z > 6.1 toward the z = 7.1 QSO ULAS J1120+0641. Here we display a remarkably long dark gap toward PSO J323+12. It covers z_gap = 5.88–6.43 and has a length of 154h⁻¹ Mpc, as shown in the bottom row of Figure 2.

For reference, we overplot in green the regions of spectra corresponding to Lyβ for the Lyα shown in Figure 2. ²⁰ In many cases, the Lyβ forest also includes higher order Lyman series absorption, as indicated in the figure. Although dark gaps are highly opaque to Lyα, there are often narrow transmission peaks corresponding to Lyβ. These peaks demonstrate that the dark gaps in Lyα typically cannot arise from continuous regions of neutral gas, which would be highly opaque to all Lyman series lines. Broken regions of neutral gas may still be present, however, with the Lyβ transmission corresponding to gaps between neutral sections (e.g., Keating et al. 2020b; Nasir & D'Aloisio 2020).

In total, we detected 1329 dark gaps from the sample, of which 50 have a length of L ≥ 30h⁻¹ Mpc. Properties of all dark gaps detected are summarized in Table 2. Details on dark gap detection for each QSO sightline are shown in Figure 1.

As an overview, Figure 3 plots all dark gaps identified in this work according to their central redshift and length. Dark gaps with associated metal absorbers are labeled in red. This figure has excluded dark gaps that are completely inside the 7 pMpc proximity zone and/or inside the 30h⁻¹ Mpc "buffer zone" beyond the proximity zone. Not surprisingly, as redshift increases, there are more long dark gaps and a larger scatter in dark gap length. The lowest-redshift gaps with L ≥ 30h⁻¹ Mpc appear around z = 5.3.

Zoom In Zoom Out Reset image size

Figure 4 displays the Lyα forest coverage and all dark gaps identified for every line of sight in our sample. At z ≲ 5.2, most QSO sightlines are highly transmissive; a few gaps with L ∼ 10–20 h⁻¹ Mpc appear but these tend to contain metal absorbers and are likely to be DLAs. Dark gaps longer than 30h⁻¹ Mpc appear in the sightlines of PSO J340-18 and PSO J183-12 at z ≃ 5.3 and 5.4, respectively. The frequency of long dark gaps increases with redshift, such that most lines of sight at z ≃ 5.8 show gaps longer than 30h⁻¹ Mpc in the Lyα forest. Interestingly, the J1535+1943 sightline is relatively transmissive at z ∼ 6 compared to others at the same redshift. Although J1535 has a reddened spectrum, the continuum reconstruction is acceptable and most of the transmission peaks in the Lyα forest appear to be real.

Zoom In Zoom Out Reset image size

We introduce the fraction of QSO spectra exhibiting long (L ≥ 30h⁻¹ Mpc) gaps as a function of redshift, F₃₀(z), as a new Lyα forest statistic. As mentioned in Section 3.1, in order to deal with the finite length of the spectra for this statistic we cut off each QSO sightline at the blue edge of the 30h⁻¹ Mpc buffer zone. F₃₀ quantifies how common the large Lyα-opaque regions are and how they evolve with redshift. We choose 30h⁻¹ Mpc because we found that this length most effectively distinguishes between the models described in Section 4, especially between the homogeneous-UVB and other models. The comparison of the dark gap length distribution, P(L), predicted by different models in Section 4.5 also implies that dark gaps with L ≥ 30h⁻¹ Mpc are potentially good probes for H i if the late reionization scenario is indeed preferred. We note that we include all long dark gaps regardless of the presence of associated metal absorbers since the dense absorption systems alone are not likely to create troughs longer than 30h⁻¹ Mpc.

Figure 5 displays the evolution of F₃₀ with redshift measured from the QSO spectra. The result is averaged over Δz = 0.02 bins. The mean, 68% limits, and 95% limits of F₃₀ are calculated based on 10,000 bootstrap resamplings of the whole sample. In each realization, we randomly select 55 QSO spectra, with replacement, and add up the number of sightlines showing L ≥ 30h⁻¹ Mpc dark gaps at a given redshift. The total is then normalized by the number of QSO sightlines at each redshift, which yields F₃₀(z) for this realization. F₃₀ starts to be nonzero from z ≃ 5.3 and increases strongly with redshift. At z = 6, ∼90% of sightlines present long gaps.

Zoom In Zoom Out Reset image size

We noted above that a deep spectrum of SDSS J1250+3130 was obtained based on preliminary indications from shallower data of a long gap in its spectrum. This is the only QSO in the sample for which the selection is related to the foreknowledge of dark gaps. We include J1250 for completeness, but note that excluding this line of sight from our sample would only decrease (increase) F₃₀ by ≲ 0.02 (0.05) over 5.50 < z < 5.90 (5.90 < z < 5.93).

Finally, we test whether metal absorbers could be linking adjacent dark gaps in a way that would impact our F₃₀ statistic. For this, we calculate a "pessimistic" F₃₀ by dividing dark gaps at the redshifts of DLAs and other metal systems (Appendix D). The resulting change in F₃₀ is minor, with a maximum decrease of ∼0.1 at z ∼ 5.8. The differences between the observations and model predictions (Section 5.1; Figure D1) can still be well distinguished. We therefore conclude that this potential impact of metal absorbers on F₃₀ is not significant.

In addition to F₃₀, we investigate the cumulative distribution function (CDF) of gap length, P( < L). Figure 6 plots P( < L) in redshift bins of Δz = 0.2. Dark gaps are assigned to a bin based on the central redshift of the gap, and we do not truncate gaps extending beyond the edges of the redshift windows. We treat the dark gaps truncated by the 7 pMpc proximity zone cut by plotting the most pessimistic and optimistic bounds on P( < L). The pessimistic bound is calculated by considering the lengths of dark gaps are as measured. The optimistic bound, however, is given by assuming the lengths of truncated dark gaps are infinite, which indicates the most extreme dark gap length possible in the absence of the QSO. In the latter case, we still use the measured central redshift of each dark gap to assign it to a redshift bin.

Zoom In Zoom Out Reset image size

Figure 6 demonstrates that longer dark gaps become more common toward higher redshifts. This is consistent with the result of F₃₀. Moreover, similar to the rapid redshift evolution in F₃₀ near z ≃ 5.7, P( < L) shows a large change between 5.5 < z < 5.7 and 5.7 < z < 5.9.

To test the effects of metal absorbers on P( < L), we calculate the distribution by excluding dark gaps with known associated metal absorbers. We find the difference is minor. The maximum increment on the most pessimistic P( < L) over 5.7 < z < 5.9 is less than 0.03, and the difference is less than 0.005 over the other redshift bins.

We first include a baseline model, wherein reionization is fully completed at z > 6 and the UVB is spatially uniform. For this, we use a run from the Sherwood simulation suite, which successfully reproduces multiple characteristics of the observed Lyα forest over 2.5 < z < 5 (Bolton et al. 2017). The Sherwood suite uses a homogeneous Haardt & Madau (2012) UV background. Reionization occurs instantaneously at z = 15, allowing the IGM to fully relax hydrodynamically by z = 6. The simulations were run with the parallel smoothed particle hydrodynamics code P-GADGET-3, which is an updated and extended version of GADGET-2 (Springel 2005). We use the simulation with 2 × 2048³ particles and box size of ${\left(160\,{h}^{-1}\,\mathrm{Mpc}\right)}^{3}$ to build mock spectra for the homogeneous-UVB model, as described in Section 4.4.

We use two sets of models wherein reionization continues significantly below redshift six. In these models, long dark gaps in Lyα transmission at z < 6 arise from a combination of neutral islands and regions of suppressed UVB, which are often adjacent to one another.

The first late reionization models are from Keating et al. (2020a). They include three models with different ionization and/or thermal histories. We denote the fiducial model as K20-low- τ_CMB, wherein the volume-filling fraction of ionized gas reaches 95% at z = 5.6 and 99.9% at z = 5.2. Two other runs, the K20-low- τ_CMB -hot and K20-high- τ_CMB models, are also included. Briefly, the K20-low- τ_CMB -hot model uses a higher temperature for the input blackbody ionizing spectrum, namely T = 40,000 K instead of T = 30,000 K as used in the K20-low- τ_CMB model. They have a volume-weighted mean temperature at the mean density at z = 6 of T₀ ≃ 10,000 K and 7000 K, respectively. The K20-high- τ_CMB model shares a similar IGM thermal history with the K20-low- τ_CMB model, but it has an earlier reionization midpoint of z_mid = 8.4.

The K20 simulations are modified versions of the late reionization model published in Kulkarni et al. (2019a). The model was modified, such that (i) the IGM temperature evolution is in better agreement with recent observations (Boera et al. 2019; Walther et al. 2019; Gaikwad et al. 2021), and (ii) the mean Lyα transmission is in better agreement with data at z < 4.7 (Becker et al. 2015). The ionization state of the IGM is modeled using the radiative transfer code ATON (Aubert & Teyssier 2008, 2010) that postprocesses underlying hydrodynamic simulations performed with P-GADGET-3. The simulations use the identical initial condition and box size of the Sherwood simulation suite. The radiative transfer, however, leads to an extended and self-consistent reionization history. This produces scatter in the Lyα τ_eff. The simulations also contain fluctuations in temperature and photoionization rates. A light cone from the radiative transfer simulation was extracted on the fly. Using sightlines through this light cone, Keating et al. (2020a) computed the optical depths continuously spanning 4.0 ≲ z ≲ 7.5 for each model, which allows us to avoid having to do any interpolation.

The second set of late reionization models is from Nasir & D'Aloisio (2020). In these models, the volume-filling factor of ionized gas reaches 95% at z = 5.3–5.4. As in the Keating et al. (2020a) models, fluctuations in both the UVB and temperature are present. The UVB fluctuations are driven by a short and spatially variable mean free path, similar to the model in Davies & Furlanetto (2016). In the two Nasir & D'Aloisio (2020) models, which we denote as ND20-late-longmfp and ND20-late-shortmfp, the volume-weighted average mean free path for 912 Å photons at z = 5.6 is $\langle {\lambda }_{\mathrm{mfp}}^{912}\rangle =30\,{h}^{-1}\,\mathrm{Mpc}$ and 10h⁻¹ Mpc, respectively. As a result of the shorter mean free path, ND20-late-shortmfp contains stronger fluctuations in the UVB. The shorter $\langle {\lambda }_{\mathrm{mfp}}^{912}\rangle$ is also more consistent with the recent mean free path measurement of Becker et al. (2021).

The Nasir & D'Aloisio (2020) simulations use a modified version of the Eulerian hydrodynamics code from Trac & Pen (2004). They use 2 × 2048³ gas and dark matter resolution elements and a box size of L = 200h⁻¹ Mpc. To model the effects of reionization on the forest, they postprocess the hydrodynamics simulations using seminumeric methods. Optical depth skewers are available at z = 5.6, 5.8, and 6.0, and neutral fraction information is available at z = 5.6 and 5.8. A sample of 4000 lines of sight were extracted at each redshift, with each optical depth skewer having a length of 500h⁻¹ Mpc by making use of the periodic boundary conditions (F. Nasir, private communication).

Finally, we include a model from Nasir & D'Aloisio (2020) wherein the volume-filling factor of ionized gas reaches ∼ 98% by z = 6, but the UVB retains large spatial fluctuations to somewhat lower redshifts. ²¹ It has $\langle {\lambda }_{\mathrm{mfp}}^{912}\rangle =10\,{h}^{-1}\,\mathrm{Mpc}$ as in the ND20-late-shortmfp model. We refer to this model as ND20-early-shortmfp. It is essentially a modified version of the fluctuating UVB model proposed by Davies & Furlanetto (2016) with temperature fluctuations included. Compared to the ND20-late-shortmfp model mentioned previously, the ND20-early-shortmfp model has a similarly broad UVB distribution but a much earlier end of reionization. In this model, long dark gaps at z < 6 primarily correspond to regions with a low UVB. Since the IGM is not technically fully ionized in this model until down to z ≃ 5, however, a small fraction of dark gaps may still contain some neutral hydrogen.

In order to directly compare the observations to the models we construct mock spectra from the simulations with properties similar to the real data. We first describe how we create mock spectra for the homogeneous-UVB model.

The snapshots for the homogeneous-UVB model are available on every Δz = 0.1 interval over 3.9 ≤ z ≤ 8.9. To be consistent with the simulations from Nasir & D'Aloisio (2020) we only use snapshots from every Δz = 0.2, and the same snapshots are used for every sightline. We have verified, however, that using snapshots spaced every Δz = 0.1 would not significantly impact our results. Each snapshot was used to extract 5000 160h⁻¹ Mpc skewers along which the native Lyα optical depths have been calculated (Bolton et al. 2017). For a mock spectrum centered at redshift z₀ we combine skewers from redshifts z₀ − 0.2, z₀, and z₀ + 0.2 ²² after shifting the periodic lines of sight by random amounts. The resulting mock spectra are still 160h⁻¹ Mpc in length but contain information about the redshift evolution of the Lyα-opaque regions. We fit the τ_eff evolution over 5 ≤ z ≤ 6 from Bosman et al. (2018) with a power law of ${\tau }_{\mathrm{eff}}\propto {\left(1+z\right)}^{12.34}$ and renormalize the optical depths of the mock spectra such that their average Lyα transmission matches this evolution. We have also checked that the mean transmission measured directly from our observed sample is within the 1σ uncertainties of the measurement in Bosman et al. (2018). We create 5000 mock spectra matching each of our 55 lines of sight. For each QSO, we bin the mock spectra using exactly the same wavelength arrayas the observed spectrum.We then add Gaussian noise to the mock spectra based onthe correspondingflux error array.

Because each optical depth skewer from the Nasir & D'Aloisio (2020) models has a length of 500h⁻¹ Mpc, we first clip them to 160h⁻¹ Mpc and then follow a similar procedure to build the mock spectra set at z₀=5.8 as described above, including rescaling the effective optical depth. In order to cover the full redshift range of the simulation outputs, we extend the mock spectra down to z = 5.6 and up to z = 6.0 by making use of the unclipped skewers to create mock spectra sets centered at z₀=5.6 and 6.0. However, because the spatial structure of the IGM is only recovered over 5.6 ≤ z ≤ 6.0, we restrict our dark gap analysis to this redshift range. As for K20 models, Keating et al. (2020a) ran many radiative transfer simulations until they converged on a reionization history that self-consistently reproduces the mean flux of the Lyα forest as measured by Bosman et al. (2018). We therefore only needed to rebin the skewers and add noise in order to match them to each individual observed QSO spectrum. We note that continuum errors are not considered for the mock spectra. This is because the continuum errors for the observed spectra are estimated to be small (≲10%; Section 2.2), and partly because we are primarily concerned with very low flux levels, which are less affected in an absolute sense by continuum uncertainties.

In Figure 8, we display mock spectra randomly selected from all the models with S/N chosen to match the the Lyα forest of ULAS J0148+0600 as examples. The homogeneous-UVB model exhibits more small transmission peaks than the other models, as expected because the IGM is fully ionized by a uniform UVB. The other models tend to show longer dark gaps interspersed with regions of high transmission.

Zoom In Zoom Out Reset image size

Here we examine the connection between dark gaps and regions of neutral hydrogen. For this we calculate the dark gap length distribution P(L) predicted by models. We use the method described in Section 3.1 to find dark gaps in mock spectra generated in Section 4.4, but with no noise added, and identify gaps that contain regions of neutral hydrogen. The frequency of dark gaps with length L for each model in each redshift bin is calculated based on 10,000 realizations and normalized by the total count of dark gaps in each redshift bin, with P(L) averaged over bins of ΔL = 5h⁻¹ Mpc. We consider a dark gap to contain neutral hydrogen if any pixels inside this gap have x_HI > 0.9. Over each redshift bin, dark gaps extending beyond the boundaries of the Δz = 0.2 window are truncated at the edge. We do so to avoid artifacts in P(L) caused by the finite length of the mock spectra.

As shown in Figure 9, P(L) varies significantly between models. First, no dark gaps with neutral pixels are found in the homogeneous-UVB model because the IGM is fully ionized. In the ND20-early-shortmfp model, the IGM is 98% ionized by z = 6, and therefore only a small fraction of dark gaps contain neutral islands. Dark gaps with no neutral islands also dominate in the K20-high- τ_CMB model that has an extended reionization history. The situation is very different in the rapid late reionization scenarios, however. Dark gaps with neutral islands become dominant for L ≥ 15–20 h⁻¹ Mpc in both ND20-late models. Similarly, in the K20-low- τ_CMB (-hot) model, dark gaps with neutral islands start to be the majority for L ≳ 25–30 h⁻¹ Mpc at z > 5.4. Long dark gaps with L ≳ 30h⁻¹ Mpc are therefore of potentially high interest in terms of identifying regions of the IGM that may contain neutral gas. This paper is therefore largely focused on these long gaps.

Zoom In Zoom Out Reset image size

We further investigate the correlation between neutral islands coverage and dark gap length in the K20-low- τ_CMB model at different redshift, as shown in Figure 10. The histogram is calculated based on 10,000 realizations, and we include all dark gaps regardless of whether they contain neutral pixels. The neutral islands coverage shown here is the sum of the line-of-sight length of neutral pixels inside a dark gap. The mean neutral islands coverage is proportional to the dark gap length, meaning that long dark gaps may contain more neutral gas. Nevertheless, the neutral islands coverage is, on average, significantly less than the dark gap length. This suggests that UVB fluctuations also play a significant role in producing the dark gaps in the late reionization models.

Zoom In Zoom Out Reset image size

We now compare our results to predictions from the simulations described in Section 4. Figure 11 plots the dark gap length versus central redshift for representative mock samples drawn from the homogeneous-UVB model and the K20-low- τ_CMB model. Qualitatively, as redshift increases, the homogeneous-UVB model predicts a milder increase in long dark gaps than is seen in either the K20-low- τ_CMB model or the observations (Figure 3). To quantify the differences, we compute the relevant statistics by drawing mock samples from the simulations that match our observed QSO spectra in redshift and S/N ratio. We then compute the dark gap statistics described in Section 3. We repeat this process 10,000 times for each model and compute the mean, 68% limit, and 95% limit on the expected scatter for the present sample size. Figure 12 compares F₃₀ predicted by models to that calculated from data. The jagged edges of the simulation confidence intervals are caused by the combined effects of step changes in the number of sightlines with redshift and the quantization of F₃₀ for a finite sample size.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

The top left panel shows that the homogeneous-UVB model is highly inconsistent with the observations over 5.3 ≲ z ≲ 5.9. At z ∼ 5.8, the homogeneous-UVB model under-predicts F₃₀ by a factor of 3. At z ∼ 5.4 and over 5.5 ≲ z ≲ 5.8, this model is rejected by the data with > 99.9% confidence.

On the other hand, the K20-low- τ_CMB and K20-low- τ_CMB -hot models, wherein reionization ends at z ≃ 5.3, produce F₃₀ results that are generally consistent with the observations over 5 < z < 6. One exception is that these models underpredict the small number of long dark gaps observed at z ∼ 5.4. The K20-high- τ_CMB model is consistent with the observations at z ≥ 5.75 but underpredicts F₃₀ at lower redshifts. This is a natural consequence of the earlier reionization in this model, which leads to a lower neutral hydrogen fraction and smaller UVB fluctuations at these redshifts.

As shown in the right panels, F₃₀ values from the Nasir & D'Aloisio (2020) models are consistent with the observations within their 95% limits over the available redshift range. Among the ND20 models, ND20-early-shortmfp gives lower F₃₀ values compared to ND20-late, but the difference is within the 68% range for the present sample size.

We compare the cumulative distributions of dark gap length in Figure 13, and give the differences between the observation and the model predictions in Figure 14. In order to facilitate a direct comparison between the observations and simulations, we divide the data into redshift bins of Δz = 0.2. Here, dark gaps extending beyond the boundaries of a redshift bin are truncated at the edge when calculating P( < L) for both the observation and models. Similar to our approach in Section 4.5, we do this to avoid artifacts from the finite length of the mock spectra.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

We present numerical convergence tests for the homogeneous-UVB model in Appendix A. We find that the results for both F₃₀ and P( < L) are relatively insensitive to box size, but that the number of small gaps increases with increasing mass resolution. The impact of mass resolution is more significant for P( < L) at smaller gap lengths than for F₃₀. For P( < L) measured from the homogeneous-UVB model, therefore, we display predictions based on a higher-resolution run with 2 × 2048³ particles and a box size of L = 40h⁻¹ Mpc (hereafter 40_2048) instead of the fiducial configuration of 2 × 2048³ particles and box size of L =160h⁻¹ Mpc (hereafter 160_2048). Because Keating et al. (2020a) use postprocessed radiative transfer simulations, and Nasir & D'Aloisio (2020) simulations are based on an Eulerian code instead of a SPH code, mass resolution effects may be significantly different for these models than for the homogeneous-UVB model. We therefore present results as they are, although mass resolution corrections may be needed.

Over z = 5.6–6.0, the homogeneous-UVB model predicts significantly fewer long gaps than are observed in the data. The discrepancies between the data and the homogeneous-UVB model persist down to the z = 5.2–5.4 bin.

In contrast, the late reionization models, K20-low- τ_CMB, K20-low- τ_CMB -hot, and K20-high- τ_CMB, predict P( < L) values that are generally consistent with the data. Nevertheless, over z = 5.7–5.9, we note that these models, especially the K20-high- τ_CMB model, systematically yield higher P( < L), i.e., fewer long gaps, than the observed for some L, though the discrepancies are less conspicuous compared to those for the homogeneous-UVB. At lower redshifts, there are minor differences between the K20 models and the observation. The ND20-early-shortmfp and ND20-late models are generally consistent with the observation in the redshift range (5.6 ≤ z ≤ 6.0) currently probed by the simulations.

Combining the results for F₃₀ and P( < L), it is evident that a fully ionized IGM with a homogeneous UV background is disfavored by the observations down to z ∼ 5.3. This result is consistent with the large-scale inhomogeneities in IGM Lyα opacity seen in recent τ_eff measurements (Becker et al. 2015; Bosman et al. 2018; Eilers et al. 2018; Yang et al. 2020b; Bosman et al. 2021a). Our results also agree with the early indication from Gnedin et al. (2017) that models, wherein reionization ended well before z = 6, struggled to produce enough long dark gaps.

The late reionization models from Keating et al. (2020a) and Nasir & D'Aloisio (2020) are generally consistent with dark gap statistics in the Lyα forest. In these models, the residual neutral islands at z < 6 coupled with UVB fluctuations can naturally explain the appearance of long dark gaps in the Lyα forest. Among these models, the data tend to prefer those with later and more rapid reionization histories. For example, the K20-low- τ_CMB and K20-low- τ_CMB -hot models, which have a reionization midpoint of z₅₀ = 6.7, is somewhat more consistent (see curves and shades near 5.6 ≤ z ≤ 5.8 in Figures 12, 13, and 14) with the dark gap statistics at z < 6 than the K20-high- τ_CMB mode, for which z₅₀ = 8.4. A late and rapid reionization is also suggested by the recent mean free path measurement from Becker et al. (2021) (see also Cain et al. 2021; Davies et al. 2021).

Alternatively, long dark gaps can arise from a fully reionized IGM provided that there are large UVB fluctuations. The early reionization model from Nasir & D'Aloisio (2020), which retains postreionization fluctuations in the UV background and IGM temperature, is consistent with the data over at least 5.6 < z < 6.0, where the available simulation outputs allow mock spectra to be compared to the data using the methods described above. Extending these simulations down to lower redshifts would be helpful for testing the pure fluctuating UVB model further.