1 Introduction

Measurements of heavy-flavour hadron production in hadronic collisions provide crucial tests for calculations based on quantum chromodynamics (QCD). Calculations of \(p_{\textrm{T}}\)-differential heavy-flavour hadron production cross sections in hadronic collisions are factorised into three separate components: the parton distribution functions (PDFs), which describe the Bjorken-x distributions of quarks and gluons within the incoming hadrons for a given transferred momentum squared \(Q^2\); the hard-scattering cross section for the partons to produce a charm- or beauty-quark pair; and the fragmentation functions, which characterise the hadronisation of a quark to a given hadron species [1]. The hadronisation process is typically described via two different mechanisms: fragmentation and recombination (also known as coalescence). In the former, colour neutrality is reached by means of gluon radiation and gluon splitting into quark pairs, with a final phase in which quark pairs or triplets bind together to form mesons or baryons. In the latter, two or three quarks close in the velocity and coordinate space, bind to form color-neutral hadrons. As charm and beauty quarks have masses much larger than \(\Lambda _\textrm{QCD}\), which is the energy scale in QCD at which quarks and gluons are confined within hadrons, the parton–parton hard-scattering cross sections can be calculated perturbatively [2]. In contrast, the fragmentation functions cannot be calculated with perturbative QCD (pQCD) methods, so they are determined from measurements in \(\textrm{e}^{+}\textrm{e}^{-}\)collisions. They are then applied in cross section calculations, assuming that the relevant hadronisation processes are “universal”, i.e. independent of the collision energy and system. A recent review with a more comprehensive overview of heavy-quark hadronisation can be found in Ref. [3]. Factorisation can be implemented in pQCD-based calculations in different ways, for example, in terms of the transferred momentum squared (collinear factorisation) [1]. Calculations for LHC energies, like the general-mass variable-flavour-number scheme (GM-VFNS) [4, 5] and the fixed order plus next-to-leading logarithms (FONLL) approach [6, 7] provide a next-to-leading order (NLO) accuracy with all-order resummation of next-to-leading logarithms. These calculations describe the production of heavy-flavour mesons within uncertainties in wide kinematic and pp collision-energy ranges [8,9,10,11]. The dominant source of theoretical uncertainty in these calculations is related to the choice of the energy scales for the validity of the perturbative regime (factorisation and renormalisation scales). Charm-hadron production cross sections are also compared with the POWHEG pQCD calculations of charm-quark production with NLO accuracy [12], matched with PYTHIA 6 [13] to generate the parton shower and fragmentation. The POWHEG+PYTHIA 6 simulations describe the charm-meson production cross sections but largely underpredict the production of charm baryons [14]. To isolate the effects of hadronisation, hadron-to-hadron production ratios within the charm sector, such as \(\mathrm D_{s}^+/\mathrm {D^0} \), \(\mathrm {\Lambda _{c}^{+}}/\mathrm {D^0} \), and \(\mathrm {\Xi _{c}^{0}}/\mathrm {D^0} \) are particularly effective, since in pQCD calculations the PDFs and the choice of the factorisation and renormalisation scales are common to all charm-hadron species and their effects almost fully cancel in the yield ratios.

Previous measurements of charm-meson production cross sections in pp and p–Pb collisions at the LHC [9, 10, 15,16,17,18,19] show that the \(\mathrm D^+/\mathrm {D^0} \) ratio is independent of the transverse momentum within uncertainties, while a hint of increase with \(p_{\textrm{T}}\) is visible for the \(\mathrm D_{s}^+/\mathrm {D^0} \), \(\mathrm D_{s}^+/\mathrm D^+ \), and \(\mathrm D_{s}^+/(\mathrm {D^0} + \mathrm D^+)\) ratios in the interval \(p_{\textrm{T}}\) < 8 GeV/c. The ratios are also described well by pQCD calculations and by the PYTHIA 8 event generator using the Monash tune [20, 21], with the fragmentation tuned on \(\textrm{e}^{+}\textrm{e}^{-}\)collisions. However, the charm baryon-to-meson ratios \(\mathrm {\Lambda _{c}^{+}}/\mathrm {D^0} \), \(\mathrm {\Xi _{c}^{0,+}}/\mathrm {D^0} \), \(\Omega _\textrm{c}^{0}/\mathrm {D^0} \), and \(\Sigma _\textrm{c}^{0,++}/\mathrm {D^0} \) measured at midrapidity at the LHC [14, 22,23,24,25,26,27,28,29,30] show significant deviations from the values measured in \(\textrm{e}^{+}\textrm{e}^{-}\)collisions, and the Monash tune of PYTHIA 8 significantly underpredicts the production rates of charm baryons. These results pose a challenge to the assumption of a universal hadronisation mechanism [31]. Models that incorporate other effects such as string formation beyond the leading-colour approximation [32, 33], coalescence or recombination of charm quarks with quarks or di-quarks from a thermal medium [34,35,36], and statistical hadronisation including higher charm-resonant states not yet discovered [37] provide a better description of the data. Measurements of beauty-baryon production in pp and p–Pb collisions by the CMS, LHCb, and ALICE Collaborations [38,39,40,41,42,43] also indicate similar differences in hadronisation mechanisms in the beauty sector between hadronic and leptonic collision systems [31,32,33]. The enhancement of the relative abundance of baryons compared to that of mesons has a strong impact on the charm-quark fragmentation fractions, as demonstrated in pp collisions at \({\sqrt{s} = 5.02}\) TeV and \(\sqrt{s}\) = 13 TeV  [18, 44].

Measurements in proton–nucleus collisions allow an assessment of the various effects, denoted as cold-nuclear-matter (CNM) effects, related to the presence of one or more nuclei in the colliding system. In the initial state, the PDFs are modified in bound nucleons compared to free nucleons, depending on x and \(Q^2\) [45, 46]. At LHC energies, the most relevant effect is shadowing: a reduction of the parton densities at low x, which intensifies when \(Q^2\) decreases and the nucleus mass number A increases. This effect, induced by the high phase-space density of small-x partons, can be described within the collinear factorisation framework using phenomenological parametrisations of the modification of the PDFs (denoted as nPDFs) [47,48,49]. If the parton phase-space reaches saturation, PDF evolution equations are not applicable, and the most appropriate theoretical description is the Colour Glass Condensate effective theory (CGC) [50]. The modification of the small-x parton dynamics can significantly reduce charm-hadron production at low \(p_{\textrm{T}}\). Furthermore, the multiple scattering of partons in the nucleus before and/or after the hard scattering can modify the kinematic distribution of the produced hadrons: partons can lose energy in the initial stages of the collision via initial-state radiation [51], or experience transverse momentum broadening due to multiple soft collisions before the heavy-quark pair is produced [52, 53]. These initial-state effects are expected to have a small impact on charm-hadron production at high \(p_{\textrm{T}} \) (\(p_{\textrm{T}} >3\)–4 GeV/c), but they can induce a significant modification of the yield and momentum distribution in the lower \(p_{\textrm{T}}\) region.

The nuclear modification factor \(R_{\textrm{pPb}}\) (the ratio of the cross section in p–Pb collisions to that in pp interactions scaled by the mass number of the Pb nucleus) of D mesons measured by ALICE in p–Pb collisions at a centre-of-mass energy per nucleon–nucleon collision \(\sqrt{s_{\textrm{NN}}}\) = 5.02 TeV is consistent with unity for \(0<p_{\textrm{T}} <36\) GeV/\(c\)  [54], suggesting that the cold-nuclear-matter effects that influence charm-meson production at midrapidity are small. However, measurements of \(\mathrm {\Lambda _{c}^{+}}\) baryons in p–Pb collisions [23] indicate a \(p_{\textrm{T}}\)-dependent \(R_{\textrm{pPb}}\), with values lower than unity for \(0<p_{\textrm{T}} <2~\)GeV/\(c\) and systematically above unity for \(p_{\textrm{T}} >2\) GeV/\(c\), indicating an increase in the \(\mathrm {\Lambda _{c}^{+}}\) mean \(p_{\textrm{T}}\) in p–Pb collisions with respect to pp collisions. A POWHEG+PYTHIA 6 simulation, which is coupled with the EPPS16 nPDF set [55] for p-Pb collisions, is in fair agreement with the \(\mathrm {\Lambda _{c}^{+}}\) measurements for \(p_{\textrm{T}} <3\) GeV/\(c\), but it does not describe the \(R_{\textrm{pPb}}\) increase above unity in the region \(4<p_{\textrm{T}} <8\) GeV/\(c\). Measurements in the light-flavour sector in high-multiplicity pp and p–Pb collisions at different energies have revealed strong long-range correlations [8], resembling those measured in Pb–Pb collisions, the latter being understood as due to a collective flow related to the formation of a deconfined QCD medium, the quark–gluon plasma. A modification of the \(p_{\textrm{T}}\) shape as a function of multiplicity is observed in the strangeness sector by the ALICE and CMS Collaborations in p–Pb collisions [56, 57] and is consistent with the effect of radial flow in hydrodynamic models such as EPOS LHC [58]. In this picture, particles of larger mass are boosted to higher transverse momenta by a common velocity field [59]. In the heavy-flavour sector, differential studies of \(\mathrm {\Lambda _{c}^{+}}\) and \(\mathrm {D^0}\) production as a function of charged-particle multiplicity in pp collisions at \(\sqrt{s}\) = 13 TeV by ALICE [60] revealed a multiplicity dependence of baryon-to-meson yield ratios. In addition, baryon production at intermediate \(p_{\textrm{T}}\) may be enhanced due to hadronisation via quark recombination [34, 36, 61]. This can be further examined by measuring the \(\mathrm {\Xi _{c}^{0}}\)-baryon production in p–Pb collisions and comparing it with the results published in pp collisions [26, 27].

In this paper, the measurement of the \(p_{\textrm{T}}\)-differential production cross section of prompt \(\mathrm {\Xi _{c}^{0}}\) baryons in the region 2 \(< p_{\textrm{T}}<\) 12 GeV/c in p–Pb collisions at \(\sqrt{s_\textrm{NN}}\) = 5.02 TeV is reported. The term prompt refers to charm-hadrons produced directly in the hadronisation of a charm quark or the strong decay of a directly produced excited charm-hadron state, in contrast to feed-down charm-hadrons, produced in the decay of a hadron containing a beauty quark. To gauge possible modifications of the \(p_{\textrm{T}}\) distribution of promptly produced \(\mathrm {\Xi _{c}^{0}}\) baryons due to the presence of nuclei in the collision system, the nuclear modification factor is also evaluated. Furthermore, information on the hadronisation process can be extracted by comparing the production cross sections of different hadron species. The \(p_{\textrm{T}}\)-dependent yield ratios of \(\mathrm {\Xi _{c}^{0}}/\mathrm {D^0} \) and \(\mathrm {\Xi _{c}^{0}}\)/\(\mathrm {\Lambda _{c}^{+}}\) are reported in this article. Finally, by integrating the \(p_{\textrm{T}}\)-differential results and extrapolating them to \(p_{\textrm{T}}\)  = 0, the \(p_{\textrm{T}}\)-integrated prompt \(\mathrm {\Xi _{c}^{0}}\) production cross section is computed. This allows the calculation of the charm fragmentation fractions and of the total charm cross section in p–Pb collisions, reported in Ref. [62].

2 Experimental setup and data samples

The ALICE detector system and its performance are described in detail in Refs. [63, 64]. The reconstruction of charm baryons from their hadronic decay products at midrapidity primarily relies on the inner tracking system (ITS) [65] for the reconstruction of charged-particle trajectories and determination of primary and decay vertices, the time projection chamber (TPC) [66] for track reconstruction and particle identification (PID) through specific energy loss measurements, and the time-of-flight detector (TOF) [67], which extends the PID capabilities of the TPC by measuring the flight time of the charged particles from the interaction point. These detectors are located inside a solenoidal magnet of field strength 0.5 T directed along the beam axis. In addition, the two V0 scintillator arrays [68] are used to trigger collision events and determine the luminosity when used in conjunction with the T0 detector [69]. The zero-degree calorimeter (ZDC) is employed for offline event selection in p–Pb collisions [64].

The analysis was performed in the pseudorapidity interval \(|\eta |<0.8\) on data from p–Pb collisions at \(\sqrt{s_{\textrm{NN}}}~=~5.02\) TeV collected with a minimum-bias (MB) trigger during Run 2 of the LHC. For p–Pb collisions, the rapidity in the nucleon–nucleon centre-of-mass system \((y_\textrm{cms})\) is shifted by 0.46 units in the direction of the proton beam due to the energy asymmetry of the colliding beams. The results are reported for the rapidity interval \(|y_\textrm{lab}|<0.5\) in the laboratory system, which corresponds to \(-0.96<y_\textrm{cms}<0.04\).

The MB trigger requires a pair of coincident signals in the two V0 scintillator arrays, which are located on each side of the interaction point. Further offline selections were applied to suppress the background originating from beam–gas collisions and other machine-related background sources [70]. In order to maintain uniform ITS acceptance in pseudorapidity, only events with a reconstructed vertex position within 10 cm along the beam axis from the nominal interaction point were analysed. The primary vertex position was determined using tracks reconstructed in the TPC and ITS detectors. Events with multiple interaction vertices reconstructed from TPC and ITS tracks were tagged as pileup from several collisions and removed from the analysed sample [64]. Using these selection criteria, the p–Pb sample comprised approximately 600 million events, corresponding to an integrated luminosity of \(\mathscr {L}_\textrm{int} =287\pm 11~\mu \textrm{b}^{-1}\).

3 Data analysis

In this analysis, \(\mathrm {\Xi _{c}^{0}}\) baryons were reconstructed via the decay channel \(\mathrm {\Xi _{c}^{0}} \rightarrow \mathrm{\pi ^{+}}\Xi ^{-}\) and its charge conjugate, with branching ratio BR \(=\) (1.43 ± 0.32)% [71]. The \(\Xi ^{-}\) baryons were selected from the decay chain \(\Xi ^{-} \rightarrow \pi ^{-}\Lambda \), BR \(=\) (\(99.887 \pm 0.035\))%, followed by \(\Lambda \rightarrow \mathrm{\pi ^{-} p}\), BR \(=\) (\(63.9 \pm 0.5\))% [71]. The \(\Xi ^{-}\) and \(\Lambda \) baryons were reconstructed by exploiting their characteristic decay topologies as reported in Refs. [72, 73]. Charged-particle tracks and particle-decay vertices were reconstructed using the ITS and the TPC. The particle trajectories in the vicinity of the primary vertex and the decay vertices were reconstructed with the KFParticle package [74], which allows a direct estimate of their parameters and the associated uncertainties. The package allows one to set constraints on the mass and the production point of the reconstructed particles, using information about the uncertainties of the decay-product trajectories to improve the reconstruction accuracy of the parent particle. The mass constraint improves the mass and momentum reconstruction of the particle, while the production point constraint helps to determine whether the particle is coming from a given vertex. These constraints were applied to the \(\Lambda \) and \(\Xi ^{-}\) decay vertices in the \(\mathrm {\Xi _{c}^{0}}\) decay chain reconstruction.

Several selection criteria were applied for the initial filtering of \(\mathrm {\Xi _{c}^{0}}\) candidates. To ensure good quality of the tracks used to reconstruct the \(\mathrm {\Xi _{c}^{0}}\) candidates, track quality and kinematic selection criteria were applied. The tracks were required to be within the pseudorapidity interval \(|\eta | < \) 0.8 and to have crossed at least 70 TPC pad rows out of a maximum of 159. The number of clusters in the TPC used for the energy loss determination was required to be larger than 50 to enhance the precision of the mean specific energy loss (\(\text {d}E/\text {d}x\)) measurement. Moreover, the candidate tracks of the \(\pi ^+\) produced in the \(\mathrm {\Xi _{c}^{0}}\) decay were required to have a minimum of three (out of six) hits in the ITS.

The PID selections were based on the difference between the measured and expected detector signals for a given particle species hypothesis in units of the detector resolution (\(n^\textrm{det}_\sigma \)). For candidate tracks of the pions from the \(\Lambda \) and \(\Xi ^-\) decays and of the proton from the \(\Lambda \) decay, a selection on the measured dE/dx in the TPC of \(|n^\textrm{TPC}_\sigma |<4\) was applied. For the pion from the \(\Xi ^-\) decay, the transverse momentum was required to be larger than 150 MeV/\(c\), and, if the track has an associated hit in the TOF detector, a further PID selection of \(|n^\textrm{TOF}_\sigma |<5\) was applied to its flight time. The deviation of the measured invariant masses from the world-average value of the \(\Lambda \) and \(\Xi ^-\) masses [71] was required to be within 10 MeV/\(c^2\). To further reduce the background in the \(\mathrm {\Xi _{c}^{0}}\) candidate sample, a selection on the \(p_{\textrm{T}}\) of the pion produced in the \(\mathrm {\Xi _{c}^{0}}\) decay was applied. It was required to be greater than 1.6, 1.2, 1.0, and 1.0 GeV/\(c\) in the 2–4, 4–6, 6–8, and 8–12 \(\mathrm {\Xi _{c}^{0}}\) \(p_{\textrm{T}}\) intervals, respectively.

After applying the selections described above, further rejection of background \(\mathrm {\Xi _{c}^{0}}\) candidates was obtained using a boosted decision tree (BDT) algorithm. The BDT implementation provided by the XGBoost library was used [75, 76]. During the training process, the BDT is presented with labelled samples, where the true classifications of signal and background are known. The algorithm is trained to optimise the classification of the two classes based on the differences in the topology, kinematics, and PID information. Once the BDT is trained, it is applied to the data sample where the classifications are unknown. With the machine learning approach, multiple selection criteria are combined into a single response variable representing the algorithm’s confidence in classifying a candidate as a true \(\mathrm {\Xi _{c}^{0}}\) baryon. After applying a trained BDT model to the data sample, a selection in the BDT response was applied to reduce the large combinatorial background. Independent BDTs were trained for each \(p_{\textrm{T}}\) interval in the analysis. The signal samples for the training were obtained from simulated events using the PYTHIA 8.243 [20] Monte Carlo (MC) generator with the Monash tune [21] embedded into an underlying p–Pb collision generated with HIJING 1.36 [77]. The transport of simulated particles within the detector was performed with the GEANT 3 package [78]. The LHC beam conditions and the conditions of the ALICE detectors in terms of active channels, gain, noise level, and alignment, and their evolution with time during the data taking, were taken into account in the simulations. Each PYTHIA 8 event was required to contain a c\(\overline{\textrm{c}}\) quark pair, with at least one hadronising into a \(\mathrm {\Xi _{c}^{0}}\) baryon, which is forced to decay via the decay channel of interest. Only prompt \(\mathrm {\Xi _{c}^{0}}\) signal candidates were selected for the training, while feed-down ones were not used since they have a different decay vertex topology. The background sample was selected from a fraction of real data using the same selection criteria described above, with the additional requirement that the invariant mass of the \(\mathrm {\Xi _{c}^{0}}\) candidate was within the intervals \(2.17< M < 2.39\) GeV/\(c^2\) or \( 2.55< M < 2.77\) GeV/\(c^2\) (sidebands) to ensure that the signal region was excluded.

The training variables in the BDT related to the \(\Xi ^-\) candidates were (i) the reconstructed invariant mass, (ii) the pointing angle of its momentum to the primary vertex, i.e. the angle between the momentum vector of the reconstructed \(\Xi ^-\) particle and the vector pointing from the reconstructed primary vertex to the \(\Xi ^-\) decay vertex, (iii) the decay length normalised by its uncertainty, (iv) a normalised \(\chi ^2\) value, which is obtained by evaluating whether the momentum vector of the \(\Xi ^-\) candidate points back to the reconstructed primary vertex, and is provided by the KFParticle package. The training variables describing the pion emerging from the \(\mathrm {\Xi _{c}^{0}}\) decay were the \(n\sigma ^\textrm{TOF}_{\pi } \), the \(n\sigma ^\textrm{TPC}_{\pi }\) and the distance of closest approach to the primary vertex. The selection on the BDT response was tuned in each \(p_{\textrm{T}}\) interval to maximise the expected statistical significance, which is calculated using (i) an estimated value for the signal extrapolated from the \(\mathrm {\Xi _{c}^{0}}\) production cross section reported in Ref. [26] using a Lévy-Tsallis fit, multiplied by the reconstruction and selection efficiencies for each BDT selection threshold, (ii) an estimate of the background within the signal region obtained by interpolating a fit to the invariant mass distribution in the sidebands of the \(\mathrm {\Xi _{c}^{0}}\) signal region.

After applying the BDT selections, the raw \(\mathrm {\Xi _{c}^{0}}\) yield in each \(p_{\textrm{T}}\) interval under study was obtained by fitting the invariant mass distribution of the candidates. Examples of those distributions are shown in Fig. 1 for the \(p_{\textrm{T}}\) intervals 2–4 and 6–8 GeV/c. A Gaussian function was used to model the signal peak and an exponential (for \(p_{\textrm{T}}\) \(> 4\) GeV/\(c\)) or parabolic function (in the \(2< p_{\textrm{T}} <4\) GeV/\(c\) interval) was used to model the background. Due to the small signal-to-background ratio, the standard deviation of the Gaussian signal function was fixed to the value obtained from simulations to improve the fit stability. A \(\mathrm {\Xi _{c}^{0}}\) signal, with a statistical significance larger than 3, could be extracted in the four considered \(p_{\textrm{T}}\) intervals in the range \(2<p_{\textrm{T}} <12\) GeV/\(c\).

Fig. 1
figure 1

Invariant mass distributions of \(\mathrm {\Xi _{c}^{0}} \rightarrow \mathrm{\pi ^{+}}\Xi ^{-}\) candidates (and charge conjugates) in \(2< p_{\textrm{T}} < 4\)  GeV/\(c\) (left), and in \(6< p_{\textrm{T}} < 8\) GeV/\(c\) (right) in p–Pb collisions at \(\sqrt{s_{\textrm{NN}}}~=~5.02\) TeV. The red dashed curves represent the background fit functions, and the blue curves represent the total fit functions

4 Corrections

The \(p_{\textrm{T}}\)-differential production cross section of prompt \(\mathrm {\Xi _{c}^{0}}\) baryons per unit of rapidity in the interval \(-0.96< y_{\textrm{cms}} < 0.04\) was calculated from the raw yields as

$$\begin{aligned} \frac{\mathrm {d^2}\sigma }{\textrm{d}y~\textrm{d}p_{\textrm{T}}} = \frac{1}{2} \frac{N^{\mathrm {\mathrm {\Xi _{c}^{0}} +\overline{\mathrm {\Xi _{c}^{0}}}}}_{\textrm{raw}}(p_{\textrm{T}})\!\times \! f_{\textrm{prompt}}(p_{\textrm{T}})}{\Delta y_\textrm{lab}\,\Delta p_{\textrm{T}} \!\times \!(\textrm{Acc}\times \varepsilon )_{\textrm{prompt}}(p_{\textrm{T}})\!\times \!\textrm{BR}\times \mathscr {L}_\textrm{int}}, \end{aligned}$$
(1)

where \(N^{\mathrm {\mathrm {\Xi _{c}^{0}} +\overline{\mathrm {\Xi _{c}^{0}}}}}_{\textrm{raw}}\) is the raw yield (sum of particles and antiparticles) extracted from the invariant mass fit, \(f_{\textrm{prompt}}\) is the fraction of prompt \(\mathrm {\Xi _{c}^{0}}\) in the measured raw yield, BR is the branching ratio of the considered decay chain, and \(\mathscr {L}_\textrm{int}\) is the integrated luminosity. The factor 2 accounts for the presence of particles and antiparticles in the raw yields, and \(\Delta y_\textrm{lab}\,\Delta p_{\textrm{T}}\) accounts for the widths of the rapidity and transverse momentum intervals. The measurement of \(\mathrm {\Xi _{c}^{0}}\) is performed for \(\Delta y_\textrm{lab} = 1.6\), under the assumption that the cross section per unit of rapidity of \(\mathrm {\Xi _{c}^{0}}\) baryons does not significantly change between \(|y_\textrm{lab} |< 0.5\) and \(|y_\textrm{lab} |< 0.8\). This has been verified using PYTHIA 8 simulations [20] and FONLL pQCD calculations [2, 6]. The factor \((\textrm{Acc}\times \varepsilon )_{\textrm{prompt}}\) is the product of the geometrical acceptance (Acc) and the reconstruction and selection efficiency (\(\varepsilon \)) for prompt \(\mathrm {\Xi _{c}^{0}}\) candidates and is evaluated using the PYTHIA 8 simulations described in Sect. 3. To account for differences in the \(p_{\textrm{T}}\)-distribution of prompt \(\mathrm {\Xi _{c}^{0}}\) baryons between the MC simulation and the actual data distribution, weighting factors were applied to the generated \(\mathrm {\Xi _{c}^{0}}\) \(p_{\textrm{T}}\)-distribution before calculating \((\textrm{Acc}\times \varepsilon )_{\textrm{prompt}}\). These weighting factors were determined by comparing the \(p_{\textrm{T}}\)-distribution of \(\mathrm {\Xi _{c}^{0}}\) generated from the Quark Combination Model (QCM) [79] to the PYTHIA 8 distribution described above. The QCM was chosen because it best reproduces the measured \(\mathrm {\Xi _{c}^{0}}\) \(p_{\textrm{T}}\) distribution. Figure 2 shows the final \((\textrm{Acc}\times \varepsilon )\) correction factors of prompt and feed-down \(\mathrm {\Xi _{c}^{0}}\) as a function of \(p_{\textrm{T}}\). The \((\textrm{Acc}\times \varepsilon )\) of prompt \(\mathrm {\Xi _{c}^{0}}\) baryons is slightly larger than that of feed-down baryons. This difference arises from the inclusion of the distance of closest approach (DCA) to the primary vertex of the pion produced in the \(\mathrm {\Xi _{c}^{0}}\) decay as one of the variables in the BDT training. Since the pions from decays of feed-down \(\mathrm {\Xi _{c}^{0}}\) present a wider DCA distribution as compared to those from prompt \(\mathrm {\Xi _{c}^{0}}\), due to the displacement by a few hundred micrometres of the feed-down \(\mathrm {\Xi _{c}^{0}}\) decay vertices, prompt signals tend to exhibit higher BDT output scores compared to feed-down signals.

The fraction of prompt baryons in the raw \(\mathrm {\Xi _{c}^{0}}\) yield extracted from the selected candidate sample was calculated as:

$$\begin{aligned} f_{\textrm{prompt}}&= 1 - \frac{N^{\mathrm {\mathrm {\Xi _{c}^{0}} +\overline{\mathrm {\Xi _{c}^{0}}}}}_{\mathrm {feed\text {-}down}}}{N^{\mathrm {\mathrm {\Xi _{c}^{0}} +\overline{\mathrm {\Xi _{c}^{0}}}}}_{\textrm{raw}}} \nonumber \\&= 1 - \frac{1}{N^{\mathrm {\mathrm {\Xi _{c}^{0}} +\overline{\mathrm {\Xi _{c}^{0}}}}}_{\textrm{raw}}} \frac{\mathrm {d^2}\sigma ^{\mathrm {\Xi _{c}^{0}}}_{\mathrm {feed\text {-}down}}}{\textrm{d}y~\textrm{d}p_{\textrm{T}}}\nonumber \\&\quad \times 2 \times (\textrm{Acc}\times \varepsilon )_{\mathrm {feed\text {-}down}} \times R_{\mathrm {pPb,~feed\text {-}down}}\nonumber \\&\quad \times \Delta p_{\textrm{T}} \times \Delta y_\textrm{lab} \times \textrm{BR} \times \mathscr {L}_\textrm{int} \end{aligned}$$
(2)

The yield of feed-down \(\mathrm {\Xi _{c}^{0}}\) (\(\mathrm {d^2}\sigma ^{\mathrm {\Xi _{c}^{0}}}_{\mathrm {feed\text {-}down}}/\textrm{d}p_{\textrm{T}} \textrm{d} y\)) is estimated starting from the cross section of \(\mathrm {\Lambda _{c}^{+}}\) baryons originating from \(\mathrm {\Lambda _{b}^{0}}\) decays, which is obtained using the beauty-quark production cross section from FONLL calculations [6, 7], the fraction of beauty quarks fragmenting into beauty hadrons taken from the LHCb measurement of beauty fragmentation fractions in pp collisions at \(\sqrt{s}~=~13\) TeV  [39], and the decay kinematics of beauty hadrons decaying into a final state with a \(\mathrm {\Lambda _{c}^{+}}\), which is taken from PYTHIA 8 (\((\mathrm {d^2}\sigma /\textrm{d}p_{\textrm{T}} \textrm{d} y)^{\mathrm {\Lambda _{c}^{+}}}_{\mathrm {feed\text {-}down,FONLL}}\)). The predicted feed-down \(\mathrm {\Lambda _{c}^{+}}\) cross section is scaled by the ratio of the measured \(p_{\textrm{T}}\)-differential production cross sections of prompt \(\mathrm {\Xi _{c}^{0}}\)  [26] \((\mathrm {d^2}\sigma /\textrm{d}p_{\textrm{T}} \textrm{d} y)^{\mathrm {\Xi _{c}^{0}}}_{\textrm{prompt}}\), and prompt \(\mathrm {\Lambda _{c}^{+}}\)  [24], where the \((\mathrm {d^2}\sigma /\textrm{d}p_{\textrm{T}} \textrm{d} y)^{\mathrm {\Lambda _{c}^{+}}}_{\textrm{prompt}}\) was measured in pp collisions at \(\sqrt{s}~=~5.02\) TeV to obtain an estimation for the \(p_{\textrm{T}}\)-differential feed-down \(\mathrm {\Xi _{c}^{0}}\) production cross section

$$\begin{aligned} \frac{\mathrm {d^2}\sigma ^{\mathrm {\Xi _{c}^{0}}}_{\mathrm {feed\text {-}down}}}{\textrm{d}y~\textrm{d}p_{\textrm{T}}} = \left( \frac{\mathrm {d^2} \sigma }{\textrm{d}p_{\textrm{T}} \textrm{d} y}\right) ^{\mathrm {\Lambda _{c}^{+}}}_{\mathrm {feed\text {-}down,FONLL}}\nonumber \\ \times \frac{(\mathrm {d^2}\sigma /\textrm{d}p_{\textrm{T}} \textrm{d} y)^{\mathrm {\Xi _{c}^{0}}}_{\textrm{prompt}}}{(\mathrm {d^2}\sigma /\textrm{d}p_{\textrm{T}} \textrm{d} y)^{\mathrm {\Lambda _{c}^{+}}}_{\textrm{prompt}}} \end{aligned}$$
(3)

The scaling factor \((\mathrm {d^2}\sigma /\textrm{d}p_{\textrm{T}} \textrm{d} y)^{\mathrm {\Xi _{c}^{0}}}_{\textrm{prompt}}/(\mathrm {d^2}\sigma /\textrm{d}p_{\textrm{T}} \textrm{d} y)^{\mathrm {\Lambda _{c}^{+}}}_{\textrm{prompt}}\) approximates the term

$$\begin{aligned} & \frac{\sum _{\mathrm {H_B}} \textrm{b} \rightarrow \mathrm {H_{b}} \rightarrow \mathrm {\Xi _{c}}}{\sum _{\mathrm {H_B}} \textrm{b} \rightarrow \mathrm {H_{b}} \rightarrow \mathrm {\Lambda _{c}}} \simeq \frac{\textrm{b} \rightarrow \mathrm {\Xi _{b}} \rightarrow \mathrm {\Xi _{c}}}{\textrm{b} \rightarrow \mathrm {\Lambda _{b}}\rightarrow \mathrm {\Lambda _{c}}} \end{aligned}$$
(4)

considering that the only beauty baryons that contribute to the yield of \(\Xi _\textrm{c}\) (\(\mathrm {\Lambda _{c}^{+}} \)) are \(\Xi _\textrm{b}\) (\(\mathrm {\Lambda _{b}^{0}})\). This scaling relies on the assumptions that the \(p_{\textrm{T}}\) shapes of the feed-down \(\mathrm {\Lambda _{c}^{+}}\) and \(\mathrm {\Xi _{c}^{0}}\) production cross sections are similar, and the ratio of production cross sections of \(\mathrm {\Xi _{c}^{0}}\) and \(\mathrm {\Lambda _{c}^{+}}\) is similar for prompt and feed-down baryons. Finally, a hypothesis on the value of the nuclear modification factor \(R_{\textrm{pPb}}\) of feed-down \(\mathrm {\Xi _{c}^{0}}\) is needed. It is assumed that the \(R_{\textrm{pPb}}\) of prompt \(\mathrm {\Xi _{c}^{0}}\) is equal to that of prompt \(\mathrm {\Lambda _{c}^{+}}\) and that the \(R_{\textrm{pPb}}\) of prompt and feed-down \(\mathrm {\Xi _{c}^{0}}\) are equal. The evaluated prompt fraction ranges between 0.91 and 0.96 depending on the \(p_{\textrm{T}}\) interval.

Fig. 2
figure 2

Product of detector acceptance and efficiency for \(\mathrm {\Xi _{c}^{0}}\) baryons in p–Pb collisions at \(\sqrt{s_{\textrm{NN}}}~=~5.02\) TeV as a function of \(p_{\textrm{T}}\). The solid line corresponds to the \((\textrm{Acc}\times \varepsilon )\) for prompt \(\mathrm {\Xi _{c}^{0}}\), while the dotted line represents \((\textrm{Acc}\times \varepsilon )\) for \(\mathrm {\Xi _{c}^{0}}\) baryons originating from beauty-hadron decays. The statistical uncertainties are smaller than the marker size

5 Systematic uncertainties

The contributions to the systematic uncertainty on the prompt \(\mathrm {\Xi _{c}^{0}}\) production cross section are summarised in Table 1.

Table 1 Systematic uncertainties on the prompt \(\mathrm {\Xi _{c}^{0}}\) production cross section

The systematic uncertainty on the raw yield extraction was evaluated by repeating the fit to the invariant mass distributions varying: (i) the function used to describe the background, (ii) the minimum and maximum of the invariant mass intervals considered for the fit, and (iii) the Gaussian width of the mass peak by \(\pm 10\)% compared to the value obtained from simulations. The systematic uncertainty was assigned by adding in quadrature the root mean square error (RMS) of the resulting distribution of the raw yield values and the shift of the mean of the distribution with respect to the value obtained with the default fit configuration.

The systematic uncertainty on the track reconstruction efficiency was estimated by (i) varying the track selection criteria in the TPC and (ii) comparing the ITS-TPC matching efficiency in data and simulations. These contributions are reported as “Tracking efficiency” and “Matching efficiency” in Table 1, respectively. The first contribution to the tracking systematic uncertainty was defined as the RMS of the \(\mathrm {\Xi _{c}^{0}}\) cross section values obtained by repeating the analysis with different TPC track selection criteria. The matching efficiency affects only the pion from the \(\mathrm {\Xi _{c}^{0}}\) decay since no ITS condition was imposed for tracks coming from the \(\Xi ^-\) and \(\Lambda \) decays. The per-track uncertainty on the matching efficiency is \(p_{\textrm{T}}\) dependent, and it was propagated to the \(\mathrm {\Xi _{c}^{0}}\) taking into account the decay kinematics.

The systematic uncertainty on the \(\mathrm {\Xi _{c}^{0}}\) selection efficiency arises due to possible differences between the real detector resolutions and alignment, and their description in the simulation. This uncertainty was assessed by comparing the production cross sections obtained using different selection criteria. In particular, the selections on the BDT outputs were varied in a range corresponding to a modification of about 40% in the efficiency for the prompt \(\mathrm {\Xi _{c}^{0}}\). The systematic uncertainty was assigned as the RMS of the resulting production cross section distribution. An additional source of systematic uncertainty was assigned due to the dependence of the efficiencies on the generated \(p_{\textrm{T}}\) distribution of \(\mathrm {\Xi _{c}^{0}}\) in the simulation (“MC \(p_{\textrm{T}}\) shape” in Table 1). To estimate this effect, the efficiencies were evaluated modifying the weights to match the \(p_{\textrm{T}}\) spectrum obtained from PYTHIA 8 simulations using a tune with colour-reconnection topologies beyond the leading-colour approximation [32], which includes so-called “junctions” that fragment into baryons and lead to an increased baryon production with respect to the Monash tune. An uncertainty was assigned in each \(p_{\textrm{T}}\) interval based on the difference between the central and the varied efficiency.

The systematic uncertainty on the subtraction of feed-down from beauty-hadron decays was estimated by varying: (i) the factorisation and renormalisation scales, and the beauty quark mass in the FONLL calculations (according to the prescriptions in Ref. [80]), and the hypothesis on the fragmentation functions within the uncertainties of the LHCb measurement (taken from Ref. [81]), (ii) the \(\mathrm {\Xi _{c}^{0}} \)/\(\Lambda ^+_\textrm{c}\) ratio, scaling it up by a factor of 2 and down by a factor tuned to accommodate the \(\Xi _\textrm{b}^\mathrm{-}/\Lambda _\textrm{b}^\textrm{0}\) ratio measured by the LHCb Collaboration [39], (iii) the hypothesis on the \(R_{\textrm{pPb}}\) of feed-down \(\mathrm {\Xi _{c}^{0}}\) in the range \(0.9< R_{\mathrm {pPb, feed-down}}/R_{\textrm{pPb, prompt}} < 2\). This range is chosen to cover the uncertainties of the measured \(\mathrm {\Lambda _{c}^{+}}\) \(R_{\textrm{pPb}}\). The theoretical calculation from QCM was also considered for the systematic variation of the \(R_{\textrm{pPb}}\).

The uncertainty on the luminosity measurement is 3.7% for p–Pb collisions [82] and the uncertainty on the decay chain branching ratio is 22% [71].

6 Results

The \(p_{\textrm{T}}\)-differential cross section of prompt \(\mathrm {\Xi _{c}^{0}}\)-baryon production in p–Pb collisions at \(\sqrt{s_{\textrm{NN}}}~=~5.02\) TeV measured in the transverse momentum interval \(2<p_{\textrm{T}} < 12\) GeV/\(c\) and in the rapidity interval \({-0.96< y_\textrm{cms} < 0.04}\) is shown in Fig. 3. In the left panel of Fig. 3, the measured cross section is compared with POWHEG+PYTHIA 6 calculations and with QCM predictions [79].

Fig. 3
figure 3

Prompt \(\mathrm {\Xi _{c}^{0}}\)-baryon \(p_{\textrm{T}}\)-differential cross section in p–Pb collisions at \(\sqrt{s_{\textrm{NN}}}~=~5.02\) TeV. The statistical uncertainties are shown as vertical bars and the systematic uncertainties are shown as boxes. Left: comparison to predictions from POWHEG+PYTHIA 6 [12, 13] simulations matched with the EPPS16 nPDF parameterisation [55], PYTHIA 8 Angantyr calculations [33], and the QCM [79]. The uncertainties on the POWHEG calculation are due to the choice of the pQCD scales and the charm quark mass as described in the text. Right: Comparison with the cross section measured in the semileptonic decay channel in pp collisions at \(\sqrt{s}~=~5.02\) TeV  [26] and in the hadronic decay channel in pp collisions at \(\sqrt{s}~=~13\) TeV  [27] scaled to \(\sqrt{s}~=~5.02\) TeV (the energy-scaling factor is described in the text). Both the pp measurements are scaled by the atomic mass number A of the Pb nucleus

In POWHEG+PYTHIA 6 calculations, the CT14NLO parton distribution functions [83] are used for the proton, while for the Pb nucleus the nuclear modification of the PDFs is modelled with the EPPS16 nPDF parameterisation [55]. The factorisation and renormalisation scales, \(\mu _{\textrm{F}}\) and \(\mu _{\textrm{R}}\), were taken to be equal to the transverse mass of the quark, \(\mu _{\textrm{0}}=\sqrt{m^2+p_{\textrm{T}}^2}\), and the charm-quark mass was set to \({m_{\textrm{c}}=1.5}\) GeV/\(c^2\). The theoretical uncertainties were estimated by varying these scales in the intervals \(0.5 \mu _{0}< \mu _{\textrm{F,R}} < 2 \mu _{\textrm{0}}\), with \(0.5 \mu _{0}< \mu _{\textrm{R}}/\mu _{\textrm{F}} < 2 \mu _{\textrm{0}}\) based on the prescriptions of Ref. [6]. The uncertainties on the nPDF were not included in the calculation as they are considerably smaller than the scale uncertainties. The POWHEG+PYTHIA 6 calculations underestimate the measurement by about a factor of 50. This discrepancy is mostly attributed to the description of the hadronisation process in PYTHIA 6, which is tuned on \(\mathrm {e^+ e^-}\) collisions and underestimates baryon production. This is also supported by the \(\mathrm {\Xi _{c}^{0}}/\textrm{D}^{0} \) and \(\mathrm {\Xi _{c}^{0}}/\mathrm {\Lambda _{c}^{+}} \) ratios described in the following. Notably, a large difference between the measured \(\mathrm {\Xi _{c}^{0}}\) production and the predictions from PYTHIA 8 with the Monash tune, in which the hadronisation is tuned on \(\mathrm {e^+ e^-}\) collisions, was already observed in pp collisions both for the \(\mathrm {\Xi _{c}^{0}}\) cross section [26] and the baryon-over-meson ratios [26, 27]. In the PYTHIA 8 Angantyr calculations [33] the p–Pb collision is treated as a superposition of multiple nucleon–nucleon collisions. In addition, this model implements the colour-reconnection mechanism beyond leading-colour approximation which includes the formation of junctions that fragment into baryons. The junction formation and fragmentation for the colour dipoles containing heavy quarks has been improved. The rope hadronization model, needed to increase the strange quark production in strings breakups, is not yet implemented and its effect is emulated by increasing the overall relative probability of having strange quarks in string breakups. The PYTHIA 8 Angantyr calculations underestimate the measured \(\mathrm {\Xi _{c}^{0}}\) production cross section in p–Pb collisions by a factor of about 3. The QCM calculations implement charm-quark hadronisation via a simplified approach to hadronisation by coalescence, developed solely in the momentum space, without considering the coordinate space. The model estimates the \(p_{\textrm{T}}\) distributions of quarks from a fit to the measured \(\pi \), K, and D meson spectra, and assumes that hadronisation occurs exclusively through coalescence at all momenta [35, 79]. The charm quark is combined with a co-moving light antiquark or two co-moving quarks to form a charm meson or baryon. A free parameter, \(R^{\mathrm {(c)}}_{\mathrm {B/M}}=0.425\), characterises the relative production of single-charm baryons to single-charm mesons and it is tuned to reproduce the \(\mathrm {\Lambda _{c}^{+}}\)/\(\textrm{D}^{0}\) ratio measured by ALICE in pp collisions at \(\sqrt{s}~=~7\) TeV  [22]. The relative abundances of the different charm-baryon species are determined by thermal weights from the statistical hadronisation approach [84]. The QCM prediction, tuned on the \(\mathrm {\Lambda _{c}^{+}}\) production in pp collisions, underestimates the measured \(\mathrm {\Xi _{c}^{0}}\) production cross section in p–Pb collisions by a factor of about 2. A similar discrepancy was observed in pp collisions [27] when comparing the results to a statistical hadronisation approach [37], which also employs thermal weights to determine the abundances of different charm-baryon species [27].

In the right panel of Fig. 3 the prompt \(\mathrm {\Xi _{c}^{0}}\)-baryon production cross section measured in p–Pb collisions is compared with the measurement in the semileptonic decay channel in pp collisions at \(\sqrt{s}~=~5.02\) TeV  [26] and with the measurement in the hadronic decay channel \(\mathrm {\Xi _{c}^{0}} \rightarrow \mathrm{\pi ^{+}}\Xi ^{-}\) in pp collisions at \(\sqrt{s}~=~13\) TeV  [27] scaled to \(\sqrt{s}~=~5.02\) TeV. Both the pp cross sections are scaled by the atomic mass number \(A=208\) of the lead nucleus. The prompt \(\mathrm {\Xi _{c}^{0}}\) production cross section measured in pp collisions at \(\sqrt{s}~=~13\) TeV was first rebinned to match the \(p_{\textrm{T}}\) intervals of the p–Pb measurement. In the rebinning, the raw yield uncertainty was considered uncorrelated across \(p_{\textrm{T}}\) intervals, while all other systematic uncertainties were treated as correlated. The \(p_{\textrm{T}}\)-dependent energy-scaling factor from \(\sqrt{s}~=~13\) TeV to \(\sqrt{s}~=~5.02\) TeV was computed as the ratio of \(\textrm{D}^{0}\)-meson production cross section from FONLL calculations in pp collisions at the same collision energies [85]. The ALICE Collaboration has shown that the energy scaling factors for mesons and baryons are compatible within the uncertainties [18]. The uncertainty on the energy-scaling factor was calculated by varying the renormalisation and factorisation scales, the mass of the charm quark, and the nuclear PDFs consistently at the two collision energies. The total uncertainty is calculated as the envelope of the variations. The two results for the \(\mathrm {\Xi _{c}^{0}}\) cross sections in pp collisions at \(\sqrt{s}~=~5.02\) TeV are in agreement with each other within uncertainties. The cross section measured via the hadronic decay channel in pp collisions at \(\sqrt{s}~=~13\) TeV scaled to \(\sqrt{s}~=~5.02\) TeV is used as the pp reference for the calculation of the nuclear modification factor because the two measurements are performed in the same decay channel and the systematic uncertainty due to the branching ratio, which is the dominant systematic uncertainty, cancels out in the ratio. Furthermore, the cross section in pp collisions at \(\sqrt{s}~=~13\) TeV was measured in the \(p_{\textrm{T}}\) interval 8–12 GeV/c, allowing the computation of the \(R_{\textrm{pPb}}\) up to higher \(p_{\textrm{T}}\) as compared to the measurement in pp collisions at \(\sqrt{s}~=~5.02\) TeV, which extends only up to \(p_{\textrm{T}}\) = 8 GeV/c.

Fig. 4
figure 4

Nuclear modification factor \(R_{\textrm{pPb}}\) of prompt \(\mathrm {\Xi _{c}^{0}}\) baryons in p–Pb collisions at \(\sqrt{s_{\textrm{NN}}}~=~5.02\) TeV as a function of \(p_{\textrm{T}}\) compared to the \(R_{\textrm{pPb}}\) of \(\mathrm {\Lambda _{c}^{+}}\) baryons [14]. The measured \(R_{\textrm{pPb}}\) is also compared to POWHEG+PYTHIA 6 with EPPS16 simulations, to PYTHIA 8 Angantyr calculations [33], and to QCM predictions. The statistical uncertainties are shown as vertical bars and the systematic uncertainties are shown as boxes. The uncertainties on the POWHEG calculation are due to the choice of the pQCD scales as described in the text

To better investigate any effect due to the collision system, the nuclear modification factor \(R_{\textrm{pPb}}\) was calculated as the ratio between the \(p_{\textrm{T}}\)-differential \(\mathrm {\Xi _{c}^{0}}\) cross section in p–Pb collisions and the reference pp measurement scaled by the nuclear mass number A of the lead nucleus and corrected to account for the rapidity shift between pp and p-Pb collisions using FONLL calculations [6]. The result is shown in Fig. 4. The systematic uncertainties on the branching ratio and beauty feed-down are treated as fully correlated between the two collision systems, and all other systematic uncertainties are considered as uncorrelated. The central values of the \(\mathrm {\Xi _{c}^{0}}\)-baryon \(R_{\textrm{pPb}}\) are larger than unity in the full \(p_{\textrm{T}}\) interval of the measurement, even though all data points are compatible with unity within the large uncertainties (the maximal deviation is 1.5\(\sigma \) for the \(p_{\textrm{T}}\) interval 8–12 GeV/\(c\)). This also prevents to establish if an increasing trend with \(p_{\textrm{T}}\) is present. The \(\mathrm {\Xi _{c}^{0}}\) \(R_{\textrm{pPb}}\) is compatible with the \(\mathrm {\Lambda _{c}^{+}}\) \(R_{\textrm{pPb}}\)  [14] within uncertainties pointing to a similar modification of the production of these two baryon species in p–Pb collisions with respect to pp collisions. The measured \(R_{\textrm{pPb}}\) is compared to POWHEG+PYTHIA 6 simulations, PYTHIA 8 Angantyr calculations, and QCM predictions. The only nuclear effect included in the POWHEG + PYTHIA 6 calculations is due to the EPPS16 modification of the PDFs. The resulting \(R_{\textrm{pPb}}\) is lower than unity with a mild \(p_{\textrm{T}}\) dependence, and it reproduces within the large uncertainties the measured \(\mathrm {\Xi _{c}^{0}}\) \(R_{\textrm{pPb}}\). On the other hand, some tension is visible for the \(\mathrm {\Lambda _{c}^{+}}\) in the \(4< p_{\textrm{T}} < 8\) GeV/\(c\) interval. The QCM reproduces both measurements within their uncertainties. The comparison with POWHEG+PYTHIA 6 calculations suggests that the deviation of \(R_{\textrm{pPb}}\) from unity (significant for the \(\mathrm {\Lambda _{c}^{+}}\)) may have influences beyond the alteration of the PDFs of nucleons within the nuclei’s structure compared to those of protons. Additional effects, possibly related to the hadronisation process and the presence of an expanding medium, may play a role, as suggested by the modification of the \(\mathrm {\Lambda _{c}^{+}}\) \(p_{\textrm{T}}\) shape between pp and p–Pb collisions [14]. However, the large uncertainties associated with the \(\mathrm {\Xi _{c}^{0}}\) results prevent conclusive interpretations, because the measured \(R_{\textrm{pPb}}\) exhibits a trend compatible with both the \(\mathrm {\Lambda _{c}^{+}}\) and a flat trend. The \(R_{\textrm{pPb}}\) obtained from PYTHIA 8 Angantyr calculations shows a mild increasing trend with \(p_{\textrm{T}}\), reaching unity at \(p_{\textrm{T}} \sim 3\) GeV/\(c\) for both \(\mathrm {\Xi _{c}^{0}}\) and \(\mathrm {\Lambda _{c}^{+}}\) and is compatible with the measurements within the large uncertainties.

The ratio of the production cross sections of different hadron species is sensitive to the modification of the hadronisation mechanisms in a partonic environment. The \(p_{\textrm{T}}\)-dependent \(\mathrm {\Xi _{c}^{0}}/\textrm{D}^{0} \) baryon-to-meson yield ratio measured in pp and p–Pb collisions at \(\sqrt{s_{\textrm{NN}}}~=~5.02\) TeV  [26] is reported in the left panel of Fig. 5. For the measurement in p–Pb collisions, the prompt \(\textrm{D}^{0}\) cross section reported in Ref. [54] is used. The systematic uncertainty on the \(\mathrm {\Xi _{c}^{0}}/\textrm{D}^{0} \) yield ratio is calculated assuming all the uncertainties of the \(\mathrm {\Xi _{c}^{0}}\) and \(\textrm{D}^{0}\) cross sections as uncorrelated, except for the tracking and feed-down systematic uncertainties, which partially cancel in the ratio, and the uncertainty on the luminosity which fully cancels in the ratio. The \(\mathrm {\Xi _{c}^{0}}/\textrm{D}^{0} \) ratio in p–Pb collisions hints at a slightly decreasing trend with \(p_{\textrm{T}}\), similar to the one measured in pp collisions, albeit with large uncertainties. The theoretical predictions from QCM are also shown and they underpredict the \(\mathrm {\Xi _{c}^{0}}/\textrm{D}^{0} \) ratios by a similar amount in both collision systems. This discrepancy is predominantly due to the low \(\mathrm {\Xi _{c}^{0}}\)-baryon yield predicted by this model, as it can be seen by the fact that QCM undershoots the \(\mathrm {\Xi _{c}^{0}}\) cross section (see Fig. 3) and the \(\mathrm {\Xi _{c}^{0}}/\mathrm {D^0} \) yield ratio by the same amount. PYTHIA 8 Angantyr calculations [33] are also shown for both pp and p–Pb collisions. This PYTHIA 8 implementation improves the description for \(\mathrm {\Xi _{c}^{0}}\) with respect to previous calculations [32] available for pp collisions. However, the model still underestimates the \(\mathrm {\Xi _{c}^{0}}/\mathrm {D^0} \) yield ratio by a similar amount as the QCM model in both pp and p–Pb collisions. One difference between the two models is that the PYTHIA 8 Angantyr calculation does not exhibit a crossing at low \(p_{\textrm{T}}\) between pp and P–Pb collisions, indicating signs of larger radial flow in p–Pb collisions, but the calculation in p–Pb collisions is larger than the pp one in the full \(p_{\textrm{T}}\) interval. POWHEG+PYTHIA 6 calculations predict a slightly increasing trend with \(p_{\textrm{T}}\), and undershoot the measured \(\mathrm {\Xi _{c}^{0}}/\textrm{D}^{0} \) yield ratio by approximately a factor of 20. Given that these predictions underestimate the \(\textrm{D}^{0}\) production only by a factor of about 2, it is reasonable to conclude that the large discrepancy in the description of the \(\mathrm {\Xi _{c}^{0}}\)-baryon production lies primarily in the hadronisation provided by PYTHIA 6, which is tuned on \(\mathrm {e^+e^-}\) collisions, rather than in the description of charm production or in an inaccurate modelling of initial state CNM effects. Furthermore, given that a difference of \(1.7\sigma \) between the \(\mathrm {\Xi _{c}^{0}}\)/\(\textrm{D}^{0}\) yield ratio in pp and p–Pb collisions is measured in the 6–8 GeV/\(c\) \(p_{\textrm{T}}\) interval, it is not possible to conclude on a possible enhancement of this baryon-over-meson yield ratio in p–Pb collisions with respect to pp collisions.

Fig. 5
figure 5

Left: \(\mathrm {\Xi _{c}^{0}}\)/\(\textrm{D}^{0}\) ratio as a function of \(p_{\textrm{T}}\) in p–Pb collisions at \(\sqrt{s_{\textrm{NN}}}~=~5.02\) TeV and in pp collisions at \(\sqrt{s}~=~5.02\) TeV  [26] compared to QCM, POWHEG + PYTHIA 6, and PYTHIA 8 Angantyr predictions. Right: \(\mathrm {\Xi _{c}^{0}}\)/\(\mathrm {\Lambda _{c}^{+}}\) ratio as a function of \(p_{\textrm{T}}\) in p–Pb collisions at \(\sqrt{s_{\textrm{NN}}}~=~5.02\) TeV and in pp collisions at \(\sqrt{s}~=~13\) TeV  [27] compared to QCM, POWHEG + PYTHIA 6, and PYTHIA 8 Angantyr calculations. The BR uncertainties for the \(\mathrm {\Lambda _{c}^{+}}\) are evaluated as the weighted average on the two decay channels [22] and are included in the systematic uncertainty box for each \(p_{\textrm{T}}\) bin

The right panel of Fig. 5 reports the \(\mathrm {\Xi _{c}^{0}}\)/\(\mathrm {\Lambda _{c}^{+}}\) baryon-to-baryon yield ratio measured in p–Pb collisions at \(\sqrt{s_{\textrm{NN}}}~=~5.02\) TeV compared with the ratio measured in pp collisions at \(\sqrt{s}~=~13\) TeV  [27]. The two measurements are in agreement within uncertainties, and they exhibit no significant \(p_{\textrm{T}}\) dependence within the current measurement uncertainties. This suggests that there is no appreciable additional modification of the hadronisation process when moving from pp to p–Pb collisions. The QCM calculations show a slightly increasing trend as a function of \(p_{\textrm{T}}\) and underestimate the measured ratio by a factor of about 2.5 (1.6) in the 2–4 (6–8) GeV/c \(p_{\textrm{T}}\) intervals. PYTHIA 8 Angantyr calculations in pp and p–Pb collisions are compatible and show a slightly increasing trend as a function of \(p_{\textrm{T}}\). These calculations underestimate the measured ratio by a factor of about 2.4 (1.7) in the 2–4 (6–8) GeV/c \(p_{\textrm{T}}\) intervals. POWHEG+PYTHIA 6 predictions are also included in the figure, underestimating the \(\mathrm {\Xi _{c}^{0}}\)/\(\mathrm {\Lambda _{c}^{+}}\) yield ratio by a factor of about 4 at all \(p_{\textrm{T}}\). This reflects a similar underprediction in the production of both baryon species. However, the \(\mathrm {\Xi _{c}^{0}}\)-baryon production appears to be underpredicted more than the \(\mathrm {\Lambda _{c}^{+}}\)-baryon one, suggesting that additional effects, such as recombination, could be considered in order to provide a better description of the results, although the large uncertainties preclude definitive conclusions.

Fig. 6
figure 6

\(\mathrm {\Xi _{c}^{0}}\)/\(\textrm{D}^{0}\) (left) and \(\mathrm {\Xi _{c}^{0}}\)/\(\mathrm {\Lambda _{c}^{+}}\) (right) ratios as a function of \(p_{\textrm{T}}\) in p–Pb collisions at \(\sqrt{s_{\textrm{NN}}}~=~5.02\) TeV measured by ALICE, compared to the \(\mathrm {\Xi _{c}^{+}}\)/\(\textrm{D}^{0}\) and the \(\mathrm {\Xi _{c}^{+}}\)/\(\mathrm {\Lambda _{c}^{+}}\) ratio measured by LHCb at \(\sqrt{s_{\textrm{NN}}}~=~8.16\) TeV  [86], and PYTHIA 8 Angantyr predictions [33]

In Fig. 6 the particle ratios measured by ALICE in the central rapidity region (\(-0.96< y_\textrm{cms} < 0.04\)) in p–Pb collisions at \(\sqrt{s_{\textrm{NN}}}~=~5.02\) TeV are compared to the \(\mathrm {\Xi _{c}^{+}}/\textrm{D}^{0} \) (left panel) and \(\mathrm {\Xi _{c}^{+}}/\mathrm {\Lambda _{c}^{+}} \) (right panel) yield ratios measured by LHCb at forward (\(1.5< y_\textrm{cms} < 4.0\)) and backward (\(-5.0< y_\textrm{cms} < 2.5\)) rapidities in p–Pb collisions at \(\sqrt{s_{\textrm{NN}}}~=~8.16\) TeV  [86]. A direct comparison between \(\mathrm {\Xi _{c}^{0}}\) and \(\mathrm {\Xi _{c}^{+}}\) results is possible because these two baryons are expected to be produced in equal amounts due to isospin symmetry and it was indeed demonstrated in pp collisions at \(\sqrt{s}~=~13\) TeV that the production cross sections of these two baryon species are compatible within uncertainties [18]. While the baryon-over-meson yield ratio provides an indication of a rapidity dependence, with the yield ratio at midrapidity being larger than the ones at forward and backward rapidities in the full \(p_{\textrm{T}}\) range (a difference ranging from \(1.5 \sigma \) to \(2.0 \sigma \) is measured across the different \(p_{\textrm{T}}\) intervals), the baryon-over-baryon yield ratios are compatible at mid, forward, and backward rapidity within the uncertainties (a difference of \(1.1 \sigma \) is measured for the 2–4 GeV/\(c\) \(p_{\textrm{T}}\) interval). Conducting measurements of baryon-over-meson and baryon-over-baryon yield ratios at the same centre-of-mass energy per nucleon pair for the same hadron reconstructed in the same decay channel will allow the removal of potential ambiguities in the interpretation of the results, helping to unravel possible rapidity dependence of the charm baryon enhancement. PYTHIA 8 Angantyr calculations, computed for p–Pb collisions for different rapidity intervals, are also shown in Fig. 6. A small dependence on the rapidity is observed for the \(\mathrm {\Xi _{c}^{0}}/\mathrm {D^0} \) ratio. The computed ratio is largest at midrapidity and lowest at backward rapidity. This dependence might be introduced by the different parton and string densities in this asymmetric collision system. However, such a difference is not observed for the baryon-to-baryon yield ratio. Although the baryon-to-meson and baryon-to-baryon ratios measured at midrapidity are underestimated by PYTHIA 8 Angantyr, the model agrees with the forward-rapidity measurements within uncertainties.

The visible prompt \(\mathrm {\Xi _{c}^{0}}\)-baryon cross section is computed by integrating the \(p_{\textrm{T}}\)-differential cross section in the measured \(p_{\textrm{T}}\) region. In the integration, the systematic uncertainties were propagated considering the uncertainty due to the raw-yield extraction as fully uncorrelated and all the other sources as fully correlated between the \(p_{\textrm{T}}\) intervals. The visible \(\mathrm {\Xi _{c}^{0}}\)-baryon cross section is

$$\begin{aligned} & \textrm{d}\sigma ^{\mathrm {\Xi _{c}^{0}}}_{\textrm{pPb}}/\textrm{d} y \big |^{(2< p_{\textrm{T}}< 12 \text {~GeV}/c)}_{-0.96< y < 0.04} \\ & \quad = 8.6~\pm ~1.6\,\mathrm {(stat.)}~\pm ~1.3\,\mathrm {(syst.)}\\ & \qquad \pm ~1.9\, \mathrm {(BR)}~\pm ~0.3\,\mathrm {(lumi.)}~\textrm{mb}. \end{aligned}$$

The \(p_{\textrm{T}}\)-integrated \(\mathrm {\Xi _{c}^{0}}\) cross section at midrapidity is obtained by extrapolating the visible cross section to the full \(p_{\textrm{T}}\) range. The QCM spectra were chosen to perform the extrapolation, as they provide a reasonable description of the measured \(p_{\textrm{T}}\) shapes for both the production cross section and the \(\mathrm {\Xi _{c}^{0}}/\mathrm {D^0} \) baryon-to-meson yield ratio. The extrapolation factor was calculated as the ratio between the QCM predictions in the full \(p_{\textrm{T}}\) range and those in the \(p_{\textrm{T}}\) interval of the measurement. The scaling factor is 1.74. As QCM does not provide any theory uncertainties, an extrapolation uncertainty was determined using two contributions: (i) the maximum difference among the extrapolation factors obtained with QCM, POWHEG+PYTHIA 6, and PYTHIA 8 Angantyr (\( ^{+20.6}_{-0.0}\)%) and (ii) varying the factorisation and renormalisation scales (\(\mu _\textrm{F,R}\)) and the EPPS16 parameters used in the POWHEG+PYTHIA 6 calculations and determining the maximal relative differences (\( ^{+23.0}_{-8.6}\)%). For the variations of \(\mu _\textrm{F,R}\), the standard variation ranges were used (\(0.5 \mu _0< \mu _\textrm{R,F} < 2 \mu _0\), with \(0.5< \mu _{\textrm{R}}/\mu _{\textrm{F}} < 2\), where \(\mu _0 = \sqrt{m_\textrm{c}^2 + p_\textrm{T}^2})\). These contributions were added in quadrature to obtain an overall extrapolation uncertainty of \( ^{+30.9}_{-8.6}\)% on the total \(\mathrm {\Xi _{c}^{0}}\) production cross section. The resulting \(p_{\textrm{T}}\) integrated cross section for the \(\mathrm {\Xi _{c}^{0}}\) is

$$\begin{aligned} & \textrm{d}\sigma ^{\mathrm {\Xi _{c}^{0}}}_{\textrm{pPb}}/\textrm{d} y \big |_{-0.96< y < 0.04} = 15.0~\pm ~2.8\,\mathrm {(stat.)}~^{+2.2}_{-2.3}\,\mathrm {(syst.)}~\\ & \pm ~3.3\,\mathrm {(BR)}~\pm ~0.6\,\mathrm {(lumi.)}~^{+4.6}_{-1.3}\,\mathrm {(extr.)}~\textrm{mb}. \end{aligned}$$

7 Summary

The measurement of prompt \(\mathrm {\Xi _{c}^{0}}\)-baryon production at midrapidity in p–Pb collisions at \(\sqrt{s_{\textrm{NN}}}~=~5.02\) TeV with the ALICE detector at the LHC is reported. The \(p_{\textrm{T}}\)-differential cross section for the production of prompt \(\mathrm {\Xi _{c}^{0}}\) baryons is compared to POWHEG+PYTHIA 6 calculations. As seen in pp collisions, the results indicate a significant underestimation, by a factor of about 50, of the \(\mathrm {\Xi _{c}^{0}}\)-baryon yield. This is mainly due to the small fraction of charm quarks hadronising into baryons in the PYTHIA 6 fragmentation stage, which is tuned on results from e\(^+\)e\(^-\) collisions. PYTHIA 8 Angantyr predictions, with colour reconnections beyond the leading-colour approximation, include the formation of junctions that fragment into baryon. Therefore, the production of \(\mathrm {\Xi _{c}^{0}}\) baryons is enhanced as compared to POWHEG+PYTHIA 6 calculations and the discrepancy with the measured \(\mathrm {\Xi _{c}^{0}}\) production cross section is reduced to a factor of about 3. The QCM calculations, which implement charm-quark hadronisation via coalescence and are tuned to reproduce the \(\mathrm {\Lambda _{c}^{+}}\)/\(\mathrm {D^0}\) yield ratio measured by ALICE in pp collisions at \(\sqrt{s}~=~7\) TeV, provide a better description of the measured \(\mathrm {\Xi _{c}^{0}}\) production cross section and \(\mathrm {\Xi _{c}^{0}}\)/\(\mathrm {D^0}\) yield ratio than POWHEG+PYTHIA 6 calculations, albeit they still underestimate the \(\mathrm {\Xi _{c}^{0}}\)-baryon yield by a factor of about 2. Neither the introduction of colour reconnections beyond the leading-colour approximation in PYTHIA 8 Angantyr, nor the coalescence mechanism as implemented in QCM quantitatively captures the \(\mathrm {\Xi _{c}^{0}}\) enhancement relative to the \(\mathrm {\Lambda _{c}^{+}}\).

The \(p_{\textrm{T}}\)-differential \(R_{\textrm{pPb}}\) is larger than unity in the full \(p_{\textrm{T}}\) interval of the measurement, even though all data points are compatible with unity within the large uncertainties. The \(R_{\textrm{pPb}}\) is compatible within uncertainties with both a flat and a slightly increasing trend as the \(p_{\textrm{T}}\) increases, similar to the one observed for \(\mathrm {\Lambda _{c}^{+}}\) baryons [14]. It is underpredicted by POWHEG+PYTHIA 6 calculations with the EPPS16 parameterisation of the nuclear PDFs, suggesting that additional final-state effects might be relevant for the \(\mathrm {\Xi _{c}^{0}}\)-baryon production in p–Pb collisions, while the QCM predictions are compatible with the results within the uncertainties. PYTHIA 8 Angantyr predictions provide a similar \(p_{\textrm{T}}\)-differential \(R_{\textrm{pPb}}\) for \(\mathrm {\Xi _{c}^{0}}\) and \(\mathrm {\Lambda _{c}^{+}}\) baryons, exhibiting a mild increasing trend with \(p_{\textrm{T}}\) and similar magnitudes to the POWHEG+PYTHIA 6 calculations. The \(\mathrm {\Xi _{c}^{0}}\) \(R_{\textrm{pPb}}\) is compatible with that of \(\mathrm {\Lambda _{c}^{+}}\) baryons within uncertainties, possibly hinting to a similar modification of the production of the two baryon species in p–Pb collisions with respect to pp collisions.

The \(\mathrm {\Xi _{c}^{0}}/\mathrm {D^0} \) baryon-to-meson yield ratio shows a slightly decreasing trend with increasing \(p_{\textrm{T}}\). It is underestimated by the QCM predictions by a factor of about two. This discrepancy originates from the underestimation of the \(\mathrm {\Xi _{c}^{0}}\) production cross section. The POWHEG+PYTHIA 6 simulations underestimate the measured ratio by a factor of 20, mainly due to the description of the charm-quark fragmentation in PYTHIA 6. The introduction of colour reconnections beyond the leading-colour approximation implemented in PYTHIA 8 Angantyr reduces the data-model discrepancy, but still leads to an underestimation of the \(\mathrm {\Xi _{c}^{0}}/\mathrm {D^0} \) yield ratio similar to that of QCM. The \(\mathrm {\Xi _{c}^{0}}\)/\(\mathrm {\Lambda _{c}^{+}}\) ratio is also reported. It is compared to the results obtained in pp collisions at \(\sqrt{s}~=~13\) TeV. The two measurements are compatible within the uncertainties and do not present a significant \(p_{\textrm{T}}\) dependence. The measured ratios are underestimated by QCM, POWHEG+PYTHIA 6, and PYTHIA 8 Angantyr calculations. The reported \(\mathrm {\Xi _{c}^{0}}/\mathrm {D^0} \) yield ratio is found to be larger than the measurements of \(\mathrm {\Xi _{c}^{+}}\)/\(\mathrm {D^0}\) performed by LHCb at forward rapidity, while the \(\mathrm {\Xi _{c}^{0}}\)/\(\mathrm {\Lambda _{c}^{+}}\) and \(\mathrm {\Xi _{c}^{+}}\)/\(\mathrm {\Lambda _{c}^{+}}\) yield ratios measured in the two rapidity intervals are found to be compatible within uncertainties. Lastly, the \(p_{\textrm{T}}\)-integrated \(\mathrm {\Xi _{c}^{0}}\) production cross section is measured by extrapolating the visible cross section to the full \(p_{\textrm{T}}\) range using the \(p_{\textrm{T}}\) shape from QCM predictions.

These measurements provide important inputs and constraints to theoretical model calculations of the hadronisation process. Moreover, these results will provide key inputs for the computation of the charm quark fragmentation fractions in p–Pb collisions.