1 Introduction

In ultra-relativistic heavy-ion collisions, the impact of two colliding nuclei creates an extremely hot and dense medium, in which quarks and gluons are liberated from confinement inside hadrons and form a new state of matter, referred to as quark–gluon plasma (QGP) [1, 2]. In the past 20 years, extensive experimental evidence from the Relativistic Heavy Ion Collider (RHIC) and Large Hadron Collider (LHC), including the jet quenching and strong particle flow of light-flavor hadrons (consisting of the light quarks u, d and s), has shown that the QGP matter is strongly coupled and behaves like a liquid with a small viscosity-to-entropy-density ratio [3,4,5,6]. However, most of the light-flavor hadrons are produced late in the collision process, along with final state effects; thus, information about the QGP created during early stages of the collision may be smeared.

Heavy-quark masses (\(m_{c} \sim 1.3\,\hbox {GeV}/c^{2}\), \(m_{b} \sim 4.8\,\hbox {GeV}/c^{2}\)) are much larger than those of light quarks and exceed the quantum chromodynamics (QCD) energy scale (\(\Lambda _{\mathrm{QCD}}\)). They are predominately produced via initial hard processes during the early stages of ultra-relativistic heavy-ion collisions; here, the probability of thermal production is negligible, especially at RHIC energies. Thus, heavy quarks experience the whole evolution of the QCD matter created in heavy-ion collisions, making them ideal probes to study QGP matter properties. Most of the heavy quarks hadronize into open heavy flavor mesons (e.g., \(D^0\), \(D^{\pm }\), \(D_{s}^{\pm }\), \(B^0\), \(B^{\pm }\), and \(B_{s}^{\pm }\)) and baryons (e.g., \(\Lambda _{c}\)); however, a small fraction (\(<1\)% of the total heavy quarks) form hidden heavy flavor quarkonium states (e.g., \(J/\psi\) and \(\Upsilon\)) and their families.

Theoretical models predict that heavy quarks loose less energy compared with light quarks because their large masses suppress the gluon radiation angle [7]. Measurement of the open charm/bottom nuclear modification factor (\(R_{\mathrm{AA}}\))—defined as the yield measured in Au + Au collisions divided by that obtained in \(p+p\) collisions and scaled by the average number of binary collisions (\(N_{\mathrm{coll}}\))—is commonly used to evaluate the medium effects, which are characterized as a deviation from unity. The strong suppression of open charm hadrons \(R_{\mathrm{AA}}\) in central heavy-ion collisions at a high transverse momentum (\(p_{\mathrm{T}}\)), recently measured by the STAR and ALICE experiments [8, 9], indicates strong interactions between charm quarks and the hot–dense medium. The similar suppression levels of charmed and light hadrons can be explained by model calculations incorporating both elastic and inelastic energy losses [10, 11]. Meanwhile, open bottom hadron measurements are difficult to achieve owing to poor production rates and small hadronic decay branching ratios. An effective way to measure bottom hadrons is via their decay products.

On the other hand, the second-order coefficient of the particle azimuth distribution’s Fourier expansion in the momentum space, \(v_2\), is commonly used to measure medium bulk properties and assess how the medium transports partons [12]. Heavy quarks are expected to struggle to participate in the partonic collectivity due to their large masses. However, recent measurements of a large elliptic flow of D mesons indicate that charm quarks have a similar flow to light quarks and may achieve thermalization [13,14,15].

By comparing experimental data and theoretical model calculations, the transport diffusion coefficients of charm quarks traversing the medium can be obtained with large uncertainties [13]. To better understand the interactions between heavy quarks and the medium, experiments continue to focus on upgrading detectors, to pursue high spatial resolutions and fast responses for precise, next-generation measurements of heavy flavor hadron production.

Quarkonium is a tightly bound state consisting of a heavy quark and its antiquark. Charmonium (bottomonium) refers to the bound state of a charm (bottom) quark and its antiquark. Table 1 shows the masses, binding energies, and radii of various quarkonium states [16]. When QGP is formed, the potential between the heavy quark and its antiquark is expected to be modified by the deconfined medium. The real part of the potential can be color-screened statically by the medium, resulting in a broadening of the wave function of the heavy quark–antiquark pair; simultaneously, the imaginary part of the potential relates to the dissociation of quarkonium, which arises from the scattering of quarkonium with medium constituents such as gluons. The suppression of \(J/\psi\) as a result of color screening was proposed as a signature of QGP formation [17], and it was considered to be strong experimental evidence of deconfinement in the medium produced in Pb + Pb collisions at the Super Proton Synchrotron (SPS) [18].

The temperature required to dissociate a quarkonium state (dissociation temperature, \(T_{\mathrm{d}}\)) depends on its binding energy. A more loosely bounded state has a lower \(T_{\mathrm{d}}\). In both charmonium and bottomonium sectors, \(T_{\mathrm{d}}\) decreases with increasing quarkonium mass, and the excited states have a lower \(T_{\mathrm{d}}\) than the 1S state. From the radii of the quarkonium shown in Table 1, it is expected that \(T_{\mathrm{d}}^{\Upsilon (1S)}> T_{\mathrm{d}}^{\chi _b} \sim T_{\mathrm{d}}^{J/\psi } \sim T_{\mathrm{d}}^{\Upsilon (2S)}> T_{\mathrm{d}}^{\chi _b^{\prime }} \sim T_{\mathrm{d}}^{\chi _c} \sim T_{\mathrm{d}}^{\Upsilon (3S)} > T_{\mathrm{d}}^{\psi (2S)}.\) The systematical measurements of quarkonium suppression can also help to constrain the temperature profile and dynamic evolution of the fireball produced in ultra-relativistic heavy-ion collisions.

Table 1 The masses, binding energies, and radii of charmonium and bottomonium states [16]

In contrast to color-screening, the quarkonium production yield can be enhanced through the (re)combination of (un)associated heavy quarks and their antiquarks during QGP evolution and/or hadronization. The dissociation rate and/or recombination probability depends on the properties of the QGP—including the temperature profile and evolution of the fireball—and the size of the quarkonia. Although the (re)combination effect counteracts the QGP melting effect, both require deconfinement and can be used to search for QGP and study its properties.

In addition to these two hot nuclear matter effects, quarkonium production in heavy-ion collisions is also affected by cold nuclear matter (CNM) effects, including the modification of the parton distribution functions of nuclei (nPDF), the breakup by hadrons, and the scattering and/or energy losses of the partons evolved during quarkonium production. The CNM effects can be experimentally studied via the collisions of p or light nucleus and heavy nucleus. Other effects must be considered when interpreting the experimental results. One important effect is the feeddown contribution of quarkonium production.

The relative contributions of these effects have different dependencies on a range of variables, including the initial energy density, system size, total heavy quark cross section, and the size and transverse momentum (\(p_{\mathrm{T}}\)) of the quarkonium state; thus, a comprehensive study of the quarkonium yield as a function of the collision energy, collision system, quarkonium \(p_{\mathrm{T}}\), and rapidity of different quarkonium states, as well as the collectivity of heavy flavor hadrons and quarkonium, is essential for a complete understanding of quarkonium production in heavy-ion collisions.

In the following sections, we refer to the average of a particle and antiparticle when using the term “particle,” unless otherwise specified.

2 Open heavy flavor production

2.1 Open charm production

The charm production cross sections in high-energy \(p+p\) collisions can be evaluated using perturbative QCD [19, 20]. The differential transverse momentum (\(p_{\mathrm{T}}\)) spectra of \(D^0\) mesons in a wide energy range (from \(\sqrt{s} = 200\,\hbox {GeV}\) up to 7 TeV) in \(p+p\) collisions as measured by STAR [21, 22], CDF [23], and ALICE [24,25,26] experiments, respectively, are in good agreement with the upper limit of fixed-order-next-to-leading-logarithm (FONLL) calculations [19, 27,28,29]. In heavy-ion collisions, charm quarks interact with the hot–dense medium, and their transverse momenta are modified via energy loss, collective flow, or CNM effects. The charmed hadrons are formed via charm hadronization arising from fragmentation, coalescence, or recombination until chemical freeze-out occurs. After kinetic freeze-out, the final state interactions cease, and the charmed hadrons’ spectra are fixed at their measured values. Figure 1 shows the centrality dependence of the charmed hadron \(p_{\mathrm{T}}\) spectra; these measurements were taken by STAR and were facilitated by identifying the secondary decay vertices of charmed hadrons with a recently developed state-of-the-art silicon pixel detector, the heavy flavor tracker (HFT) [30, 31]. The \(D^0\)\(p_{\mathrm{T}}\) spectra at mid-rapidity (\(|y|<1\)) in 0–10%, 10–20%, 20–40%, 40–60%, and 60–80% Au + Au collisions [32] are shown in the left-hand panel. The \(D_s\)\(p_{\mathrm{T}}\) spectra (triangles) in 0–10%, 10–40%, 40–80%, and \(\Lambda _c\) spectrum (stars) in 10–80% Au + Au collisions at \(|y|<1\) [33, 34] are shown in the right-hand panel. The spectra in some centrality bins are scaled using arbitrary factors, which are indicated on the figure for clarity.

Fig. 1
figure 1

Left-hand panel: The \(D^0\)\(p_{\mathrm{T}}\) spectra at \(|y|<1\) in 0–10%, 10–20%, 20–40%, 40–60%, and 60–80% Au + Au collisions [32]. Right-hand panel: The \(D_s\)\(p_{\mathrm{T}}\) spectra (triangles) in 0–10%, 10–40%, 40–80%, and \(\Lambda _c\) spectrum (stars) in 60–80% Au + Au collisions at \(|y|<1\) [33, 34]. The statistical and systematic uncertainties are represented by vertical error bars and brackets, respectively

The nuclear modification factor \(R_{\mathrm{AA}}\) is calculated as the ratio of \(N_{\mathrm{bin}}\)-normalized yields between Au + Au and \(p+p\) collisions. The \(R_{\mathrm{AA}}\) of \(D^0\) mesons in 0-10% central Au + Au collisions at \({\sqrt{s_{\mathrm{NN}}} = \mathrm {200\,GeV}}\) [32] is compared to that of (a) an average D-meson from ALICE [35] and (b) charged hadrons from ALICE and \(\pi ^{\pm }\) from STAR [36, 37], as shown in Fig. 2. The \(D^0\)\(R_{\mathrm{AA}}\) from this measurement is comparable to that taken from LHC measurements for Pb + Pb collisions at \(\sqrt{s_{_{\mathrm{NN}}}}\) = 2.76 TeV, despite the large energy difference between these measurements. A significant suppression is visible at \(p_{\mathrm{T}}>\) 5 GeV/c. The suppression level is similar to that of light-flavor mesons, indicating strong interactions of the charm quark with the medium and energy loss. At \(p_{\mathrm{T}}\,<\,5\,\hbox {GeV}/c\), the \(D^0\)\(R_{\mathrm{AA}}\) shows a characteristic “bump” structure. The Duke model and the linearized Boltzmann transport (LBT) model [38, 39] calculations, which predict sizable collective motions for charm quarks during the medium evolution, can qualitatively describe the STAR data. The uncertainties from the \(p+p\) reference [21] dominate the systematic uncertainty for STAR \(R_{\mathrm{AA}}\).

Fig. 2
figure 2

(Color online) \(D^{0}\)\(R_{\mathrm{AA}}\) in 0–10% Au + Au collisions at \({\sqrt{s_{\mathrm{NN}}} = \mathrm {200\,GeV}}\), compared against a the ALICE D-meson results in 0–10% Pb + Pb collisions at \(\sqrt{s_{_{\mathrm{NN}}}}\) = 2.76 TeV and b charged hadrons from ALICE and \(\pi ^{\pm }\) from STAR. a Also shows the model calculations from the LBT and Duke groups [38, 39]. The notations for statistical and systematic uncertainties are the same as those in previous figures

Several functions, including the Levy, power law, \(m_{\mathrm{T}}\) exponential, blast wave [40], and Tsallis blast wave [41], are used to fit the \(D^0\) data in the above centrality bins, to extract the collectivity and thermal properties. In this paper, only the physical results and conclusions are discussed. Detailed analyses can be found in Ref. [32]. The slope parameter \(T_{\mathrm{eff}}\) obtained for \(D^0\) mesons is compared to other light and strange hadrons measured at the RHIC. The left-hand panel of Fig. 3 summarizes the slope parameter \(T_{\mathrm{eff}}\) for various identified hadrons (\(\pi ^{\pm }\), \(K^{\pm }\), p/\(\bar{p}\), \(\phi\), \(\Lambda\), \(\Xi ^-\), \(\Omega\), \(D^0\), and \(J/\psi\)) in central Au + Au collisions at \(\sqrt{s_{_{\mathrm{NN}}}} \mathrm{= 200\,GeV}\) [42,43,44,45]. Point-by-point statistical and systematic uncertainties are added as a quadratic sum when fitting. All fits are performed up to \(m_{\mathrm{T}} - m_{0} <1\,\mathrm {GeV}/c^2\) for \(\pi ,\ K,\ p\); \(<2\,\hbox {GeV}/c^2\) for \(\phi ,\ \Lambda ,\ \Xi\); and \(<3\,\hbox {GeV}/c^2\) for \(\Omega ,\ D^{0},\ J/\psi\), respectively.

The slope parameter \(T_{\mathrm{eff}}\) in a thermalized medium can be characterized by the random (generally interpreted as the kinetic freeze-out temperature \(T_{\mathrm{fo}}\)) and collective (radial flow velocity \(\langle \beta _{\mathrm{T}}\rangle\)) components, using a simple relation [3, 46, 47]:

$$\begin{aligned} \begin{aligned} T_{\mathrm{eff}} = T_{\mathrm{fo}} + m_0 \langle \beta _{\mathrm{T}}\rangle ^2. \end{aligned} \end{aligned}$$
(1)

Therefore, \(T_{\mathrm{eff}}\) will exhibit a linear dependence on the particle mass \(m_0\), with a slope that can be used to characterize the radial collective flow velocity.

The data points of \(\phi ,\ \Lambda ,\ \Xi ^{-},\ \Omega ^{-},\ D^0\) follow linear dependencies with different slopes compared to those of \(\pi ,\ K,\ p\), as represented by the dashed lines shown in the left-hand panel of Fig. 3. Light-flavor hadrons, such as \(\pi ,\ K,\ p\) gain radial collectivity through the whole system’s evolution; therefore, the linear dependence exhibits a larger slope. Meanwhile, hadrons contains strange or heavy quarks, such as \(\phi ,\ \Lambda ,\ \Xi ^{-},\ \Omega ^{-},\ D^0\) may freeze out from the system earlier; therefore, it receives less radial collectivity and results in a smaller slope of linear dependence between \(T_{\mathrm{eff}}\) and mass.

Fig. 3
figure 3

(Color online) Left-hand panel: Slope parameter \(T_{\mathrm{eff}}\) for different particles in 0–10% central Au + Au collisions [42,43,44,45]. The dashed lines depict linear fits to \(\pi ,K,p\) and \(\phi ,\Lambda ,\Xi ^{-},\Omega ^{-},D^0\), respectively. Right-hand panel: Results of \(T_{\mathrm{kin}}\) versus \(\langle \beta \rangle\) from the blast-wave model, fitted to different groups of particles. The data points for each group of particles represent the results from different centrality bins. Those in the most central collisions exhibit the largest \(\langle \beta \rangle\)

The right-hand panel of Fig. 3 summarizes the fit parameters for \(T_{\mathrm{kin}}\) vs. \(\langle \beta \rangle\) from the blast-wave model, fitted to different groups of particles: black markers indicate the simultaneous fit to \(\pi ,\ K,\ p\); red markers indicate the simultaneous fit to \(\phi ,\ \Xi ^-\), and blue markers indicate the fit to \(D^0\).   The data points for each group of particles represent the fit results from different centrality bins, with the most central data points exhibiting the largest \(\langle \beta \rangle\) value.   As seen in the fit to the \(m_\mathrm{T}\) spectra, point-by-point statistical and systematic uncertainties are added in the quadrature when fitting. The fit results for \(\pi ,\ K,\ p\) are consistent with previously published results [41]. The fit results for multi-strange particles \(\phi ,\ \Xi ^{-}\), as well as for \(D^0\), show much smaller mean transverse velocities \(\langle \beta \rangle\) and larger kinetic freeze-out temperatures, suggesting that these particles decouple from the system earlier and gain less radial collectivity compared with light hadrons. The resulting \(T_{\mathrm{kin}}\) parameters for \(\phi ,\ \Xi ^-\), and \(D^0\) are close to the pseudocritical temperature \(T_\text {c}\) calculated from lattice QCD calculations at zero baryon chemical potential [48], indicating a negligible contribution from the hadronic stage to the observed radial flow of these particles.  Therefore, the collectivity that \(D^0\) mesons obtain is predominantly through partonic stage re-scatterings in the QGP phase.

Another observable through which to measure bulk collectivity is the elliptic flow; this is characterized by \(v_2\), the second-order coefficient of the particle azimuth distribution in the momentum space. The elliptic flow measurements of multi-strange hadrons and \(\phi\) mesons indicate that partonic collectivity accumulates in the top-energy heavy-ion collisions at the RHIC [50]. Recently, using the silicon vertex detector HFT, the STAR experiment measured the \(D^0\)\(v_2\) [13] in Au + Au collisions at \(\sqrt{s_{_{\mathrm{NN}}}}\) 200 GeV. Figure 4 compares the \(D^0\)\(v_2\) measured using the event-plane method in the 10–40% centrality bin with the \(v_2\) of \(K_{s}^0\), \(\Lambda\), and \(\Xi ^-\) [49]. The comparison between \(D^0\) and light hadrons must be performed in a narrow centrality bin, to avoid the bias caused by the fact that the \(D^0\) yield scales with the number of binary collisions, while the yield of light hadrons scales approximately with the number of participants [51]. Panel (a) shows \(v_2\) as a function of \(p_{\mathrm{T}}\), where a clear mass ordering for \(p_{\mathrm{T}} \,<\,2\,\hbox {GeV}/c\) including \(D^0\) mesons is observed. For \(p_{\mathrm{T}}\)  > 2 GeV/c, the \(D^0\) meson \(v_2\) follows that of other light mesons, indicating a significant charm quark flow at the RHIC [49, 50, 52]. Recent ALICE measurements show that the \(D^0\)\(v_2\) is comparable to that of charged hadrons in 0–50% Pb + Pb collisions at \(\sqrt{s_{_{\mathrm{NN}}}}\) = 2.76 TeV [14, 15], suggesting sizable charm flows at the LHC. Panel (b) shows \(v_2/n_q\) as a function of the scaled transverse kinetic energy, \((m_{\mathrm{T}}-m_0)/n_q\); here, \(n_q\) is the number of constituent quarks in the hadron, \(m_0\) is the rest mass, and \(m_{\mathrm{T}}=\sqrt{p_{\mathrm{T}}^2+m_0^2}\). We find that the \(D^0\)\(v_2\) exhibits the same characteristics as all other light hadrons [49, 52], particularly for \((m_{\mathrm{T}}-m_0)/n_q\,<\,1\,\hbox {GeV}/c^2\). This suggests that charm quarks gain significant flow through interactions with the QGP medium in 10–40% Au + Au collisions at \(\sqrt{s_{_{\mathrm{NN}}}}\) = 200 GeV.

Fig. 4
figure 4

(Color online) a \(v_2\) as a function of \(p_{\mathrm{T}}\) and b \(v_2/n_q\) as a function of \((m_{\mathrm{T}}-m_0)/n_q\) for \(D^0\) in 10–40% centrality Au + Au collisions, compared with \(K^0_S\), \(\Lambda\), and \(\Xi ^-\) [49]. The vertical bars and brackets represent statistical and systematic uncertainties, and the gray bands represent the estimated non-flow contribution

In heavy-ion collisions, charm quarks interact with the QGP matter when traversing the medium. The transverse momentum of the charm quark is modified by the medium through energy loss or collective flow. However, the total number of charm quarks may remain conserved because they are produced in initial hard processes (before QGP formation), and no more charm quarks are created later via thermal production at RHIC energies. Figure 5a, b shows the \(p_{\mathrm{T}}\)-integrated cross section for \(D^0\) production per nucleon–nucleon collision \(\mathrm {d}\sigma ^{\mathrm{NN}}/\mathrm {d}y|_{y=0}\) for different centrality bins in \(\sqrt{s_{_{\mathrm{NN}}}} = \mathrm{200\,GeV}\) 200 GeV Au + Au collisions for the full \(p_{\mathrm{T}}\) range and \(p_{\mathrm{T}}\,>\,4\,\hbox {GeV}/c\), respectively [32]. Results of \(p+p\) measurements at the same collision energy are also shown in both panels [21].

Fig. 5
figure 5

(Color online) Integrated \(D^0\) cross section per nucleon–nucleon collision at mid-rapidity in \(\sqrt{s_{\mathrm{NN}}} = 200\) GeV Au + Au collisions for a \(p_{\text {T}} > 0\) and b \(p_{\text {T}} > 4\) GeV/c, as a function of the average centrality \(N_{\text {part}}\). The statistical and systematic uncertainties are shown as error bars and brackets on the data points. The green boxes on the data points indicate the overall normalization uncertainties in the \(p+p\) and Au + Au data, respectively

The high \(p_{\mathrm{T}}\) (> 4 GeV/c) \(\mathrm {d}\sigma ^{\mathrm{NN}}/\mathrm {d}y|_{y=0}\) shows a clear decreasing trend from the peripheral to the mid-central and central collisions, and the peripheral collision results are consistent with \(p+p\) collisions within uncertainties. This suggests that the charm quark loses more energy in more central collisions at high \(p_{\mathrm{T}}\). However, the \(\mathrm {d}\sigma ^{\mathrm{NN}}/\mathrm {d}y|_{y=0}\), integrated over the full \(p_{\mathrm{T}}\) range, shows an approximately flat distribution as a function of \(N_{\mathrm{part}}\). The values for the full \(p_{\mathrm{T}}\) range in mid-central to central Au + Au collisions are smaller than those observed in \(p+p\) collisions with \(\sim 1.5\sigma\) effects, considering the large uncertainties from the \(p+p\) measurements. The total charm quark yield in heavy-ion collisions is expected to follow a number-of-binary-collision scaling because charm quarks are conserved at RHIC energies. However, CNM effects (including shadowing) could also play an important role. In addition, hadronization through coalescence might alter the hadrochemistry distributions of charm quarks in various charm hadron states, which may reduce the observed \(D^0\) yields in Au + Au collisions [53]. For instance, hadronization through coalescence can enhance the charmed baryon \(\Lambda _{c}^+\) yield over the \(D^0\) yield [54,55,56]. Combining this with the strangeness enhancement in the hot QCD medium and sequential hadronization can also lead to an enhancement in the charmed strange meson \(D_\text {s}^+\) yield relative to \(D^0\) [55,56,57,58].

The STAR HFT, equipped with a silicon pixel detector, achieved a \(\sim {30}\upmu \hbox {m}\) spatial resolution for the track impact parameter to the primary vertex; this allows a topological reconstruction of the decay vertices of open charm hadrons, in particular the \(\Lambda _{c}\) baryons with a lifetime of 60 microns. The left-hand panels of Figure 6 show (a) the charmed baryon to meson ratio compared with the light and strange baryon to meson ratios [60, 61] and (b) various models. The \(\Lambda _c/D^0\) ratio is comparable in magnitude to the \(\Lambda /K^0_s\) and p/\(\pi\) ratios, and it shows a similar \(p_{\mathrm{T}}\) dependence in the measured region. A significant enhancement is observed when compared against calculations from the latest PYTHIA 8.24 release (Monash tune [62]) without (solid green curve) and without (dot-dashed magenta curve) color reconnections (CR) [63]. The implementation with CR is found to enhance the baryon production with respect to meson production. However, neither calculations can fully describe the data and their \(p_{\mathrm{T}}\) dependence. Figure 6b also shows a comparison to various models of the coalescence hadronization of charm quarks [54,55,56,57,58]. The comparisons suggest that coalescence hadronization plays an important role in charm-quark hadronization in the presence of QGP. Furthermore, the data can be used to constrain the coalescence model calculations and their model parameters.

Fig. 6
figure 6

(Color online) Left-hand panels: The \(\Lambda _{c}/D^0\) ratio measured at mid-rapidity (\(|y|<\) 1) as a function of \(p_{\mathrm{T}}\) for Au + Au collisions at \(\sqrt{(}s_{\mathrm{NN}}) =\) 200 GeV in 10–80% centrality, compared with the baryon-to-meson ratios for a light and strange hadrons and b various model calculations. The \(p_{\text {T}}\)-integrated \(\Lambda _{c}/D^0\) ratio from the THERMUS [59] model calculations with a freeze-out temperature of \(T_{\mathrm{ch}}=160\) MeV is shown as a horizontal bar on the left-hand axis of the plot. Right-hand panels: c The integrated \(D_{s}/D^{0}\) ratio (solid black circles) of \(1.5< p_{\text {T}} < 8\) GeV/c as a function of \(p_{\text {T}}\), compared to the model calculations (curves) in 0–10% Au + Au collisions at \(\sqrt{s_{_{\mathrm{NN}}}}\) = 200 GeV. d The same \(D_{s}/D^{0}\) ratio as (c) but for 10–40% centrality. The vertical lines and brackets on the data points indicate statistical and systematic uncertainties, respectively

The right-hand panel of Fig. 6 shows the \(D_{s}/D^{0}\) ratio as a function of \(p_{\mathrm{T}}\) compared to coalescence model calculations for 0–10% (c) and 10–40% (d) collision centralities. Several models incorporating the coalescence hadronization of charm quarks and strangeness enhancements are used to describe the \(p_{\mathrm{T}}\) dependence of \(D_{s}/D^{0}\) ratios. These models assume that \(D_s^{\pm }\) mesons are formed by the recombination of charm quarks with equilibrated strange quarks in the QGP [54,55,56,57,58]. In particular, the sequential coalescence model, together with charm quark conservation [55], considers that more charm quarks are hadronized to \(D_s^{\pm }\) mesons than to \(D^{0}\) because the former is created earlier in the QGP, which results in further enhancement of the \(D_{s}/D^{0}\) ratio in Au + Au collisions relative to \(p+p\) collisions.

The STAR experiment extracted the total charm production cross section per binary nucleon collision at mid-rapidity in 200 GeV Au + Au collisions by summing all yields of the open charm hadron states [64]; the results were consistent with those seen in \(p+p\) collisions [21], within uncertainties. The numbers are reported as

$$\begin{aligned} {\text {AuAu:}}&\mathrm {d}\sigma ^{\mathrm{NN}}/\mathrm {d}y|_{y=0} = 152 \pm 13 ({\text {stat}}) \pm 29 ({\text {sys}}) \mu \text {b}, \end{aligned}$$
(2)
$$\begin{aligned} {pp:}&\mathrm {d}\sigma /\mathrm {d}y|_{y=0} = 130 \pm 30 ({\text {stat}}) \pm 26 ({\text {sys}}) \mu \text {b} \end{aligned}$$
(3)

These results are consistent with charm quark conservation in heavy-ion collisions at RHIC top energies.

2.2 Open bottom production

Theoretical calculations predict that the heavy quark energy loss is less than that of light quarks, due to suppression of the gluon radiation angle by the quark mass. The bottom quark mass is a factor of three larger than the charm quark mass; thus, less bottom quark energy loss is expected compared to charm quarks when they traverse the hot–dense medium created in heavy-ion collisions [7, 10, 11]. However, the low-production cross sections of bottom quarks at RHIC energies and the very small hadronic decay branching ratio prevent direct measurement of open bottom hadrons in RHIC experiments. Fortunately, the different lifetimes of open charm hadrons and open bottom hadrons allow us to separate their decay products by using the STAR HFT to distinguish their decay vertices and provide the impact parameter (or the distance of closest approach to the primary collision vertex) distributions.

Recently, the non-prompt products from open bottom decays \(B{\rightarrow }J/\psi\), \(B{\rightarrow }D^0\), and \(b{\rightarrow }e\) were measured by the STAR experiment at mid-rapidity for \(\sqrt{s_{\mathrm{NN}}} =\) 200 GeV Au + Au collisions; the experiment employed a template fit method using the different shapes of impact parameters between the signal and background [65,66,67]. The results of \(R_\mathrm{AA}\) as a function of \(p_{\mathrm{T}}\) are shown in Fig. 7. The data of \(B{\rightarrow }J/\psi\), represented by solid blue squares, were observed to be suppressed in the whole \(p_{\mathrm{T}}\) region from 2 to 8 GeV/c. A similar suppression was also observed for \(B{\rightarrow }D^0\) (solid red circles) and \(b{\rightarrow }e\) (open blue circles) at high \(p_{\mathrm{T}}\). These results indicate that interactions between the bottom quarks and hot–dense nuclear matter lead to bottom quark energy losses in the medium. It should be noted that the non-prompt \(J/\psi\), \(D^0\), and electrons shown here are in the 0–80%, 0–10%, and 0–80% Au + Au collisions, respectively. Comparing with \(c{\rightarrow }e\) (shown as open red squares), \(b{\rightarrow }e\) is systematically less suppressed, which indicates that bottom quarks lose less energy than charm quarks. The calculations of a transport model from the Duke group [68] reproduce the data within the bounds of uncertainty. Furthermore, the non-prompt \(D^0\) at 4 GeV/c shows no suppression, which is again consistent with the smaller energy loss of the bottom quark as a result of its heavier mass compared with charm and light quarks. These two independent measurements provide important evidence of mass-dependent parton energy losses in the hot QCD medium created by high-energy heavy-ion collisions.

Fig. 7
figure 7

(Color online) The \(R_{\mathrm{AA}}\) of \(B{\rightarrow }J/\psi\) (solid blue squares), \(B{\rightarrow }D^0\) (solid red circles), \(b{\rightarrow }e\) (open blue circles), and \(c{\rightarrow }e\) (open red squares) at mid-rapidity in \(\sqrt{s_{\mathrm{NN}}} = 200\) GeV Au + Au collisions from the STAR experiment [65,66,67]. Vertical bars and bands represent statistical and systematic uncertainties, respectively. Dashed and dot dashed curves represent Duke model calculations [68] for \(b{\rightarrow }e\) and \(c{\rightarrow }e\), respectively

Recently, an experimental data-driven approach was applied to extract the bottom elliptic flow from heavy flavor semileptonic decay channels [69]. Using silicon vertex detectors, the STAR experiment precisely measured the open charm hadrons; this allowed the bottom contribution to be extracted by subtracting the contributions of open charm decays from the inclusive heavy flavor electron spectrum. Figure 8 shows the \(v_2\) results of electrons from open charm (\(v_2^{\mathrm{c}\rightarrow \mathrm{e}}\)) and open bottom (\(v_2^{\mathrm{b}\rightarrow \mathrm{e}}\)) decays as a solid blue curve with an uncertainty band and red circles, respectively. The \(v_2\) of \(\mathrm \phi \rightarrow e\) (\(v_2^{\phi \rightarrow \mathrm{e}}\)) is shown as a long-dashed red curve with an uncertainty band. DUKE model predictions [68] are also shown as dot dashed curves for comparison. On average, the electron \(v_2\) from the beauty hadron decays at \(p_{\mathrm{T}}> 3.0\,\hbox {GeV}/c\) is observed with a 4-sigma significance (\(\chi ^2/ndf = 29.7/6\)) deviation from zero. Furthermore, this is consistent with electrons from charmed or strange hadron decays (within uncertainties) at \(p_{\mathrm{T}}> 4.5\,\hbox {GeV}/c\). This flavor-independent \(v_2\) at high \(p_{\mathrm{T}}\) could be attributed to the initial geometric anisotropy or the path-length dependence of the energy loss in the medium. A smaller \(v_2^{\mathrm{b}\rightarrow \mathrm{e}}\) (compared with \(v_2^{\mathrm{c}\rightarrow \mathrm{e}}\)) is observed at \(p_{\mathrm{T}} < 4.0\, \hbox {GeV}/c\), which may be driven by the larger mass of the beauty quark compared to the charm quark. The \(v_2^{\mathrm{b}\rightarrow \mathrm{e}}\) deviates from the hypothesis that the B-meson \(v_2\) follows the number of constituent quark (NCQ) scaling (black curve) at 2.5 GeV/\(c< p_{\mathrm{T}} < 4.5\,\hbox {GeV}/c\) with a confidence level of 99% (\(\chi ^2/ndf\) = 14.3/4); this indicates that the beauty quark elliptic flow is smaller than that of the light quark, unlike the \(D^0\)\(v_2\) when scaled with the elliptic flow of light flavor hadrons by dividing the NCQ in both \(v_2\) and \(\left( m_{\mathrm{T}}-m_0\right)\) as previously presented in Fig. 4. This suggests that the beauty quark is unlikely to thermalize and is too heavy to follow the collective flow of lighter partons in heavy-ion collisions at RHIC energies.

Fig. 8
figure 8

(Color online) The elliptic flows (\(v_2\)) of electrons from open charm (blue band) and open bottom (red circles) decays at mid-rapidity (\(\left| \eta \right| < 0.7\)) in minimum bias Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV. The \(v_2^{\mathrm{b\rightarrow e}}\) (with the B-meson \(v_2\) NCQ scaling assumption) and the \(v_2^{\mathrm{\phi \rightarrow e}}\) are shown as a dashed curve and open squares, respectively. Results from Duke [68] model predictions are shown for comparison. This figure is taken from Ref. [69]

3 Quarkonium production in \(p+p\) collisions

For various reasons, quarkonium production in \(p+p\) collisions is a crucial baseline for the study of quarkonium production in medium. First, to quantify the modification of quarkonium production in heavy-ion collisions, we typically compare the production yield of quarkonium in heavy-ion collisions to that observed in \(p+p\) collisions, by calculating the nuclear modification factor:

$$\begin{aligned} R_{\text {AA}}=\frac{y_{\text {AA}}}{y_{pp}~N_{\text {coll}}}, \end{aligned}$$
(4)

where \(y_\text {AA}\) and \(y_{pp}\) denote the quarkonium yield in heavy ion and \(p+p\) collisions, respectively; and \(N_\text {coll}\) is the number of binary nucleon–nucleon collisions in heavy-ion collisions. If the heavy ion collision is a superposition of nucleon–nucleon collisions exclusively, the quarkonium yield should follow the \(N_\text {coll}\)-scaling and \(R_\text {AA}=1\). The deviation of \(R_{\mathrm {AA}}\) from unity indicates a modification of quarkonium production in heavy-ion collisions. Secondly, understanding the production mechanism of quarkonium in \(p+p\) collisions is essential to interpreting the \(R_{\mathrm {AA}}\) measurements in heavy-ion collisions. For example, quarkonium has a relatively large formation time; thus, the question of whether the \(q\bar{q}\) interstates (before the quarkonium is fully formed) is color-singlet or color-octet may determine a different modification of the quarkonium yield when it traverses the medium in heavy-ion collisions.

Quarkonium production in hadron collisions also provides important test of QCD. The heavy quark–antiquark pair is predominantly produced from the initial hard scattering and can be calculated within the framework of perturbative QCD down to a low \(p_{\mathrm{T}}\). However, when the heavy quark pair forms a physical quarkonium bound state, the process involves long distances and soft momentum scales; consequently, it is non-perturbative. Such problems are solved via modeling. The detailed study of quarkonium production with hadron colliders and the comparisons against theoretical calculations provide an important testing ground for both perturbative and non-perturbative aspects of QCD calculations.

Experimental studies of the quarkonium production mechanism are typically conducted by measuring the production yield and polarization with respect to different kinematic variables (such as rapidity and \(p_{\text {T}}\) ) and comparing the results against different theoretical calculations.

Furthermore, it is important to note that not all of the quarkonia are produced directly; instead, a large fraction are produced via the decay of other hadrons, such as the higher quarkonium states and the B-hadrons for charmonium. The contribution from the decay of other hadrons is referred to as the feeddown contribution. Under different properties (e.g., different binding energies), the directly produced and hadron-decay produced quarkonia should undergo different modifications in heavy-ion collisions. The feeddown contribution must be considered when extracting physical information from measurements of quarkonium suppression in heavy-ion collisions. The feeddown contribution is typically studied in \(p+p\) collisions and/or small systems such as \(p+\hbox {A}\) collisions.

3.1 \(J/\psi\) production cross section

Figure 9 shows the inclusive \(J/\psi\) production cross section at mid-rapidity (\(|y|<1\)) as a function of \(p_{\text {T}}\) in non-single-diffractive \(p+p\) collisions at \(\sqrt{s}\) = 200 GeV; this was measured by the STAR Collaboration [70] via the di-electron decay channel, by combining the minimum-bias trigger events with those triggered by the electromagnetic calorimeter for various thresholds, taken in 2012. The results were found to be consistent with STAR’s [71, 72] and PHENIX’s previously published results (\(|y|<0.35\)) [73] but with improved precision at \(p_\mathrm{T}>2 \,\text {GeV}/c\). The decay branching ratio is not corrected for. The total production cross section per unit rapidity for inclusive \(J/\psi\) is extracted as

$$\begin{aligned} B_\text {ee} \frac{\mathrm {d}\sigma }{\mathrm {d}y}|_{y=0} = 43.2 \pm 3.0(\text {stat.}) \pm 7.5(\text {syst.})\,\text {nb}. \end{aligned}$$
(5)

The precise \(p_{\text {T}}\) spectrum is compared to theoretical calculations. The theory-to-data ratios are shown in the lower panels of Fig. 9. The green band represents the calculations from the color evaporation model (CEM) for prompt \(J/\psi\) at \(|y|<0.35\) [74]. The orange and magenta bands represent the calculations performed within the framework of next-to-leading order (NLO) nonrelativistic QCD (NRQCD) with different treatments, labeled as NRQCD A [75] and NRQCD B [76]. NRQCD A is for prompt \(J/\psi\) and NRQCD B is for direct \(J/\psi\). The blue band shows the calculations from NRQCD A, which incorporates a color-glass condensate (CGC) effective theory describing small-x re-summation for prompt \(J/\psi\) [77]. The CEM and NRQCD calculations describe the data in the applicable \(p_{\text {T}}\) ranges (within uncertainties). The CGC+NRQCD calculations at low \(p_{\text {T}}\) are systematically higher than the data, but the lower boundary touches the data. It should be noted that the contribution from B-hadron decay is not included in the theoretical calculations. As discussed in the next subsection, its contribution increases with increasing \(p_{\mathrm{T}}\), but less than 20% at \(p_{\text {T}}\) below 5 GeV/c.

Fig. 9
figure 9

(Color online) Top panels: Inclusive \(J/\psi\) production cross section multiplied by the decay branching ratio, as a function of \(p_{\text {T}}\) at mid-rapidity in non-single diffractive \(p+p\) collisions at \(\sqrt{s}\) = 200 GeV, as measured by the STAR [70,71,72] and PHENIX [73] Collaborations. The bars and boxes on the data points denote the statistical and systematical uncertainties, respectively. The bands display theoretical calculations from the color evaporation model (CEM) for direct \(J/\psi\) [74] (green), nonrelativistic QCD (NRQCD) for prompt \(J/\psi\) [75] (orange), NRQCD for direct \(J/\psi\) [76] (magenta), and color-glass condensate (CGC)+NRQCD [77] (blue). Bottom panels: The ratios with respect to the central value of STAR measurements, taken from the 2012 dataset. The figure is taken from [70]

Figure 10 shows the inclusive \(J/\psi\) production cross section in mid-rapidity, as a function of \(p_{\text {T}}\) in non-single diffractive \(p+p\) collisions at \(\sqrt{s}\) = 500 and 510 GeV, as measured by the STAR Collaboration [78] via both di-muon and di-electron decay channels. It covers a broad \(p_{\text {T}}\) range (\(0<p_\mathrm{T}<20\,\text {GeV}/c\)). The data points denote the results obtained under the unpolarized assumption, and the gray band denotes the polarization envelope. The total production cross section per unit rapidity for inclusive \(J/\psi\) within \(0<p_{\mathrm{T}}<9\,\text {GeV}/c\) is

$$\begin{aligned} B_{\mu \mu } \frac{\mathrm{d}\sigma }{\mathrm{d}y}|_{y=0} = 67 \pm 6(\text {stat.}) \pm 10(\text {syst.}) ^{+100}_{-18}(\text {pol.}) \pm 7(\text {lumi.}) \,\text {nb}. \end{aligned}$$
(6)

The data are compared to NRQCD [75], CGC+NRQCD [77], and improved CEM (ICEM) [79] calculations. All model calculations are for prompt \(J/\psi\), and the feeddown from B-hadrons is not included. To fairly compare between the data and theoretical calculations, the feeddown contribution from B-hadrons is estimated using FONLL calculations [19, 27, 28] and added to the theoretical calculations. Figure 10(b-d) shows the ratio between the theoretical calculations and a fit to data using an empirical function. At high \(p_{\mathrm{T}}\), the NRQCD and ICEM describe the data reasonably well. At low \(p_{\mathrm{T}}\), both the CGC+NRQCD and ICEM calculation results lie above the data but are within the large uncertainties expressed via the polarization envelope.

Fig. 10
figure 10

(Color online) a Inclusive \(J/\psi\) production cross section multiplied by the decay branching ratio, as a function of \(p_{\text {T}}\) at mid-rapidity in non-single diffractive \(p+p\) collisions at \(\sqrt{s}\) = 500 and 510 GeV, as measured by the STAR Collaboration [78]. bd The ratios between the data and the different model calculations used to produce a fit thereto. See text for details. The figure is taken from [78]

3.2 Feeddown contribution of \(J/\psi\)

The inclusive \(J/\psi\) production includes prompt \(J/\psi\) and non-prompt \(J/\psi\). The former includes the directly produced \(J/\psi\) and a contribution from the decay of excited charmonium states, including \(\psi (2S)\) and \(\chi _{c0,1,2}\); the latter refers to the contribution from the decay of B-hadrons. It is important to understand the different component fractions of inclusive \(J/\psi\) , to interpret the measurements of inclusive \(J/\psi\) production mechanisms in both \(p+p\) and A + A collisions.

\(\psi (2S)\) and \(J/\psi\) are typically reconstructed in the same dilepton decay channel. The systematic uncertainties can be largely canceled out when calculating the ratio of their yields. The measurement is very challenging in practice because the study of \(J/\psi\) at the RHIC is statistically limited at present, the yield of \(\psi (2S)\) is considerably lower than that of \(J/\psi\) , and the dilepton decay branching ratio is also lower. This results in a reconstructed \(\psi (2S)\) signal approximately 50 times smaller than the \(J/\psi\) signal for similar combinatorial and correlated backgrounds. The yield ratios of inclusive \(\psi (2S)\) and inclusive \(J/\psi\) (after correcting for the differences in acceptance and efficiency), as measured at the RHIC by the STAR and the PHENIX Collaborations, are shown in Fig. 11 and compared to world data. The uncertainties are predominately statistical. Notably, the decay branching ratio is not corrected for; it is approximately

$$\begin{aligned} \frac{\text {BR}_{\psi (2S)}}{\text {BR}_{J/\psi }} \sim 7.53 \pm 0.16~(e^+e^-) \text { or } 7.5 \pm 0.6~(\mu ^+\mu ^-) , \end{aligned}$$
(7)

[81]. The ratio increases with \(p_{\text {T}}\) but exhibits very little energy dependence for a center-of-mass energy of 40 GeV to 7 TeV. The ICEM calculations [79] at RHIC can describe the data well. The branching ratio of \(\psi (2S) \rightarrow J/\psi + X\) is \((61.4 \pm 0.6)\%\) [81]. The feeddown contribution of \(\psi (2S)\) to inclusive \(J/\psi\) can be approximately estimated by multiplying the ratio shown in Fig. 11 by a factor of \(\sim 4.6 \pm 0.5\). This fraction is \(<\sim 10\%\) at low \(p_{\text {T}}\) and increases to around 15% for \(p_{\text {T}}\) up to 10 GeV/c.

Fig. 11
figure 11

(Color online) The ratio between the production cross section multiplied by the branching ratio of \(\psi (2S)\) and \(J/\psi\) , as a function of \(p_{\text {T}}\) in \(p+p(\bar{p}\)) or p + A collisions from various experiments. The two panels of this figure are taken from [78] and [80], respectively

The feeddown contribution of \(\chi _c\) to \(J/\psi\) is typically studied via the radiative decay of \(\chi _c\) (\(\chi _c \rightarrow J/\psi + \gamma\)). These measurements are also very challenging because the photon from the decay typically has very low energy; thus, the electromagnetic calorimeter must have a very good energy resolution and a low threshold. So far, only the PHENIX Collaboration has successfully performed this measurement at the RHIC [83]. The feeddown fraction of \(\chi _c\) decays in the inclusive \(J/\psi\) was measured to be \((32 \pm 9)\%\). Statistics suitable for studying the \(p_{\text {T}}\) dependence of the fraction are currently unavailable. The LHCb Collaboration precisely measured the feeddown fraction of \(\chi _c\) decays in prompt \(J/\psi\) as a function of \(p_{\text {T}}\) ; they found that the fraction is only 14% at \(p_{\mathrm{T}}=2\) GeV/c and monotonically increases to about 25% at \(p_\mathrm{T}=10\) GeV/c in \(p+p\) collisions at \(\sqrt{s}\) = 7 TeV in forward rapidity [84].

The feeddown contribution of non-prompt \(J/\psi\) (\(J/\psi \leftarrow B\)) can be measured using two methods. The STAR Collaboration measured the feeddown fraction of B-hadron decays in mid-rapidity (\(|y|<1\)) at \(p_T > 5\) GeV/c in \(p+p\) collisions at \(\sqrt{s}\) = 200 GeV; they used the correlation functions of \(J/\psi\) and charged hadrons because non-prompt \(J/\psi\) is typically associated with more charged hadrons than prompt \(J/\psi\) [71]. The PHENIX Collaboration measured the fraction in forward rapidity (\(1.2<|{y}|<2.2\)) for \(p+p\) collisions at \(\sqrt{s}\) = 510 GeV; they used the silicon vertex detector to statistically distinguish \(J/\psi\) originating from the primary and secondary vertexes [82]. Figure 12 shows the fraction as a function of energy and \(p_{\mathrm{T}}\). The data from the Tevatron and LHC experiments are also shown. The fraction shows very little center-of-mass energy dependence but a significant \(p_{\text {T}}\) dependence. The fraction is \(8.1\% \pm 2.3\%\) (stat.) \(\pm ~1.9\%\) (syst.) for low \(p_{\text {T}}\) (\(0<p_{\mathrm{T}}<5\) GeV/c) and \(1.2<|y|<2.2\); it increases to approximately 15% at \(p_{\mathrm{T}}=5\) GeV/c and about 25% at \(p_\mathrm{T}=10\) GeV/c.

Fig. 12
figure 12

(Color online) The feeddown fraction of B-hadron decays to inclusive \(J/\psi\) as a function of energy and \(p_{\text {T}}\) in \(p+p(\bar{p})\) collisions. The figure is taken from [82]

For more details regarding the feeddown contribution to quarkonium production, please refer to a recent review [85].

3.3 \(J/\psi\) polarization

The measurements of \(J/\psi\) polarization are important because the acceptance and efficiency used in the production cross-section measurements depend on the polarization; moreover, they help to distinguish or constrain different theoretical models. At present, no available model can simultaneously describe \(J/\psi\) production cross sections and polarization. \(J/\psi\) polarization (spin alignment) can be measured via the angular distribution of the dilepton decay from \(J/\psi\); it can be parameterized as

$$\begin{aligned} \begin{aligned}&W(\cos \theta ,\varphi ) \propto \frac{1}{3+\lambda _{\theta }} (1+\lambda _{\theta }\cos ^{2}\theta \\&\quad +\lambda _{\varphi }\sin ^{2}\theta \cos 2\varphi +\lambda _{\theta \varphi }\sin 2\theta \cos \varphi ), \end{aligned} \end{aligned}$$
(8)

where \(\lambda _{\theta }\), \(\lambda _{\varphi }\), and \(\lambda _{\theta \varphi }\) are the polarization parameters. \(\theta\) and \(\varphi\) are the polar and azimuthal angles of a lepton in the \(J/\psi\) rest frame with respect to the chosen quantization axis, upon which the coefficients depend. The names of the reference frames and their corresponding quantization axes are as follows:

  1. 1.

    Helicity (HX) frame: The direction along the \(J/\psi\) momentum in the center-of-mass system of the colliding beams;

  2. 2.

    Collins–Soper (CS) frame: The bisector of the angle formed by one beam direction and the opposing direction of the other beam in the \(J/\psi\) rest frame;

  3. 3.

    Gottfried–Jackson (GJ) frame: The direction of the beam momentum boosted into the \(J/\psi\) rest frame.

Figure 13 shows the polarization parameter measurements for inclusive \(J/\psi\) within various reference frames, for forward rapidity in \(p+p\) collisions at \(\sqrt{s}\) = 510 GeV [86]. In all frames, the polarization parameter \(\lambda _\theta\) is significantly negative at low \(p_{\text {T}}\) and consistent with no polarization at high \(p_{\mathrm{T}}\). In contrast, the polarization parameter \(\lambda _\phi\) is close to zero at low \(p_{\text {T}}\) and becomes slightly negative at high \(p_{\mathrm{T}}\). For comparison, we also show the theoretical calculation of \(\lambda _\theta\) for prompt \(J/\psi\) in the HX frame, under the NRQCD factorization approach by Chung et al. [88] at \(2<p_\mathrm{T}<5\,\hbox {GeV}/c\) and by Shao [89] at \(p_{\text {T}}\) above 5 GeV/c. Both calculations are consistent with the data for high \(p_{\mathrm{T}}\). However, a discrepancy persists between the data and the theoretical calculations at low \(p_{\mathrm{T}}\). The theory predicts a small but positive \(\lambda _\theta\); however, the measured results are significantly negative.

Fig. 13
figure 13

(Color online) \(J/\psi\) polarization parameters \(\lambda _\theta\) and \(\lambda _\phi\) as a function of \(p_{\text {T}}\) in various reference frames at forward rapidity in \(p+p\) collisions at \(\sqrt{s}\) = 510 GeV. The figure is taken from [86]

STAR recently measured the \(J/\psi\) polarization parameters \(\lambda _\theta\) and \(\lambda _\phi\) at mid-rapidity in \(p+p\) collisions at \(\sqrt{s}\) = 200 GeV, as shown in Fig. 14 [87]. In the HX frame, the measured \(\lambda _\theta\) is slightly negative but is consistent with zero at \(p_\mathrm{T}<6\) GeV/c, whereas \(\lambda _\phi\) is slightly positive. In the CS frame, \(\lambda _\theta\) is slightly positive but consistent with zero when considering uncertainties. The bands in Fig. 14 depict NRQCD calculations using two sets of long distance matrix elements (LDME). The two calculations are labeled as NRQCD1 [90] and NRQCD2 [91]. They exhibit significant differences, especially at low \(p_{\mathrm{T}}\). With the current precision, the data are consistent with both calculations; however, in the future, measurements with improved precision should be able to constraint the LDMEs.

Fig. 14
figure 14

(Color online) \(J/\psi\) polarization parameters \(\lambda _\theta\) and \(\lambda _\phi\) as a function of \(p_{\text {T}}\) in various reference frames at mid-rapidity in \(p+p\) collisions at \(\sqrt{s}\) = 200 GeV. The figure is taken from [87]

3.4 \(\Upsilon\) production cross section

The production cross section of \(\Upsilon\), multiplied by the di-lepton decay branching ratio, is approximately three orders of magnitude lower than that of \(J/\psi\) and nine orders lower than that of inelastic \(p+p\) collisions at the RHIC. Approximately half a billion minimum bias \(p+p\) collision events are needed to produce one \(\Upsilon \rightarrow l^+l^-\) decay at mid-rapidity. To enhance the recorded integral luminosity for \(\Upsilon\) study, a special trigger based on the barrel electromagnetic calorimeter was designed and makes measuring \(\Upsilon\) at STAR possible. However, the statistics and momentum resolution are insufficient to separate the 1S, 2S, and 3S states in \(p+p\) collisions. The 1S, 2S, and 3S states are measured collectively. Figure 15 shows the measured \(\Upsilon (1S+2S+3S)\) cross section per unit rapidity at mid-rapidity multiplied by the \(\Upsilon \rightarrow e^+e^-\) branching ratio, as a function of the center-of-mass energy [87]. Theoretical calculations from the NLO CEM [74] describe the cross section for a center-of-mass energy of 20 GeV to 7 TeV, and the RHIC data follow the world-wide trend. Furthermore, PHENIX measured \(\Upsilon (1S+2S+3S)\) at forward rapidity (\(1.2<|{y}|<2.2\)), using the di-muon trigger [92]. The rapidity distribution was obtained by combining measurements from STAR (\(|{y}|<1\)) and PHENIX (\(1.2<|{y}|<2.2\)); it is narrower than the NLO CEM prediction [74].

Fig. 15
figure 15

(Color online) \(\Upsilon (1S+2S+3S)\) production cross section per unit rapidity per binary nucleon–nucleon collision at mid-rapidity, multiplied by the \(\Upsilon \rightarrow l^+l^-\) branching ratio, in \(p + p(\bar{p}, \mathrm{A})\) collisions. The figure is taken from [87]

The binding energies of \(\Upsilon (1S)\), \(\Upsilon (2S)\), and \(\Upsilon (3S)\) are very different. The modifications of their production in nuclear medium are expected to vary for different \(\Upsilon\) states. It is of particular interest to separate the 1S, 2S, and 3S states. This is currently unachievable in \(p+p\) collisions at 200 GeV; however, it is possible to separate 1S from 2S+3S (or even 1S, 2S, and 3S) in p(d) + A and A + A collisions, due to the larger quantities of statistics and the better momentum resolutions thanks to the better primary vertex resolutions. However, the production cross sections of 1S and 2S+3S (or 2S and 3S) states in \(p+p\) collisions are needed as a reference, to study the various nuclear matter effects of different \(\Upsilon\) states. The authors of Ref. [93] performed a systematic study of \(\Upsilon\) production in \(p + p(\bar{p}, \text {A})\) collisions from worldwide data and developed a method of predicting \(\Upsilon (2S)/\varUpsilon (1S)\) and \(\Upsilon (3S)/\varUpsilon (1S)\) in \(p+p\) collisions for a given center-of-mass energy. The production cross section of 1S, 2S, and 3S in \(p+p\) collisions can be obtained by combining the derived ratios and the measurements of the \(\Upsilon (1S+2S+3S)\) production cross section. The STAR Collaboration also established a method to derive the \(p_{\text {T}}\) spectrum of \(\Upsilon (1S)\), \(\Upsilon (2S)\), and \(\Upsilon (3S)\) in \(p+p\) at 200 GeV, to study the \(p_{\text {T}}\) dependence of the nuclear modification factor of \(\Upsilon\) states. Recently, the STAR Collaboration managed to measure the \(p_{\text {T}}\) spectrum of \(\Upsilon (1S)\) and \(\Upsilon (2S+3S)\) in \(p+p\) collisions at 500 GeV [94]. The newly installed inner time projection chamber (TPC) in STAR will improve the mass resolution of \(\Upsilon\) [95]. The possibility of further improving the mass resolution, by changing the working gas of the TPC to suppress transverse diffusion, is under investigation.

4 \(J/\psi\) production in medium

Although the production mechanism in \(p+p\) collisions is not fully understood, quarkonia in heavy-ion collisions are one of several important probes of QGP. The suppression of \(J/\psi\) production yield in relativistic heavy-ion collisions with respect to the yield in \(p+p\) collisions, scaled by the number of binary nucleon–nucleon collisions, was proposed as a “smoking gun” signature of QGP formation by Matsui and Satz [17]. This suppression is the result of \(J/\psi\) dissociation caused by the screening of the potential between charm and anti-charm quarks in the deconfined hot dense medium. Since 1986, \(J/\psi\) production in nucleus–nucleus collisions has been extensively studied at the CERN SPS. The pioneer experiment NA38 found that \(J/\psi\) production in S+U collisions is suppressed relative to \(p+\hbox {U}\) collisions as a function of the transverse energy \(E_{\mathrm{T}}\), which relates to the collision centrality. However, it was later found that the suppression pattern was compatible with the extrapolation of the trend observed in \(p+\hbox {A}\) collisions and can be accounted for by normal nuclear absorption. The NA50 experiment collected high statistics data using p beams of 450 and 400 GeV energies on Be, Al, Cu, Ag, W, and Pb targets. The normal nuclear absorption of \(J/\psi\) production was obtained by systematically studying the \(p+\hbox {A}\) data [96].

Furthermore, the NA50 experiment collected data using a Pb beam with an energy per nucleon of 158 GeV on a Pb target (\(\sqrt{s_{_\mathrm {NN}}}\)= 17.3 GeV) in 1995, 1996, 1998, and 2000. The analysis of these data showed that \(J/\psi\) production, relative to Drell-Yan production, is anomalously suppressed with respect to the normal nuclear absorption pattern, which was computed by extrapolating the \(J/\psi\) suppression in \(p+\hbox {A}\) collisions as previously described [18]. To extract \(p+\hbox {A}\) data under the same conditions as the A + A data, the NA60 experiment operated with a proton beam on nuclear targets (Be, Al, Cu, In, W, Pb, and U) and an In beam on an In target, with an energy per nucleon of 158 GeV [97]. The nuclear absorption was found to be beam-energy dependent. The anomalous \(J/\psi\) suppression was calculated using the new nuclear absorption cross section, and it also considers the differences in the parton distribution function between the nucleus (nPDF) and nucleon (shadowing effect). The observed suppression of \(J/\psi\) was compatible with the extrapolation of CNM effects up to \(N_{\text {part}} \sim 200\). When \(N_{\text {part}} > 200\), an anomalous suppression of up to \(\sim 20{-}30\%\) was observed in the most central Pb + Pb collisions.

The PHENIX Collaboration at RHIC measures \(J/\psi\) production in Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV (more than a factor of ten larger than that of SPS) in \(|y| < 0.35\) and \(1.2< |y| < 2.2\) rapidities by using the di-electron and di-muon decay channels, respectively. The left-hand panel of Fig. 16 shows the nuclear modification factor \(R_{\mathrm {AA}}\) as a function of \(N_{\text {part}}\) in mid-rapidity for Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV (measured by the PHENIX Collaboration [99]) and in Pb + Pb, In + In, and S + U collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 17.3 GeV (measured by the NA50, NA60, and NA38 Collaborations [18, 97, 100], respectively). Although the center-of-mass energies differ by one order of magnitude, the level of suppression is very similar at RHIC and SPS energies. This was puzzling because a stronger suppression was expected at the RHIC due to the high energy density and/or initial temperature. Furthermore, the suppression was observed to be much stronger at forward rapidity than at mid-rapidity, as shown in the right-hand panel of Fig. 16. It was expected that the suppression at mid-rapidity (where the energy density is higher) would be stronger. The (re)combination of the charm quark and anti-charm quark was introduced as an additional \(J/\psi\) production mechanism [101,102,103,104,105]. The concept of this (re)combination mechanism is as follows: if charm and anti-charm quarks not initially produced as a bound state get close enough in space and momentum after transportation in the QGP, they may form a bound state such as \(J/\psi\). The production yield of \(J/\psi\) is only about 1% of the total number of charm and anti-charm quark pairs in \(p+p\) collisions; thus, the (re)combination mechanism may have a sizable effect despite the small (re)combination probability. The probability is proportional to the square of the number of charm and anti-charm quark pairs produced in one event; it is negligible for \(p+p\) collisions at the RHIC energy; however, because the number of charm and anti-charm quark pairs is roughly scaled by the number of binary nucleon–nucleon collisions (which can be as large as 1000 in central Au + Au collisions), the probability is much larger in heavy-ion collisions than in \(p+p\) collisions. Unlike the color-screening (or QGP melting) mechanism, the (re)combination mechanism could enhance \(R_{\mathrm {AA}}\) in heavy-ion collisions. Because the charm and anti-charm quark pair production cross section increases dramatically with the center-of-mass energy, the (re)combination mechanism plays a more important role in heavy-ion collisions at higher center-of-mass energies. Based on a theoretical calculation [106] on charm and anti-charm quark pair production cross sections for \(p+p\) collisions, we estimated that the number of charm and anti-charm quark pairs in a 0–10% Pb + Pb collision at a SPS energy (\(\sqrt{s_{_\mathrm {NN}}}\)= 17.3 GeV) is \(0.13 \pm 0.03\). This number increases to \(18 \pm 4\) in 0–10% Au + Au collisions at RHIC energies (\(\sqrt{s_{_\mathrm {NN}}}\)= 200) and to over 100 in 0–10% Pb + Pb collisions at LHC energies. The contribution of (re)combination is negligible at SPS but could have sizable effect at the RHIC and LHC. The theoretical models (e.g., the transport models) that include both QGP melting and (re)combination can explain the \(R_{\mathrm {AA}}\) observed at the RHIC in both mid- and forward rapidities, as well as those observed at the SPS.

Fig. 16
figure 16

(Color online) \(R_{\mathrm {AA}}\) as a function of \(N_{\text {part}}\) for inclusive \(J/\psi\) at the SPS and RHIC. The panels are taken from [98, 99]

\(J/\psi\) production rates in heavy-ion collisions at the RHIC and LHC are the result of an interplay of QGP melting, CNM effects, and (re)combination effects. To study the properties of the QGP via \(J/\psi\), we need a good understanding of each of the three effects. The CNM effects are normally studied experimentally in \(p+\hbox {A}\) collisions or in the collisions of light ions, for which the QGP melting and/or (re)combination effects are unlikely to occur (at least at RHIC energies). However, separating QGP melting and (re)combination effects is very difficult. Fortunately, these two effects have very different collision energies, collision systems, and \(p_{\text {T}}\) dependencies. A systematic study of \(J/\psi\) production in heavy-ion collisions is helpful to understand the \(J/\psi\) production mechanism in heavy-ion collisions and to study the properties of QGP using \(J/\psi\).

4.1 Collision energy dependence

RHIC launched the Beam Energy Scan (BES) program in 2010, to explore the QCD phase diagram. Both STAR and PHENIX collected data for Au + Au collisions at 62.4 and 39 GeV in 2010, and at 27 and 19 GeV in 2011, as phase-I of the BES program (BES-I). These center-of-mass beam energies help to fill the large gap between the SPS energy and the RHIC top energy. These BES data can be used to study the evolution of the CNM effects, QGP melting, and (re)combination between the SPS and RHIC. The production cross section of \(J/\psi\) decreases dramatically with a decreasing center-of-mass energy, and the luminosity of RHIC also decreases quickly with decreasing beam energy. We were only able to measure \(J/\psi\) production in the 39 and 62.4 GeV collisions. To obtain \(R_{\mathrm {AA}}\) at these two energies, the \(J/\psi\) cross section for \(p+p\) collisions is needed. Several measurements from p + A fixed-target experiments and \(p+p\) collider experiments have been performed near these two energies in the last century, in the Intersection Storage Ring (ISR). Unfortunately, some data at mid-rapidity were found to be inconsistent with each other. At mid-rapidity, STAR uses the \(J/\psi\) production cross section derived from world-wide experimental data [108], to calculate \(J/\psi\)\(R_{\mathrm {AA}}\) for Au + Au collisions at 39 and 62.4 GeV [107]. For the forward rapidity, PHENIX used reference data derived using data from the Fermilab fixed-target experiment, ISR collider experiment, and CEM model calculations [109]. The left-hand panel of Fig. 17 shows the inclusive \(J/\psi\)\(R_{\mathrm {AA}}\) as a function of \(N_{\text {part}}\) for Au + Au collisions at 39 and 62.4 GeV, in both mid- and forward rapidities, comparing it to that at 200 GeV. The results indicate no suppression in peripheral collisions but strong suppression in the more central collisions. In mid-rapidity, no significant energy dependence was observed (within uncertainties) between the SPS (\(\sqrt{s_{_\mathrm {NN}}}\)= 17.3 GeV) and RHIC (\(\sqrt{s_{_\mathrm {NN}}}\)=200 GeV) top energies. However, in forward rapidity, the suppression seems weaker for Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 39 and 62.4 GeV than at 200 GeV. The rapidity dependence of the suppression can be obtained by comparing the STAR data for mid-rapidity (\(|y| < 1\)) and the PHENIX data for forward rapidity (\(1.2< |y| < 2.2\)). The suppression is much stronger at forward rapidity than at mid-rapidity for Au + Au collisions at 200 GeV. However, at 39 and 62.4 GeV, the suppression of \(J/\psi\) shows no significant rapidity dependence (within uncertainties). This could be a result of the energy and rapidity dependence of the (re)combination contribution, which is larger at higher collision energies and mid-rapidity. To understand the collision energy dependence, the inclusive \(J/\psi\)\(R_{\mathrm {AA}}\) for central heavy-ion collisions is plotted as a function of the center-of-mass energy. The \(R_{\mathrm {AA}}\) is flat at \(\sim 0.4\) from \(\sqrt{s_{_\mathrm {NN}}}\)= 17.3 to 200 GeV; however, it dramatically increases to \(>0.6\) at the LHC. The curves shown in Fig. 17 are theoretical calculations from a transport model [110] that implements CNM effects, QGP melting, and (re)combination. The dashed curve represents the \(R_{\mathrm {AA}}\) of primordially produced \(J/\psi\), whose production yield suffers from the QGP melting and CNM effects. The \(R_{\mathrm {AA}}\) is fairly flat from \(\sqrt{s_{_\mathrm {NN}}}\)= 17.3 to 62 GeV and then decreases with center-of-mass energy. This trend is the result of counterbalancing between the CNM effects and QGP melting. The nuclear absorption cross section decreases with increasing center-of-mass energy, resulting in an increased \(R_{\mathrm {AA}}\) at larger center-of-mass energies. Meanwhile, the QGP melting results in a decreasing trend due to the increasing energy density. The QGP melting plays a significant role at RHIC top energies. The \(R_{\mathrm {AA}}\) calculated using only CNM effects is estimated to be about 0.6 for central Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV [110]. The QGP melting reduces the \(R_{\mathrm {AA}}\) from 0.6 to 0.2. The dotted curve in Fig. 17 shows the \(R_{\mathrm {AA}}\) for the \(J/\psi\) produced from (re)combination; it is negligible from the SPS energy to the RHIC BES energy of \(\sqrt{s_{_\mathrm {NN}}}\)\(<\sim 50\) GeV and starts to play a role at higher center-of-mass energies. At the RHIC top energy, the contribution to \(J/\psi\) from (re)combination is comparable to the survived primordial \(J/\psi\), and it becomes dominant at the LHC. The solid curve in Fig 17 shows the sum of the two components. It can describe the inclusive \(J/\psi\)\(R_{\mathrm {AA}}\) for central heavy-ion collisions from the SPS (CNM effects domain) to LHC ((re)combination domain) within uncertainties.

Fig. 17
figure 17

(Color online) Left: Inclusive \(J/\psi\)\(R_{\mathrm {AA}}\) as a function of centrality in mid- and forward rapidities for Au + Au collisions at 39, 62.4, and 200 GeV. Right: Inclusive \(J/\psi\)\(R_{\mathrm {AA}}\) as a function of center-of-mass energy in 0–20% central heavy-ion collisions (Au + Au at the RHIC and Pb + Pb at the SPS and LHC). The figure is taken from [107]

It is particularly interesting to study the \(J/\psi\) suppression in heavy-ion collisions at center-of-mass energies around 50 GeV; here, the (re)combination contribution remains negligible (as in the SPS), but the energy density is higher and the expected CNM effects (such as nuclear absorption) are smaller than at the SPS. The STAR experiment has collected a large dataset of Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 54 GeV. The quantity of available statistics is one order of magnitude larger than those for the 62 GeV dataset. This allows for more precise and differential measurements of the \(J/\psi\) suppression for around 50 GeV. A fixed-target experiment using the LHC beams (AFTER@LHC [111]), or current LHC experiments in fixed-target mode, will be able to collect comprehensive statistical heavy ion collision data at \(\sqrt{s_{_\mathrm {NN}}}\)= 72 GeV with a Pb beam of 2.76 TeV per nucleon. The \(p+\hbox {A}\) data can also be extracted at the same energy using a Pb beam on a proton target, to study the CNM effects.

4.2 Collision system dependence

The measurements of Pb + Pb collisions at the SPS show anomalous \(J/\psi\) suppression from semi-peripheral to central Pb + Pb collisions (\(N_{\text {part}}>\sim 100\)) at \(\sqrt{s_{_\mathrm {NN}}}\)= 17.3 GeV. At RHIC energies, the energy density is expected to surpass the threshold value for QGP formation, based on lattice QCD calculations at \(N_{\text {part}}\)s below 100. However, the Au + Au data in this QGP transition threshold region are limited. To provide the necessary information for such an important region, PHENIX has measured \(J/\psi\) suppression in Cu+Cu collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV. Figure 18a, b shows the inclusive \(J/\psi\)\(R_{\mathrm {AA}}\) as a function of \(N_{\text {part}}\) in Cu+Cu collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV in the mid-rapidity (\(|y| < 0.35\)) and forward/backward rapidity (\(1.2< |y| < 2.2\)), respectively. The results for Au + Au collisions at the same energy are also shown for comparison. The \(R_{\mathrm {AA}}\) for Cu+Cu and Au + Au is consistent with each other (within uncertainties) at comparable \(N_{\text {part}}\). The Cu+Cu data cover the \(N_{\text {part}}\) range up to 100, with much finer bins than those used for Au + Au collisions. The observed suppression receives a large contribution from CNM effects. To extract the possible QGP melting effects of Cu+Cu collisions, the CNM effects are estimated by extrapolating the suppression measurement for \(d+\hbox {Au}\) collision up to the same energy, considering nPDF and nuclear absorption. The nPDF is taken from the shadowing model EKS98 [114] or nDSg [115]. The nuclear absorption cross sections are optimized separately for mid- and forward rapidities from the fit to \(d+\hbox {Au}\) data. The CNM effects estimated with EKS98 (method 1) are shown in Fig. 18a, b as solid lines. The dashed lines depict the calculation performed by varying the nuclear absorption cross section by \(1\sigma\). The predicted CNM \(R_{\mathrm {AA}}\) shows almost no difference between Cu+Cu and Au + Au collisions for the same \(N_{\text {part}}\). Figure 18c shows the measured \(R_{\mathrm {AA}}\) in Cu+Cu collisions divided by the predicted CNM \(R_{\mathrm {AA}}\) with EKS98 parameterization in both mid- and forward/backward rapidities. At \(N_{\text {part}}\)\(<\sim 50\), the measured \(R_{\mathrm {AA}}\) in Cu+Cu collisions is seen to be consistent with the CNM projection to within an approximate uncertainty of 15%. When \(N_{\text {part}}\) exceeds 50, the centroid of the measured ratios at both mid- and forward rapidities are smaller than unity. However, no strong conclusions can be drawn with the large uncertainties. More precise measurements for \(p(d)+\hbox {A}\) collisions are needed, along with a better understanding of how to extrapolate the CNM effects from \(p(d)+\hbox {A}\) collisions to A + A collisions. Nevertheless, the CNM effects dominate \(J/\psi\) production in Cu+Cu collisions and peripheral and semi-peripheral Au + Au collisions.

Fig. 18
figure 18

Inclusive \(J/\psi\)\(R_{\mathrm {AA}}\) as a function of \(N_{\text {part}}\) in a mid-rapidity and b forward/backward rapidity, as well as c the ratio of \(R_{\mathrm {AA}}\) to the expectation from CNM effects in Cu+Cu collisions at 200 GeV. The figure is taken from [112]

In 2012, the RHIC collided Cu + Au collisions at 200 GeV. The rapidity dependence of \(J/\psi\) suppression in the asymmetric collision system may provide key insights into the balance of CNM and hot nuclear matter effects. The parton distribution functions were more strongly modified for heavier Au nucleus than for lighter nuclei. In the forward rapidity (Cu-going direction), the \(J/\psi\) probes gluons at lower Bjorken x in the Au nucleus and higher x in the Cu nucleus. This is reversed in backward rapidity. The shadowing effects are expected to be stronger in forward rapidity than in backward rapidity. On the other hand, the \(J/\psi\) produced in forward rapidity have a large rapidity relative to the Au nucleus; thus, they have a shorter proper time. In the forward rapidity, this could result in a reduced nuclear absorption of \(J/\psi\) or energy losses. Furthermore, the energy density and hadron multiplicity are also asymmetric in Cu + Au collisions; they are higher in the backward rapidity (Au-going direction). The asymmetric energy density and hadron multiplicity may result in different CNM and hot matter effects. The breakup of \(J/\psi\) by comovers depends on the density of comovers and is expected to be stronger in backward rapidity than in forward rapidity. The asymmetric hot nuclear matter effects are not as straightforward. The QGP melting effect is stronger in the backward rapidity, reducing \(R_{\mathrm {AA}}\) . However, the (re)combination effect is also stronger in the backward rapidity, increasing \(R_{\mathrm {AA}}\) .

The upper panel of Fig. 19 shows the inclusive \(J/\psi\)\(R_{\mathrm {AA}}\) as a function of \(N_{\text {part}}\) in forward (\(1.2< y < 2.2\)) and backward (\(-2.2< y < -1.2\)) rapidities for Cu + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV. The combined Au + Au results for forward and backward rapidities at the same center-of-mass energy are also shown for comparison. The \(R_{\mathrm {AA}}\) for Cu + Au collisions in the backward rapidity (Au-going direction) resembles that for Au + Au collisions at similar \(N_{\text {part}}\); however, the \(R_{\mathrm {AA}}\) for Cu + Au collisions in the forward rapidity (Cu-going direction) is systematically lower. The difference between the forward and backward rapidities is clearer in the bottom panel of Fig. 19, in which the ratio of \(R_{\mathrm {AA}}\) in forward rapidity to that in backward rapidity is shown. The ratios are 20–30% lower than unity and show no significant \(N_{\text {part}}\) dependence. The gray band depicts the prediction from a simple Glauber model, incorporating gluon distribution function modifications from the EPS09 [116] parameterization, as well as a rapidity independent effective \(c\bar{c}\) breakup cross section of 4 mb to account for the nuclear absorption effects. The expected shadowing difference has the same sign as the data; the difference is comparable with the data (within uncertainties) but is systematically smaller, especially in peripheral collisions. The QGP melting is expected to result in a forward-to-backward ratio exceeding unity. However, the predicted (re)combination effects have the same sign as the data and decrease toward more peripheral collisions. It seems that neither QGP melting or (re)combination can explain the possible differences between the data and the \(R_{\mathrm {AA}}\) estimated from the gluon shadowing effect.

Fig. 19
figure 19

(Color online) Top: Inclusive \(J/\psi\)\(R_{\mathrm {AA}}\) as a function of \(N_{\text {part}}\) in forward (\(1.2< y < 2.2\)) and backward (\(-2.2< y < -1.2\)) rapidity for Cu + Au collisions at 200 GeV. The data for Au + Au collisions at 200 GeV are also shown for comparison. Bottom: The ratio of \(R_{\mathrm {AA}}\) in forward and backward rapidities for Cu + Au collisions. The figure is taken from [113]

The energy density or particle multiplicity dependence of \(J/\psi\) suppression can also be studied in U + U collisions. The energy density for U + U collisions is about 20% larger than that seen for Au + Au collisions with similar numbers of participants. Figure 20 shows the inclusive \(J/\psi\)\(R_{\mathrm {AA}}\) as a function of \(N_{\text {part}}\) for U + U collisions at 193 GeV in the forward rapidity, compared against that measured for Au + Au collisions [117]. The U + U data were taken in 2012. Unlike Au nuclei, U nuclei are deformed and their shape is not well understood. The number of participants and the number of binary nucleon–nucleon collisions in U + U collisions depend on the shape of the U nucleus. The U + U results shown in the upper and lower panels of Fig. 20 were obtained using two parameterization of the deformed Woods–Saxon distribution for U (set 1 [118] and set 2 [119]). The parameterization of set 2 has a smaller surface diffuseness, resulting in a notably more compact nucleus (and larger number of binary nucleon–nucleon collisions).

Fig. 20
figure 20

(Color online) Inclusive \(J/\psi\)\(R_{\mathrm {AA}}\) as a function of \(N_{\text {part}}\) in the forward rapidity (\(1.2< |y| < 2.2\)) for U + U collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 193 GeV, compared to Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV. The figure is taken from [117]

In both parameterizations, the observed \(R_{\mathrm {AA}}\) for U + U collisions is similar to that seen for Au + Au collisions with the same number of participants in peripheral and semi-peripheral collisions; however, it exhibits less suppression than in central Au + Au collisions. The CNM effects due to shadowing are expected to resemble those for Au + Au and U + U collisions. The difference between Au + Au and U + U collisions is likely due to hot nuclear matter effects. The increase in \(R_{\mathrm {AA}}\) from Au + Au collisions to U + U collisions supports the hypothesis that the enhancement due to (re)combination becomes more significant than the suppression due to QGP melting.

4.3 Transverse momentum dependence

The QGP melting, (re)combination, and CNM effects do not only depend on the collision energy and system; they also depend on the transverse momentum of \(J/\psi\). The \(R_{\mathrm {AA}}\) from CNM effects usually exhibits an increasing trend as a function of \(J/\psi\)\(p_{\mathrm{T}}\). The E866 [121] and HERA-B [122] experiments found that the \(J/\psi\) suppression factor \(\alpha\), which was obtained by assuming a cross-sectional dependence on nuclear mass A, is of the form \(\sigma _A = \sigma _N \times A^\alpha\) in fixed-target \(p+\hbox {A}\) collisions; it features a clear increasing trend as a function of \(p_{\mathrm{T}}\) and crosses unity at of around \(2-3\) GeV/c. The increasing trend is typically attributed to the multiple scatterings of the incident parton before hard scattering and to the nascent \(c\bar{c}\) in the final state. This effect is sometimes also referred to as the Cronin effect. At the RHIC, the PHENIX Collaboration presented the \(J/\psi\) suppression as a function of \(p_{\text {T}}\) for \(d+\hbox {Au}\) collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV, in both mid- and forward/backward rapidities [123]; furthermore, they recently submitted for publication the results for \(p+\hbox {Al}\), \(p+\hbox {Au}\), and \(^3\)He+Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV [124]. The STAR Collaboration also measured the \(p_{\text {T}}\) dependence of \(J/\psi\) suppression for \(p+\hbox {Au}\) collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV [125]. For all collision systems using Au, the suppression of inclusive \(J/\psi\) also shows an increasing trend. The suppression severity is approximately 30% at low-\(p_{\text {T}}\) but consistent with no suppression at \(p_{\text {T}}\) above 3–4 GeV/c. A transport model [110] predicted a \(R_{\mathrm {AA}}\) that is approximately 0.4 for \(p_{\mathrm{T}}\) around zero and lies on unity for \(p_{\mathrm{T}}\) from 4.5 to 10 GeV/c; these predictions were in mid-rapidity for 0–20% central Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV, including CNM effects such as nuclear absorption and the feeddown contribution from B.

The (re)combination effect is expected to decrease with \(p_{\mathrm{T}}\); this is mainly because the \(J/\psi\) yield arising from (re)combination is approximately proportional to the square of the number of charm quarks, which falls fast with \(p_{\mathrm{T}}\). The transport models [110, 126] show that the contribution from (re)combination is comparable with the primordial \(J/\psi\) at \(p_{\mathrm{T}}\) below 1 GeV/c and is negligible at \(p_{\mathrm{T}}\) above 5 GeV/c. The \(p_{\text {T}}\) dependence of QGP melting is not well understood. The formation time effect predicts an increasing trend because, at higher \(p_{\mathrm{T}}\), \(J/\psi\) is more likely to form outside of the hot dense medium and will be less affected by it. However, the dissociation temperature of \(J/\psi\) may depend on the relative velocity between \(J/\psi\) and the medium, and its \(p_{\text {T}}\) dependence is model dependent. \(J/\psi\) with higher \(p_{\text {T}}\) may have higher or lower dissociation temperatures in different models [127, 128]. A detailed differential measurement of \(J/\psi\) suppression over a broad kinematic range might shed new light on \(J/\psi\) production mechanisms in heavy-ion collisions, as well as the properties of QGP.

Since 2006, the STAR Collaboration has attempted to extend \(J/\psi\) measurements in heavy-ion collisions to \(p_{\mathrm{T}}\) above 5 GeV/c [71, 129]. The \(J/\psi\) production at high \(p_{\mathrm{T}}\)—for which the CNM and (re)combination effects are negligible—is found to be consistent with no suppression in Cu+Cu and peripheral Au + Au collisions but is significantly suppressed in (semi-)central Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV. However, because of the limited statistics, no firm conclusions have been drawn.

In 2014 and 2016, the STAR Collaboration collected large datasets of Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV, utilizing the Muon Telescope Detector (MTD). The MTD detector is designed to trigger and identify muons; it was completed in early 2014. Compared to the previous measurements taken in mid-rapidity through the di-electron channel [45, 71, 107], the new data allow the kinematic reach to be extended toward high \(p_{\text {T}}\) with better precision. Figure 21 shows the inclusive \(J/\psi\)\(R_{\mathrm {AA}}\) as a function of \(p_{\text {T}}\) for Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV, using the data taken in 2014. The filled stars denote the new results from STAR, through the di-muon channel, and the open and filled circles depict the previous results through the di-electron channel. The results for low \(p_{\text {T}}\) from PHENIX are also shown as hollow crosses [99]. The new results are consistent with previous results in the overlapping kinematic region; however, they have better precision and cover a wider kinematic range. Within the uncertainties, \(J/\psi\) suppression shows little \(p_{\text {T}}\) dependence from \(p_\mathrm{T} \sim 0\) up to 14 GeV/c. For comparison, the \(J/\psi\) suppression in mid-rapidity for Pb + Pb collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 2.76 TeV, as measured by the ALICE [130] and CMS [131] Collaborations, is shown in panel (a). The \(p_{\text {T}}\) dependences at the LHC and RHIC are significantly different. The \(J/\psi\)\(R_{\mathrm {AA}}\) at the RHIC is lower than at the LHC for low \(p_{\text {T}}\) , but it is systematically higher than at the LHC for high \(p_{\mathrm{T}}\). The difference in the \(R_{\mathrm {AA}}\) is due to different (re)combination contributions in \(J/\psi\) production and the initial temperature and lifetime of the QGP in heavy-ion collisions at different collision energies. The shaded bands and dashed lines represent two transport model calculations for Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV, from the Tsinghua (TM1) [126] and TAMU (TM2) [110] groups. The CNM effects, QGP melting, and (re)combination are considered in both models, although the detailed treatments are different. The TM1 model describes the data reasonably well at low \(p_{\mathrm{T}}\); however, it shows a steeper increasing trend toward high \(p_{\mathrm{T}}\) than the data indicate. The TM2 model’s \(p_{\text {T}}\) dependence better matches the data; however, the absolute values are systematically lower than the data between intermediate and high \(p_{\mathrm{T}}\)s. The two solid bands covering \(3.5<p_{\mathrm{T}}<15\) GeV/c in panel (b) show theoretical calculations using the vacuum \(J/\psi\) wave function in the absence of color-screening, and it includes both the radiative energy loss of color-octet \(c\bar{c}\) pairs and the collisional dissociation of \(J/\psi\) [132]. The two bands correspond to two different values of the \(J/\psi\) formation time; both of them are consistent with the data. All calculations include a feeddown contribution constrained by the measurements for \(p+p\) collisions, as well as CNM effects constrained by the measurements for \(p(d)+\hbox {A}\) collisions.

Fig. 21
figure 21

Inclusive \(J/\psi\)\(R_{\mathrm {AA}}\) as a function of \(p_{\text {T}}\) for Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 through di-electron and di-muon decay channels. The results from Pb + Pb collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 2.76 GeV are also shown for comparison. The curves and shaded bands depict theoretical calculations. See text for detailed descriptions. The figure is taken from [120]

Figure 22 shows the centrality dependence of \(J/\psi\) suppression in heavy-ion collisions at both RHIC and LHC energies for low-\(p_{\text {T}}\) (upper) and high-\(p_{\text {T}}\) (lower) \(J/\psi\). For low-\(p_{\text {T}}\)\(J/\psi\), the suppression decreases toward central collisions at the RHIC but is flatter at the LHC. The low-\(p_{\text {T}}\)\(J/\psi\) suppression arises from the interplay of CNM effects, QGP melting, and (re)combination. The greatly reduced suppression observed in central heavy-ion collisions at the LHC compared to the RHIC is likely due to the different fractions of (re)combination between the RHIC and LHC, which are expected as a result of the different charm quark production cross sections at these energies. The high-\(p_{\text {T}}\)\(J/\psi\) is suppressed by a factor of 3.1 with a significance of 8.1\(\sigma\) in 0–10% Au + Au collisions. The CNM effects and (re)combination contribution are expected to be minimal in this \(p_{\text {T}}\) range (\(p_\mathrm{T}>5\) GeV/c). The significant suppression of high-\(p_{\text {T}}\)\(J/\psi\) in central Au + Au collisions provides strong evidence for the color-screening effect of QGP. Unlike the low-\(p_{\text {T}}\)\(J/\psi\), the high-\(p_{\text {T}}\)\(J/\psi\) is more suppressed at the LHC than at the RHIC. This could be because the temperature of the medium created at the LHC is higher than that at the RHIC.

Fig. 22
figure 22

(Color online) Inclusive \(J/\psi\)\(R_{\mathrm {AA}}\) as a function of \(N_{\text {part}}\) for Au + Au collisions at 200 GeV, for low-\(p_{\text {T}}\) and high-\(p_{\text {T}}\)\(J/\psi\). The figure is taken from [120]

4.4 Collective flow

The collective flow measurements of \(J/\psi\) may also shed light on the relative contributions of primordial \(J/\psi\) and \(J/\psi\) from (re)combination. The primordial \(J/\psi\) is predominantly produced before QGP formation; thus, it does not have an initial collective flow. In non-central collisions, the primordial \(J/\psi\) may exhibit different suppressions along different azimuthal angles with respect to the reaction plane, owing to the different path lengths. However, the azimuthal anisotropy should be limited. On the other hand, the \(J/\psi\) produced by the (re)combination of the charm quark and its antiquark should inherit the flow of charm quarks and may possess considerable flow characteristics.

As discussed in Sect. 2.1, the \(D^0\)\(v_2\) shown in Fig. 4 is found to follow mass ordering at low \(p_{\mathrm{T}}\) (as expected from hydrodynamics) and NCQ-scaling as the light and strange hadrons in the intermediate \(p_{\mathrm{T}}\) (as expected from quark coalescence). It is concluded that the charm quarks gain a significant flow in QGP.

The radial flow of \(D^0\) mesons in heavy-ion collisions is also studied by precisely measuring the \(p_{\text {T}}\) or \(m_{\mathrm{T}}\) spectra for Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV [32], as discussed in Sect. 2.1. The \(T_{\text {eff}}\) slope parameter of the exponential fit to the \(m_{\mathrm{T}}\) spectra, as a function of particle mass for light-flavor hadrons, strange hadrons, and \(D^0\) mesons, clearly shows two different systematic trends. The data for the light-flavor hadrons \(\pi\), K, and p follow a linear dependence, whereas the data for the strange and charm hadrons \(\phi\), \(\Lambda\), \(\Omega\), and \(D^0\) follow another linear dependence. The \(T_{\text {fo}}\) and \(\langle \beta _t \rangle\) of \(D^0\), extracted by fitting the \(p_{\text {T}}\) spectra using blast-wave (BW) [133] or Tsallis blast-wave (TBW) [41] models, are found to group with the multi-strangeness particles \(\phi\), \(\Xi\), and \(\Omega\); they show a much smaller \(\langle \beta _t \rangle\) and larger \(T_{\text {fo}}\) compared to the light hadrons \(\pi\), K, and p. This suggests that the \(D^0\) flows with the medium and its collectivity is mostly obtained via partonic re-scattering in the QGP phase. If the \(J/\psi\) produced from the (re)combination of charm quarks and their antiquarks is the dominant process, it should have a significant \(v_2\) and radial flow.

In 2010, the STAR Collaboration measured \(J/\psi\)\(v_2\) for Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV using a combination of various triggers. The inclusive \(J/\psi\)s were reconstructed through the di-electron channel. The upper panel of Fig. 23 shows \(v_2\) as a function of \(p_{\text {T}}\) for inclusive \(J/\psi\), \(\phi\), and charged hadrons, which is dominated by \(\pi\). The gray box on the \(J/\psi\) data indicates the estimated maximum possible range of \(v_2\) if the influence of non-flow is corrected for; it is estimated using the measurement of \(J/\psi\) -hadron correlation in \(p+p\) collisions at the same energy. Unlike the \(D^0\), the \(J/\psi\)\(v_2\) is significantly lower than that observed for \(\phi\) and charged hadrons at \(p_{\mathrm{T}}\) above 2 GeV/c.

Fig. 23
figure 23

(Color online) \(J/\psi\)\(v_2\) as a function of \(p_{\text {T}}\) for 0–80% Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV, compared to the results for \(\phi\) and charged hadrons, as well as theoretical calculations. The figure is taken from [134]

The lower panel of Fig. 23 compares the \(J/\psi\)\(v_2\) data and theoretical calculations. The solid line shows the calculation results for the \(J/\psi\) produced from the initial hard scattering; this is nonzero but limited in the \(p_{\text {T}}\) range of 0–5 GeV/c. Although significant suppression of \(J/\psi\) is observed in Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV, the azimuthally different suppression along the different path lengths in the azimuth is limited, beyond the sensitivity capabilities of current measurements. The dotted line shows the prediction for \(J/\psi\) produced by the coalescence of fully thermalized charm quarks at the freeze-out ((re)combination). Its maximum is similar to that of light and strange hadrons; however, it is shifted to higher \(p_{\text {T}}\) due to the significantly larger mass of the charm quark and \(J/\psi\) particle. The prediction for \(J/\psi\) from coalescence is systematically higher than the data at \(p_{\text {T}}\) > 2 GeV/c. The \(\chi ^2/ndf\) is as large as 16.2/3, corresponding to a small p value of \(1.0\times 10^{-3}\). The transport models incorporating contributions from both primordial production and (re)combination predict a much smaller \(v_2\) and are consistent with the data. The p values are 0.58 and 0.38 for the TAMU [135] and Tsinghua [136] transport models, respectively. The small \(v_2\) in the transport model arises because the \(v_2\) of the charm quark is small at low \(p_{\mathrm{T}}\), and the (re)combination contribution is small at high \(p_{\mathrm{T}}\). Although both transport models describe the data reasonably well, a sizable difference exists between them. The TAMU model is closer to \(v_2\) for initially produced \(J/\psi\). Measurements with improved precision will help distinguish or constrain the two transport models. The hydrodynamics model, tuned to describe the \(v_2\) of light hadrons, predicts a \(J/\psi\)\(v_2\) that strongly increases with \(p_{\text {T}}\) below 4 GeV/c; thus, it fails to describe the data. The \(\chi ^2\)/ndf is 7.0/3, corresponding to a p value of 0.072 [137]. Based on the data and model comparisons, it is concluded that the \(J/\psi\)\(v_2\) data disfavor the scenario that \(J/\psi\) with \(p_{\text {T}}\)\(>2\) GeV/c are predominantly produced through coalescence from charm and anti-charm quarks that thermalize and flow with the QGP.

The \(J/\psi\) radial flow at the SPS and RHIC is systematically studied in [138] using the Tsallis blast-wave (TBW) model. The \(p_{\text {T}}\) spectra of light and strange hadrons for Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV at the RHIC, as well as for Pb + Pb collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 17.3 at the SPS, are fitted with the TBW model to extract the radial flow, kinetic freeze-out temperature, and non-extensive parameter. The \(p_{\text {T}}\) spectrum for \(J/\psi\) , predicted from the TBW with the same set of parameters as light and strange hadrons, is much softer than the measurement. It overestimates the yield at low \(p_{\text {T}}\) and underestimates it at high \(p_{\mathrm{T}}\). Fitting to just the \(J/\psi\)\(p_{\text {T}}\) spectrum shows that the radial flow of \(J/\psi\) at both the RHIC and SPS is consistent with zero. This provides further evidence that \(J/\psi\) production at the RHIC and SPS does not predominantly arise from the (re)combination of thermalized charm quarks.

4.5 \(J/\psi\) photoproduction with nuclear overlap

\(J/\psi\) can also be generated by the intense electromagnetic fields that accompany relativistic heavy ions [139]. The intense electromagnetic field can be treated as a spectrum of equivalent photons using the equivalent photon approximation [140]. The quasi-real photon emitted by one nucleus fluctuates into a \(c\bar{c}\) pair, scatters off the other nucleus, and emerges as a real \(J/\psi\). The coherent nature of these interactions determines the distinctive characteristics of the process; the final products consist of a J/\(\psi\) with a very low transverse momentum, two intact nuclei, and nothing else. Conventionally, these reactions are only visible and studied in ultra-peripheral collisions (UPCs), in which the impact parameter (b) is more than twice the nuclear radius (\(R_\mathrm{A}\)), because these prevent any hadronic interactions. Several results of \(J/\psi\) production in UPCs are already available at the RHIC [141] and LHC [142,143,144]; they provide valuable insights into the gluon distribution in the colliding nuclei [145].

Can coherent photon products also exist in hadronic heavy-ion collisions (HHICs, \(b < 2R_{\mathrm{A}}\)), where violent strong interactions occur in the overlapping region? The answer originates from the measurements taken at ALICE: significant excesses of \(J/\psi\) yield at very low \(p_{\mathrm{T}} (< 0.3\) GeV/c) have been observed in peripheral Pb + Pb collisions at \(\sqrt{s_{\mathrm{{NN}}}} =\) 2.76 TeV [146]; these cannot be explained by the hadronic \(J/\psi\) production with the known cold and hot medium effects. STAR made measurements of di-electron production [147] for Au + Au collisions at \(\sqrt{s_{\mathrm{{NN}}}} =\) 200 GeV, and they also observed significant enhancement at very low \(p_{\mathrm{T}}\) in peripheral collisions. The observed anomalous excesses have the characteristics of coherent photon interactions and can be quantitatively described by the theoretical calculations using coherent photon–nucleus [148,149,150,151] and photon–photon [152,153,154] production mechanisms; this suggests evidence of coherent photon reactions in HHICs. The observed excesses may originate from coherent photon-induced interactions, which imposes considerable challenges for the existing models: for example, how the broken nuclei satisfy the requirement of coherence. Measurements of \(J/\psi\) production at very low \(p_{\mathrm{T}}\) for different collision energies, collision systems, and centralities can shed new light on the origin of the excess.

The STAR Collaboration measured \(J/\psi\) production yields at very low \(p_{\mathrm{T}}\) for Au + Au collisions at \(\sqrt{s_{\mathrm{{NN}}}} =\) 200 GeV, as well as U + U collisions at \(\sqrt{s_{\mathrm{{NN}}}} =\) 193 GeV, in mid-rapidity via the di-electron decay channel. Figure 24 shows the \(J/\psi\)\(R_{\mathrm {AA}}\) as a function of \(p_{\mathrm{T}}\) for Au + Au collisions and U + U collisions in different centrality classes. Suppression of \(J/\psi\) production was observed for \(p_{\mathrm{T}}>\) 0.2 GeV/c in all collision centrality classes; this is consistent with the previous measurements [45, 71, 99, 107] and can be well described by the transport models [110, 126] incorporating cold and hot medium effects. However, in the extremely low \(p_{\mathrm{T}}\) range (for instance, \(p_{\mathrm{T}} < 0.2\) GeV/c), large enhancement of \(R_{\mathrm{{AA}}}\) (above unity) was observed in peripheral collisions (40–80%) for both Au + Au and U + U collisions. In this \(p_{\mathrm{T}}\) range, the color-screening and CNM effects would suppress \(J/\psi\) production; the only gain effect is regeneration, which is negligible in peripheral collisions [126]. The overall effect would lead to \(R_{\mathrm{{AA}}}<\) 1 for hadronic production, which is far below the current measurement. For \(p_{\mathrm{T}} < 0.05\) GeV/c in the 60–80% centrality class, the \(R_{\mathrm{{AA}}}\) is \(24\pm 5 (\mathrm {stat.}) \pm 9 (\mathrm {syst.})\) for Au + Au collisions and \(52\pm 18 (\mathrm {stat.}) \pm 16 (\mathrm {syst.})\) for U + U collisions, significantly deviating from the hadronic \(p+p\) reference with \(N_{\mathrm{{coll}}}\) scaling; this strongly suggests an additional production mechanism.

Fig. 24
figure 24

(Color online) The J/\(\psi\)\(R_{\mathrm{{AA}}}\) as a function of \(p_{\mathrm{T}}\) for Au + Au collisions at \(\sqrt{s_{\mathrm{{NN}}}} = 200\) GeV and U + U collisions at \(\sqrt{s_{\mathrm{{NN}}}} = 193\) GeV. The figure is taken from [155]

By assuming that the observed excess originates from coherent photoproduction, STAR also reported the differential cross section \(\mathrm {d}\sigma /\mathrm {d}t\), where t is the negative momentum transfer squared, \(-t \sim p_{\mathrm{T}}^2\); this indicates the distribution of interaction sites and is closely related to the parton distribution in the nucleus. Figure 25 shows the \(J/\psi\) yield with the expected hadronic contribution subtracted, as a function of \(-t\) in the 40–80% centrality class for Au + Au and U + U collisions in the low \(p_{\mathrm{T}}\) range. The shape of the dN/dt distribution resembles that observed in UPC for \(\rho ^{0}\) mesons [156]. An exponential fit has been applied to the distribution in the \(-t\) range of \(0.001-0.015~(\text {GeV}/c)^{-2}\) for Au + Au collisions. The slope parameter of this fit corresponds to the volume of the interaction sites within the target. The extracted slope parameter is \(177 \pm 23~(\text {GeV}/c)^{-2}\), which is consistent with that expected for an Au nucleus (199 \((\text {GeV}/c)^{-2}\)) [157] within uncertainties. As shown in the figure, the data point at \(-t < 0.001 ~(\text {GeV}/c)^{-2}\) is significantly lower (\(3.0 \sigma\)) than the extrapolation of the exponential fit. This suppression may suggest interference, which has been confirmed by STAR [158] in the UPC case for the \(\rho ^{0}\) meson. The theoretical calculations with interference from [149], shown as the blue curve in the plot, can describe the Au + Au data reasonably well (\(\chi ^{2}\)/ndf = 4.8/4) for \(-t < 0.015~(\text {GeV}/c)^{-2}\). It should be noted that there also exists a possible contribution from incoherent \(J/\psi\) photoproduction. The fitting \(-t\) range is chosen to ensure that the coherent production (\(\langle -t \rangle \sim 0.005~(\text {GeV}/c)^{-2}\)) dominates over the incoherent production (\(\langle -t \rangle \sim 0.250~(\text {GeV}/c)^{-2}\)). Owing to the different nuclear profile, the \(-t\) distribution for U + U collisions is expected to differ from that of Au + Au collisions; however, as shown in the figure, the difference is not observable due to the large uncertainties.

Fig. 25
figure 25

(Color online) The \(J/\psi\) yield as a function of the negative momentum transfer squared \(-t\) ( \(-t \sim p_{\mathrm{T}}^{2}\) ) in the 40–80% collision centrality class for Au + Au and U + U collisions. The figure is taken from [155]

Figure 26 shows the \(p_{\mathrm{T}}\)-integrated \(J/\psi\) yields for \(p_{\mathrm{T}}<\) 0.1 GeV/c, with the expected hadronic contribution subtracted as a function of \(N_{\mathrm{{part}}}\) for 30–80% Au + Au and 40–80% U + U collisions. The expected hadronic contributions for Au + Au collisions are also plotted for comparison. As depicted in the figure, the contribution from hadronic production is non-dominant for the low-\(p_{\mathrm{T}}\) range in the measured centrality classes. Furthermore, the hadronic contribution increases dramatically toward more central collisions, while the measured excess shows no sign of significant centrality dependence (within uncertainties). Under the assumption of coherent photoproduction, the excess in U + U collisions should exceed that seen in Au + Au collisions. Indeed, the central value of measurements in U + U collisions exceeds that in Au + Au collisions. However, limited by the current experimental precision, the observed difference (2.0\(\sigma\)) is not significant. The model calculations for Au + Au collisions with the coherent photoproduction assumption [149] are also plotted for comparison. In the model calculations, the authors consider either the whole nucleus or only the spectator nucleons as photon and Pomeron emitters; this results in four configurations for the photon emitter + Pomeron emitter: (1) Nucleus + Nucleus; (2) Nucleus + Spectator; (3) Spectator + Nucleus; and (4) Spectator + Spectator. All four scenarios can describe the data points in the most peripheral centrality bins (60–80%). However, in more central collisions, the Nucleus + Nucleus scenario significantly overestimates the data, which suggests a partial disruption of coherent production by the violent hadronic interactions in the overlapping region. The measurements in semi-central collisions seem to favor the Nucleus + Spectator or Spectator + Nucleus scenarios. The approach used in the model effectively incorporates the shadowing effect, which can describe the UPC results in the x-ranges probed by RHIC measurements. However, the coherently produced J\(/\psi\) can be modified by hot medium effects (e.g., QGP melting), which are not included in the model. More precise measurements in more central collisions, as well as advanced modeling incorporating hot medium effects, are essential to distinguishing the different scenarios.

Fig. 26
figure 26

(Color online) The \(p_{\mathrm{T}}\)-integrated J/\(\psi\) yields (\(p_{\mathrm{T}}<\) 0.1 GeV/c) as a function of \(N_{\mathrm{{part}}}\) for 30–80% Au + Au collisions and 40–80% U + U collisions. The figure is taken from [155]

5 \(\Upsilon\) production in medium

The \(b\bar{b}\) cross section is much smaller than that of the \(c\bar{c}\) at both the RHIC and LHC. From an FONLL calculation [19, 27, 28], the number of \(b\bar{b}\) pairs per event is estimated to be less than 0.1 for 0–10% Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV and \(2.3 \pm 0.4\) (\(4.9 \pm 0.9\)) for 0–10% Pb + Pb collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 2.76 (5.5) TeV. b quarks struggle to achieve thermalization due to their large masses. Thus, the contribution of (re)combination is negligible at RHIC for \(\Upsilon\). In terms of CNM effects, Ref. [159] points out that the dissociation of \(\Upsilon (1S)\) by comovers is much smaller than that of \(J/\psi\) and can be neglected at the RHIC. Thus, compared to J/\(\psi\), \(\Upsilon\) provides a cleaner probe of QGP melting, at least at the RHIC.

The binding energy or the radii of \(\Upsilon (1S)\), \(\Upsilon (2S)\), and \(\Upsilon (3S)\) are quite different. Because the dissociation temperature depends on the radii of the quarkonium states, measurements on the suppression of various \(\Upsilon\) states can be used to study the properties of the color screening and QGP.

5.1 \(\Upsilon\) production in \(p+\hbox {Au}\) collisions

The CNM effects on \(\Upsilon\) production can be studied in \(p+\hbox {A}\) or \(d+\hbox {A}\) collisions. Figure 27 shows the suppression of \(\Upsilon (1S+2S+3S)\) as a function of rapidity for \(p+\hbox {Au}\) and \(d+\hbox {Au}\) collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV. The STAR results are measured at mid-rapidity through the di-electron channel [160, 161], and the PHENIX results are measured in the forward/backward rapidity through the di-muon channel [92]. The data points at \(|y|<0.5\) in \(p+\hbox {Au}\) collisions have the highest precision. It is \(R_{p\text{Au}} = 0.82 \pm 0.10\,\text {(stat.)} ^{+0.08}_{-0.07}\,\text {(syst.)} \pm 0.10\,\text {(global)},\) indicating the suppression of \(\Upsilon\) due to CNM effects. The shaded area in the figure represents the calculation from the CEM using the EPS09 nuclear parton distribution function [74]. It predicts enhancement at \(y \sim 0\), mainly due to the anti-shadowing effect. The long dashed line depicts the calculations including parton energy loss only [162]. It more closely fits the data. The dashed line shows the calculations including both EPS09 nPDF and parton energy loss [162]; these are closer to the calculations using EPS09 alone in mid-rapidity and those using parton energy loss alone in forward rapidity. The data are systematically lower than the calculations, particularly in the mid-rapidity. This suggests that other CNM effects besides the nPDF effect are needed to describe the data.

Fig. 27
figure 27

(Color online) \(\Upsilon\) suppression as a function of rapidity for \(p(d)+\hbox {Au}\) collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV, compared to theoretical calculations. The figure is taken from [160]

5.2 \(\Upsilon\) production in Au + Au collisions

\(\Upsilon\) measurements at the RHIC are very challenging due to the small production cross section. The STAR Collaboration collected numerous data samples in 2014 and 2016, to study \(\Upsilon\) for Au + Au collisions; they used a di-muon trigger by employing the MTD detector installed in early 2014. Thanks to the large statistics and good momentum resolution for Au + Au collisions, the separation of \(\Upsilon (1S)\) and \(\Upsilon (2S+3S)\) from the invariant mass spectrum of the di-muon is possible. Figure 28 shows \(R_{\mathrm {AA}}\) as a function of \(N_{\text {part}}\) for \(\Upsilon (1S)\) and \(\Upsilon (2S+3S)\) for Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV. The suppression of \(\Upsilon (1S+2S+3S)\) in \(p+\hbox {Au}\) collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV is also shown for comparison. The suppression for both \(\Upsilon (1S)\) and \(\Upsilon (2S+3S)\) increases toward central collisions. In central collisions, a significant suppression of both \(\Upsilon (1S)\) and \(\Upsilon (2S+3S)\) is observed. The observed suppression for inclusive \(\Upsilon (1S)\) could be due to the suppression of the feeddown contribution of higher bottomonium states. The fraction of direct \(\Upsilon (1S)\) in the inclusive \(\Upsilon (1S)\) is estimated to be \((71 \pm 5)\%\) at low \(p_{\text {T}}\) and \((45.5 \pm 8.5)\%\) at high \(p_{\mathrm{T}}\) for \(p+p\) collisions [85]. With current precision, it is unclear whether the direct \(\Upsilon (1S)\) is suppressed. The suppression for \(\Upsilon (2S+3S)\) is found to be larger than that for \(\Upsilon (1S)\) in (semi-)central collisions, supporting the “sequential” suppression picture.

Fig. 28
figure 28

(Color online) \(R_{\mathrm {AA}}\) as a function of \(N_{\text {part}}\) for \(\Upsilon (1S)\) and \(\Upsilon (2S+3S)\) for Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV. The figure is taken from [163]

5.3 Comparison between RHIC and LHC

The results from the RHIC are compared to the results from the LHC for Pb + Pb collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 2.76 TeV, as measured by the CMS Collaboration [164]. Both measurements were performed in mid-rapidity through the di-muon channel.

The upper panel of Fig. 29 shows \(\Upsilon (1S)\). The suppression of \(\Upsilon (1S)\) at the RHIC and CMS is similar from peripheral to central heavy-ion collisions, although the center-of-mass energies differ by one order of magnitude. It is plausible that the inclusive \(\Upsilon (1S)\) suppression arises mainly from the CNM effects and the suppression of the feeddown from excited bottomonium states, while the direct \(\Upsilon (1S)\) remains unaffected by the deconfined medium in both the RHIC and LHC.

Fig. 29
figure 29

(Color online) \(R_{\mathrm {AA}}\) as a function of \(N_{\text {part}}\) for \(\Upsilon (2S+3S)\) in Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV and for \(\Upsilon (2S)\) and \(\Upsilon (3S)\) in Pb + Pb collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 2.76 TeV. The figure is taken from [163]

The lower panel of Fig. 29 compares \(R_{\mathrm {AA}}\) as a function of \(N_{\text {part}}\) for \(\Upsilon (2S+3S)\) in Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV and for \(\Upsilon (2S)\) and \(\Upsilon (3S)\) in Pb + Pb collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 2.76 TeV. The \(\Upsilon (2S+3S)\) appears less suppressed at the RHIC than at the LHC, particularly in peripheral collisions. This could be due to the different temperature profiles of the medium produced in the heavy-ion collisions at the RHIC and LHC. It is predicted that the initial temperature is higher in central collisions and at higher center-of-mass energies. If the initial temperatures achieved in the heavy-ion collisions at the LHC and RHIC both well exceed the dissociation temperature of \(\Upsilon (2S)\) and \(\Upsilon (3S)\), no significant difference is expected at the RHIC and LHC. However, if the initial temperature is close to the dissociation temperature of \(\Upsilon (2S)\), the suppression of inclusive \(\Upsilon (2S)\) will be sensitive to the temperature profile of the medium, and this could result in differing behaviors between the RHIC and LHC.

5.4 Comparison between experiment and theory

For better understanding of the \(\Upsilon\) production and the temperature constraints of the medium produced in heavy ion collision at the RHIC, the \(\Upsilon\) suppression data are compared to two theoretical calculations. In the TAMU transport model (Rapp) [159], the QGP melting and (re)combination of the \(\Upsilon\) mesons are controlled by a kinetic-rate equation. The binding energies in the medium are predicted by thermodynamic microscopic T-matrix calculations, using the internal energy from lattice QCD as the potential. The space-time evolution of the fireball is dictated by a lattice-QCD-based equation of state. The initial temperature of the fireball is approximately 310 MeV in the most central Au + Au collisions. CNM effects are also considered in this calculation. The model by Rothkopf and his collaborators [165] uses a lattice QCD-vetted, complex-valued, heavy-quark potential coupled with a QGP background following an anisotropic hydrodynamic evolution. The initial temperature is set at approximately 440 MeV in the most central collisions. No (re)combination or CNM effects are included in the Rothkopf calculations.

Figure 30 compares the STAR measurements for \(\Upsilon (1S)\) and \(\Upsilon (2S+3S)\), as well as the corresponding theoretical calculations from the two models mentioned above. Both model calculations are consistent with the data for the ground and excited \(\Upsilon\) states, within experimental and theoretical uncertainties. To extract the temperature achieved in the heavy-ion collisions, the precision of the data and the theoretical calculations must be improved via a systematic study of quarkonium suppression.

Fig. 30
figure 30

(Color online) \(J/\psi\)\(v_2\) as a function of \(p_{\text {T}}\) for 0–80% Au + Au collisions at \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV, compared to the results for \(\phi\) and charged hadrons, as well as theoretical calculations. The figure is taken from [134]

6 Summary

This paper presents a review of recent experimental measurements of open heavy flavors and quarkonia production at the RHIC. Heavy quarks, owing to their large masses, are expected to behave differently to light flavors when interacting with the nuclear matter created in high-energy heavy-ion collisions; this includes differences in the production mechanisms, hadronization, thermalization, and interactions with the medium. By taking advantage of the developments in silicon vertex detector technology, precise measurements have been achieved and will provide better constraints to theoretical calculations. In the following, we summarize several key points in open heavy flavor and quarkonium production.

Open heavy flavor production Open charm hadrons’ \(p_{\mathrm{T}}\) spectra—including those of \(D^0\), \(D_s\), and \(\Lambda _c\)—in various centrality bins at mid-rapidity \(|y|<1\) for \(\sqrt{s_{_\mathrm {NN}}}\)= 200 GeV Au + Au collisions, are presented. The thermal parameters \(T_{\mathrm{eff}}\) and \(T_{\mathrm{kin}}\), as well as the radial flow velocity extracted from \(D^0\)\(p_{\mathrm{T}}\) spectra, show that \(D^0\) freezes out earlier than light hadrons. The strong suppression of the \(D^0\) nuclear modification factor \(R_\mathrm{AA}\) at high \(p_{\mathrm{T}}\) and the large elliptic flow \(v_2\) exhibit similar behaviors to light-flavor hadrons, indicating strong interactions between charm quarks and the medium; furthermore, charm quarks may be thermalized as light quarks. The enhancement of the \(\Lambda _c/D^0\) and \(D_{s}/D^{0}\) ratios in Au + Au collisions, compared to those in \(p+p\) collisions, provides important data for understanding the charm quark hadronization mechanisms. The comparison to various models incorporating charm-quark coalescence suggests that coalescence mechanisms play an important role in charm-quark hadronization in the presence of QGP.

Open bottom hadrons were indirectly measured via their decay products \(B{\rightarrow }J/\psi\), \(B{\rightarrow }D^0\), and \(b{\rightarrow }e\) by the STAR experiment in mid-rapidity for \(\sqrt{s_{\mathrm{NN}}} =\) 200 GeV Au + Au collisions. The suppression observed for non-prompt \(J/\psi\) and non-prompt \(D^0\)/e at high \(p_{\mathrm{T}}\) indicates bottom-medium interactions and bottom energy losses. The smaller suppression of \(b{\rightarrow }e\) compared with \(c{\rightarrow }e\), and the lack of suppression of \(B{\rightarrow }D\) at 4 GeV/c, indicates the smaller energy loss of the bottom quark as a result of its extremely large mass; this is consistent with flavor-dependent parton energy loss mechanisms. Nonzero \(b\)\(v_2\) was observed at \(p_{\mathrm{T}} > 3\) GeV/c. The smaller \(b\)\(v_2\) compared with \(c\)\(v_2\) and the NCQ scaling hypothesis indicate that bottom quark is unlikely thermalized and struggles to participate in the light flavor partonic collectivity at RHIC energy.

Quarkonium production The production of \(J/\psi\) in \(p+p\) and heavy-ion collisions is being intensively studied at the RHIC. In \(p+p\) collisions, the measured \(p_{\text {T}}\) spectra and polarization (spin alignment) are found to be consistent with the theoretical calculations from the (I)CEM and NRQCD. The ICEM and NRQCD calculations using different LDMEs predict different polarizations for \(J/\psi\); however, current measurements are unable to identify the difference. The measurement of \(J/\psi\) polarization with higher statistics will be helpful for constraint models or LDMEs in NRQCD.

The collision energy, collision size, and \(p_{\text {T}}\) dependence of \(J/\psi\) production in heavy-ion collisions are measured at the RHIC and compared to results from the SPS and LHC; at low \(p_{\mathrm{T}}\), they concur with the picture that QGP melting, (re)combination, and CNM effects play important roles. Their relative contributions vary with the collision energy, collision centrality, system size, and kinematic variables of \(J/\psi\). Using a transport model calculation, the low-\(p_{\text {T}}\)\(J/\psi\) suppression in central Au + Au collisions is found to be approximately 0.6, and QGP melting further suppresses it to 0.2; however, (re)combination increases it back to approximately 0.4. At high \(p_{\mathrm{T}}\), where the contributions from CNM effects and (re)combination are negligible, significant suppression is observed; this provides strong evidence of QGP melting at the RHIC.

The collectivity of \(J/\psi\) has been studied in Au + Au collisions via radial and elliptic flows. The results disfavor the scenario that at the RHIC, \(J/\psi\) is predominantly produced via the (re)combination of thermalized charm quarks.

\(\Upsilon\), a cleaner probe of QGP melting, is found to be significantly suppressed in central Au + Au collisions. The sequential suppression (stronger suppressions of \(\Upsilon (3S)\) and \(\Upsilon (2S)\) than of \(\Upsilon (1S)\)) observed at the LHC is confirmed at the RHIC, which provides further strong evidence of QGP melting. The comparisons of the RHIC and LHC data, as well as the theoretical calculations, are useful for extracting the properties of QGP.

In around 2023, sPHENIX will begin high-luminosity runs with a high-speed silicon vertex detector, which is based on a state-of-the-art monolithic active pixel sensor technology. The quantity of statistics collected will be increased 100-fold; this will facilitate dedicated bottom measurements via hadronic decay channels, including precision measurements of nuclear modification factors and flows for B-mesons and b-tagged jets [166].

Using the enhanced statistics of the Zr+Zr and Ru+Ru (3B events for each collision system) data collected in 2018, the \(J/\psi\)\(R_{\mathrm {AA}}\) and elliptic flow will be measured with good precision, deepening our understanding of the interplay of QGP melting, (re)combination, and CNM effects on \(J/\psi\) production. The Z dependence of \(J/\psi\) photoproduction can also be studied by using the isobaric collision data and by comparison with Au + Au collisions. The Cu + Au collision data taken by STAR in 2012 have recently been fully released, and \(J/\psi\) production measurements via the di-electron decay channel will soon be possible. The STAR forward upgrade program—including a finished inner time projection chamber, endcap time-of-flight upgrades, and ongoing forward tracking system and forward calorimeter system upgrades—will extend the rapidity coverage of quarkonium measurements for STAR up to \(y=4\); this will facilitate many unique physics opportunities using quarkonium in \(p+p\) , \(p(d)+\hbox {A}\), and A + A collisions at very forward rapidities. With sPHENIX conducting high-luminosity A + A runs, the precision of \(\Upsilon\) measurements is expected to be significantly improved.

These facility upgrades will further our understanding of the interactions of heavy flavors and quarkonia with the hot–dense medium created in heavy-ion collisions at the RHIC.