Introduction

To curb the spread of COVID-19, nations responded by implementing various restrictions including mandatory lockdown, masking in public areas, social distancing, closure of non-essential businesses, and mandatory quarantine for infected or exposed individuals. As a result of the pandemic and virus containment measures, adults experienced a wide array of mental health concerns such as loneliness, stress [36], depression, and anxiety [39]. While pandemic-related literature supports that many adults experienced negative mental health outcomes during this time, parents/caregivers had higher stress during the first year of the pandemic compared to those who were not parents [1]. Caregivers had to adjust to the demands of the pandemic while balancing work-related stressors, such as working from home, coping with potential income loss, whilst responding to increased childcare demands [43].

Caregivers of school-aged children had reduced time to attune to their own needs as they were also responsible for assisting their children with remote educational tasks [17]. This vulnerability is further exacerbated for single parent households due to increased economic precarity along with decreased opportunities to offset caregiving demands [42]. Concurrently, for some families, caregivers reported increased distress during the pandemic [72]. This is of concern in the context of child mental health as caregiver distress impacts caregivers’ ability to effectively care for their children and attune to their needs [65]. As a result, caregiver distress can negatively impact childhood development [71], and contribute to internalizing symptoms [35] as well as behavioural concerns [45].

Young children are especially vulnerable to global disasters such as the COVID-19 pandemic (hereafter referred to as pandemic) because of their lack of understanding of disaster-related information, immature self-protection and coping skills, and their reliance on caregivers for physical care and emotional support [33]. When examining mental health outcomes during the pandemic, children experienced depression, anxiety, stress, clinginess, irritability, hyperactivity, and disturbed sleep [8, 9, 34, 52]. These findings are concerning, given that access to child mental health services declined [50, 70] and opportunities for children to connect with extended family and peers were reduced all while there was an increase in familial, parental, and child stress levels during the pandemic [30, 44, 46]. Previous research has investigated the impacts of COVID-19 on samples of young children and their families, however there is a need to contribute empirical research in samples of clinically referred young children. It is imperative that we evaluate service access trends and the experiences of young children and their families receiving mental health treatment during the pandemic to identify potential areas for support and intervention.

Current Study

This study aimed to address current limitations in the literature and utilized a comprehensive assessment tool to investigate the influence of the pandemic on clinically referred children between 12 and 47 months of age in the Province of Ontario, Canada. The comprehensive assessment tool utilized in this study is grounded in the biopsychosocial approach [15] to support clinical decision-making and reasoning [63]. This integrative framework combines the World Health Organization’s International Classification of Function, Disability and Health (WHO-ICF) model with important concepts from interRAI (www.interRAI.org) and Elven and colleagues [14]. The child is placed at the centre of the model and in alignment with a biopsychosocial approach, the biological, psychological, and social factors surround the child, interacting and influencing the child’s wellbeing [31]. Surrounding these influences are various factors related to service access and use (e.g., screening, assessment, care planning). The clinical reasoning model demonstrates the interactive nature of biological, psychological, and sociocultural contexts and how they can be monitored with an assessment-to-intervention process.

In alignment with the clinical reasoning model, and similar to previous findings with older children and youth aged 4–18-years, it was anticipated that the number of clinical assessments would be lower during the pandemic, attributed to COVID-19 contagion fears [66], and a lack of access to mental health services and agencies [27]. With respect to parenting issues, it was expected that there would be higher caregiver distress in caregivers during the pandemic, compared to caregivers in the pre-pandemic period. Due to the stressful environment of the pandemic, reduced emotional capacity, and lack of external resources [1, 17, 43] it was also expected that parents assessed during the pandemic would report fewer strengths compared to parents assessed prior to its start. Lack of socialization, social support, disruptions to daily life, and parental stress were expected to detrimentally impact child adjustment [34, 52, 72]. Hence, compared to pre-pandemic levels, it was expected that young children would experience greater issues related to distractibility/inattention, emotional dysregulation, and behavioural issues during the pandemic. Findings were anticipated to provide valuable information about the multifaceted impact that the pandemic had on child mental health outcomes and underscore the need for increased child mental health support.

Method

Sample

Data were obtained from assessments of children 12 to 47 months who were referred to child mental health and developmental services agencies which provide early intervention services in the Province of Ontario. The clinical assessments were collected from responses on the interRAI Early Years assessment, completed by clinicians from participating mental health service agencies across Ontario [59]. Completed interRAI Early Years assessments received a randomly generated case record number to de-identify the records. Data were accessed on a protected data repository at a participating university. The study was approved by the University of Western Ontario’s Research Ethics Board (REBs #106415 and #108024).

A total of 1343 assessments from 1174 unique children were captured across 11 agencies in Ontario and were included in the sample. Children were referred for an assessment through clinic intake processes by teachers, parents, clinicians, and other healthcare service providers. The average age of these children was 32 months (SD = 9.3), with 68.8% of the sample identified as male sex (n = 924), and 31.2% identified as female sex (n = 419).

Measures

interRAI Early Years Assessment

interRAI (www.interRAI.org) is a not-for-profit collective consisting of researchers, clinicians, and policy experts from over 35 countries who collaborate to create and review evidence-based clinical care assessments. The interRAI Early Years is a comprehensive assessment tool that underwent a multi-step peer review process [66] and multi-year piloting. The assessment is used for children 0 to 47 months with developmental, social, behavioural, or emotional concerns, and consists of approximately 400 items divided into 19 subsections. Items are used to evaluate children’s functioning, mental and physical health, social support, and access to services. Information is collected on the Early Years through a clinician-rated semi-structured interview, taking approximately 45–90 min depending on case complexity [26]. A detailed manual supports this assessment and is accompanied with coding rules for the items. As a result of stringent coding rules, assessments within the interRAI suite of instruments demonstrate robust reliability and validity psychometric properties (e.g., [3, 18, 21, 26, 32, 53, 54, 56, 58, 60, 61, 65]. Currently, it is used in mental health and developmental service agencies across Ontario which provide early intervention for young children [26]. Assessment information is used to inform care planning, outcome measurement, service urgency, as well as resource allocation.

Early Years Applications

Caregiver Distress Algorithm

The Caregiver Distress Algorithm is a built-in Early Years algorithm which includes 17 unique items on the Early Years instrument. Items were chosen for inclusion based on their utility to predict feelings of distress, anger, or depression experienced by the child’s parent or primary caregiver [65]. Sample items include “parent/primary caregiver expresses feelings of distress, anger, or depression”, “parent(s)/primary caregiver has current developmental or mental health issues”. The algorithm was created using decision tree modeling, with the completed decision tree consisting of 17 items, 5 levels (where age of child represents the first level), and 22 terminal nodes [64]. Item responses were coded as ‘yes’ or ‘no’ at each branch level and were split based on those responses until a terminal node was reached. Each terminal node has a corresponding number between 0 and 6, where higher scores indicate higher proportions of caregiver distress. Scores were dichotomized such that a cut point of 3 and higher reflected high levels of caregiver distress based on other interRAI studies [64].

Parenting Strengths Scale

The Parenting Strengths Scale encompasses six items on the Early Years instrument, including items such as communicating effectively with child, assisting child with regulation of emotions or demonstrating warmth and support. All items are scored on a 3-point ordinal scale (0 = most of the time to 2 = rarely or never), with a possible range of 0–12, where higher scores indicate a poorer parent–child relationship. Scores were dichotomized and a cut point of 3 or higher indicated poor parenting strengths. Previous interRAI research has dichotomized the parenting strengths scale for determining service need utilizing the same cut point [61].

Distractibility and Inattention Scale

The Distractibility and Inattention Scale is comprised of five Early Years items. Sample items include “inability to tolerate frustration”, “excessive activity”, and “easily distracted”. Four items are scored on a 5-point ordinal scale (0 = not present to 4 = exhibited daily in last 3 days, 3 or more episodes or continuously). The fifth item refers to the child’s ability to attend is scored as yes or no. Scores range from 0 to 20 with a cut point of 6 or higher indicating high distractibility and inattention [55].

Emotional Dysregulation Scale

The Emotional Dysregulation Scale includes six Early Years items such as “irritability”, “extreme reactivity”, and “difficulty tolerating frustration”. These items are scored on a 5-point ordinal scale (0 = not present to 4 = exhibited daily in last 3 days, 3 or more episodes or continuously). The child’s difficulty adapting to change in routine or environment is scored on a 3-point ordinal scale (0 = adapts without difficulty to 2 = has difficulty adapting to even minor change). Scores range from 0 to 22 where higher scores indicate greater levels of emotional dysregulation. A cut point of 7 or higher represented high emotional dysregulation [55].

Behavioural Issues Scale

The Behavioural Issues Scale is composed of four Early Years items, examining constructs such as “physical aggression” and “outbursts of anger”. All items are scored on a 5-point ordinal scale (0 = not present to 4 = exhibited daily in last 3 days, 3 or more episodes or continuously), with scores ranging from 0 to 16, where higher scores indicate more pronounced behavioural issues. Scores were dichotomized using a cut point of 5 or higher indicating greater behavioural issues [55].

Procedure

Trained assessors, including mental health professionals and childcare workers, completed interRAI Early Years assessments during routine clinical practice at child mental health agencies and childcare centres. Assessors were required to complete an educational program for early child intervention, have at least 2 years of work experience with children, and participate in a 2-day Early Years training program [26]. During the semi-structured interview, the assessor utilizes a variety of sources of information to complete the assessment including interviewing the parent(s)/caregiver(s), observing the child, contacting collateral supports (e.g., family physician, therapist etc.). Assessments were generally completed in-person, but depending on client and agency needs, virtual or telephone assessments may have been completed. Supplemental clinical documents such as medical records were incorporated. Responses were then uploaded to the interRAI Canada secure web-based database. Responses were required to be in proper form and entirely complete for the record to be accepted into the system [26], such that no assessments contained missing items.

Clinical assessments were obtained from March 11, 2018–December 11, 2019 (pre-pandemic timeframe) and March 11, 2020–December 11, 2021 (pandemic timeframe) for comparison. These specific timepoints were selected to ensure each month was represented to reduce as much year-to-year variability as possible, while including timepoints that share the same general temporal trends [66]. The aim was to provide a broad snapshot of mental health outcomes.

Analyses

All assessments collected from pre-pandemic and pandemic timepoints were included in the sample, including assessments from individuals who were assessed more than once during these periods. All applicable assessments were included, even if children were assessed more than once, to provide an accurate capture of data collected from various agencies during pre-pandemic and pandemic periods [62, 66]. Demographic characteristics were compared before and during the pandemic by utilizing chi-square and a non-parametric Fisher’s exact test to evaluate if assessment frequencies differed based on specific participant groups. No cell sizes under five were reported. Upon a significant chi-square or Fishers exact test, post hoc pairwise comparisons were performed using the Bonferroni adjustment. Additionally, standardized residuals were calculated for each of these characteristics.

Before Early Years algorithms and scales were analyzed, internal consistency was measured for each of the four scales. To determine reliability, Cronbach’s alpha was utilized for each of the scales of interest. These results determined the utility of the applications for the current sample. When the alpha threshold of 0.70 was achieved [10], scale and algorithm analyses were conducted whereby chi-square test of independence analyses were performed amongst the applications to evaluate differences in mental health outcomes for children under four before and during the pandemic. For the Early Years applications, distinct score categories were created by collapsing scores into two groups, low and high risk. Previous interRAI studies have dichotomized scores on the scales and algorithms by designating cut-points based on accepted interRAI standards and previous publications to aid with consistency across studies [25, 26, 56, 57, 61]. Further rationale for dichotomizing variables is to produce findings that are easily interpreted by a wider audience [16]. Data prior to the pandemic served as a baseline for typical mental health outcomes for children, portraying mental health trends across time. SAS 9.4 was used for all analyses.

Results

Table 1 compares assessment-related information and child characteristics across the two selected timepoints of interest. As anticipated, the overall number of child assessments declined between the pre-pandemic and pandemic timepoints. Results indicated that there were no significant differences in the number of assessments completed for sex or age, suggesting that children’s ability to be clinically assessed during the pandemic was not disproportionally impacted by these demographics. Results also suggested that the pandemic did not alter caregivers’ need to offset financial costs by limiting access and provision of basic care essentials (e.g., limiting food to pay rent), and there were no differences observed for children’s enrollment in childcare before and during the pandemic.

Table 1 Demographic characteristics before and during the pandemic
Full size table

The remaining demographic characteristics displayed statistically significant chi-square or Fishers exact tests. Table 2 displays the results of pairwise comparisons that were performed for these demographic characteristics. The number of assessments was lower during the pandemic period (vs. before the pandemic) for children whose primary language was English compared to children whose primary language fell in the ‘other’ category (i.e., neither English or French). Children previously accessing care had fewer assessments in the pandemic period (vs. before) than children first accessing care. Additionally, significantly fewer children who did not live with their parents had assessments compared to those who did live with their parents. Similarly, when making assessment comparisons for legal guardianship, significantly fewer children under the care of Child Protection had assessments during the pandemic than those under the care of both caregivers. Finally, significantly less children with a history of foster placement had assessments during the pandemic than children with no prior involvement with foster services. Overall, the pandemic impacted the number of early years (12 to 47 months) assessments that were collected, with some participant groups experiencing larger declines than others.

Table 2 Residual analysis and pairwise comparisons of demographic characteristics
Full size table

Internal consistency analyses were performed for the items in all four Early Years scales. When evaluating the internal consistency statistics, the Parenting Strengths Scale, Distractibility and Inattention Scale, Emotional Dysregulation Scale, and Behavioural Issues Scale all demonstrated internal consistency for the current study (α = 0.951; α = 0.746; α = 0.789; α = 0.715, respectively). Table 3 outlines the results of the chi-square tests of independence that were performed for the Early Years applications. Results demonstrated that distributions of perceived emotional dysregulation scores differed across time, where there was a higher proportion of scores in the high-risk group during the pandemic than before the pandemic. Results revealed that presentations of caregiver distress, parenting strengths, distractibility/inattention, and behavioural issues did not significantly differ prior to and during the pandemic.

Table 3 Selected assessment characteristics before and during the pandemic
Full size table

Discussion

Data collected from 1174 children, across 11 unique agencies in Ontario indicated that there were fewer clinical assessments completed during the COVID-19 pandemic compared to pre-pandemic assessments. Virus contagion fears limited children’s access for in-person visits as caregivers remained home for the safety of themselves and their children [38], and public messaging encouraged the public to stay home except to gather essentials and to respond to emergent situations [40]. Further, during the pandemic, many mental health agencies transitioned from in-person service delivery to virtual platforms [5], which provided challenges to conduct a clinical assessment remotely with a young child [23, 67, 74]. Results revealed a significantly lower number of assessments were completed with subsequent encounters compared to first encounters, and for children with unique caregiving and housing related presentations.

It is plausible that due to declines in clinical referrals [69], and the reduced availability for clinicians to assess children [38, 50], assessments were difficult to obtain during the pandemic months. Reports and feedback from agencies suggest that COVID-19 impacted their agency as they managed low staffing, sickness, re-deployment, and staff remaining home to care for their own children who could not attend school due to closures. As virtual care and interviews were not immediately available, there was significant training required to transition staff to a virtual platform. Staff were required to follow COVID-19 cleaning/sterilization guidelines which resulted in fewer appointments offered to allow for cleaning time between appointments. Furthermore, research in the health care field suggest that COVID-19 impacted the mental health of staff, subsequently impacting their response to the pandemic and the quality of care [11, 29]. Further, the decreased number of assessments were notable for children who do not live with a parent or caregiver, those with a history of foster placement, and those who were in the care of Child Protection. This may be indicative of the systematic barriers that exist for children under this type of care to be referred for a clinical assessment [49, 73]. Pandemic period literature supports this finding as child welfare-involved children and their families experienced reduced child visitation and socialization opportunities, as well as reduced access and availability of mental health evaluations and treatment planning [7]. The current findings are worrisome as children in the care of child welfare services experience greater mental health concerns than children in the care of primary caregivers [13]. The mental health of these children is of the utmost concern as they may not have been connected to crucial mental health services during the pandemic.

As predicted, presentations of emotional dysregulation were higher in children assessed during the pandemic compared to children assessed before the pandemic. There are several potential explanations for this finding. First, daycares, preschools, parks and playgrounds are spaces where young children can be physically active, discover new stimuli, interact with peers, and adapt to new environments [6, 12]. The closures of these spaces may have impacted children and their ability to emotionally adjust when virus containment measures limited access to activities and people that were a part of their routine [34, 52, 72]. Furthermore, the limited exposure to external mental stimulation and communication opportunities during the pandemic, may have reduced opportunities to engage in meaningful interactions. As a result, it is possible that these limited interactions led to increased emotional dysregulation compared to the pre-pandemic cohort. Furthermore, it is also possible that the higher levels of emotional dysregulation during the pandemic period (vs. before the pandemic) was a result of referral patterns and triaging during the pandemic. Although there is no access to referral or triaging data, referral patterns may have shifted such that only those with more pronounced emotional or behavioural challenges were referred during the pandemic period. It is also possible that children with higher levels of emotional dysregulation were triaged and assessed more than the children with lower levels of emotional dysregulation. Nonetheless, further research is needed to investigate potential explanations for the higher levels of emotional dysregulation present during the pandemic, compared to pre-pandemic levels.

Findings indicated that caregiver distress levels were not significantly different before compared to during the pandemic. It is possible that, within clinical samples, caregiver distress was unchanged during the pandemic due to the implementation of federal financial initiatives (e.g., Canada Emergency Response Benefit, Canada Recovery Caregiving Benefit, mortgage payment deferral) to help offset pandemic-related financial strain for adults, caregivers, and families. These national supports potentially mitigated caregiver distress for this sample as caregivers spent more time at home with their families while still receiving financial support. Concurrently, these caregivers may have had fewer competing demands such as working out of the home whilst taking children to childcare centres, assisting children with schoolwork, and engaging in other caregiving and household related tasks [2, 24]. For some caregivers, virus containment measures such as lockdowns were a break from the fast-paced environment they had prior to the pandemic, and as a result, did not experience increased caregiver stress [24]. It is also possible that unchanged caregiver distress levels are a result of our sample of caregivers being higher resourced financially and/or emotionally and therefore able to pursue an assessment during the pandemic.

In line with the results observed for caregiver distress, parenting strengths did not significantly differ prior to and during the pandemic. In a previous study, the pandemic influenced caregivers of young children differently based on a variety of intersecting factors including employment status, ability to share caregiver demands, financial and housing status, and access to childcaring and personal support resources [24]. Perhaps because of individual differences, when score comparisons were made before and during the pandemic, no differences were noted with respect to parenting strengths from before to during the pandemic. It is of importance to evaluate changes in caregiver distress and parenting strengths as a facet of personal support access, as well as financial and housing statuses to further clarify this relationship.

Clinical Implications

This study provides a preliminary understanding of how clinically referred young children were impacted by the COVID-19 pandemic. This is one of the first studies to examine the mental health of young (12–47 months), clinically referred children during the pandemic. One of the main takeaways from this study is the differences in assessment completion patterns before the pandemic regarding key sociodemographic factors. Key sociodemographic factors impacting service access included children/families who spoke a language other than English, living without caregivers, having legal guardianship assigned to Child Protection services, and having a history of foster placement. Therefore, it is imperative that assessment completion disparities are reduced for these individuals. It is recommended that high risk areas are targeted, assessed, and flagged either in physicians’ offices or early years programs. Facilitating home visiting programs for infants with parents who are marginalized, low-income, and/or less resourced are also suggested. Furthermore, it is important to improve access to assessments for newcomer families, by implementing screening or assessment in newcomer centres or improving access through schools, community centres, or medical offices.

Emotional dysregulation was identified as an area where young children may require more tailored intervention and support, especially during times of worldwide crisis. It is important that clinicians continue utilizing integrated, standardized, needs-based assessments such as the interRAI Early Years assessment to identify at risk children and tailor interventions appropriately and based on need. The parent/caregiver and their relationship with their child is especially important in the early years [4]. As a result, clinicians are encouraged to support the implementation of tailored interventions for children with emotion dysregulation concerns such as attachment-based treatment with parents/caregivers (e.g., Circle of Security [22]). Parents and caregivers are integral to assisting their child with developing emotion regulation skills [47]. Although parenting strengths did not differ in this study, programming that is focused on strengthening the parent–child relationship and the attachment relationship is imperative to enhancing children’s emotional regulation skills.

For these young children and their families, results provide some optimism as caregiver distress, parenting strengths, distractibility/inattention, and behaviour did not significantly differ during the pandemic for these already vulnerable children. Several of these environmental and psychological variables showed insignificant differences prior to compared to during the pandemic. The clinical reasoning model, of which this assessment approach was grounded in, posits that various contextual and situational factors contribute to a child’s wellbeing. Since these variables showed similar findings from before to during the pandemic, it is imperative that this type of model continues to be studied and applied to aid with understanding the mental health and wellbeing of young children. Clinicians are encouraged to continue monitoring these unique and integrated assessment systems that examine needs across development to support continuity of care. High quality assessment-to-intervention systems that examine the interactive nature of biological, psychological, and sociocultural contexts to understanding the mental health and wellbeing of young children are essential.

Limitations and Future Directions

The assessments captured and included in this study consist only of clinically referred children who were assessed by clinicians in Ontario agencies who use the interRAI suite of instruments. Since our sample utilized a sample of clinically referred children, generalizability of the results to community samples is not known. Furthermore, mental health agency referral patterns were not available, and therefore comparisons between the number of referrals made and the number of assessments completed cannot be evaluated. This comparison would provide more context for the lower number of assessments completed during the pandemic period. With respect to the format of assessments, it is unknown how many assessments were done virtually, via telephone, or in-person, and as a result, this is a limitation of the study. Further, because empirically supported clinical assessments provided data for this study, there were no items specifically tailored to inquire about the impact of COVID-19 on service access. Future research, comparing mental health information from service-seeking and non-service-seeking children and their families would be of benefit. Furthermore, information on how the pandemic directly impacted young children and their families can enhance the current study to shed light on the type of services and programming that would be the most helpful to triage, support, and provide treatment plans for children in the midst of social distancing, quarantine, and parental stress.

Although significant differences for various demographic characteristics were observed across the two timepoints of interest, and children’s emotional dysregulation increased, fewer assessments occurred during the pandemic period. This potentially impacted the ability to detect differences, resulting in an inherent challenge to distinguish a null effect from a very small effect [20]. Next, due to the timeline of the current study and its exploratory nature, only assessments up to December 11, 2021, were included in analyses. A longitudinal study including data up to the present can evaluate children’s long-term adjustment and resilience capacity as Ontario transitions out of a pandemic period of social distancing, masking, and long duration quarantining to a period that emulates life prior to the pandemic. Finally, although previous interRAI instruments completed either via telephone or in-person produced similar data [48], there is an opportunity to evaluate in person compared to other interview modes (e.g., virtual, telephone) using interRAI Early Years assessments specifically. However, it is of importance to mention that there are potential limitations with collecting assessment information virtually or via telephone such that families may be less likely to disclose marital difficulties or domestic violence if they lack privacy in their home. Other challenges associated with alternative interview modes may include hearing impairment, language barriers, and difficulties building rapport.

Summary

The current study is among the first to evaluate how clinically referred children between 12 and 47 months were impacted by the pandemic. Overall, the number of clinical assessments obtained from these children reduced by almost half between the pre-pandemic and the pandemic period. Various demographic characteristics were related to significant assessment declines including having a primary language other than English, obtaining a subsequent assessment, living with individuals other than caregivers, having legal guardianship assigned to Child Protection services, and having a history of foster placement. A higher proportion of young children assessed during the pandemic had elevated emotional dysregulation compared to the proportion of assessed pre-pandemic. However, there were no significant differences when parental outcomes and when children’s distractibility/inattention and behaviour were compared before and during the pandemic. These findings emphasize the importance of maintaining access to child mental health agencies during a global crisis so that children’s wellbeing can be monitored. Additionally, this study highlights the need to balance the implementation of local public health guidelines while also finding opportunities to foster child socialization opportunities with extended family members, other meaningful adults, and same-age peers.