Introduction

Oral health is an essential component of daily functioning and well-being. Healthy People 2020 [1] emphasizes the importance of prevention and control of oral and craniofacial diseases, conditions, and injuries, and of enhancing access to preventive services and dental care. Children and adolescents are in the transitional stage for development and often have diets with high levels of sugar in United States and globally [2,3,4], making them vulnerable to dental problems and oral diseases [5].

It is challenging to develop oral health measures for cost effectively evaluating large populations for their current oral health status and the treatment needs, especially for children and adolescents. One of the most commonly used oral health surveys is the child version of Oral Impact on Daily Performances (Child-OIDP) [6,7,8,9], based on social dental approach for the consequences of oral health diseases [10]. The Child-OIDP measures the impact of oral health on children’s life in a Europe population, and it was highly concentrated on the impact of “eating, smiling and speaking” and the impact of the oral health condition on quality of life [10,11,12]. A number of other questionnaires for measuring oral health-related quality of life (OHRQoL) of children have been developed [13], but none of them provide treatment referral recommendations. For example, the Child Perceptions Questionnaire (CPQ) [14,15,16] mainly links the frequency of events in the past 3 months to oral health conditions. The Child Oral Health Impact Profile (COHIP) [17, 18] centers on measuring social impact of oral disorders. The Early Child Oral Health Impact Scale (ECOHIS) [19] primarily focuses on the negative influence of dental disease and treatment experience in very young children. Other surveys, such as the Scale of Oral Health Outcomes for 5-year-old children (SOHO-5) [20, 21], the Michigan Oral Health-Related Quality of Life scale (MOHRQoL) [22], the PedsQL Oral Health Scale [23, 24], and the Pediatric Oral Health-Related Quality of Life Measure (POQL) [25], do not identify the levels of treatment need for children. In short, these existing surveys above were created without comprehensive psychometric analysis. Surveys that can measure the current oral health status and estimate treatment needs, and can be used to evaluate the large population efficiently and effectively, are in need for children and adolescents. But these oral health surveys should be developed through rigorous psychometric analysis encompassing global health, physical, mental, and social health components to establish their reliability and validity.

High-quality patient-reported measures are essential for patient-centered care [26,27,28]. The Patient-Reported Outcome Measurement Information System (PROMIS), a United States National Institutes of Health roadmap initiative, began in 2004 to create self-reported health measures using state-of-the-science qualitative and quantitative methods [29, 30]. The PROMIS framework conceptualizes health as consisting of physical, mental, and social components [29, 31, 32], following the well-recognized World Health Organization model [29, 33]. Each component is then classified into subcomponents, such as function and symptoms in physical health; affect, behavior, and cognition in mental health; and relationships and functions in social health [29, 32, 34]. The subcomponents are further expanded into domains and sub-domains with respect to a particular concept [32]. These domains are intended to be mutually exclusive and collectively exhaustive.

We have developed self-report items assessing oral health by children and adolescents using the PROMIS framework and methodology [35, 36]. Specifically, we employed four operational phases to develop the oral health items [35, 36]: (i) systematic review of the literature to identify instruments and items associated with oral health; (ii) focus groups with racially/ethnically and geographically diverse children, parents, and oral health professionals to help conceptualize oral health [35, 36]; (iii) draft oral health items administered in a series of face-to-face cognitive interviews to ensure that subjects understand the intent of each question; and (iv) field test to evaluate the resulting oral health items.

The aim for this paper is to develop a static fixed-length short-form that measures current oral health status and provides recommendations for different levels of treatment needs, which could eventually serve as a screening tool for efficiently and effectively assessing oral health of children and adolescents. This paper describes the calibration of this static oral health short-form that can be used to provide a reduced list of survey questions along with a scoring table, and to enable score mapping to clinical categories (e.g., referral treatment categories) guided by our oral health conceptual model (see Fig. 1).

Fig. 1
figure 1

The domain structure of child oral health conceptual model based on PROMIS framework

Full size image

Methods

Conceptual model

Guided by the PROMIS framework, the oral health conceptual model (Fig. 1) was developed by our expert panel of pediatric dentists, general dentists, social scientists, and PROMIS experts. The conceptual model is divided into three main components: physical, mental, and social health. Each component is further extended to subcomponent, domain, and sub-domain. The colors indicate different levels of the structure, from left to right: component (orange), subcomponent (green), domain (purple), and sub-domain (blue). The gray colored domains and sub-domains were not included in the analysis because they indirectly measure oral health status.

Dental examination and referral recommendations

Data were obtained from (1) dental exams performed by calibrated dentists [35] (i.e., dentists who use the same standards in the exams) and (2) survey questions answered by children (age 8–17) during field testing in dental clinics located in Los Angeles County from August 2015 to October 2016. Details of the study, such as the development of oral health items and the study flowchart, are described elsewhere [35, 36]. Patients (children age 8–17) without orthodontic appliances were recruited with their parents or guardians from participating dental clinics located in Los Angeles County for on-site dental exams and administration of survey questions. Only one child was chosen from each family to form a stratified systematic convenient sample, which aimed at reflecting the race/ethnicity distribution of the general U.S. population [37].

The clinical exams were performed and surveys were administered in the same clinic setting. The data collected at dental clinics in the field test were used to evaluate and calibrate the oral health items by IRT modeling, and to refine the item pool. The oral health items were administered to children and parents by computer using Questionnaire Development System™ (QDS™), developed by Nova Research Company [38]. Children and their parents completed the surveys separately at the clinics. In this paper, we only focused on the oral health items for children.

Using professional dental chairs, the tooth-based dental exam was performed by two experienced dentists who were part of the research team. The dental exam yielded the clinical outcome measures, including Children’s Oral health Status Index (COHSI) and treatment referral recommendations (RRs). COHSI assesses clinical oral health and reflects diseases (missing teeth, decay, and abnormal position) and occlusal status, estimating overall oral health status. The COHSI has a best possible score of 100 and decrements from that are estimated by multiplying previously derived regression coefficients by number of missing teeth (− 2.27 for primary anterior teeth, − 4.55 for primary posterior teeth and permanent teeth), number of decayed teeth (− 1.12 for primary anterior teeth, − 2.24 for primary posterior teeth and permanent teeth), occlusion status (− 4.38), and abnormal positions (− 1.73) (see [32] for details). The worst possible COHSI score is − 27.4 for adolescents with all permanent teeth missing and 18.16 for younger children with all primary teeth missing. The regression coefficients, which are derived from a paired preference approach [39], are higher for posterior teeth and permanent teeth that are decayed or missing. The observed range of the COHSI in the field test was 59.18–100. The RR is based on cavities and gingival bleeding and has four categories: see a dentist (i) immediately, (ii) within 2 weeks, (iii) at earliest convenience, or (iv) with regular schedule [40, 41].

RRs employ the 4-level guidelines used in the National Health and Nutrition Examination Survey (NHANES) [42, 43]. Because the NHANES 2009–2014 survey revealed few emergency dental conditions among those who had a dental visit within the past year, we revised the referral conditions to include urgent as well as emergency conditions. In addition to NHANES guidelines, the following criteria were used for RRs: level 1 emergency dental condition—if any permanent tooth has decay with cavitation or more than three primary teeth have decay, see dentist immediately; level 2 urgent condition—if a child has 1–2 primary teeth decayed, or more than half of teeth (12+) have bleeding on probing, or during mixed dentition, a primary tooth is missing between two abnormally positioned permanent teeth, see a dentist within 2 weeks; level 3 earliest convenience—any evidence of malocclusion, space loss, cross bites or crowding; level 4 continue regular care—applies when none of the above conditions exist.

Statistical analyses

We constructed 88 questions to evaluate oral health based on existing (n = 187) and newly written (n = 9) items from focus group [36]. If a response option was selected by fewer than three field test participants, we collapsed the category with the adjacent lower level option for scoring. Item skewness was examined as one indicator of poor fit of the item to the sample. Highly skewed items (skewness<− 7.25) were excluded [44]. We examined product–moment correlations of the remaining 88 items with the COHSI and RR and found 25 items that were significantly positively correlated or had high correlation (r > 0.20) with one or both of the clinical outcomes. To ensure full coverage of the oral health conceptual model, we selected ten other items (by criterion of relative high correlation within a given domain) for domains that had no items strongly correlating with the clinical outcomes, resulting in 35 items:11 items correlated with the COHSI only, 10 items with the RR only, and 14 items with both.

Exploratory factor analysis (EFA) was performed on the 35 items, and 6 items with standardized loadings < 0.30 were then excluded, resulting in 29 items. Then, two confirmatory factor analysis (CFA) models were fitted to evaluate the unidimensionality of the (a) 19 items that correlated substantially with the COHSI; and (b) the 22 items that correlated with the RR. Nine items were included in both factors. While these two sets of items were hypothesized to assess a common higher-order factor, it is possible for the “COHSI” score to be low due to occlusal conditions, such as overbite, crowded, and spacing, but no need to see a dentist urgently—that is, the child could wait until the occlusal issues are ready for future orthodontic intervention. The practical fit of the CFA models was evaluated using the comparative fit index (CFI) (> 0.90), the root mean square error of approximation (RMSEA) (< 0.06), and standardized root mean square residual (SRMR) (< 0.08) [45,46,47,48]. Local independence was evaluated by looking for non-trivial residual correlations (0.20 or above).

Samejima’s graded response model was used to estimate item discrimination and difficulty parameters for the 19 items in the “COHSI” scale and the 22 items in the “RR” scale [49,50,51,52]. The monotonicity assumption of item response theory (IRT) was evaluated by checking item characteristics curves (ICCs) to see if the probability of choosing response options representing more positive oral health increases with better underlying oral health. Finally, we evaluate differential item functioning (DIF) [53, 54] regarding age (8–12 vs. 13–17) and gender and their impact, if any, on IRT modeling using ordinal logistic regression (OLR) and nested OLR with pseudo R2 [55].

We identified short-form versions of the 19-item “COHSI” and 22-item “RR” scales based on item parameters and inputs from content experts. We gave preference to items with higher discrimination and items with a range of difficulties. We also wanted the subset of items selected to represent a wide range of domains in the conceptual model. The extent to which the short-forms capture the information in the full-length scales was assessed by estimating intra-class correlations between estimated scores and comparing reliability (information) of measurement [50].

The readability of long- and short-form items was evaluated by the Flesch–Kincaid (F–K) grade level and flesch reading ease (FRE) scores using Microsoft Word Software [56, 57]. The F–K and FRE are based on average number of syllables per word and number of words per sentence [58].

All items were transformed linearly to a 0–100 possible range with a higher score representing better oral health [50, 59]. The COSHI and RR scales were scored on a t score metric with a mean of 50 and standard deviation of 10 in the field test sample.

SAS, STATA, and Mplus software [60,61,62] were used to calibrate the items and generate comparison plots.

Results

The sample included 334 individuals recruited from August 2015 to October 2016 at 12 dental clinics and practices in Greater Los Angeles, including general clinics and pediatric clinics. Participant characteristics are shown in Table 1. The overall mean COHSI was 88.97 with a standard deviation 8.8 (COHSI ranged from 59.18 to 100). There were 24 (7%) children who were classified as needing immediate care, while 52% children were classified to continue their regular routine care. The age groups and gender groups were balanced with 58% age 8–12 and 52% males. The sample had 42% Hispanic and 8% multiracial children.

Table 1 Characteristics of the children and adolescents in the field test
Full size table

The CFI, SRMR, and RMSEA were 0.94, 0.04, and 0.06, respectively, for the 15-item COHSI scale. These fit indices were 0.90, 0.04, and 0.08, respectively, for the 22-item RR scale. The residual correlations range from − 0.55 to 0.37 and − 0.69 to 0.52 for COHSI and RR, respectively. There are 26 pairs with absolute residual correlation larger than 0.2 for COHSI and 33 pairs for RR.

Item parameters from the GRM are shown in Tables 2 and 3. Discrimination (slope) parameters ranged from 1.47 to 2.62 for the COHSI scale and from 0.00 to 3.18 for the RR scale. Item characteristic curves provided support for the monotonicity assumption.

Table 2 Item calibration statistics for Children’s Oral Health Status Index
Full size table
Table 3 Item calibration statistics for referral
Full size table

Ordinal logistic regression (OLR) on the IRT-calibrated latent trait, oral health items selected in long form, with and without age group (8–12 vs. 13–17), or gender (male vs. female) were performed to check the no DIF assumption [55]. Items with DIF between age group 8–12 and 13–17 are presented in Tables 2 and 3.

The eight items for COHSI and seven items for RR selected for the short-forms are bolded in Tables 2 and 3 and shown in Table 4. In Table 4, the items are divided into two parts showing the measures of COHSI and RRs separately or together. Using Tables 5 and 6, we can convert the summation of the short-form score to corresponding clinical outcomes—COHSI and RRs. The first column of the two tables is the raw summation of short-form score. The second column and third column are the corresponding latent variable t score and standard error to estimate COHSI and RRs. We also report the sample size (% of the sample) in both of the two tables. As noted above, these items were selected based on the slope (higher slope), threshold parameters (wider range), domain representation, and expert panel opinions. The overall F–K and FRE scores are 5.0 and 85.7 for long form, and 3.3 and 95.3 for short-form, which correspond to 3rd or 4th grade reading level (children 8–9 years old).

Table 4 PROMIS child oral health short-form items
Full size table
Table 5 T score conversion table for Children’s Oral Health Status Index
Full size table
Table 6 T score conversion table for referral recommendations
Full size table

Information curves for the full-length and short-form scales are provided in Fig. 2. Intra-class correlations between the estimated scores for the corresponding full-length and short-form scales were 0.86 for COHSI and 0.87 for RR. The estimated correlation between the COHSI and the RR scale was 0.90 (full-length) and 0.74 (short-form).

Fig. 2
figure 2

Information curve for both COHSI and referral (long form vs. short-form). Conversion of raw scores to t scores are given in Table 5 (COHSI) and Table 6 (RR)

Full size image

Discussion

Guided by the PROMIS framework, the initial items for measuring children’s oral health were developed from a literature review of existing survey questions that measure oral health status of children and adolescents [35]. Newly designed questions from focus group interview [36] were also included in the initial item pool. The revised items serve as the foundation for developing static short-forms, and later computer adaptive testing (CAT) with different purposes for oral health evaluation, screening, and surveillance monitoring. A sample size of 250 or more was enough to estimate the parameters for GRM [63], but a sample size 500 and over was recommended for estimating latent variable accurately and calibrating stable parameters of the items. Although this is ideal, a smaller sample can still provide useful information. In the sample, we did not include those who have orthodontic appliances and who could not speak English.

The on-site clinical examinations were performed independently by two dentists on the research team who were calibrated with regard to the examination instrument and standards prior to the examination. The exam checked the entire mouth for occlusion, oral mucosal tissue, overbite, overjet, cross bite, space loss, and crowding issue for primary and permanent teeth. Examination was done on a tooth-by-tooth basis, including missing, decayed, filled, sound but fractured, abnormal positioned, intrinsic/extrinsic enamel opacity, and bleeding. The RRs are given immediately at the site to the participants and their parent or guardian. The two outcomes, COHSI score and RRs, represent two perspectives of the oral health status, namely the overall evaluation of the child’s current oral health status and the examiner’s determination as to whether a dental appointment is necessary beyond regular dental visit. This dentist-based clinical examination model is, however, not feasible for evaluating oral health of large populations of children and it has been very challenging to find a feasible and cost effective alternative approach to monitor oral health of large populations of school children.

The aim for this paper is to develop a static short-form that measures current oral health status and provides recommendations for different levels of treatment needs, which could eventually serve as a screening tool for efficiently assessing oral health of children and adolescents. The domain hierarchy for oral health developed in this paper is designed to comprehensively cover different aspects of oral health from physical, mental, and social perspectives. The long form includes all domains in the conceptual model that directly measure oral health conceptually from physical, mental, and social aspects of health. The final combined short-forms of COHSI and RRs include 12 items, which minimize the number of questions and significantly reduce the burden for subjects, while at the same time maintain as much information in the long form as possible. The short-form can be effectively applied to large populations of children to evaluate their oral health status and provide a treatment recommendation for dental care. However, one does not have to administer both COHSI and RR together. If needed, one can only administer items for COHSI or RR to obtain its corresponding t score. The summation score for the short-form responses, used together with Table 5 for COHSI and Table 6 for RRs, can estimate the current overall oral health status and treatment needs, respectively.

This study has limitations. We recruited 334 children/adolescents with age 8–17, and all of them had access to dental care, with fewer oral health problems than the general population of children. Very few subjects needed a recommendation to see a dentist immediately or within 2 weeks. Given the complexity of residential mobility in Los Angeles, we did not take into account the variation among different areas, either demographic differences or differential oral health status in the analysis. We pooled the data collected from different clinics together to conduct the analysis and develop long and short-forms.

The long and the short-forms created in this study provide a possible alternative solution to the long-lasting challenge of assessing oral health in large populations of children in care, who have already visited a dentist and has a dental clinic (dental home) to visit again when in need. For more general populations, we will need to incorporate additional questions. For example, we will need to consider the unique situations of those who are not in care yet. This may include items related to frequency of dental visits, and some clinical questions, such as dry mouth, gingival bleeding when brushing or flossing, etc.

We recommend that people use the short-forms as they are. However, it is also possible for future users to add other items and evaluate whether they contribute additional value (information) beyond the existing items. Furthermore, future evaluation of the existing items in other samples is needed. In addition, those items that are excluded for score estimation due to skewness may need to be included for other purposes, e.g., smoking or tobacco use as an important indicator for poor oral health. The short-form was developed based on the sample recruited from dental clinics located within Los Angeles County, with higher percentages of Latinos and Asians than the national proportion. This, together with the higher COHSI score and fewer subjects with an urgent need to see a dentist, made it more difficult to directly generalize the current short-form (as well as the current long form) to a broader general population without modifications. Specific adjustments that consider the variation of a particular subgroup should be considered when using the short-form. This version of the short-form not only provides recommendations for needed care but also estimates current overall oral health status. While the screening by short-forms can never totally replicate the examination by dental professionals, it can provide a cost-effective way to make oral health screening feasible for large populations of children and adolescents. It can be used in practices at local, state, and even national level for triaging the large children and adolescent populations with oral health needs and setting the priorities within populations with varying urgency for dental and oral health care. This is especially true with school-aged children and adolescents who may need immediate care, and therefore contribute to gaining more timely access to needed care.

The initial short-forms presented in this paper are not disease-targeted, although it could be designed to focus on dental caries, missing teeth, periodontal diseases, etc. The generic version of the short-forms for oral health can be used to compare the relative burden of oral-related diseases, and evaluate overall oral health status and need for care of large populations. In the process, we flagged items with DIF instead of directly excluding them from the design, as these items provide the insights of fully measuring oral health, for example, the aesthetic domain might not be the same for males and females.

In conclusion, this is the initial paper for developing the generic version of oral health short-forms that can cover the different domains in the oral health conceptual structure including components of global health, physical, mental, and social health. The overall 12-item short-form can be effectively used in large populations of children to evaluate their current overall oral health status and treatment needs at different levels. Using a standardized t score metric, it is easy to compare oral health status across individuals and populations. With further tuning, the short-form can potentially have better accuracy and higher sensitivity and it can be eventually be implemented through computer, internet, and smart phone apps at subject’s preferred place and time. With the created long form shown in Tables 2 and 3, CAT system, a dynamic and efficient survey tool, could be developed in the next step, in order to be more effective and accurate in survey process, notably for ease of scoring and immediate plotting results in real-time [31]. The oral health item bank system created in this paper provides the foundation for other purposes such as creating specific targeted short-form(s) for program evaluation and/or oral health policy planning and others.