Introduction

Dental caries is a common preventable disease and is still a major oral health problem around the world, affecting up to 90% of children and adolescents [1,2,3]. Caries can be arrested in early stages but, without proper prevention, the disease can progress until destruction or tooth loss, affecting the quality of life of the child, with potential negative social and economic repercussions in the family [4, 5]. Periodical dental control, oral hygiene, antimicrobial agents, fluorides, low-carbohydrate diet and dental sealants are preventive measures that can help prevent irreversible problems on the teeth such as dental caries [6].

Many countries have implemented preventive programs to decrease the incidence of caries in the general population. To standardise the dental caries preventive measures, evidence-based clinical practice guidelines (CPGs) have been developed, facilitating clinician and patient decisions about the most appropriate health care [7]. These are instruments that provide specific recommendations to optimise results, minimise risks and promote a cost-effective practice for health care professionals [8,9,10,11]. CPG recommendations should be based on the best available scientific evidence to obtain the desirable outcomes in patient care [12].

There are multiple CPGs for dental caries prevention, primarily developed by governmental health institutions and dental societies. Some countries adopt and/or adapt existing CPGs to their settings, leading to a vast variability in the CPG quality. The quality assessment of guidelines in other medical specialties has demonstrated high variability and is rarely considered optimal [12,13,14].

We found only one study that assessed the quality of CPGs in the prevention of dental caries; however, it was limited to European guidelines about fissure sealants [13]. The authors concluded that, according to the AGREE II instrument, the three guidelines evaluated showed high variability in the quality. Therefore, we conducted a systematic assessment of the quality of the CPGs about dental caries prevention in children and adolescents.

Materials and methods

Study design

We conducted a systematic assessment of the quality of CPGs for dental caries prevention in children and adolescents with the AGREE II instrument.

Search strategy and selection of CPGs

We performed a systematic search of scientific literature published between 2005 and 2016 to identify CPGs about caries preventive measures. The search was conducted in MEDLINE, EMBASE, LILACS and Trip databases. It was complemented by hand searching websites of guideline developers (Scottish Intercollegiate Guidelines Network (SIGN) and National Institute for Clinical Excellence (NICE)), CPG compilers (National Guidelines Clearinghouse (NGC) and Guidelines International Network (GIN)), scientific societies of paediatric dentistry and Ministries of Health (Online Resource 1).

We included dental CPGs in English and Spanish that included at least one recommendation related to caries prevention measures (fluoride, sealants, oral hygiene and oral health counselling) and carried out a systematic search of the literature. We excluded CPGs developed exclusively for the adult population or patients with special care needs, previous versions of the same guideline, letters to the editor, policies and conference summaries. Two reviewers (A.S. and C.A.A.) independently evaluated the search results to determine eligibility of the references. In case of differences between the reviewers, consensus was reached by discussion or in consultation with a third reviewer (C.Z.).

Quality assessment

A pilot test in five potentially eligible CPGs was performed to calibrate the reviewers. As a minimum, information that involves subjective interpretation and information that is critical to the interpretation of results should be extracted independently by at least two reviewers [15]. This is also what AGREE II recommends to increase the reliability of the assessment [16, 17]. Three calibrated reviewers (A.S., C.A.A., C.Z. or A.O.) independently evaluated the CPGs using the online AGREE tool website My AGREE PLUS [17]: two paediatric dentists and two experts in research methodology. The AGREE II instrument includes 23 items divided into six domains: (1) Scope and purpose, refers to the aim of the guidelines; (2) Stakeholder involvement, represents the views of the intended users; (3) Rigour of development, reflects the quality of the CPG elaboration process and its recommendations; (4) Clarity of presentation, is about the structure of the guidelines; (5) Applicability, shows the barriers and facilitators for the implementation of the CPG, considering also the financial implications to implement the recommendations; and (6) Editorial independence, illustrates the transparency in the formulation of the guidelines recommendations. Each of the 23 items is rated from 1 to 7 points in the Likert scale [16].

Data analysis

We calculated the domain scores by summing up all scores of the individual items included in a domain and by scaling the total as a percentage of the maximum possible score for that domain [16]. If discrepancies were more than 3 points, or if the standard deviation (SD) in an item was greater than or equal to 1.5 SD, the item of the guideline was reassessed by the reviewers and agreement was reached [18]. The guidelines were examined separately according to their scope: fluorides, pit and fissure sealants, oral hygiene or CPGs that considered more than one preventive measure.

Inter-rater reliability

We calculated the intraclass coefficient (ICC) with its 95% confidence interval (95%CI) as an indicator of overall agreement between reviewers for each of the 23 items of the AGREE II instrument. According to the scale proposed by Landis and Koch, the degree of agreement between 0.01 and 0.20 is slight, from 0.21 to 0.40 is fair, from 0.41 to 0.60 is moderate, from 0.61 to 0.80 is substantial and from 0.81 to 1.00 is very good [19]. We used the statistical package IBM SPSS (version 23.0).

Identifying high-quality CPGs

The reviewers considered the “overall quality” of the CPGs and made a final recommendation about each guideline (“recommended”, “recommended with modifications” or “not recommended”) [16]. They also classified the CPG as high quality when at least three of the domains showed a score of 60% or higher, including the Rigour of development domain [18, 20, 21].

Results

Literature search

Of the 878 citations systematically searched, duplicates were firstly eliminated, leaving 637 citations for title and abstract review. Five hundred fifty-six titles and 56 abstracts were excluded because they did not fulfil the inclusion criteria. Of the 25 full-text articles assessed for eligibility, 3 were excluded because they did not declare a systematic search of evidence [22,23,24,25]. Characteristics of the excluded studies can be found in the Online Resource 2. Finally, 22 CPGs were included (Fig. 1).

Fig. 1
figure 1

Flow chart of the included CPGs

Full size image

Characteristics of the included CPGs

The characteristics of the CPGs are listed in Table 1. Thirteen of the included CPGs were in English: seven from the USA [26,27,28,29,30,31,32], three from Ireland [33,34,35], one from Scotland [36], one from Malaysia [37] and one from New Zealand [38]. The other nine were in Spanish: three from Chile [39,40,41], two from Colombia [42, 43], two from Mexico [44, 45], one from Ecuador [46] and one from Spain [47].

Table 1 Characteristics of the included CPGs
Full size table

One CPG was exclusively about infants, one only considered preschool children and one was about caries in adolescents. Nine CPGs included infants, children and adolescents and eight guidelines aimed for the general population, including children and/or adolescents. Nineteen CPGs included recommendations about fluorides, but only five of them were exclusive to this topic. The fluoride preventive measure recommendations included varnish, toothpaste, foam, mouth rinse, gel and dietary supplements.

Thirteen CPGs formulated recommendations about pit and fissure sealant, but only three were exclusive about this topic. Other preventive measures considered in the recommendations of the guidelines with more than one preventive measure were diet, oral hygiene, oral health education, chlorhexidine varnish and mouth rinse, periodical dental examinations, casein phosphopeptide and xylitol chewing gum. We found only one guideline exclusive to oral hygiene, and there were no CPGs limited to oral health counselling only. For analysis, the CPG of oral hygiene was assessed in the group of guidelines with more than one preventive measure.

Of the 22 guidelines, 15 declared an instrument to assess the quality of the included studies to develop the recommendations: Three used the Scottish Intercollegiate Guidelines Network (SIGN) checklist, two used the United States Preventive Services Task Force (USPSTF) system, two used the AGREE instrument to assess the quality of other CPGs and eight used more than one system. The level of evidence and the grade of recommendation used were declared in 18 of the guidelines.

Appraisal of CPGs

The final standardised scores by domain, CPG and preventive measure, in addition to the overall recommendation, are presented in Tables 2, 3 and 4. Figure 2 shows the mean scores for all preventive measures separated by domains. The overall agreement between the three reviewers for the CPG evaluation with the AGREE II instrument was very good (ICC = 0.90; 95%CI 0.89–0.92).

Table 2 Sealant guidelines: standardised scores across CPGs per domain (AGREE II)
Full size table
Table 3 Fluoride guidelines: standardised scores across CPGs per domain (AGREE II)
Full size table
Table 4 Guidelines with more than one preventive measure: Standardised scores across CPGs per domain (AGREE II)
Full size table
Fig. 2
figure 2

Mean quality score for each domain of the AGREE II instrument for all the CPGs

Full size image

Scope and purpose

The mean score for this domain was 89.5 ± 12.3% (range 76–100%) in the sealant CPGs, 96.7 ± 3% (range 93–100%) in the fluoride CPGs and 87.1 ± 13.5% (range 54–100%) in the CPGs with more than one preventive measure. The mean score of all the 22 guidelines was 89.6 ± 12% (range 54–100%). Only one (4.5%) guideline scored below 60% in this domain [31].

Stakeholder involvement

For this domain, the main score for the sealant guidelines was 59.3 ± 3.7% (range 56–63%), for the fluoride guidelines was 57.8 ± 12.9% (range 37–72%) and for the CPGs with more than one preventive measure was 53 ± 18.1% (range 11–91%). The global mean score was 55 ± 15.6% (range 11–91%). Seven CPGs (31.8%) scored above 60% in this domain [32,33,34,35,36, 38, 41].

Rigour of development

The mean score for this domain was 73.8 ± 19.6% (range 53–92%) in the sealant CPGs, 76.5 ± 21.2% (range 39–90%) in the fluoride guidelines and 58.9 ± 20.4% (range 29–90%) in the CPGs with more than one preventive measure. The global mean score was 64.9 ± 21.2% (range 29–92%). Nine of the included guidelines (40.9%) scored below 60% [27, 30, 31, 39, 43,44,45,46,47].

Clarity of presentation

The mean score for this domain was 93.8 ± 5.3% (range 91–100%) in the sealant CPGs, 91.5 ± 7.9% (range 78–98%) in the fluoride guidelines and 80.4 ± 15.5% (range 50–100%) in the CPGs with more than one preventive measure. The global mean score was 84.8 ± 14.1% (range 50–100%). Only two guidelines (9.1%) scored below 60% in this domain [30, 31].

Applicability

The mean score for this domain was 29.2 ± 41% (range 3–76%) in the sealant CPGs, 26.7 ± 34% (range 0–86%) in the fluoride guidelines and 32.3 ± 31.4% (range 0–93%) in the CPGs with more than one preventive measure. The global mean score was 30.6 ± 31.5% (range 0–93%). Four of the assessed CPGs (18.2%) scored above 60% in this domain [33,34,35,36], and two guidelines (9.1%) scored 0% [27, 30].

Editorial independence

The mean score for this domain was 50.9 ± 22.5% (range 25–64%) in the sealant CPGs, 57.2 ± 33.4% (range 6–92%) in the fluoride guidelines and (61.9 ± 24.6% (range 11–86%) in the CPGs with more than one preventive measure. The global mean score was (59.3 ± 25.5% (range 6–92%). Nine of the elected guidelines (40.1%) scored below 60% [26,27,28, 30, 31, 38, 39, 41, 45].

Overall assessment

Thirteen of the 22 assessed CPGs (59.1%) were “recommended” by the reviewers (2 of sealants [26, 35], 4 of fluorides [28, 29, 33, 38] and 7 of the CPGs with more than one preventive measure [32, 34, 36, 37, 40,41,42]); eight (36.4%) were “recommended with modifications (1 of sealants [44], 1 of fluorides [27] and 6 of the CPGs with more than one preventive measure [30, 39, 43, 45,46,47]); and one (4.5%) was “not recommended” [31]. The same 13 CPGs evaluated as “recommended” were classified as high quality, scoring ≥ 60% in at least three domains, including the Rigour of development domain. The one “not recommended” guideline scored below 60% in all the domains.

Discussion

The evaluation of 22 worldwide CPGs in dental caries prevention in children and adolescent patients showed a moderate quality. Thirteen were classified as “recommended” and as high quality. However, only 4 CPGs scored above 60% in all six domains (1 published by the SIGN and 3 by the Irish Oral Health Services Guideline Initiative).

The guideline of the American Academy of Pediatric Dentistry (AAPD) on adolescent oral health care [31] was the only one evaluated as “not recommended”. The main difference between this guideline and the other three AAPD guidelines, classified as “recommended with modifications”, was in the Scope and purpose domain because it did not have a clear objective.

Of all the guidelines included, the domain with the highest overall score was Scope and purpose (90%), followed by Clarity of presentation (85%) and Rigour of development (65%). In the other three domains, the overall score was below 60%, with the Applicability domain being the lowest (15%).

The CPGs considered in our study showed an overall higher quality than the ones assessed in other topics, with a 59.1% of “recommended” guidelines compared to a 37% reported in the literature [48]. Studies assessing the quality of CPGs are practically new in dentistry and have been reported since 2015 in the areas of dental radiology, implants, orthodontics and preventive dentistry [13, 49,50,51,52]. The guideline assessment of these studies in dentistry also obtained the highest scores in the Scope and purpose and Clarity of presentation domains, with the Applicability domain being the worst [49,50,51,52].

The Scope and purpose domain was the best assessed, which shows the overall aim of the guideline. The guidelines with the lower scores in this domain failed in giving a detailed description of the health questions covered by the CPG. The aim of the CPG needs to be clear and in coherence with the key recommendations to guide the user in implementing the most favourable care for a specific health problem [17].

Analysing the Stakeholder involvement domain, it appears that user opinions and preferences were not considered, or the results were simply not reported. Additionally, most guidelines did not include or report the specialist or expert involved in their development. The evidence shows that guidelines improve when specialists, experts and patients participate effectively in the process, thereby keeping it transparent, helping to define the objectives and showing that the recommendations are not based on bias decisions [53, 54].

The Rigour of development domain is usually considered the main domain in the assessment of the clinical guidelines. It directly evaluates the gathering of information and the quality of the development process of the recommendations. In our assessment, this domain obtained a high score, except in nine of the CPGs that scored below 60% in this domain. The main problems in our assessment were not indicating the limitations of the evidence, not reporting the possible harmful effects and not specifying the results of the expert’s external review, and their implications in the development process. It seems easier to highlight the positive aspects of preventive measures; nonetheless, every procedure has risks that need to be declared. For instance, fluorides can produce fluorosis; a defective sealant can put the teeth at further risk of caries than possibly not sealing them at all [55,56,57]. The guidelines need to be clear on this point, so that users and patients can make informed decisions about their dental health. On the other hand, the importance of the results of the external review is significant, as some guideline development groups can show bias in the development of their own guidelines [58, 59].

The Clarity of presentation domain was the second best evaluated, where the main problems in some of the CPGs were the ambiguity and the format in the presentation of the recommendations, which is important to make the guideline easier to use [16].

The low scores in the Applicability domain are mostly because the assessed CPGs did not consider economic analysis for the implementation of the recommendation guidelines. The evidence shows that economic analysis is seldom incorporated in the development of CPGs [60]. Further economic evaluations in preventive measures in dentistry are needed and should be incorporated in the guidelines [61, 62]. The economic analysis in the guidelines can impact both the quality of care and the efficient allocation of resources. Studies in medicine assessing cost-effectiveness of the guidelines have shown that there was an increment of QALYs using the evidence-based recommendations of the guidelines [8,9,10, 63].

There is also a lack of information as to how the guidelines are going to be disseminated and implemented. These measures should be available to patients, clinicians and the general population. However, there is no reliable evidence on which implementation method is the best [64].

The problem presented in the Editorial independence domain was in the declaration of the conflicts of interest, although, this was included in almost all the guidelines, it only considered financial conflicts of interest. For instance, political, intellectual or social conflicts of interest were not declared. Guideline members should make the interests of the target population their sole concern and set aside any competing financial or professional interests. It is particularly important that the funding body and the group members of the CPGs state their conflicts interest, because CPGs are being used in decisions regarding insurance coverage and standards of care [58]. The best way to avoid the influence of external interests is to link the recommendations to the evidence, particularly systematic reviews and avoiding those panellists with conflicts in the decision-making process [59, 65].

There is only one study in dentistry that focused on dental caries prevention and assessed European pit and fissure sealant CPGs [13]. In view of the differences in the inclusion criteria, it was difficult to compare our work with the above study; nonetheless, in both studies, the best results were in the Scope and purpose domain. They analysed three guidelines; the Irish CPG was the only one coinciding with our evaluation [35]. The scores of the Irish guideline are very similar between both studies; the main difference is that we classified this guideline as “recommended” and the European study as “recommended with modifications”.

We found a lack of uniformity in the grading systems used in the CPGs, and four of them did not rate the quality of the evidence and strength of their recommendations (three from the AAPD and one from the New Zealand Guidelines Group). Clinical guidelines must be based on the best available evidence and need to use validated recommendation rating systems, to provide an explicit connection with the evidence. It helps to avoid bias in the recommendation development process with the goal of developing high-quality guidelines [59].

Strengths and limitations

We identified some limitations in our review process. First, the study did not include all preventive measures in dental caries, such as diet assessment or chemical agents (for example xylitol and chlorhexidine). We decided to include the measures directly related to dental caries prevention and those that are usually used in public health measures. And second, we only considered guidelines in Spanish and English language, which prompted the omission of CPGs.

Access to the CPGs was not an obstacle in this study. The CPGs were all available in open access and free of charge, allowing easy dissemination of the information. The search of the methodological/procedure and manuals of the guidelines was sometimes difficult, particularly in guidelines that were developed with older versions of an updated manual. The data analysis was much easier in those guidelines that specified where to locate these manuals, or those that included a clear and complete methodology.

The main strengths of this study are that information regarding development process of CPGs was obtained in a systematic search of the literature and was evaluated independently by three calibrated reviewers using a standardised tool. The AGREE II instrument is currently the only validated and reliable instrument that enables a quantitative comparison of CPGs, and is designed to help users to evaluate their methodological quality. This tool also provides a methodological strategy for the development of guidelines and informs about the type of information to be reported [16].

To our knowledge, we are the first to report the quality assessment of dental guidelines in Spanish. These will be useful in Spanish-speaking countries, with private and public health services and dental societies able to readily access the “recommended” guidelines in the development and adaptation of their own guidelines. Access to this information is especially important in Latin American countries where the prevalence of dental caries continues to be a public health problem [4]. We carried out an extensive systematic search of scientific evidence, and three calibrated evaluators did the assessment of the CPGs, resulting in a high level of concordance between them. Finally, our team included clinical experts and methodologists with vast experience in CPGs.

Implications for practice and research

The variability of the guidelines stresses the importance of the clinicians’ need to identify high-quality guidelines before implementing the recommendations. A low-quality CPG may not meet effective health results in the use of its recommendations or might not consider the risks or disadvantages of using those recommendations in a specific scenario. Eight of the guidelines were “recommended with modifications”, which hopefully will prompt a discussion in the CPG development process to improve future updates.

Improvements in the quality can be made by using a standardised framework to present the recommendations and by including patient preferences in the development process and also by incorporating economic evaluations and cost-effectiveness studies of the measures, as well as through evaluations of the quality of the evidence available. Our results facilitate the decision-making process in the selection of the most suitable CPG in dental caries prevention.

Conclusion

CPGs in dental caries prevention in children and adolescents during the last 10 years showed a high quality in the assessment with the AGREE II instrument. However, they presented some deficiencies, in the domains of Stakeholder involvement, Applicability and Editorial independence.