Abstract
Aim
To compare the diagnostic performance of the automated caries detection system (ACDS) for the detection and diagnosis of occlusal caries with the histological appearance of the lesions.
Methods
Eighteen posterior permanent teeth were used, out of which 40 sections were made and 53 areas were evaluated. Teeth with hypoplastic and/or hypomineralised areas or sealants on the occlusal surfaces were excluded from the study. The teeth that were used for this study were a subgroup of the teeth used in the study that introduced ACDS system. This subgroup consisted of teeth having in their occlusal surfaces early carious lesions classified as international caries detection and scoring system (ICDAS) 0, 1, 2 and 3 after clinical examination by the examiners. Histological preparations were classified by experienced examiners based on the Ekstrand, Ricketts and Kidd (ERK) system and for the respective occlusal surfaces by the ACDS system based on ICDAS II system. There were two threshold limits considered as carious in either system ICDAS ≥ 2 or ≥ 3 and ERK index ≥ 2 or ≥ 3 and all possible combinations were analysed. Statistical methods of weighted version of kappa coefficient, Kendall’s tau-b correlation coefficient and p-values using the Fisher’s exact method were used at the confidence level of 0.05.
Results
Intra-examiner kappa coefficient agreement was 0.87 and 0.89 while the inter-examiner for the two trials were 0.87 and 0.92. The ICDAS3-ERK3 combination between the ACDS and histological sections presented the best agreement with kappa coefficient 0.76, agreement 92.5%, sensitivity 100% and specificity 91.1%. ICDAS3-ERK3 combination between the optical examination of the examiners compared to the histological preparations showed kappa coefficient 0.87, agreement 96.2%, sensitivity 100%, Specificity 95.6%.
Conclusion
The evidence supports the view that ACDS classification of occlusal surfaces based on the ICDAS system are comparable with classification to that of an examiner and with the histology of the lesion. The use of ACDS has the distinct advantage though of removing the subjectivity of the examiner since it performs the classification without any intervention by him.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Introduction
Dental caries is one of the most prevalent chronic diseases affecting 60–90% of school children and the vast majority of adults (Selwitz et al. 2007; Dental Health Foundation Ireland 2018). In the past few decades though there is a consistent reduction of caries incidence with the greatest reduction occurring on the proximal and smooth surfaces (Ripa et al. 1985; Hannigan et al. 2000). This pattern of reduction made occlusal the most frequent tooth surface affected by caries. Whilst occlusal surface is the most readily accessible tooth surfaces, detection and diagnosis of occlusal caries is still very challenging for the dental clinician (Sawle and Andlaw 1988), with diverging decisions on its management. In this direction and in an attempt to make diagnosis more accurate and eliminate divergence on diagnostic opinions, several instruments and tools have been developed to substitute the dentist’s optical assessment of the lesions. However, since most of these approaches are using instruments analysing the compositional changes of the lesions they have failed to reach high scores of agreement, due to several artefacts accompanying these methods. Therefore, it is important to develop sensitive, specific and reproducible diagnostic tools for precise caries management (Lussi 2003) especially in the early stages of the disease. One of these tools that was developed the past 15 years is the ICDAS system which has become, as a visual tool, an established system of classifying carious lesions visually. There is extensive literature describing the theoretical and clinical background of the system (Stookey 1996, 2000, 2004; Pitts and Stamm 2004; International Caries Detection and Assessment System Coordinating Committee 2012) as well as its clinical performance (Ekstrand et al. 1997, 2001, 2005; Fyffe et al. 2000; Ricketts et al. 2002).
In 2015, Berdouses et al. proposed the Automated Caries Diagnosis System (ACDS) which used digital colour images of the occlusal surfaces, in order to diagnose and classify occlusal carious lesions according to visual ICDAS system. The performance of the ACDS has been compared against experienced examiners using the visual ICDAS II system with very promising results (Berdouses et al. 2015). However, in order to validate the performance of ACDS, there is a need to validate it against a golden standard, which is the appearance of the carious lesions in histological sections.
The purpose of this study is to compare the diagnostic performance for the detection and diagnosis of occlusal caries of the ACDS system, compared with the histological appearance of the lesions in tooth sections which will be used as the golden standard.
Materials and methods
For the needs of this study 18 posterior permanent extracted teeth were used, out of which 40 sections were made and 53 areas were evaluated. The extracted teeth were collected from the Maxillofacial Surgery Clinic of the Dental School of the National and Kapodistrian University of Athens, as well as from private dental offices in Athens. Teeth with hypoplastic and/or hypomineralised areas or sealants on the occlusal surfaces were excluded from the study. The teeth that were used for this study are a subgroup of the teeth used in the study that introduced ACDS system (Berdouses et al. 2015). This subgroup consisted of teeth having in their occlusal surfaces early carious lesions classified as ICDAS 0, 1, 2 and 3 after clinical examination by the examiners.
Then the roots of the teeth were removed with a high speed handpiece under water spray. The teeth then were immersed in acrylic resin and allowed to set into round blocks with 25 mm diameter and 20 mm height. A chemically transparent acrylic resin was used (powder: Jet Tooth Shade Acrylic Powder Clear, liquid: Jet Tray Liquid clear, Lang Dental Mfg Co Inc, Wheeling, Il, USA).
The teeth were stored, cleaned, photographed and evaluated by the ACDS as described by Berdouses et al. (2015).
The teeth were sectioned with the use of a microtome (Microtome 2, Metal Research, Cambridge, England) using a disk of 0.1 mm thickness at slow speed. In every crown, sections were made along the areas of the more serious lesions or the pit and fissures of the surface. The sections had mesial-distal direction for the mandibular molars and all premolars and buccal-lingual for the maxillary molars. Figure 1 presents an example of the clinical picture of a tooth (Fig. 1a), the design of the section (Fig. 1b) and the histological section from the optical microscope with 90° polarised light and 50× magnification. The sections were then polished in a polisher device (EcoMet III, Buehler Ltd, Lake Bluff, IL, USA). The polishing process started with polishing SiC papers with 180–600 grit in order to achieve a uniform flat surface and uniform thickness that was measured with digital thickness gauge (± 0.01 mm). Then SiC papers with 800, 1000, 1200, 2400 and 4000 grit were used to polish the surface and prepare it for histological examination.
The histological sections were examined from pictures taken in incident light optical microscope (DM 4000B, Leica Microsystems) with 90° polarised light and 50× magnification so the details of every pit and fissure were evident. There were 2 examiners that evaluated the sections with 2 weeks interval. The samples were classified based on the criteria of the Ekstrand, Ricketts and Kidd (ERK) system (Ekstrand et al. 1997). If after the second evaluation was a disagreement between the two examiners, that case was discussed and the two examiners concluded in one value that was used to compare with the ACDS classification.
ACDS classifies occlusal areas based on the ICDAS system and the histological sections were classified based on the ERK classification. In order to investigate the agreement between the ACDS and the histology of the lesions a threshold limit was used. Lesions classified with higher score than the limit, were considered as caries. Therefore, the lesions were classified into two groups as carious or sound. Two threshold limits were used for the two systems. ICDAS 2 threshold considers as carious areas that have ICDAS value, as it is classified by ACDS system, ≥ 2. ICDAS3 considers caries that have ICDAS value 3. Respectively, ERK2 and ERK3 thresholds were considered areas that have ERK ≥ 2 and ERK ≥ 3. These thresholds divided the data into four groups and all four possible combinations were analysed.
Statistical analysis
Intra- and inter-examiner reliability was assessed using the weighted version of kappa coefficient of agreement. Also the Kendall’s tau-b correlation coefficient and its associated p-value were calculated to quantify and assess the strength of the association between the results.
The agreement of all four combinations between the ACDS and histological classifications was analysed by calculating the overall agreement, the unweighted kappa coefficient of agreement (as there were only two categories), the sensitivity and the specificity. p values given in the corresponding 2 × 2 tables were calculated using the Fisher’s exact method. The confidence level of 0.05 was used in all statistical tests. All analyses were performed using Stata 13 (Stata Corp. TX, USA).
Results
Tables 1 and 2 present the confusion matrix, the kappa coefficient and the agreement between the first and the second examination of the examiner A and the examiner B, respectively. Intra-examiner kappa coefficient for the examiner A was 0.87 and for examiner B 0.89 while the agreement was 84.9% and 86.8% respectively. Kendall’s correlation coefficient τ (tau) was statistically significant for both examiners suggesting that each examiner classified the areas with a similar value at the two examinations.
Tables 3 and 4 present the inter-examiner reliability at the first and second examinations respectively. The agreement was 84.9% for the first examination and 90.6% for the second while kappa coefficient was 0.87 and 0.92, respectively. Kendall’s tau correlation coefficient was statistically significant for both examinations.
Table 5 presents the unweighted kappa coefficient, the agreement, the sensitivity and the specificity of the comparison between the different cut off points of the ICDAS classification of the ACDS and the ERK systems. It is apparent that ERK 3 presented the higher values of sensitivity for both ICDAS cut off points while ERK2 presented better specificity values for both ICDAS II cut off points. The most balanced agreement was between ICDAS 3 and ERK 3 cut off points. We can observe similar results when we compare the ICDAS classifications of the examiners that evaluated the occlusal surfaces clinically and the histological sections. The results from this comparison are presented in Table 6.
Discussion
The present study estimated the validity of the Automated Caries Diagnosis System (ACDS) (Berdouses et al. 2015) using the histological appearance of the lesions as the golden standard. The ERK classification system was used to classify the histological sections (Ekstrand et al. 1997). The ACDS uses a digital image of the occlusal surface of the tooth and using an algorithm identifies and classifies different areas of the occlusal surface according to the ICDAS II system.
The teeth used in this study were clinically classified by two examiners with occlusal carious lesions having ICDAS scores from 0 to 3. The selection of lesions between ICDAS 1–3 was decided because they pose the most difficult categories of the ICDAS system to be diagnosed correctly and code 0 for comparison as a control.
In the literature there are three suggested methods that use digital pictures and try to classify the occlusal lesions based on ICDAS (Kositbowornchai et al. 2006; Umemori et al. 2010; Ghaedi et al. 2014). All these methods use different optical methodologies to classify the occlusal areas and some of them have limitations in the design of the study. Kositbowornchai et al. (2006) developed an artificial neural network for the detection of simulated dental caries. They used artificial caries to evaluate the method. A significant limitation is the fact that artificial caries lesions are very different in their histochemical composition as they have not gone through the extensive demineralisation/remineralisation process and the absorption of stains as it happens with natural caries in the oral environment. These differences produce different optical characteristics between the artificial and natural occurring carious lesions that obscure the classification of the lesions. Umemori et al. (2010) suggested, an analysis of digital pictures that combines statistical test of 2 way ANOVA and Games-Howell. In this in vivo study the authors did not describe the criteria they used to classify the lesions as sound (C0), caries in the enamel (C1) and caries in dentine (C2). They verified the classification of the areas by opening the lesions with a bur. The authors did not use histological sections of the lesions because it was an in vivo study so they did not prepare areas that were classified as sound. Thus, it was not possible to evaluate if the sound areas were classified correctly. Histological validation was not performed in any of these two studies.
The algorithm proposed by Ghaedi et al. (2014), was very similar with the algorithm that was used for the present ACDS study. The results from the clinical evaluation of both systems are very close, but as with previous studies there is not histological validation for the method proposed by Ghaedi et al. (2014). Therefore, ACDS is the first system that is designed to classify occlusal caries lesions based on a digital image that has been validated in comparison to histological appearance of the lesions.
For the histological classification of the sections the system ERK was used (Ekstrand et al. 1997). The ERK system was chosen because the authors showed that the five (5) score system presented high correlation (rs = 0.87–0.93) between visual and histological evaluations. Moreover, in a more recent study (Jablonski-Momeni et al. 2008), where the ERK classification was compared with the classification proposed by Downer (1975), it was found that they had very similar correlation values. The rs value for the ERK system was found to be between 0.43 and 0.68 and for the Downer system 0.48–0.72.
In order to compare the agreement between the classification of ACDS and ERK systems different cut off points for caries existence were evaluated for both systems (Table 5). The comparison of ERK3 to both ICDAS2 and ICDAS3 presents better sensitivity values than ERK2. On the contrary, ERK2 presents better specificity values compared to both ICDAS 2 and ICDAS 3 than ERK3. It is apparent that the most balanced agreement is when ERK3 is compared to ICDAS 3.
Reduced specificity value suggests that this combination of cut off points falsely classifies an increased number of healthy surfaces as carious. Such combinations are ERK2—ICDAS2 and ERK3—ICDAS2. In clinical practice reduced specificity leads to increased unnecessary treatment which is not acceptable. In the literature it is suggested that in some particular cases though, especially when sound surfaces are given a low ICDAS score not requiring operative treatment, is not as important since the clinical outcome for these surfaces would be to receive preventive treatment which in high risk patients would be beneficial (Jablonski-Momeni et al. 2008). As a principle though, low specificity of any diagnostic technique is not desired for any diagnostic technique.
The results show that the combination ERK3 and ICDAS3 presents the best balance between specificity and sensitivity values. A very interesting finding was a very high sensitivity value (100%). This high value maybe is affected by the fact that there were 8 surfaces classified as ICDAS3 that all were correctly classified based on the histological section. It is very important that the best agreement of ACDS classification and the histological sections have the best agreement at the cut off points ERK3 and ICDAS3. These cut off points in both systems suggest that there is initial cavitation of the lesion and they should receive operative treatment. Therefore, ACDS can classify occlusal areas at the level of initial cavitation correctly when compared with the histological section of the lesion. Similar results are presented in the literature when having compared optical evaluation of the occlusal surfaces by examiners and the histology of the lesions. Jablonski-Momeni et al. (2009), reported 0.71 sensitivity and 0.88 specificity between ICDAS 3 and the histological cut off point of Downer 3 (lesion at the outer half of the dentine). Ekstrand et al. (1997) reported a range of sensitivity values of 0.92–0.97 and specificity values of 0.85–0.93 between the ERK 3 and Ekstrand 3 (same as ICDAS3) cut off points.
The performance of the ACDS has been previously compared with the optical evaluation of the occlusal surfaces by examiners and had very good results (Berdouses et al. 2015). The results showed 80% sensitivity and 96% specificity, precision 82%, accuracy 83% and k = 0.78. The results from this study also support a good performance when ACDS was compared with the histology of the lesions as the golden standard. Both comparisons suggest that the ACDS is a system that can identify occlusal caries with a similar accuracy of an experienced examiner.
Conclusions
ACDS classification of occlusal surfaces based on the ICDAS system are comparable to that of an examiner and with the histology of the lesion. The use of the ACDS system has some important advantages compared to an examiner because it removes the subjectivity of the examiner since the system needs only the picture of the occlusal surface and then performs the classification without any other intervention. ACDS can improve its performance by incorporating into the algorithm future knowledge on image processing or on caries progression and if a larger number of pictures with known ICDAS classification are incorporated into the system.
References
Berdouses ED, Koutsouri GD, Tripoliti EE, et al. A computer-aided automated methodology for the detection and classification of occlusal caries from photographic color images. Comput Biol Med. 2015;62:119–35. https://doi.org/10.1016/j.compbiomed.2015.04.016.
Dental Health Foundation Ireland. Dental caries (Tooth decay). 2018. http://www.dentalhealth.ie/dentalhealth/causes/dentalcaries.html. Accessed 24 Nov 2018.
Downer CM. Concurrent validity of an epidemiological diagnostic system for caries with the histological appearance of extracted teeth as validating criterion. Caries Res. 1975;9:231–46.
Ekstrand KR, Ricketts DN, Kidd EA. Reproducibility and accuracy of three methods for assessment of demineralization depth on the occlusal surface: an in vitro examination. Caries Res. 1997;31:224–31.
Ekstrand KR, Ricketts DN, Kidd EA. Occlusal caries: pathology, diagnosis and logical management. Dent Update. 2001;28:380–7.
Ekstrand KR, Ricketts D, Longbottom C, Pitts NB. Visual and tactile assessment of arrested initial enamel carious lesions: an in vivo pilot study. Caries Res. 2005;39:173–7.
Fyffe HE, Deery CH, Nugent ZJ, Nuttall NM, Pitts NB. Effect of diagnostic threshold on the validity and reliability of epidemiological caries diagnosis using the Dundee Selectable Threshold Method for caries diagnosis (DSTM). Comm Dent Oral Epidemiol. 2000;28:42–51.
Ghaedi L, Gottlieb R, Sarrett DC, et al. An automated dental caries detection and scoring system for optical images of tooth occlusal surface. Conf Proc IEEE Eng Med Biol Soc. 2014;1925–8. https://doi.org/10.1109/EMBC.2014.6943988.
Hannigan AOD, Barry D, Schaffer F, Roberts AJ. A caries susceptibility classification of tooth surfaces by survival time. Caries Res. 2000;34:103–8.
International Caries Detection and Assessment System Coordinating Committee. Rationale and Evidence for the International Caries Detection and Assessment System (ICDAS II). 2012. https://www.icdas.org/. Accessed 24 Nov 2018.
Jablonski-Momeni A, Stachniss V, Ricketts DN, Heinzel-Gutenbrunner M, Pieper K. Reproducibility and accuracy of the ICDAS-II for detection of occlusal caries in vitro. Caries Res. 2008;42(2):79–87. https://doi.org/10.1159/000113160.
Jablonski-Momeni A, Ricketts DN, Heinzel-Gutenbrunner M, et al. Impact of scoring single or multiple occlusal lesions on estimates of diagnostic accuracy of the visual ICDAS-II system. Int J Dent. 2009;2009:7 (Article ID 798283).
Kositbowornchai S, Siriteptawee S, Plermkamon S, Bureerat S, Chetchotsak D. An artificial neural network for detection of simulated dental caries. Int J Comput Assist Radiol Surg. 2006;1(2):91–6. https://doi.org/10.1007/s11548-006-0040-x.
Lussi A. Performance of conventional and new methods for the detection of occlusal caries in deciduous teeth. Caries Res. 2003;37:2–7.
Pitts NB, Stamm J. International consensus workshop on caries clinical trials (ICW-CCT)—final consensus statements: agreeing where the evidence leads. J Dent Res. 2004;83(suppl 1):C125–8.
Ricketts DN, Ekstrand KR, Kidd EA, Larsen T. Relating visual and radiographic ranked scoring systems for occlusal caries detection to histological and microbiological evidence. Oper Dent. 2002;27:231–7.
Ripa LW, Leske GS, Sposato A. The surface-specific caries pattern of participants in a school-based fluoride mouthrinsing program with implications for the use of sealants. J Pub Health Dent. 1985;45(2):90–5.
Sawle R, Andlaw RJ. Has occlusal caries become difficult to diagnose? A study comparing clinically undetected lesions in molar teeth of 14-16-year old children in 1974 and 1982. Br Dent J. 1988;164(7):209–11.
Selwitz R, Ismail AI, Pitts NB. Dental caries. Lancet. 2007;369(9555):51–9.
Stookey GK. In Stookey GK, editor, Proceedings of the first annual indiana conference: early detection of dental caries. Indianapolis: Indiana University Press; 1996.
Stookey GK. In Stookey GK, editor, Proceedings of the Second International Indiana Co. Indianapolis: Indiana University Press; 2000.
Stookey GK. In Stookey GK, editor, Early caries detection III, 2004; Indianapolis: Indiana University Press.
Umemori S, Tonami K, Nitta H, Mataki S, Araki K. The possibility of digital imaging in the diagnosis of caries. Int J Dent. 2010. https://doi.org/10.1155/2010/860515.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Rights and permissions
About this article
Cite this article
Berdouses, E.D., Oulis, C.J., Michalaki, M. et al. Histological validation of the automated caries detection system (ACDS) in classifying occlusal caries with the ICDAS II system in vitro. Eur Arch Paediatr Dent 20, 249–255 (2019). https://doi.org/10.1007/s40368-018-0389-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40368-018-0389-x