Introduction

Dental caries is one of the most prevalent chronic diseases affecting 60–90% of school children and the vast majority of adults (Selwitz et al. 2007; Dental Health Foundation Ireland 2018). In the past few decades though there is a consistent reduction of caries incidence with the greatest reduction occurring on the proximal and smooth surfaces (Ripa et al. 1985; Hannigan et al. 2000). This pattern of reduction made occlusal the most frequent tooth surface affected by caries. Whilst occlusal surface is the most readily accessible tooth surfaces, detection and diagnosis of occlusal caries is still very challenging for the dental clinician (Sawle and Andlaw 1988), with diverging decisions on its management. In this direction and in an attempt to make diagnosis more accurate and eliminate divergence on diagnostic opinions, several instruments and tools have been developed to substitute the dentist’s optical assessment of the lesions. However, since most of these approaches are using instruments analysing the compositional changes of the lesions they have failed to reach high scores of agreement, due to several artefacts accompanying these methods. Therefore, it is important to develop sensitive, specific and reproducible diagnostic tools for precise caries management (Lussi 2003) especially in the early stages of the disease. One of these tools that was developed the past 15 years is the ICDAS system which has become, as a visual tool, an established system of classifying carious lesions visually. There is extensive literature describing the theoretical and clinical background of the system (Stookey 1996, 2000, 2004; Pitts and Stamm 2004; International Caries Detection and Assessment System Coordinating Committee 2012) as well as its clinical performance (Ekstrand et al. 1997, 2001, 2005; Fyffe et al. 2000; Ricketts et al. 2002).

In 2015, Berdouses et al. proposed the Automated Caries Diagnosis System (ACDS) which used digital colour images of the occlusal surfaces, in order to diagnose and classify occlusal carious lesions according to visual ICDAS system. The performance of the ACDS has been compared against experienced examiners using the visual ICDAS II system with very promising results (Berdouses et al. 2015). However, in order to validate the performance of ACDS, there is a need to validate it against a golden standard, which is the appearance of the carious lesions in histological sections.

The purpose of this study is to compare the diagnostic performance for the detection and diagnosis of occlusal caries of the ACDS system, compared with the histological appearance of the lesions in tooth sections which will be used as the golden standard.

Materials and methods

For the needs of this study 18 posterior permanent extracted teeth were used, out of which 40 sections were made and 53 areas were evaluated. The extracted teeth were collected from the Maxillofacial Surgery Clinic of the Dental School of the National and Kapodistrian University of Athens, as well as from private dental offices in Athens. Teeth with hypoplastic and/or hypomineralised areas or sealants on the occlusal surfaces were excluded from the study. The teeth that were used for this study are a subgroup of the teeth used in the study that introduced ACDS system (Berdouses et al. 2015). This subgroup consisted of teeth having in their occlusal surfaces early carious lesions classified as ICDAS 0, 1, 2 and 3 after clinical examination by the examiners.

Then the roots of the teeth were removed with a high speed handpiece under water spray. The teeth then were immersed in acrylic resin and allowed to set into round blocks with 25 mm diameter and 20 mm height. A chemically transparent acrylic resin was used (powder: Jet Tooth Shade Acrylic Powder Clear, liquid: Jet Tray Liquid clear, Lang Dental Mfg Co Inc, Wheeling, Il, USA).

The teeth were stored, cleaned, photographed and evaluated by the ACDS as described by Berdouses et al. (2015).

The teeth were sectioned with the use of a microtome (Microtome 2, Metal Research, Cambridge, England) using a disk of 0.1 mm thickness at slow speed. In every crown, sections were made along the areas of the more serious lesions or the pit and fissures of the surface. The sections had mesial-distal direction for the mandibular molars and all premolars and buccal-lingual for the maxillary molars. Figure 1 presents an example of the clinical picture of a tooth (Fig. 1a), the design of the section (Fig. 1b) and the histological section from the optical microscope with 90° polarised light and 50× magnification. The sections were then polished in a polisher device (EcoMet III, Buehler Ltd, Lake Bluff, IL, USA). The polishing process started with polishing SiC papers with 180–600 grit in order to achieve a uniform flat surface and uniform thickness that was measured with digital thickness gauge (± 0.01 mm). Then SiC papers with 800, 1000, 1200, 2400 and 4000 grit were used to polish the surface and prepare it for histological examination.

Fig. 1
figure 1

Clinical and histological picture of an occlusal surface. a Clinical picture of occlusal surface, b design of the section, c picture of the histological section taken with optical microscope of polarised light in 90° and magnification ×50

Full size image

The histological sections were examined from pictures taken in incident light optical microscope (DM 4000B, Leica Microsystems) with 90° polarised light and 50× magnification so the details of every pit and fissure were evident. There were 2 examiners that evaluated the sections with 2 weeks interval. The samples were classified based on the criteria of the Ekstrand, Ricketts and Kidd (ERK) system (Ekstrand et al. 1997). If after the second evaluation was a disagreement between the two examiners, that case was discussed and the two examiners concluded in one value that was used to compare with the ACDS classification.

ACDS classifies occlusal areas based on the ICDAS system and the histological sections were classified based on the ERK classification. In order to investigate the agreement between the ACDS and the histology of the lesions a threshold limit was used. Lesions classified with higher score than the limit, were considered as caries. Therefore, the lesions were classified into two groups as carious or sound. Two threshold limits were used for the two systems. ICDAS 2 threshold considers as carious areas that have ICDAS value, as it is classified by ACDS system, ≥ 2. ICDAS3 considers caries that have ICDAS value 3. Respectively, ERK2 and ERK3 thresholds were considered areas that have ERK ≥ 2 and ERK ≥ 3. These thresholds divided the data into four groups and all four possible combinations were analysed.

Statistical analysis

Intra- and inter-examiner reliability was assessed using the weighted version of kappa coefficient of agreement. Also the Kendall’s tau-b correlation coefficient and its associated p-value were calculated to quantify and assess the strength of the association between the results.

The agreement of all four combinations between the ACDS and histological classifications was analysed by calculating the overall agreement, the unweighted kappa coefficient of agreement (as there were only two categories), the sensitivity and the specificity. p values given in the corresponding 2 × 2 tables were calculated using the Fisher’s exact method. The confidence level of 0.05 was used in all statistical tests. All analyses were performed using Stata 13 (Stata Corp. TX, USA).

Results

Tables 1 and 2 present the confusion matrix, the kappa coefficient and the agreement between the first and the second examination of the examiner A and the examiner B, respectively. Intra-examiner kappa coefficient for the examiner A was 0.87 and for examiner B 0.89 while the agreement was 84.9% and 86.8% respectively. Kendall’s correlation coefficient τ (tau) was statistically significant for both examiners suggesting that each examiner classified the areas with a similar value at the two examinations.

Table 1 Confusion matrix of the intra-examiner reliability of the examiner A
Full size table
Table 2 Confusion matrix of the intra-examiner reliability of the examiner B
Full size table

Tables 3 and 4 present the inter-examiner reliability at the first and second examinations respectively. The agreement was 84.9% for the first examination and 90.6% for the second while kappa coefficient was 0.87 and 0.92, respectively. Kendall’s tau correlation coefficient was statistically significant for both examinations.

Table 3 Inter-examiner reliability at the first examination
Full size table
Table 4 Inter-examiner reliability at the second examination
Full size table

Table 5 presents the unweighted kappa coefficient, the agreement, the sensitivity and the specificity of the comparison between the different cut off points of the ICDAS classification of the ACDS and the ERK systems. It is apparent that ERK 3 presented the higher values of sensitivity for both ICDAS cut off points while ERK2 presented better specificity values for both ICDAS II cut off points. The most balanced agreement was between ICDAS 3 and ERK 3 cut off points. We can observe similar results when we compare the ICDAS classifications of the examiners that evaluated the occlusal surfaces clinically and the histological sections. The results from this comparison are presented in Table 6.

Table 5 Comparison of classification between the automated caries detection system (ACDS) and histological sections for the different cut off points
Full size table
Table 6 Comparison of classification between optical examination of the occlusal surfaces by the examiners and the histological sections for the different cut off points
Full size table

Discussion

The present study estimated the validity of the Automated Caries Diagnosis System (ACDS) (Berdouses et al. 2015) using the histological appearance of the lesions as the golden standard. The ERK classification system was used to classify the histological sections (Ekstrand et al. 1997). The ACDS uses a digital image of the occlusal surface of the tooth and using an algorithm identifies and classifies different areas of the occlusal surface according to the ICDAS II system.

The teeth used in this study were clinically classified by two examiners with occlusal carious lesions having ICDAS scores from 0 to 3. The selection of lesions between ICDAS 1–3 was decided because they pose the most difficult categories of the ICDAS system to be diagnosed correctly and code 0 for comparison as a control.

In the literature there are three suggested methods that use digital pictures and try to classify the occlusal lesions based on ICDAS (Kositbowornchai et al. 2006; Umemori et al. 2010; Ghaedi et al. 2014). All these methods use different optical methodologies to classify the occlusal areas and some of them have limitations in the design of the study. Kositbowornchai et al. (2006) developed an artificial neural network for the detection of simulated dental caries. They used artificial caries to evaluate the method. A significant limitation is the fact that artificial caries lesions are very different in their histochemical composition as they have not gone through the extensive demineralisation/remineralisation process and the absorption of stains as it happens with natural caries in the oral environment. These differences produce different optical characteristics between the artificial and natural occurring carious lesions that obscure the classification of the lesions. Umemori et al. (2010) suggested, an analysis of digital pictures that combines statistical test of 2 way ANOVA and Games-Howell. In this in vivo study the authors did not describe the criteria they used to classify the lesions as sound (C0), caries in the enamel (C1) and caries in dentine (C2). They verified the classification of the areas by opening the lesions with a bur. The authors did not use histological sections of the lesions because it was an in vivo study so they did not prepare areas that were classified as sound. Thus, it was not possible to evaluate if the sound areas were classified correctly. Histological validation was not performed in any of these two studies.

The algorithm proposed by Ghaedi et al. (2014), was very similar with the algorithm that was used for the present ACDS study. The results from the clinical evaluation of both systems are very close, but as with previous studies there is not histological validation for the method proposed by Ghaedi et al. (2014). Therefore, ACDS is the first system that is designed to classify occlusal caries lesions based on a digital image that has been validated in comparison to histological appearance of the lesions.

For the histological classification of the sections the system ERK was used (Ekstrand et al. 1997). The ERK system was chosen because the authors showed that the five (5) score system presented high correlation (rs = 0.87–0.93) between visual and histological evaluations. Moreover, in a more recent study (Jablonski-Momeni et al. 2008), where the ERK classification was compared with the classification proposed by Downer (1975), it was found that they had very similar correlation values. The rs value for the ERK system was found to be between 0.43 and 0.68 and for the Downer system 0.48–0.72.

In order to compare the agreement between the classification of ACDS and ERK systems different cut off points for caries existence were evaluated for both systems (Table 5). The comparison of ERK3 to both ICDAS2 and ICDAS3 presents better sensitivity values than ERK2. On the contrary, ERK2 presents better specificity values compared to both ICDAS 2 and ICDAS 3 than ERK3. It is apparent that the most balanced agreement is when ERK3 is compared to ICDAS 3.

Reduced specificity value suggests that this combination of cut off points falsely classifies an increased number of healthy surfaces as carious. Such combinations are ERK2—ICDAS2 and ERK3—ICDAS2. In clinical practice reduced specificity leads to increased unnecessary treatment which is not acceptable. In the literature it is suggested that in some particular cases though, especially when sound surfaces are given a low ICDAS score not requiring operative treatment, is not as important since the clinical outcome for these surfaces would be to receive preventive treatment which in high risk patients would be beneficial (Jablonski-Momeni et al. 2008). As a principle though, low specificity of any diagnostic technique is not desired for any diagnostic technique.

The results show that the combination ERK3 and ICDAS3 presents the best balance between specificity and sensitivity values. A very interesting finding was a very high sensitivity value (100%). This high value maybe is affected by the fact that there were 8 surfaces classified as ICDAS3 that all were correctly classified based on the histological section. It is very important that the best agreement of ACDS classification and the histological sections have the best agreement at the cut off points ERK3 and ICDAS3. These cut off points in both systems suggest that there is initial cavitation of the lesion and they should receive operative treatment. Therefore, ACDS can classify occlusal areas at the level of initial cavitation correctly when compared with the histological section of the lesion. Similar results are presented in the literature when having compared optical evaluation of the occlusal surfaces by examiners and the histology of the lesions. Jablonski-Momeni et al. (2009), reported 0.71 sensitivity and 0.88 specificity between ICDAS 3 and the histological cut off point of Downer 3 (lesion at the outer half of the dentine). Ekstrand et al. (1997) reported a range of sensitivity values of 0.92–0.97 and specificity values of 0.85–0.93 between the ERK 3 and Ekstrand 3 (same as ICDAS3) cut off points.

The performance of the ACDS has been previously compared with the optical evaluation of the occlusal surfaces by examiners and had very good results (Berdouses et al. 2015). The results showed 80% sensitivity and 96% specificity, precision 82%, accuracy 83% and k = 0.78. The results from this study also support a good performance when ACDS was compared with the histology of the lesions as the golden standard. Both comparisons suggest that the ACDS is a system that can identify occlusal caries with a similar accuracy of an experienced examiner.

Conclusions

ACDS classification of occlusal surfaces based on the ICDAS system are comparable to that of an examiner and with the histology of the lesion. The use of the ACDS system has some important advantages compared to an examiner because it removes the subjectivity of the examiner since the system needs only the picture of the occlusal surface and then performs the classification without any other intervention. ACDS can improve its performance by incorporating into the algorithm future knowledge on image processing or on caries progression and if a larger number of pictures with known ICDAS classification are incorporated into the system.