Jaw and Teeth Segmentation on the Panoramic X-Ray Images for Dental Human Identification

Abstract

Due to the damage to biometric properties in the event of natural disasters, like fire or earthquakes, it is very difficult to identify human remains. As teeth are more durable than other biometric properties, identifying information obtained from them is much more reliable. Therefore, in cases where alternative biometric properties cannot be obtained or used, information taken from teeth may be used to identify a person’s remains. In recent years, many studies have shown how the identification process, previously performed manually by a forensic dental specialist, can be made faster and more reliable with the assistance of computers and technology. In these studies, the x-ray image is subdivided into meaningful parts, including jaws and teeth, and dental properties are extracted and matched. In order to extract the features accurately and ensure better matching, it is important to segment images properly. In this study, (i) lower and upper jaw and (ii) tooth separation was performed to segment panoramic dental x-ray images to assist in identifying human remains. To separate the jaws, a novel meta-heuristic optimization-based model is proposed. To separate teeth, a user-assisted, semi-automatic approach is presented. The proposed methods have been performed with a computer program. The results of the implementation of these methods of jaw and tooth separation in panoramic tooth images are encouraging.

Introduction

Biometric systems play an important role in the identification of individuals according to certain individual characteristics [1]. Fingerprint, face, voice, and iris are some of the biometric features commonly used. Although these features can be used successfully, they are susceptible to damage in accidents or catastrophic events. For this reason, these biometric properties often cannot be used to identify remains in which the body is seriously damaged after incidents such as fire and earthquake. On the other hand, teeth are more durable than the other abovementioned biometric features, and in their absence, identification can be made by a forensic dentist, examining oral and dental properties.

With the development of radiographic medical imaging, medical image processing has become an important tool used in clinical areas including dentistry [2]. In recent years, various studies have been carried out on the practice of identifying human remains using x-ray images to aid forensic dental identification [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17]. Radiographic dental imaging can be grouped into two main categories: intra-oral and extra-oral. Bitewing, periapical, and panoramic x-ray images are commonly used in the dental identification process. Bitewing images are used to examine a certain section of the mouth and show details of some upper and lower teeth. Periapical tooth imaging is performed to see only a few teeth in detail. Bitewing and periapical x-ray images are intra-oral x-rays. Panoramic radiographic images are x-rays in which the whole mouth is viewed in panoramic detail. These images are used to observe many dental disorders, such as tooth and bone anomalies, cysts, tumors, and infections [4].

There are three main imaging processes used in the identification of human remains using dental x-rays: teeth segmentation, tooth extraction, and tooth matching. Segmentation is performed to separate meaningful elements into each image. This is a very important process because the image analysis affects both feature extraction and classification. The subjects investigated in the segmentation stage of dental human identification studies can be separated into preprocessing, lower and upper jaw separation, and tooth separation. Unlike other types of x-rays that are frequently used in dental segmentation for identification of human remains, panoramic x-ray images show the entire mouth. The presence of other bones, such as jaw and nose bones in such images, makes segmentation more difficult. In this study, jaw and tooth separation methods have been proposed in the segmentation of panoramic dental x-ray images for identifying human remains. In this study, a meta-heuristic optimization algorithm was used. To the best of the author’s knowledge, this is the first study that utilizes the meta-heuristic approach in the literature for upper and lower jaw segmentation. A novel method based on particle swarm optimization enables automatic separation of the upper and lower jaws, and the teeth are separated with a user-supported semi-automatic system.

Related Works

Studies on automated human remain identification from dental images are fairly new. The studies forming the basis in this field began in 2003 when Jain et al. presented their first study in automated human remain identification using dental images [5]. In their study, a semi-automatic technique was presented to extract information on teeth shape. To determine best matches, the affine transformation was found, and then, a similarity rating was performed. The part of the image containing the tooth and tooth center was identified by the user. Using a radial scan, they determined tooth boundaries. In the study, they emphasized that identifying tooth shape and position is a suitable method for the purposes of identifying human remains.

In the same year, Abdel-Mottaleb et al. proposed an automated dental identification system and reported the challenges of establishing such a system [6]. In their study, iterative thresholding followed by adaptive thresholding was applied for tooth segmentation. Jaws and teeth were separated using the integral intensity projection technique.

In the study, 58 bitewing x-ray images used as dataset. The images include a total of 196 upper jaw teeth and 181 lower jaw teeth. They reported that 169 and 149 of these teeth were correctly separated, and the success of separation was 85% and 81%, respectively. Panoramic images are not evaluated in this study.

In 2004, Jain and Chen separated jaws and teeth [7] with a similar approach as in [6]. In contrast to the previous study, they designed a curve from the local integral intensity projection points determined during the jaw separation process and used lines perpendicular to this curve for separating the teeth. They found 25 of the 38 query images in the first order. Chen and Jain proposed the directional snake method to determine the contours of the teeth in [8]. They increased the first-order match ratio to about 75% in this study. Nomir et al. proposed Hierarchical Chamfer distance to match teeth features [9]. This method, which they recommended for Bitewing x-rays, was tested on 117 images that contain 391 upper jaw teeth and 361 lower jaw teeth. They managed to separate 329 and 293 of them, respectively. Success rates are 84% for upper jaw teeth and 81% for lower jaw teeth. Said et al. proposed a tooth segmentation method based on mathematical morphology [10]. In their study, some morphological filtering operations were used to improve the segmentation, and they then used connected components in order to obtain the desired region of interest. In this study, both periapical and bitewing type images were segmented. Images contains different number of teeth. Optimality rate, failure rate, and segmentation tables are used to measure success. Their optimality rate is 12%; failure rate is 1.27%. The study distinguishes with its low run time and low failure rate.

The following year, Chen and Jain used the active contour model to extract tooth contour. In addition to dental contour, they used dental works as a feature. They performed matching using tooth contour and dental work contour information. By eliminating the distorted and low-quality images, they increased the rate of finding the right person among the top 8 images to 95%.

In 2011, Frejlichowski and Wanat presented two studies on the segmentation of panoramic dental radiographic images [12, 13]. In the first study, they utilized the watershed algorithm. They performed image enhancement using the Laplacian pyramid decomposition. The jaw separation curve was determined by applying the method of [7] in a selected range. The curve was translated by a certain amount, and the locations where a sharp change was observed were selected as the position passing through the dental pulps. They then located the gaps between the adjacent teeth using the pixel values on the determined curve. Lastly, tooth contours were extracted using the watershed algorithm on the separated tooth [12]. In their second work, they used a similar procedure to find jaws and teeth gaps. In addition to this, they determined the lower parts of the tooth in the image and surrounding the teeth [13]. Segmentation results were compared visually in both studies. They showed that segmentation was better than the methods they compared.

Barboza et al. presented a semi-automatic segmentation method based on image foresting transform to determine which tooth areas to isolate in panoramic x-ray images. With this method, the image formed the basis of a graph and the shortest paths between determined points for segmentation were identified with the assistance of the user. To describe the tooth shapes, they used shape context and beam angle statistics algorithms [14]. They used 40 images taken from 20 different people. The images contain a total of 1126 teeth. Dental segmentation success measurement is not specified. They stated that they increased the first-order matching rate from 48 to 55%.

In 2013, Pushparaj et al. used the Butterworth high-pass filter and homomorphic filter for preprocessing [15]. For segmenting the upper and lower jaws and individual teeth, they utilized integral projection. Connected component labeling was used to extract the tooth shape. They improved the outputs by applying fast connected component labeling. Images were matched using Mahalanobis distance. Segmentation performance was shown visually in the study.

Le and Tran suggested that additional user-assisted parameters may improve the results of clustering algorithms. Based on this, they proposed a semi-supervised and cooperative fuzzy clustering method. They used the Otsu, fuzzy c-means, and entropy regularized fuzzy clustering algorithms together to overcome the limitations of each method [16]. They showed clustering rate as the performance metric.

Ølberg and Goodwin proposed a new dental radiographic image segmentation method that uses the lowest cost path-based method for bitewing dental x-ray images. In their work, to separate the upper and lower jaws, integral intensity projection is used and darkest points on the leftmost and rightmost of the image were selected. Between these two points, they found the lowest cost path. They separated the teeth by applying intensity projection vertically and finding the lowest costed path on points identified. Finally, the morphological active contour without edges (MorphACWE) method was used to determine the tooth boundaries [17]. They succeeded to separate 89.3% of upper teeth and 88.2% of lower teeth correctly with the shortest path method they recommend.

In 2018, Silva et al. reviewed dental x-ray image segmentation studies. In their paper, they also presented a new database of panoramic dental x-ray images. Using this database, they compared the segmentation performance of different segmentation methods [4]. In their study, they made measurements based on the pixel positions they marked manually. They observed accuracy, specificity, precision, and recall (sensitivity) values according to whether the teeth pixels are included in these positions.

Oktay adapted Jain and Chen’s method of jaw separation [7] for panoramic x-rays in her automatic human identification from x-ray image study [22]. In the study, PHOG descriptors are utilized to find tooth positions. In the study, the performance of matching people is presented. Rank 1 matching was 29 out of 36 samples.

Dibeh et al. proposed a new model for jaw and tooth separation in panoramic dental radiography images in 2018 [23]. In their model, a circular mask is created, and this mask is used to find the jaw separator. It is also utilized to extract ROIs in both mandibular and maxillary jaws. Lastly, they created lines from the center of the circle to edges of the circle. Tooth separation is performed according to sum of pixel values on these lines. Segmentation success of the study is shown with segmentation visuals.

Most of the studies mentioned above used the bitewing x-ray images as data. The gap on these studies is that they cannot be apply directly to panoramic x-rays. Panoramic x-rays are frequently used for the detection of diseases. So it is also important for dental identification as data. Another gap of the literature is the limited number of studies for teeth segmentation. Most of the studies are proposed in the automatic dental human identification studies. In these types of studies, it is often given teeth matching performance, instead segmentation performance. In teeth separation studies, count of the correct teeth separation was presented as the performance measurement. For this purpose, the ratio of the number of teeth that can be separated correctly to the total number of teeth is used.

This study presents a new automatic jaw separation and a semi-automatic teeth separation method. It is aimed to meet the need for such methods which are few in the literature. In addition, studies on panoramic x-ray images, which are used more frequently today, are rarer. Accordingly, this study contributes to literature as it provides a new method for panoramic x-ray images.

Materials and Method

Dental Radiographic Images

In this study, 20 images taken from the database presented by Gil Silva et al. [4] were used as the dataset. The dataset consists of panoramic x-ray images 1991 × 1127 in dimension. X-ray images were taken with the ORTHOPHOS XG 5/XG 5 DS/Ceph model x-ray camera, and all of the images which were originally 2440 × 1292 pixels were cropped to exclude images beyond the jaw.

Each person is identified with a different number in the dataset, and each of the 20 images taken was taken from a different sample. In the selection of samples, it was taken into account that all teeth can be found on the jaws in the x-ray image data. The selection was made by randomly taking the first 20 images.

In the study [4] where the database is presented, manually prepared tooth pixel locations are used for performance measurement. They measured the segmentation performance by evaluating the suitability of the pixels in the image found as a result of segmentation to these positions. They used accuracy, specificity, precision, and recall (sensitivity) values as the performance metrics.

Proposed Method

In the process of identifying human using dental radiographic images, teeth should be segmented as the first step in extracting tooth shape information and properties. For segmentation, the lower and upper jaw should first be separated correctly. Accurate separation of each jaw section is important to make the separation and classification of teeth more accurate. After lower and upper jaw segmentation is performed, each tooth on the jaw should be separated from each other. It is important that the jaws and teeth are well segmented so that information on the shape and properties can be obtained correctly in later stages. In this study, a preprocessing operation was applied whereby the upper and lower jaw were separated using particle swarm optimization (PSO), and then, the teeth were separated.

Preprocessing

In dental image segmentation applications, preprocessing is applied to clarify the differences in the image and distinguish teeth, soft tissue, jaw, and gaps from one another by enhancing the contrast in the same region. In this study, the homomorphic and Butterworth filters were used in preprocessing, similar to [15]. After these filters are applied, the preprocessing step was completed by increasing the contrast in the image.

Separating Upper and Lower Jaws

There are different approaches in the literature for the separation of the upper and lower jaws. In some studies, this step was not performed and the tooth was selected directly and manually entered by the user [5, 18]. In another approach, the initial positions for the best separation line were manually selected to detect the jaw separation gaps [7], and a probability function was used to separate the jaws using depth and position information in a horizontal intensity projection histogram. The image was divided into vertical strips, and this process was applied on each strip. Finally, a spline curve was generated from the separation locations determined. In another commonly used method [6], the line on the image with minimum intensity passing through the jaws was selected by rotating the image at certain angles.

In a more recently proposed study [17], the horizontal intensity projection at the beginning and end of the image was selected from the darkest points as the start and end positions, and the lowest cost and shortest path between these points were selected. The methods in these studies have been successfully used for bitewing images. However, some of these algorithms may need user assistance. Due to the nature of panoramic images, there are non-tooth portions and low-valued lines passing through the outside the jaw, so this cannot be applied on panoramic radiographic images directly.

In this study, no starting point is needed for the proposed jaw separation method. The curve fitting method is used to find the required jaw separation curves. In determining the best curve, a meta-heuristic optimization algorithm and particle swarm optimization algorithm [19] is used.

Curve Fitting with Least Square Method

In the curve fitting algorithm, a variable matrix consisting of the x values of the selected points for the nth degree polynomial is created. In determining such a matrix, the number of columns will be n + 1 and the number of rows will be the number of selected points. An output matrix is created using the y values of the selected points. Polynomial coefficients are obtained by solving the system of linear equations in Eq. (1) [20].

$$ \left[\begin{array}{c}{x}_1^n\kern0.75em {x}_1^{n-1}\kern0.75em \dots \kern0.75em 1\\ {}{x}_2^n\kern0.75em {x}_2^{n-1}\kern0.75em \dots \kern0.75em 1\\ {}\vdots \\ {}{x}_m^n\kern0.75em {x}_m^{n-1}\kern0.75em \dots \kern0.75em 1\end{array}\right]\left[\begin{array}{c}{p}_1\\ {}{p}_2\\ {}\vdots \\ {}{p}_{n+1}\end{array}\right]=\left[\begin{array}{c}{y}_1\\ {}{y}_2\\ {}\vdots \\ {}{y}_m\end{array}\right] $$

(1)

Particle Swarm Optimization

The particle swarm optimization algorithm is proposed to solve non-linear problems. This method imitates the behavior of swarms, such as flocks of bird and schools of fish, and searches for the best solutions given certain parameters. The method aims to identify an optimal solution for each new population after a randomly selected first population. While the word “particle” refers to each individual solution (bird, fish, etc.), a “swarm” refers to the collection of each individual. In the algorithm, each individual has position X and position change, that is, velocity V. The velocity variation can be limited to a value, such as V_max.

Depending on the performance of the function, the optimal value particle of the swarm is called the “global best” (gbest). The best value of each particle in previous iterations is called the “personal best” (pbest). Initially, position and velocity information is randomly selected according to Eqs. (2) and (3).

$$ {X}_{\mathrm{initial}}={X}_{\mathrm{min}}+\left({X}_{\mathrm{max}}-{X}_{\mathrm{min}}\right)\times \mathrm{random}\_\mathrm{variable}\left(0,1\right) $$

(2)

$$ {V}_i={V}_{\mathrm{max}}\times \mathrm{random}\_\mathrm{variable}\left(0,1\right) $$

(3)

These values are updated with each iteration. Current velocity information is required to update the position information. This information is determined by the formulas in Eqs. (4) and (5).

$$ {V}_{ij}^{t+1}=w{V}_{ij}^t+{c}_1{r}_1\left({P}_{ij}-{X}_{ij}\right)+{c}_2{r}_2\left({g}_{\mathrm{best}}-{X}_{ij}\right) $$

(4)

$$ {X}_{ij}^{t+1}={X}_{ij}^t+{V}_{ij}^{t+1} $$

(5)

The variables of the algorithm in these equations are “r₁” and “r₂.” These are values ranging from [0, 1]. Among other values, w is inertia weight, and c₁ and c₂ are acceleration constants. “$w{V}_{ij}^t$” in the equation prevents the sudden change in velocity from the previous value. Other operations in the equation provide for personal best and global best [21].

In the proposed method, the region of interest is narrowed. This prevents the background density portions in the image, like nasal bone or the lower-upper parts of the jay, from misleading the algorithm. The gap separating the jaws are approximately found in the middle of the images. Therefore, the region of interest is isolated by narrowing the upper and lower parts. The narrowing ratio was determined experimentally and was chosen to be 35%. In this selected range, the particle swarm optimization algorithm is used to determine the curve that best separates the jaws.

Random points were selected on the image to be optimized with the PSO algorithm. The number of random points was determined experimentally and obtained by dividing the image width by 64. Each random point has two-dimensional position values consisting of (x, y) values. A cost function is required for the PSO algorithm. In this study, the least squares algorithm and pixel intensity values are used to produce cost function.

The curve is initially fitted to the randomly generated points by the least squares method. Once the curve is determined, it is used to calculate the cost.

To calculate the cost, the coordinates of all points on the image where the polynomial curve crosses are determined. The total intensity value is found by summing the intensity values in each of the specified coordinates. This sum of intensity value is taken as the cost result of the determined third-degree polynomial curve. Since the pixels in the jaws gap area are dark-colored, the separation curve should pass over the dark pixels. Based on this, the PSO algorithm aims to minimize the polynomial cost, and the most suitable curve that can be used to separate the jaws is determined. The lower and upper limit parameters for particle swarm optimization were selected as the initial and end positions of the region of interest obtained by the narrowing process. The maximum number of iterations chosen is 1500; the population size is 100, and the acceleration constant values (c₁, c₂) are 1.49.

Teeth Separation

After the upper and lower jaws are separated, individual teeth need to be separated. When using panoramic x-ray images, firstly lower and upper separation limit curves should be determined, which comprise the lower or upper parts of teeth. To find these curves, a method similar to [13] was used. In the proposed method, the previously defined jaw separation curve is slightly translated, and the sum of pixel intensities obtained from each shift is examined. The first of the areas with a significant decrease in intensity is selected as the location of the curve through the tooth pulps. The second location with a significant decrease in intensity is selected as the lower or upper limit of the teeth.

In this study, the curve which separates the jaws is translated and sum of pixel intensity values of the curve lines is calculated. Then, peaks are found within the determined sum of pixel intensity values. Peaks that have a certain minimum distance and minimum threshold are selected to avoid side-by-side or low-rise peaks. The curve corresponding the last peak value in the selected peaks was selected as the lower or upper limit of the teeth. Figure 1 shows the sample peak identified for the selection of the lower limit curve.

Fig. 1

Sum of intensity values of the curves and selected peak

To separate the individual teeth, a semi-automatic method is proposed. In the proposed method, contrary to similar studies in the literature, the vertical intensity projection method was not used in order to prevent extra separation lines. When using this method, the user should select the points between teeth. Then, perpendicular lines are determined, similar to the method used in bitewing images [7]. These lines are drawn from each point selected by the user towards the upper or lower separation curves and perpendicular to the main jaw separation curve polynomial at the selected point. Since the orientation of all teeth is not always perpendicular to the determined polynomial curve, detected lines may not be sufficient to separate all teeth. Therefore, the direction of the determined lines needs to be optimized. For this purpose, the position of the determined vertical lines’ endpoint is shifted in the range of − 30 and 30 pixels in the x-axis direction. In the specified range, the line with the lowest average intensity cost per pixel is determined to be the tooth separation line. Finally, since the teeth are very close to each other, when each tooth region is extracted individually, the separation lines are translated by a small amount (e.g., [− 15, 15]) to avoid missing the limits.

The meta-heuristic approach proposed in the study is a new approach proposed for jaw separation. To the best of author’s knowledge, a meta-heuristic method for jaw separation has not been used before in the literature. By presenting a new semi-automatic method for tooth separation, it was aimed to achieve high teeth separation success.

Experimental Results

In order to evaluate the success of the proposed method, a computer program was built in the MATLAB environment. The results of the jaw separation performed automatically with the help of the PSO algorithm were evaluated on 20 images. There are 32 teeth, 16 in the maxillary jaw and 16 in the mandibular jaw, in each of the 20 images used. All images containing a total of 320 teeth. Sample jaw separation results are shown in Fig. 3.

The purpose of separating the upper and lower jaw is to enable the teeth of the lower jaw and the teeth of the upper jaw to be processed separately from each other. Therefore, the teeth of the lower jaw should be located in the lower region of the separation curve together with the lower jaw, and the upper jaw teeth should be located along the upper jaw at the top of the separation curve after jaw separation. Examples of correct and incorrect separations are shown in Fig. 2.

Fig. 2

Correct and incorrect separation examples

Performance in the studies presented in the literature is expressed according to whether the jaw separation is made correctly on x-ray images. Therefore, jaw separation performance in the proposed study is measured as the ratio of the number of correctly separated jaw images to the number of all images.

As a result of applying the proposed method to the dataset, upper and lower jaws are separated successfully on 19 (95%) of 20 x-ray images (Fig. 3).

Fig. 3

Example separation of mandibular and maxillary jaws

Example results of tooth extraction are shown in Fig. 4. In Fig. 4, the lower boundaries of the teeth are shown using an orange line and the jaw separation line is shown in blue. The blue dashed lines indicate lines perpendicular to the jaw separation line; the other colored lines indicate the improved direction of these lines.

Fig. 4

Teeth separation examples

Datasets are generally not publicly available for dental x-ray image processing studies on teeth separation and human identification. Studies in the literature, mostly, measure their performance in their own datasets. So, there is no common dataset used in jaw and tooth separation studies in dental radiography images. Panoramic x-ray datasets are not publicly available in the studies in which the proposed study was compared. Therefore, it is difficult to compare the studies fairly. In the study, to compare the proposed method, the studies of Jain and Chen [7] and Dibeh et al. [23] have been implemented by us. These implemented methods were tested on images in the dataset that used in the proposed study.

Jain and Chen’s work are commonly utilized in the works in subsequent years, such as [12, 22]. This study is one of the pioneers of automatic dental human identification using dental x-ray image studies. It is important for comparison.

To the best of author’s knowledge, Dibeh et al. [23] is the most recent study on jaws and teeth separation. Therefore, these two studies were selected for comparison.

In Jain and Chen’s work [7], a probabilistic approach to jaw separation was used. The same approach is also used in [12, 22] on panoramic x-rays. First point is selected manually. Using this point, the first separation point with the highest probability is determined. Images were divided into many vertical strips and a new point is extracted on these strips.

Finally, a spline curve is created with the extracted points to separate the jaws. The pixel values of the points on the lines perpendicular to the spline curve are summed. Lines with low brightness value sums are determined as lines separating teeth.

Oktay [22] adapted the recommended jaw separation method to panoramic radiography images. The starting point automatically selected from the middle part where the nose bone and bite disc are located in the image. By moving from the selected point to the beginning and end of the image, the points that will form the separation curve are extracted. Since panoramic radiography images are processed in the prepared application for comparison, Oktay’s [22] customization has been used.

In the other study selected for comparison considering the structure of panoramic radiography images, a circular mask is used. By moving circular mask on the image, most suitable points for jaw separation are extracted. A threshold value is used to the selection of points. Again, by using the mask, the lower part of the mandibular jaw and the upper part of the maxillary teeth were determined and removed from the image.

In order to separate the teeth, outward lines are created from the circle center and the pixel values of the points that the lines pass over are summed. Separation lines were determined by finding the global minimum values between the local maximums with a certain distance.

Some of jaw separation samples is shown in Figs. 5 and 6.

Fig. 5

Example jaw separation results: a, b proposed, c, d Jain and Chen [7] and Oktay [22], and e, f Dibeh [23]

Fig. 6

Example jaw separation results: a, b proposed, c, d Jain and Chen [7] and Oktay [22], and e, f Dibeh [23]

Since the studies for comparison are used in a different database, some differences were required in the parameters. For example, in [23], 1/4 and 5/6 values are used as crop values in the ROI extraction. However, this range is not suitable for the images used in this study. Therefore, different suitable values are used for some fixed parameters. Mandibular teeth separation comparison images are shown in Fig. 7. Maxillary teeth separation comparison images can be seen in Fig. 8.

Fig. 7

Example mandibular teeth separation results: a, b Dibeh [23], c, d Jain and Chen [7], and e, f proposed

Fig. 8

Example maxillary teeth separation results: a, b Dibeh [23], c, d Jain and Chen [7], and e, f Proposed

The success of the jaw separation process is measured by the separation curve passing through the gap between the lower jaw and upper jaw and not passing over the teeth. In order to measure the success of jaw separation quantitively, the number of teeth that are correctly separated in each jaw can be examined. So, for the jaw separation comparisons, the number of teeth that the separation curves can separate correctly was examined in three methods. The results are seen in Fig. 9. In Fig. 9, the x-axis shows the number label of the x-ray image and the y-axis shows the number of teeth correctly separated after jaw separation. For example, in Fig. 8, after the jaw separation, the separation curve provided completely correct separation and did not incorrectly separate any teeth. In the same image, Jain and Chen’s [7] jaw separation curve has passed over 3 teeth, so the number of teeth it can separate correctly is 29. Similarly, 31 teeth were separated correctly using the Dibeh [7] method on Fig. 8.

Fig. 9

Jaw separation result based on correctly separated teeth numbers

The separation rate results in Table 1 calculated as dividing the number of correctly separated teeth values on both jaws to number of all teeth. When the jaw separation comparisons are examined, it has been seen that the proposed method has better rate in jaw separation. All methods achieved over 90% success. Dibeh [23] has the close accuracy rate to the proposed work. Jain and Chen [7] with Oktay [22] approach has also high rate as 94%. It was also observed that the separation curve in the proposed study did not pass over any teeth in all images except one x-ray image.

Table 1 Separation rate of jaw separation curves based on correctly separated teeth numbers

In order to evaluate tooth separation success, the number of correct tooth separation lines and accuracy of these lines were examined. In x-ray images, there are no missing teeth in both the mandibular and maxillary jaws. In Fig. 10, the correct separator numbers were compared for each mandibular jaw image. In Fig. 10, the x-axis shows the image number, and y-axis shows the correct line count for each method.

Fig. 10

Correct mandibular teeth separator counts

In Fig. 11, the numbers of incorrect separation lines were compared for each lower jaw image. Separation curves that are not in the required position are counted as false separators. For example, passing through the middle of the tooth or situations like two different separation lines that separate between the same two teeth are false separators.

Fig. 11

Incorrect mandibular teeth separator counts

In Fig. 12, excess or missing separation lines are compared for each mandibular jaw image. Each jaw has 16 teeth. It is needed to separate each tooth. The minimum number of separators required for such a separation process is 17. So it is expected that there will be 17 separation lines for each jaw. The line counts are summed for each image using the values from Figs. 10 and 11. If there are fewer than 17 lines, it can be said that there is a missing separator line. Similarly, in the case of more than 17 separation lines, that means more separators then it should have been were found. In Fig. 12, negative values indicate missing line counts, and positive values indicate extra formed line counts. For example, in Jain and Chen’s method [7], 3 extra lines were found for image number 6. For same image, in Dibeh’s method [23], 5 separators out of 17 expected separators were not found. Since there is no missing or excessive separator in the proposed method, its value in the chart is 0. When all values are summed in Figs. 10, 11, and 12 for each image, it can be noticed that sum of values are 17.

Fig. 12

Excess or missing mandibular teeth separator counts

Comparison of the correct number of separators for each maxillary jaw image is shown at Fig. 13. Comparison of the incorrect separator lines for maxillary images is shown in Fig. 14. Finally, In Fig. 15, excess or missing separation lines are compared for each maxillary jaw image.

Fig. 13

Correctly produced maxillary teeth separator counts

Fig. 14

Incorrectly produced maxillary teeth separator counts

Fig. 15

Excess or missing mandibular teeth separator counts

Looking at the comparisons for both the lower and upper jaw for the dataset used, it is seen that the correct line rate of the proposed study is higher than the others. It is seen that the most separation line is produced in Jain and Chen’s [7] method. Although this increases the correct separation counts in the Jain and Chen’s [7] method, it causes extra and incorrect separation line counts to be high. In Dibeh’s method [23], the number of incorrect and excessive separator lines is low. However, in this study, it is seen that the number of missing separation lines is high.

A total of 17 separation lines are expected for each jaw. Therefore, 340 teeth separation lines will be created for 20 images. Tables 2 and 3 show how many of the 340 separator lines expected were obtained. For example, in the proposed method, 316 separators are produced correctly in the correct locations that are aimed to be produced.

Table 2 Sum of mandibular jaw lines for all images

Table 3 Sum of maxillary jaw lines for all images

The ratio of the number of correct separators produced to the expected number of separators (316/340 = 0.9294) gives the correct separator rate. Correct teeth separator rates for all images are shown in Table 4.

Table 4 Correct teeth separator rates for all images

In Table 5, considering the missing and excessive separator lines, the accuracy rate of the separator lines found was calculated. For example, in the Jain and Chen’s [7] method, 278 separators are generated correctly in the correct positions. In addition, 85 separating lines produced in faulty positions. Apart from these, 23 extra separator lines were found. So, excess separators were also included (278/(340 + 23) = 0.7658) in the calculation of the accuracy rates of the separation lines.

Table 5 Accuracy rates of obtained teeth separators

For the measurement of tooth separation performance in the literature, (i) visual separation results [6, 9, 12, 13, 15], (ii) the ratio of correctly separated teeth to total teeth [6, 9], [17], and (iii) optimality rate, failure rate metrics [10] is used. The study that the dataset is presented uses accuracy, precision, recall values [4] as the performance metrics. The correct teeth separation performances of the studies are shown in Table 8. Here, the success of accurately separating the teeth is shown instead of the teeth separation lines.

Optimality rate and failure rate metrics are based on the correct number of determinations of the objects in the images. It is more descriptive in cases where the number of objects searched for images is different. Optimality rate and failure rate metrics are shown, respectively, in Eqs. (6) and (7). Optimality rate indicates the presence of all objects in the image, and the failure rate indicates the situation where no objects can be found. Please see [10] for details.

$$ \mathrm{Optimality}\ \mathrm{rate}=100\times \frac{\sum_{i=1}^N{P}_{ii}{F}_i}{\sum_{i=1}^N{F}_i^2} $$

(6)

$$ Failure\ rate=100\ast \frac{\sum_{i=1}^N{P}_{0i}{F}_i}{\sum_{i=1}^N{F}_i^2} $$

(7)

In the method expressed in [10], the number of objects contained in the image and the number of objects found by the algorithm are shown in a table. Each row in the table contains the number of objects found by the algorithm. Each column contains the number of objects actually present in the image.

In the database used in this study, each jaw (upper-lower) image contains 16 tooth objects. For this reason, all columns will have the value 0 except column 16. Tables 6 and 7 show the 16th column values of the proposed study; [7, 23] studies to compare the studies. The first column in the table indicates the number of teeth. The other columns indicate the number of images that contain as many objects as in the first column. For example, in the first line, the number of images that algorithms find 16 objects are 9, 0, and 0, respectively. The sum of the values in the rows in the algorithm columns should be 20, which is the total number of images.

Table 6 Comparison of the algorithms by the number of objects found (Mandibular jaws)

Table 7 Comparison of the algorithms by the number of objects found (Maxillary jaws)

According to Tables 6 and 7, for mandibular teeth, the optimality rate value for the proposed study is 0.45 while the optimality rate value for the studies [7, 23] is 0. While the optimality rate value for the proposed study is 0.15, and it is 0 for [7, 23] for maxillary teeth. Failure rate value was 0 for both the lower and upper jaw in the proposed study and [23]; in [7], it is 0.1 for lower jaw and 0 for upper jaw. Failure rate value is 0 for both the lower and upper jaw in the proposed study and [23]. In [7], it is 0.1 for lower jaw and 0 for upper jaw.

The accuracy, specificity, precision, sensitivity, and F-score evaluations of the separators found by the algorithms were made. In the study in [4], algorithms were evaluated based on pixels. However, in the proposed study, separation operation is performed. Therefore, separation lines are used as the true and false sources required for the evaluations. Table 9 describes positive, negative, true positive, false positive, true negative, and false negative values in the evaluation. Since true negative value is not clear, it is determined as follows. In the study proposed in [23], a minimum distance was used when teeth were separated. The minimum distance of 30 pixels for our database gives the best results (Tables 8 and 9). So, a minimum tooth size of 30 pixels has been chosen. Accordingly, by dividing the image width by 30, the maximum number of separators that can be found is determined. This value was used while finding true negative values.

Table 8 Total number of teeth separated by algorithms

Table 9 Measurement metrics

Table 10 TP, FP, TN, and FN measurements of algorithms for mandibular teeth

Table 11 TP, FP, TN, and FN measurements of algorithms for maxillary teeth

Table 12 Accuracy, specificity, precision, sensitivity, and F-score measurement of algorithms for mandibular teeth

Table 13 Accuracy, specificity, precision, sensitivity, and F-score measurement of algorithms for maxillary teeth

Table 14 Descriptive statistics of correct teeth lines and jaw separations

Table 15 Mauchly’s test of sphericity

Table 16 Tests of within-subject effects

Tables 10 and 11 show TP, FP, TN, and FN measurements of algorithms, respectively, for mandibular and maxillary teeth. Using these values, accuracy, specificity, precision, sensitivity, and F-score measurements are calculated and shown in Tables 12 and 13.

All three algorithms compared statistically using repeated measures ANOVA. To make statistical analysis, (i) correct separator lines for mandibular teeth data, (ii) correct separator lines for maxillary teeth data, and (iii) jaw separation according to teeth numbers data are used. Table 14 contains descriptive analysis of 3 algorithms for mandibular teeth, maxillary teeth, and jaw separation data. Table 15 shows Mauchly’s test of sphericity. In the table, it is seen that the sig. value for mandibular teeth is above 0.005 and below for other data. For this reason, in analyses shown in Table 15, sphericity assumed value is examined for mandibular teeth and Greenhouse-Geiser value is examined for other data. Table 16 contains the tests of within-subject effects.

In the proposed study, a semi-automatic method is presented in order to minimize false separation lines and not to create extra tooth separation lines. In this way, no extra tooth separation line is produced.

In the Dibeh method [23], the number of separating lines is low; however, it is observed that most of the lines found are in the correct separation positions. More separators were found in [7] than [23] method; consequently, the number of correct separators obtained is high. However, in this method, extra lines reduce the accuracy rate of correct line separators.

Tables 2 and 3 show that the success of separating the mandibular teeth in all three methods was higher than the success in separating the maxillary teeth. It is seen that all three methods are below 90% success in separating maxillary teeth.

When the comparisons are examined, it is seen that the proposed method on the dataset used gives better results in terms of both jaw separation and tooth separation operations compared with other methods. Best accuracy values are obtained using proposed method. Method in [23] has best specificity values both mandibular and maxillary teeth. It has also best precision for maxillary teeth. Best recall values obtained using proposed method and [7].

Discussion and Future Works

In this study, two new methods are proposed for jaw and tooth separation, which are two sub-steps of the segmentation procedures. The proposed methods have been applied to panoramic dental x-ray images. To the best of author’s knowledge, a meta-heuristic optimization method is used in literature to separate upper and lower jaws. The advantage of this method is that it does not need additional information when searching around for an approximately identifiable location. However, errors may occur when images have areas with high noise. A semi-automatic approach proposed for tooth separation makes the method advantageous against problems such as excessive separation lines or incorrectly defined separation points. On the other hand, the disadvantage of the proposed method is that it is not a fully automatic method and is not resistant to faulty positions potentially entered by the user. In order to observe the performance of the study, comparisons were made with two different studies. The proposed study was found to be more successful than other studies in terms of tooth separation. Possible reasons for this situation can be listed as follows: (i) Since the proposed tooth separation method is a semi-automatic method, it contains low false positive, (ii) algorithms proposed for different databases are tested on a new database, (iii) the appropriate parameters specified for the different database are not compatible for the new database, (iv) circular mask idea presented in the study [23] is weak against different oriented teeth, and (v) the method proposed in study [7] is weak against the difficulties in panoramic images. The common weakness of all three algorithms uses lines for tooth separation. Curved separators will be more useful according to the teeth shapes, especially for tight teeth.

Since the databases of these studies are not publicly presented, these studies have been implemented by us. The database used in the proposed study was tested for these two studies, and the results were obtained. Although this kind of comparison is not very fair, it is the best comparison method since the databases are different used in these works. In future studies, the aim is to make tooth separation fully automated.

Conclusion

When other biometric properties are unavailable, information gathered from teeth can be used to identify the remains of a person. This process is realized manually by a forensic dental specialist. In recent years, various studies have been presented to identify remains using dental x-ray images with the aid of computers and technology [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18]. In these studies, generally, x-ray images are first segmented, and then, tooth position and shape are determined from dental x-ray images. Using this information, the most similar match for the teeth is found in a database. The first important step of identifying a person from dental images is jaw and tooth segmentation. Studies prepared in this direction in the literature are not at a sufficient level yet. In addition to this, most studies were carried out on bitewing type images Segmentation studies are rarely seen in panoramic x-ray images, which are frequently used for diagnosis of diseases today. Success rates are not at the desired level yet. Therefore, more segmentation studies are needed in panoramic x-ray images. In this paper, two new methods for jaw and tooth separation are proposed to segment panoramic dental x-ray images. In the study, the particle swarm optimization algorithm, which is a meta-heuristic optimization method, is used for jaw separation. Initially, randomly selected points on the image are optimized using the PSO algorithm to find the polynomial curve that best separates the jaws. A semi-automatic method is also proposed for the separation of teeth. In this method, the lines separating the teeth are determined using the separation points selected by the user. Experimental studies have indicated that the proposed methods can be used to segment jaws and teeth in dental x-ray segmentations.