Introduction

Complex bone injury, such as comminuted fractureFootnote 1 is the most common consequence of severe accidents (falling from heights, for example) [3, 28]. Therefore, Computed Tomographic (CT) images are crucial for such a study. To analyze and diagnose the serious bone trauma that help decide recovery plan, it requires a series of 2D X-ray images [17]. Surgeons recommend CT stack – a set of several CT slices – since it preserves the actual anatomic structure of scanned organs [8, 13]. CT images are detailed cross-sectional (tomographic) images. This means that it provides anatomically accurate information about the area under supervision without cutting.

It is not trivial to segment of bone tissues from surrounded artifacts and to accurately identify fractured pieces from CT images [16]. Further, the bone is made up of two types of tissues: i) cortical tissue and ii) cancellous tissue. Cortical tissues are present at the outer part of the bone. They are dense and provide strength to the bone. Cancellous tissues are present in the inner part of the bone and they are spongy. We observe that both the tissues show intensity variation over the slices. At the same time, exact same tissue can have different intensity values over the slices. As an example, Fig. 1 shows the intensity variation in cortical tissues over the slices in the same CT stack. Near diaphysis (i.e., at the shaft), the cortical tissues are brighter and thicker as shown in Fig. 1a and therefore, the fractured pieces can be easily identified. While, on moving nearer to joints (i.e., at epiphysis), they become thinner and fuzzy as shown in Fig. 1b. In some slices, they are invisible. This means that the joint area is more fracture prone than shaft. In such a context, fracture line detection can be a tedious job. Like we have mentioned earlier, intensity values of cancellous tissues are lesser than CT range and as a consequence, similarity with the values of surrounded soft tissues can happen. Therefore, it is challenging to accurately identify and locate fractured bone pieces in joint area [2]. Due to which, more often, we have over-segmentation or incomplete segmentation.

Fig. 1
figure 1

CT images from the same patient having fractures: a diaphysis b epiphysis

Full size image

Complex anatomical structure of bones, fuzzy fracture line, arbitrary shape and size of fractured bone pieces, dislocation and intensity variation are the few challenges in the segmentation process. In addition, experts are required to segment and label each fractured bone pieces since in complicated fracture cases, bone pieces may get wrongly connected due to dislocation or due to proximity and resolution of CT images. To find a practical solution and to minimize the expert’s intervention, orthopedics show their interest towards the developing the automated tools that can segment and label each fractured bone piece by considering patient-specific bone anatomy.

The proposed work presents a reliable framework that will automatically segment and label fractured bone pieces by considering patient-specific bone anatomy. It is composed of three steps. The first step is unwanted artifacts removal. In this step, a robust CT image pre-processing technique is devised that is based on contrast stretching and histogram modeling. The technique effectively removes the unwanted artifacts, such as soft tissue (flesh) surrounded by bone tissue in addition to bone region enhancement. In the second step, eight connected component-based segmentation method is adapted to segment each fractured bone. In the third step, an innovative tree structure-based hierarchical labeling method is applied to assign a unique label to each fracture piece by considering bone anatomy.

This paper is organized as follows. Other methods developed to segment fractured bones are discussed in the “Related works”. “Proposed framework” describes the design and development of proposed framework and it includes pre-processing, segmentation and label assignment techniques that are respectively used to remove unwanted artifacts such as flesh, to segment (healthy/fractured) regions, and to assign unique label each segmented region. “Results” provides a comparison of proposed technique with other previous methods in addition to the clinical ground truth. Further, it provides the results of a proposed method on real patient-specific images. “Conclusion and future work” concludes the paper.

Related works

In the literature, several successful researches have been made to segment healthy as well as fractured bones from CT images. However, type of bones and its anatomy, intensity variation of the same tissues over the slices, resolution of CT images are a few challenges to segment healthy bone [28]. In addition, to segment fractured bone pieces, where dislocation and wrong connection of dislocated pieces can make problem more complicated [16]. The previous solutions may not be generic enough so that it can work for all cases. For a quick review, some segmentation techniques that are used to segment medical images are enlisted in [17, 20].

Tomazevic et al. [27] proposed a manual and semi-automatic method to segment bone tissue interactively. They adapted thresholding-based segmentation technique to detect bone tissue from CT images. Tanssani et al. [25] used a global fixed threshold value to segment fracture bone tissue. However, identifying such value is hard due intensity variation problem over the slices. A reasonable threshold-based segmentation technique is adapted to perform binarization of CT image having humerus bone [7]. Several morphological operations and Gabor wavelet transform are used to extract the cortical and trabecular tissues from the studied image.

In addition to adapting different variations in thresholding-based segmentation approaches, more often, authors used 2D [3, 6, 16] or 3D [11, 12] region growing technique to segment and label fracture bones. Paulano et al. [16] used the 2D region growing approach to segment and label fractured bone pieces from CT images. In their technique, initially, curvature flow filter is applied to each CT slice to enhance bone region boundaries. Several seeds (i.e., one seed per fracture piece) are then accepted from user and region growing algorithm is executed to segment and label each fractured bone piece. Also, new seed is accepted to separate erroneously connected fractured bone pieces. Lee et. al. [14] used multi-region growing approach, where it scans the entire image and identifies several pixels (one pixel per region) as seed points whose intensity value is above a specified threshold. If in case a multi-region approach fails, manual region-segmentation is then used to separate incorrectly joined fractured bone pieces and region merging approach is used to combine over-segmented regions.

To improve the accuracy and to minimize the user interventions, few authors used a sheetness measure, which is based on image enhancement technique for cortical and cancellous bone identification (separately) [3, 6]. The enhanced images are then segmented by applying 3D region growing algorithm. Fornaro et al. [3] used an interactive graph-cut method to improve weak edges and to separate wrongly connected bone pieces. At the end, unique labels are assigned using 3D connected labeling algorithm. Huang et al. [9] adapted a constrained 3D region growing algorithm to extract the bone regions with correct bone counters. To avoid over-segmentation and to get proper bone boundaries, an appropriate threshold value needs to be identified (via the iteration process) for every slice, which is time consuming and expensive. Other than this, deformable models [4, 21], probabilistic watershed transform [22, 23], registration-based models [15] are used to segment the fractured bone piecess from CT images.

In summary, the previously reported techniques used thresholding and region growing-based segmentation algorithms to segment and label the fractured bone pieces from CT images. Commercially available medical image analysis applications like 3D slicer, DICOM Viewer, Dolphin and InVesalius are also no exception to this. In such applications, the bone region segmentation task is based on user-specified threshold values [18, 24]. In addition, these applications provide eraser and filler like tools to separate incorrectly joined fracture bone pieces and to connect fuzzy boundaries respectively. The commercial tools and most of the above-discussed techniques require a lot of user interventions, such as specifying threshold value or seed points, to achieve the desired output. In the literature, no attempts were made to provide online medical imaging dataset for computer aided diagnosis (CAD) system development for bone fracture detection and analysis. They have integrated with collaborative environment: medical and engineering programs. In addition, fewer attempts are made to develop an effective pre-processing technique that will remove all unwanted artifacts like flesh, CT bed, cables, which will help to enhance the bone regions and their boundaries. The labeling logic also has a limitation, if certain CT slice contains more than one bone region they may consider as fractured pieces. To overcome these limitations, we developed a framework which will segment and assigns a unique label automatically and accurately by considering patient-specific bone anatomy. In the next section, the design and development of the proposed framework is described in detail.

Proposed framework

Data acquisition

Developing CAD system or a Virtual Reality (VR) -based simulators for skill acquisition and training is an emerging field in orthopedics. Patient specific data collection in the form of medical images like X-ray, CT or Magnetic Resonance Imaging (MRI) is the initial step in expert system development [19]. Hospitals and radiology centers, due to legal issues, cannot share data (even for research purpose, which is not expected [26]. To overcome this difficulty, in our study, twenty-eight patient-specific CT stacks were collected from several radiology centers and hospitals: Prism Medical Diagnostics lab, Chhatrapati Shivaji Maharaj Sarvopachar Ruganalay and Ashwini Hospital (in India). Also, we have collected CT data from different sources (different CT scan machines) with different specifications and have annotated by the experts. Such annotations help measure the performance of the techniques/tools.

Each CT stack is a collection of hundreds of CT slices (ranging from one hundred to six hundred). The number of slices per stack depends upon the area under supervision, the severity of the injury, the thickness of each slice and distance between two slices. In total, 8000 DICOMFootnote 2 images are collected. Each CT image is of 512 × 512 pixels size. DICOM file is made up of two sections: i) header and ii) data. The header section contains patient-specific information like a patient number, name, and age whereas data section contains a scanned portion. Following ethical laws like HIPPAFootnote 3 laws and/or IRBFootnote 4 protocol, we do not disclose patient-specific information. We employed the MATLAB script discussed in [10] to remove patient-specific information from CT image. This means that the script uses pre-defined function dicomanon (part of the MATLAB Image Processing Toolbox) to remove confidential medical data from a DICOM file. Our collection can be provided for research purpose (available upon request).

Data annotation

In this study, expert orthopedic surgeons and radiologists annotated every tenth slice in each CT stack. The primary reason behind annotating every tenth slice is experts do not find difference in CT images (especially fractured regions) for at least ten slices. The observations are there was a marginal change in size of fracture bones/components and in their positions. The sample slices are shown in Figs. 2 and 3. Figure 2 shows ith and i + 10th slice in cortical tissue region, where we can observe that fractured bone pieces remain in the same location. Whereas in Fig. 3, ith and i + 10th slice in cancellous tissue region shows little growth in size of fractured components. During annotation process, only outer cortex of bone regions is highlighted because while dealing with fracture, experts are interested in identifying dis-connectivity in bone outer cortex.

Fig. 2
figure 2

CT images having fracture at cortical tissue region a ith and b i + 10th slice

Full size image
Fig. 3
figure 3

CT images having fracture at cancellous tissue region a ith and b i + 10th slice

Full size image

Flesh removal

Flesh i.e., soft tissues surround by bone tissue covers a maximum portion in the CT image. To develop an efficient fracture detection and segmentation, it is necessary to devise a technique that can remove unwanted flesh from an image without affecting the region-of-interest i.e., bone tissue. In the proposed framework, we implement contrast stretching and histogram modeling-based flesh removal technique to remove unwanted flesh. Besides, our technique enhances the bone tissue.

Contrast enhancement

We first convert an image into grayscale and plot the histogram (see Fig. 4b for sample CT image Fig. 4a). In Fig. 4b, we observe that the studied image is low contrast image as maximum pixels fall under the narrow range (70-150). To enhance the image, we perform contrast stretching, where it shifts and stretches the gray level range of the input image to occupy the entire dynamic range [5]:

$$\begin{array}{@{}rcl@{}} I_{out} = ((I_{in} \!&-&\! i_{il}) / \mathit{OriginalRange} \!\times\! \mathit{DesiredRange}) + i_{ol}\\ &&\mathit{OriginalRange} = i_{ih} - i_{il} \\ &&\mathit{DesiredRange} = i_{oh} - i_{ol} \end{array} $$
(1)

where

$$\begin{array}{@{}rcl@{}} i_{il} &=& \mathit{Computed Low Intensity}; \\i_{ih} &=& \mathit{Computed High Intensity};\\ i_{ol} &=& \mathit{Desired Low Intensity} ~(i.e. 0); \\i_{oh} &=& \mathit{Desired High Intensity} ~(i.e. 1) \end{array} $$
Fig. 4
figure 4

a CT image and b its histogram

Full size image

The function accepts four parameters: i) actual (input image specific) low-intensity value, ii) actual high-intensity value, iii) desired low-intensity value and iv) desired high-intensity value. To cover the entire dynamic range, desired low and high-intensity values are set to 0 and 1 respectively whereas actual low, and high-intensity values are computed using intensity computation function. Figure 5 describes the detailed procedure of intensity computation function.

Fig. 5
figure 5

Flowchart: intensity computation function

Full size image

To obtain actual (i.e., image specific) low and high-intensity values using intensity transformation function, the function accepts an input intensity value as a parameter and starts scanning image to find the first pixel whose intensity value is higher than the value of the parameter. A 5 × 5 square shaped window is then formed by considering that pixel as the first pixel in the window and variance of that window is calculated:

$$ Variance = \mathit{HighIntensity} - \mathit{LowIntensity} $$
(2)

The difference (i.e., variance) must be as less as possible because low variance indicates flesh or bone tissues are evenly spread in that region. Such a region is more suitable to identify resultant intensity value. Hence in our work, the threshold is set to 20 to obtain a window with the desired variance. If the variance is more than the predefined threshold, the same process is repeated until getting the window with variance less than the threshold. To obtain the actual low and high-intensity values, an average of that window is calculated and divided it by 255:

Histogram plots are observed to find the actual low-intensity value. By inspecting histogram beans, we skipped the first bean with the highest value as it represented dark background pixels and considered the value associated with the second highest bean is passed as a parameter for predicting actual low-intensity value. To find out actual high-intensity value, the parameter is set to 220 as cortical bone tissue ranges from 220 to 255.

Mask formation

After contrast stretching, bone regions are enhanced and at the same time, most of the artifacts get erased. However, in a few images, wherein flesh part is a little bit brighter and dense; some artifacts may remain even after pre-processing. These are the borderlines between flesh and background. To remove the remaining artifacts, the entire image is scanned once by creating a 3 × 3 square mask. If the mask/window has less number of non-zero pixels (we consider 4), then all pixels in that window are reset to 0.

In Fig. 6, the image having a label (a) is the original CT image. The image having label (b) is resultant image after applying contrast stretching and label (c) is the resultant image after applying mask. The proposed flesh removal method (having contrast stretching, histogram modeling and mask moving processes) requires 0.0160 seconds to complete one CT image.

Fig. 6
figure 6

a Original image has been enhanced by applying b intensity transformation function and c mask

Full size image

Segmentation

The result of the segmentation process largely depends on the correct selection and application of pre-processing technique. That is, how nicely the desired image contents are enhanced (in our case bone regions) and how effectively the unwanted artifacts are removed from the image after applying preprocessing technique are important. The proposed pre-processing technique exactly does the same. It removes all unwanted artifacts and enhances the bone regions. This helps come up with simpler segmentation technique with precision. It accepts the enhanced CT image and extracts the bone regions by using connected component (CC)-based segmentation method. To extract desired region from 2D image like CT slice, one of the two: four and eight CC-based segmentation technique can be applied. Initially, we opted four CC-based technique but it did not provide optimal/expected results in the CT slices containing very thin outer layer of cortical tissues. The primary reasons behind this is, four CC-based technique is sensitive to a small changes in the fractured regions. Therefore, to consider all the neighboring pixels for the accurate expansion of fractured bone pieces, eight CC-based segmentation technique is preferred, which has been designed empirically. The detailed segmentation procedure is depicted in Fig. 7.

Fig. 7
figure 7

Flowchart: segmentation process

Full size image

To extract and label the fractured bone pieces, the entire image is scanned several iterations. The scanning process starts from the top left corner, and the first pixel with intensity value more than 220 is identified. The algorithm will then inspect adjacent pixels which satisfy n8(p) and have an intensity value of the pixels more than 70 (initial value cancellous tissue). The pixels that satisfy the above two conditions are get added to the current segmented region. If the size of currently segmented region is greater than the segmented region in the previous slice (as discussed in “Introduction”, the following situation may occur: two or more fractured bone pieces may get disconnected due to dislocation or proximity and resolution problem of CT imaging). In the latter case, morphological opening operation is performed to separate wrongly connected regions. The segmented regions are stored and the process is repeated until no more fractured bone pieces (or components) are identified.

Label assignment

Sine the fracture starts from first slice, it is unusual that the machine has to deal with it without having a-priori knowledge about unfractured areas in that particular location(s). However, it is an inherent practice for the radiologists. The proposed tree structure-based hierarchical labeling method assigns unique labels by considering bone anatomy. This means that if current CT slice contains more than one bones i.e., fractured into pieces, it then assigns different labels to all bone pieces. Moreover, it assigns appropriate labels for each fractured bone pieces of single bone. The proposed labeling technique assigns the ith label to each independent bone present in the CT image. As an example, if any ith bone is broken into n pieces, we have labels: ij, where j = {1,2,…,n}. Figure 8 can help understand the labeling process.

Fig. 8
figure 8

Flowchart: label assignment process

Full size image

An iterative labeling process – an extension of segmentation logic – starts its execution from the first CT slice. For assigning an appropriate label to the currently extracted region, labeling process consults the result of the previous slice. It randomly selects a pixel from the segmented region and checks whether that pixel is present in pixel-list of the previous slice. The value of variable n is incremented by 1 if the selected pixel is present in the list and the variable N is incremented by 1, otherwise. The increment in N indicates that new bone region is found in the current slice whereas the increment in n specifies that fracture occurs in existing bone region. After assigning a suitable label to the considered segmented region, that region is subtracted from the input image and temperately stored in a file. Subtraction is done to consider the next region in the slice (if exists) and to prevent some over-segmentation cases. If subtraction results in the non-empty image, entire labeling process can then be iterated again to assign a label for the remaining region. If not, the segmented regions stored in separate temporary files are collected to reform original image with assigned labels and it will then proceed for next slice. CT slice iand i + 1 from same CT stack is shown in Fig. 9a and b respectively. The result of the labeling process per iteration is shown in Fig. 9c and d respectively.

Fig. 9
figure 9

CT slice (a) i, (b) i + 1 and (c) first, and (d) second fracture piece of CT slice i + 1 with label

Full size image

Unlike other labeling methods, the proposed method assigns specific labels by considering bone anatomy, i.e., by considering a number of individual bones and their fractures (bone pieces). Other methods in the literature, e.g., [16] accept several seeds from the user and propagate those. While accepting a seed, the propagation logic does not consider the number of individual bones nor do it takes number of fractured bone pieces for label assignment process.

Results

Our results

To validate the proposed method (for segmentation and labeling the fractured bone pieces from real patient-specific CT images), as discussed in the data acquisition section 512 × 512 sized, patient-specific CT images were used. These data were collected from several radiology centers in India. Even though we observe variations in resolution of CT images, the bone under supervision (region-of-interest), type of fracture, and fracture complexity, the proposed method gives promising results. As promised, it can segment and label each fractured bone piece by considering specific bone anatomy. Figure 10a shows the original CT image having healthy tibia and fibula. Figure 10b shows the resultant image. Note that the lebels: 11 and 21 are assigned to tibia and fibula respectively. The assigned labels can be interpreted as the CT image has two bones without fracture. Figure 10c contains original CT image having fractured patella and femur bone pieces. Figure 10d provides segmented and labeled fractured bone pieces, where it has two bones and each contains two fractured bone pieces.

Fig. 10
figure 10

Input images: a CT image having healthy tibia and fibula bones and c CT image having fractured patella and femur bones, and their corresponding outputs respectively in b and d

Full size image

Expert-based decision

The ultimate users of CAD systems are the surgeons (and technologies) in the respective domains. Therefore, we follow clinical ground truth to make sure whether our results are correct/accurate. By confirming literature for segmentation of fractured bone(s), we have not found previous works that worked with clinical ground truth. In this study, apart from the comparison with clinical ground truth provided by expert orthopedic surgeons and radiologists, we will make sure that our method has potential so that further comparison with state-of-art segmentation techniques (see “Comparison with state-of-the-art techniques”) is possible.

To visualize how accurate is the proposed method (qualitatively speaking), we superimpose both images that come from clinical ground truth (annotated image) and result of our algorithm. The common pixels in the both images are highlighted with yellow color which shows the union of both images. More number of highlighted pixels indicate that, the result of proposed method is same as that of clinical ground truth. In our study, accuracy can be calculated as

$$ \mathit{Accuracy} = \frac{\mathit{Number~of~pixels~in~union~set}}{\mathit{Number~of~pixels~in~annotation~set}} \times 100. $$
(3)

Figure 11a shows the original CT image having fracture at femur bone. Figure 11b and c respectively shows the annotated image and output of proposed method. Figure 11d shows the result of overlapping of output image and annotated image. The pixels in yellow show the similarity. In our experiment, we have used expert delineated patient-specific CT images and have achieved an accuracy of 95 %. Note that 1000 annotated images (of 8000 CT images) were used for validation.

Fig. 11
figure 11

Comparison with clinical ground truth a Original CT image b clinical ground truth c result of proposed method d overlapped image

Full size image

Comparison with state-of-the-art techniques

Apart from testing on real patient-specific CT images, the performance of the proposed technique is compared with other segmentation techniques. These techniques are successfully implemented by other researchers to segment and label healthy and fractured bones. In our comparison, three well-known techniques are used: a) thresholding-based [25, 27]; b) active contour-based [1]; and c) 2D region growing-based [13, 16] segmentation techniques. Table 1 shows the comparison of several segmentation techniques. Moreover, to visualize the results, all techniques, in Fig. 12, we have shown the results from all techniques that are used to segment fractured bone pieces in cortical and cancellous bone tissue regions.

Table 1 Comparison
Full size table
Fig. 12
figure 12

The results obtained after applying various segmentation methods. First row: CT images having fractures in cortical tissue (left) and cancellous tissue (right). In other rows, output images are shown: thresholding (second row), active contour (third row), region growing (forth row) and proposed method (fifth row)

Full size image

In Fig. 12, two different tissue regions are used to demonstrate how different techniques work:

  • Cortical tissue region (see first column of Fig. 12).

  • Cancellous tissue regions (see second column of Fig. 12)

In case of cortical tissue region segmentation, every technique has satisfactory results. If we perform close comparison, thresholding-based segmentation technique provides better results to slices in which the presence of cortical tissues are more. Also, it does not require any user intervention. On the other hand, we have observed that it is not appropriate for CT images having fractured pieces in cancellous tissue regions since it is sensitive to noise and therefore label assignment process does hold expected output. As a consequence, it may not split wrongly connected fractured bone pieces. The active contour-based segmentation method gives promising results without performing any preprocessing. However, it requires large number of iterations (more than 1000) to achieve desired results. The region growing approach is more suitable for fracture bone segmentation and labeling. Again, like before, it requires user intervention (i.e., use of seed points to start the process). It is interesting to note that the results are better when it combines with contrast-stretching CT image enhancement. In contras, the proposed method gives promising results for both tissue regions. Due to the application of the flesh removal technique, all artifacts are erased, and the image is enhanced. Further, it does not require user intervention to segment and label fracture bone pieces. Using the exact same input images (Fig. 12), following the segmentation results, labels are shown in Fig. 13.

Fig. 13
figure 13

Labeling after segmentation with the proposed method. In the first image (cortical tissue), there are three different labels: 11, 12 and 13, where the first number in any pair refers to bone number in that CT image and the second number refers to fractured bone pieces (if any). For example, 13 represents the label of the first bone with third fractured bone piece

Full size image

Conclusion and future work

In this paper, we have a presented a segmentation fretwork that involves three steps: flesh removal, segmentation and labeling. A simple and precise, histogram modeling and contrast stretching-based pre-processing technique has been devised. It is responsible for removing unwanted artifacts and/or flesh that covers bone tissues that ultimately help enhance bone regions. A connected component-based segmentation algorithm has been designed to extract and label fractured bone pieces. In our labeling process, it has assigned unique labels to fractured bone pieces while taking bone anatomy into account. Unlike other methods in the literature, it assigns labels in a hierarchal manner to several fractured bone pieces of different bones so that it is easier for the user to bifurcate several individual bones and their fractured bone pieces independently. To comment on the correctness of the proposed method, we have tested on real patient-specific CT stacks. We have compared our results with clinical ground truth, which is our primary concern and have achieved an accuracy of 95.45%. Further, we have compared with some other previous segmentation methods.

Besides, we plan to make CT database publicly available (for research purpose) along with clinical ground truth (annotated images). In this study, since 1000 images (of 8000 CT images) are annotated, for the complete annotation, we aim to develop a machine learning-based technique (semi-supervised). Also, we plan to employ modern data-driven segmentation methods, such as Retinanet and Mask-RCNN to automatically segment and label fractured bone pieces.