Keywords

1 Introduction

Ultrasonic (US) movie is widely used by pediatricians at neonatal intensive care unit (NICU) to observe the brain tissues through the anterior fontanelle of neonates with potential risks of ischemic and related diseases such as Hypoxic-Ischemic Encephalopathy (HIE) [1] and Intraventricular Hemorrhage (IVH) [2, 3]. Since early detection of these diseases is essential for timely intervention, pediatricians have diagnosed such abnormalities from the shape and motion of brain tissues in brightness mode images and movies while taking care of some factors such as choice of suitable equipment, optimal settings, appropriate scanning protocols, and scanning experience to ensure its image quality [4].

One of cardinal viewpoints in the diagnosis of ischemic diseases is to observe a pulsatile motion of brain tissues due to blood flow of neighboring arteries. In order to assist such pediatric diagnosis, we have conducted several studies in detecting and visualizing the pulsatile tissues from US movies with time-frequency analysis [5,6,7,8]. It was based on a heartbeat-frequency component I(fHB) in the frequency spectrum obtained from a time-variant echo intensity of each pixel over 64 frames in a movie fragment extracted from a scene without any motion of the US probe. I(fHB) was provided to assist pediatric diagnosis by visualizing as a superimposed image while overlaying a color gradation proportional to I(fHB) onto the original US image [8]. It was also given by statistically analyzing as a pulsatile tissue measures. These procedures are shown in Fig. 1. An extensive archive of neonatal cranial US movies was also prepared for a variety of studies, and its retrospective analysis was examined by using a conventional analysis system. However, it was problematic to handle the entire archive due to several issues in the conventional analysis system such as extensive, redundant computations and inconsistent data.

Fig. 1.
figure 1

Procedures for detection and visualization of pulsatile tissues from a movie fragment in a scene without probe-motion

Full size image

In this study, we introduce an approach for handling the entire archive by developing a novel analysis system dedicated to neonatal cranial US movies of ischemic diseases while mitigating potential challenges associated with our conventional system. Additionally, several test cases are formulated to compare the performance of two systems.

The reminder of this paper is as follows. In Sect. 2, the archive associated with the conventional image analysis system and its issues are presented. Sect. 3 proposes a novel image analysis system to alleviate such issues. Next, Sect. 4 illustrates some experiments to evaluate the novel system performance. Finally, achievements of this paper are summarized in Sect. 5.

2 Movie Archive and Issues in Its Retrospective Analysis Using the Conventional System

The movie archive utilized in this study consists of an immense volume of cranial US movies with approximately 1000 cases recorded at NICUs in the previous affiliated institutions (Himeji Red Cross Hospital and Saiseikai Hyogo-ken Hospital) of the author (Kitsunezuka, Y.) and provided anonymously under the approval of the affiliated department, or the ethics committee after its establishment. It consists of around 213 h of movie and encompassing around 1000 cases. Table 1. shows the statistics of the examined archive. Since the US apparatus was different between the groups of different recording periods, the resolution and the fan-shaped field of view (FOV) should be considered for each group.

Table 1. Statistics of the US movie archive
Full size table

The process flow in the previous studies using the conventional analysis system is illustrated in Fig. 2. In this figure, rounded and angular rectangles indicate the processes and their resultant data, respectively. The gray background indicates a process that requires manual intervention for each case or movie fragment, while the white one means what can be performed for multiple cases or fragments automatically at once.

In that conventional process flow, four types of manual process were required for each scene prior to the time-frequency analysis in order to select appropriate scenes for detecting pulsatile tissues. They are: 1) the anatomical identification of tomographic section and diagnosis annotation, 2) probe-stabilized scene detection to eliminate the visual incoherence due to the sway of probe, 3) fHB estimation according to the heartbeat frequency for each neonate, and 4) cranial ROI detection to focus on the area of brain tissues. All of them are performed by visual observation of the movie.

The case selection process was repeatedly invoked each time a certain purpose of study was considered for corresponding analysis and visualization. Therefore, it was difficult to investigate the relationship between pulsatile tissues and pathologic conditions. The problem of this process flow revolved around the uncertainty of the success or failure of all the manual processes until the visualization of the time-frequency analysis were observed. For example, the probe-stabilized scene was not valid until the I(fHB) superimposed image was observed to confirm that it did not contain motion artifacts, as shown in Fig. 3. If it contained motion artifacts, we had to re-extract the scene and perform the time-frequency analysis again. In most cases, that process required some iterations with the feedback from the results of the time-frequency analysis. Since each of the other manual processes also required its own verification, a complete manual process included complex iterations. Therefore, the analysis of the entire archive was challenging because as more scenes were processed, various problems with iterative processes arose such as extensive, redundant computations and inconsistent data.

Fig. 2.
figure 2

Process flow in the previous studies using the conventional image analysis system

Full size image
Fig. 3.
figure 3

Example of I(fHB) superimposed images obtained from (a) probe-stabilized and (b) probe-swayed scenes

Full size image

3 A Proposal of the Novel Image Analysis System

To tackle the issues in the conventional process flow, a novel image analysis system has been proposed. Figure 4 shows the systematic process flow enabled by that system. The shape and the background color of the rectangle in Fig. 4 indicate the same process type as those shown in Fig. 2. The novel system adopts a multilayered architecture to arrange processes into distinct layers including: 1) primary layer, which prepares extensive amount of intermediate data, 2) refinement layer, which filters or transforms intermediate data into more lightweight and concentrated data, and 3) exploratory layer, which discovers data patterns that are associated with the pathologic conditions.

One of the important design features of this system is that all the processes are performed on each movie fragment rather than on each movie scene. In addition, prior to any manual process, time-frequency analysis and its visualization are performed on the entire archive, regardless of the appropriateness for detecting pulsatile tissues, and the results are recorded as an intermediate data. An element of the intermediate data consists of a fragment spectrum that includes the whole spectrum I(f) corresponding to a movie fragment, and an I(fHB) superimposed image made from the initial frame of the fragment. Example of spectrum I(f) and corresponding I(fHB) superimposed image was reflected in Fig. 1. Since a huge number of intermediate data was generated to cover the entire archive, it can be directly used for subsequent processes without regeneration, which is a clear advantage over the conventional system. Therefore, this design feature allowed all the processes of intermediate data generation to be positioned as the primary, feedback-free layer in the process flow.

Fig. 4.
figure 4

Systematic process flow enabled by a novel image analysis system

Full size image

Another important feature of this system is to efficiently perform the manual processes by using both intermediate data and the US movies. Among the manual processes, the section identification and ROI detection should be performed by visual observation of US movies since it is based on anatomical knowledge. However, the probe-stabilized scene can be effectively detected, by visual observation of a series of I(fHB) superimposed images to ensure the absence of motion artifacts [9]. Furthermore, fHB can be effectively estimated from the peak frequency of the fragment spectrum. Since iteration was no longer required for all the manual processes, they were positioned as a refinement layer in the process flow to generate pulsatile tissue measures from the intermediate data. In other words, this layer refines the large amount of data from the primary layer into more focused data for further analysis.

The final output of this flow is the result of correlation analysis between the diagnosis data and the pulsatile tissue measures to assist pediatric diagnosis. Since it is expected to apply several pediatric diagnoses such as prognosis of HIE and risk assessment of IVH, this process was positioned as the exploratory layer in the process flow and designed to be performed for a certain subset of the archive.

In our system, a fragment was constructed from 64 frames (2 s), and Blackman was selected as the windowing function, by considering the appropriateness to handle the actual fHB of neonates (2 to 3 Hz) with the standard video rate (29.97 fps).

4 Evaluation of System Performance

In order to compare the performance between the conventional and the proposed systems, four test cases were designed and conducted as follows.

  1. 1.

    Time to extract the valid probe-stabilized scenes: This test aims to evaluate the time required to extract and validate all the probe-stabilized scenes from a 20-min US movie of a certain neonate.

  2. 2.

    Time to calculate the pulsatile tissue measures: This test aims to evaluate the time required to calculate the pulsatile tissue measures from the 1200 probe-stabilized scenes for 100 neonates with a common fan-shape ROI.

  3. 3.

    Applicability to the prospective studies in NICU: This test aims to examine the system applicability for a kind of prospective study in the NICU to compare an ongoing disease with the previous cases. We gave a new US movie of an ongoing case and specified 3 previous cases in the archive, and tested the ability to generate and visualize the pulsatile tissue images of them within 1 h, assuming an acceptable latency in NICU. The pulsatile tissue image means a special I(fHB) superimposed image generated from the validated probe-stabilized scenes.

  4. 4.

    Applicability to the retrospective (case-control) studies: This test aims to examine the system applicability for retrospective studies to compare two case groups in the archive. We specified two case groups (HIE positive and negative) including around 10 patients for each in the archive and tested the ability to extract a certain pulsatile tissue measure and to visualize its sectional or time dependence.

Table 2. shows the results of four test cases. The extraction of valid probe-stabilized scenes was five times faster in the novel system. The primary reason for this result is the ability of the pre-generated I(fHB) superimposed images to eliminate the iteration of manual extraction of probe-stabilized scenes. Likewise, the calculation of pulsatile tissue measures was three times faster in the novel system because it utilized the pre-generated fragment spectrum and the refinement data.

The novel system succeeded in instantly generating the pulsatile-tissue images for both ongoing and multiple previous cases, while the conventional system failed to do so due to time-consuming iterative processes. From this result, it was found that the applicability to prospective studies in the NICU was realized exclusively in the novel system.

Table 2. Performance evaluation contrasting conventional and novel archive systems
Full size table

The novel system also succeeded in extracting the pulsatile tissue measures and visualizing their sectional or time variation in specified two case groups, while it was impossible for the conventional system because the valid probe-stabilized scene was unknown. Figure 5 shows examples of pulsatile tissue images and corresponding measures obtained in HIE negative and positive case groups retrospectively. A normalized pulsatile area in a fan-shape ROI [10] was used as the pulsatile tissue measure in this experiment and noted at the upper right of each image. The pulsatile tissue image clearly revealed different distribution of pulsatile tissues slice by slice as well as case by case. The pulsatile tissue measure revealed a significant difference between HIE positive and negative groups. It should be emphasized that the symptom of HIE was diagnosed a few weeks after birth, while the US movie was taken just after birth. Therefore, this result strongly suggests the applicability of pulsatile tissue measures to predict the prognosis of HIE in the early postnatal stage.

Fig. 5.
figure 5

Pulsatile tissue images and corresponding measures (normalized pulsatile area) obtained in HIE negative and positive case groups retrospectively

Full size image

5 Conclusion

This research presented a layer-based analysis system to deal with a large-scale archive of neonatal cranial US movies while successfully eliminating the intensive and redundant computations and inconsistent data. The system was designed to include a primary layer for fully-automated preprocessing such as frequency spectrum analysis and several consequent layers for semi-automated retrospective analysis such as detection of probe-stabilized scene and assessment of pulsatile tissues. Four test cases were examined by the conventional and novel systems to evaluate the performance. The results demonstrated that the novel system performed essential tasks three to five times faster, while also providing new capabilities that the conventional system lacks. Besides, the solution also showcases the potential applications for analysis of periodical phenomena in other mass movie repositories than our archive. In summary, the proposed system successfully handled the large-scale archive of US movies, accelerating its retrospective analysis and revealing the clinical applicability of the system.