Introduction

Ultrasound examination of the kidney is a widely performed diagnostic procedure [1, 2] and helps in the clinical diagnosis of hydronephrosis, cancer, cysts, and stones [3, 4], also ensuring accuracy and safety during ultrasound-guided percutaneous needle biopsy [5]. However, particularly in the identification of small lesions or needle biopsies that require fine manipulation of the probe, high reproducibility is needed to acquire the same image by different operators or even by the same operator at different time points. The use of an external ultrasound fixator could be a simple and reasonable solution [6,7,8,9]. However, this type of device is relatively large and may have difficulty in moving around and also carries a risk of pressure sores with longer attachment periods, as it is unmanned equipment controlled remotely. This indicates that semi-automated manipulation of the probe is awaited in terms of feasibility and safety in clinical settings.

Ultrasound probe positioning is quantitatively evaluated using appropriate coordinates. Therefore, herein, we used motion-capture markers on both the subject’s body and the ultrasound probe, continuously captured by cameras, to record the three-dimensional relationship of those two objects. Thereafter, we developed a real-time probe operation assistance system using a computer interface screen that indicates the target position and angle derived by the relative coordinate between the subject’s body and the appropriate probe position [10]. By applying this auxiliary apparatus to kidney ultrasonography, ultrasound operators, even those without expertise in the diagnostic tool, can maneuver the probe to reproduce appropriate positioning.

Materials and methods

Motion-capture markers and cameras

We attached seven motion-capture markers on the back of a subject (bilateral subscapularis, thoracic spine, bilateral ribs, and bilateral iliac crests) (Fig. 1A) and another seven markers on an ultrasound probe (GE healthcare, C1-6-D for LOGIQ E9) (Fig. 1B). On the body, the markers were placed such that their distance from the kidneys remained relatively unchanged even with slight changes in body posture. Additionally, we ensured that the markers were placed at easily identifiable anatomical positions to promote reproducibility. Then, eight cameras (Naturalpoint, Flex 13) were installed around the bed to recognize the positional information of 14 motion-capture markers independently in real time (Fig. 1C).

Fig. 1
figure 1

Methods of the motion capture-facilitated kidney ultrasound guide system. A, B Seven motion-capture markers are attached on the subject’s back (A) and the ultrasound probe (B), respectively. C Eight cameras monitor the motion-capture markers from around the subject. D Our user interface screen used to guide the probe holder has two different modes for adjustment of the location (D) and angle (E). US ultrasound apparatus

Full size image

Information processing, user interface screen, and ultrasound probe handling

Our user interface provides intuitive guidance for the movement of the probe tip by appropriate coordinate transformations. The system can properly guide the probe position even when the subject's position changes as long as the relative coordinates between the subject’s markers and kidney are preserved. The marker positions on the subject and the probe obtained when the professional operator performed ultrasonography were processed to define subject and probe coordinate systems \({\Sigma }_{k1}\) and \({\Sigma }_{j1}\), respectively, relative to the world coordinate system seen by the cameras. The transformation matrix from \({\Sigma }_{k1}\) to \({\Sigma }_{j1}\) can be calculated as \({T}_{k1,j1}\). During a real-time process, in the same manner, marker positions on the subject and the probe are continuously processed to define moving coordinate systems \({\Sigma }_{k2}\) and \({\Sigma }_{j2}\), respectively, and to compute transformation matrix \({T}_{k2,j2}\). Should the components of \({T}_{k1,j1}\) and \({T}_{k2,j2}\) match, the ultrasound device displays the kidney from the same position and angle at different times. Otherwise, differences between translational components and angular components of \({T}_{k1,j1}\) and \({T}_{k2,j2}\) are shown in the user interface.

An amateur operator can move the probe while looking at the user interface screen, instead of the ultrasound device screen. The green rectangle fixed in the center of the screen shows the position of the probe when it is drawn by a professional operator. The pink rectangle frame displayed at the same time is a two-dimensional representation of the position and angle information of the probe at the hand of an amateur operator. Depending on the position of the probe, when handled by an amateur operator, this pink rectangle frame will change its shape and position in real time (Video 1). The interface consists of two modes, position adjustment (Fig. 1D) and angle adjustment (Fig. 1E), which the operator can switch between at any time.

Feasibility study

Two subjects, two professional operators, and four amateur operators joined the feasibility study. The subjects were adult men who were 170 and 169 cm in height and weighed 75 and 72 kg (body mass index; 26.0 and 25.2 kg/cm2/cm2), respectively, without any medical history of renal deformity. The professional operators were Board Certified Nephrologists of Japan with marked experience in renal ultrasonography, and the amateur operators were graduate students in the engineering field and had no official qualifications or clinical experience of ultrasonography. We obtained data of eight different sets of one subject, one professional operator, and one amateur operator.

Additionally, we performed the study to evaluate the probe position and angle after the probe maneuver without the user interface guidance. While only referred to the ultrasound screen shot of the kidney cross-section that the professional operator obtained in advance, the amateur operators handled the probe without the aid of the user interface we developed. They were allowed to move the probe at longest for the same period as they spend with the aid of the interface. Exactly when they stated that they obtained the most resembling image, the probe position and angle were recorded with a motion-capture system.

Statistical analysis

For feasibility study, all data are presented as the mean ± standard error of the mean.

Ethical committee approval

This experiment was conducted with the approval of the University of Tokyo Ethics Review Committee (19-44), and informed consent was obtained from the subjects.

Results

First, a professional ultrasound probe operator scanned a subject’s right kidney (Fig. 2A). The spatial positional information of the ultrasound probe and of the subject’s body was saved and processed. Next, an amateur operator manipulated the ultrasound equipment. The relative positional information between the amateur operator-held ultrasound probe and the subject's body was saved as spatial coordinates and reflected in the specifically developed user interface screen. Each amateur operator adjusted the probe position based on the real-time information of its location and angle displayed on the screen in two dimensions, without looking at the ultrasound equipment screen or urging the subject to not breathe. When the probe was appropriately fixed, the amateur operator asked the subject to not breathe and then recorded the ultrasound image of the kidney (Fig. 2B, Video 1). Then, six parameters on probe location (Fig. 2C) and angle (Fig. 2D) recorded for 5 s revealed that the deviation to those provided by the professional operator was of a few mm and degrees.

Fig. 2
figure 2

Reproducibility of kidney ultrasound with the motion capture-facilitated ultrasound guide system. A Kidney ultrasound scanning image acquired by a nephrologist with expertise in ultrasonography. B Then, an amateur operator without experience of clinical ultrasonography utilizes the motion capture-facilitated ultrasound guide system and acquires the ultrasound image of the kidney. C, D When the probe is manually fixed by the amateur operator, deviation of the probe location (C) and angle (D) is recoded for five seconds. (E, F) Deviation of the probe location (E) and angle (F) is shown (N = 8), obtained by switching combinations of subject, professional operator, and amateur operator. RMS root mean square

Full size image

Further, we assessed the feasibility of the system by recruiting different subjects and operators. The amateur operators could hold the probe on the subjects by utilizing this interface (N = 8), wherein the deviation of distance and angle was (x = 2.7 ± 1.2 mm, y = 3.0 ± 1.7 mm, z = 6.6 ± 1.8 mm) (Fig. 2E) and (Rx = 1.5 ± 0.3 degrees, Ry = 2.6 ± 1.1 degrees, Rz = 1.1 ± 0.3 degrees) (Fig. 2F), respectively. Thus, this procedure enables nearly equivalent imaging of a cross-section of the kidney between amateur and professional operators.

Next, to evaluate the usefulness of this device, we measured the gap of positional and angle information among amateur operators and professional operators without using the interface. By attempting to reproduce the ultrasound image which the professional operator obtained in advance, the amateur operators eventually placed the probe with the deviation of distance and angle being (x = 35.8 ± 17.7 mm, y = 63.8 ± 18.8 mm, z = 56.6 ± 27.5 mm; Fig. 3A) and (Rx = 25.7 ± 10.5 degrees, Ry = 17.8 ± 4.7 degrees, Rz = 72.6 ± 21.8 degrees; Fig. 3B), respectively (N = 6). This result demonstrates that our device is a highly effective method to reproduce the probe position and angle, compared to the maneuver to place the probe based solely on an ultrasound screen without this new device.

Fig. 3
figure 3

The probe position and angle in the kidney ultrasonography without the aid of the motion capture-facilitated ultrasound guide system. A, B The amateur operators handled the probe without the aid of the user interface until they felt that they obtained the most resembling image, Then, deviation of the probe location (A) and angle (B) from that provided previously by the professional is shown (N = 6). The data were obtained by switching combinations of professional and amateur operators. RMS root mean square

Full size image

Discussion

Our ultrasonography system enables amateur individuals without clinical skill to imitate the probe maneuver and acquire kidney ultrasound images similar to those acquired by a professional operator. This system has several clinical advantages. We also demonstrated the feasibility and reproducibility of ultrasound probe manipulation with different combinations of operators and subjects as long as one professional operator set an example in advance.

First, in percutaneous needle kidney biopsy, retroperitoneal hemorrhage is a serious complication that sometimes necessitates blood transfusion, intravascular embolization, or even unilateral nephrectomy [11]. To avoid such iatrogenic events, before the procedure, physicians perform ultrasonography to inspect the kidney shape, determine the appropriate puncture site, and estimate the distance and angle for advancing the needle. During the biopsy, it is essential to obtain the same ultrasonic image of the kidney [5]. Since our system allows any assistant to reproduce the positioning of the ultrasound probe, the physician can concentrate on advancing the needle during percutaneous kidney biopsy. Thus, it may expand task shifting of the ultrasound operation in the future.

This system provides numerical information regarding any deviation of the ultrasound probe from the professional probe position in terms of both location and angle. In other words, it enables the quantification of one aspect of an operator’s skill and may be useful for proficiency data-based ultrasonography training of beginners/students. In addition, this system may help conduct a comprehensive study to evaluate the need to fine tune ultrasound probe positioning in daily clinical practice. Further, compared to robotic systems [6], our system can be installed even if there is little space for external devices. It can also be extended to automatic examination using a robotic arm. This will permit accurate and automatic renal examination because the absolute positioning accuracy of the typical robotic arm is worse than that of motion capture.

In our study, the deviation of distance for the z axis was relatively large. This occurred because some amateur operators were reluctant to sufficiently press the probe to the subject’s body because they felt it might cause pain. In the current work, to strictly evaluate the benefit of the two-dimensional guide system alone, amateur operators were not supervised by physicians or other health professionals. Unless any event harmful to the subject occurs, amateur operator should be advised to handle the probe according to the interface.

Our system is hopefully of advantage for subjects whose kidneys are difficult to depict with ultrasonography, including those who are obese or have nephrosclerosis. In this regard, both of the subjects in the present study happened to be technically obese [12], but we successfully reproduced the probe position and angle, partially supporting the usefulness of our device in a real clinical setting. Nevertheless, a defined clinical study to enroll actual patients whose kidneys are poorly visualized with conventional ultrasonography must be conducted as a next step.

In terms of commercialization, acceptance of our innovative system by the market could be influenced by the price. Currently, the total cost of the device and camera amounts to approximately 2,500,000 JPY (approximately 2,000 USD, according to the recent foreign exchange rate), which is less expensive than a single set of ultrasound equipment. The motion-capture markers and cameras are versatile tools, not built specifically for the current system, and may therefore become available at a lower cost.

We expect that this system can be applied to the kidneys and any other target site or organ. For instance, the pancreas in particular is an anatomically difficult organ to visualize with ultrasonography. Thus, the diagnostic performance largely relies on the operator’s experience to adjust to the patient’s condition, such as posture, body weight, stomach liquid content, and presence and amount of interfering bowel gas [13, 14]. Under such circumstances, our device may help operators correctly reproduce the special relationship between the subject’s body and professional positioning of the probe. Moreover, ultrasound-guided percutaneous needle biopsy procedures may be performed to harvest tissue or fluid samples from muscles, bones, and other organs, such as the liver, thyroid, and lungs.

In summary, this study was novel in that the device system developed can semi-automate ultrasound operations to require fine manipulation of the probe, thus helping a person without experience quasi perform kidney ultrasonography and thus attenuate the burden on healthcare professionals. Because the probe control in our system does not require the conventional clinical skills needed for ultrasonography, an interface user may need to have distinctive cognitive abilities, coordination, and physical finger skills similar to those needed for video games or esports. Thus, this system holds the promise of shifting the paradigm of human skills required for performing ultrasonography on any organ.