1 Introduction

Fig. 1
figure 1

inbody, the instant photogrammetric 3D body scanner

Full size image

The fabrication of effective prostheses and orthoses devices for medical treatments of various human diseases requires accurate body parts measurements. In the past, medical practitioners have traditionally measured the body’s size and shape by hand to assess health status and guide treatment [1]. The current trend is to avoid the use of manual techniques in order to improve the patients’ comfort and experience during the measurement procedure. This motivates the use of novel scanning systems which exploit non-invasive technologies for the human body acquisition. In addition, the need of implementing technical standards to automate the body’s measurements process requires the novel body scanning systems to be as accurate as possible.

This is pushing medical people and engineers towards the research and development of novel devices for human body instantaneous acquisition. Beside the above mentioned advantages, these systems allow to overcame one of the major limitation of handy measurements, i.e. the human body digitization. It gives the users the opportunity to store structural informations of the patient that can be queried as wanted in an offline process. The novelty resides in the fact that this approach allows the storage of precious digital information useful for the continuous monitoring of the treatment.

In the recent years, several 3D body-surface scanners have been introduced to accurately measure human body size and shape. They are often based on laser or depth sensors technologies. The principle behind laser body scanners working process is the projection of a single point, line, or multiple lines of structured light onto the subject and the use of a camera sensor to acquire the laser light. On the other hand, depth sensors are constituted by an infra-red projector and an infra-red camera receiver which records the light pattern projected in the scene. The main drawback of these technologies resides in the time required for the data acquisition. Even the state-of-the-art laser-based body scanning system, i.e. the VITUS 3D Body Scanner from Vitronic,Footnote 1 requires 6−12 s for the total human body data acquisition. The slowness of the scanning process is a key issue in many healthcare applications which require interaction with patients, especially when they are affected by mobility impairments. This constitute one of the main reason which have limited the large diffusion and integration of these body scanners in real medical scenarios.

In this paper we present the design and the development of inbody, a novel full body scanning system for healthcare applications. It is based on the digital photogrammetry technology, which guarantees an instantaneous data acquisition process by synchronously capturing multiple photos of the scanned subject from different point of view. This technology, besides the time advantage, also allows to generate accurate and high resolution digitized models of body’s surfaces. In this work we explain the main steps of the design and development of the novel scanning system and its integration in a real medical scenario. We present the results obtained on the pilot study by comparing our system with current technologies. The prototype is shown in Fig. 1.

The design of the scanner has followed an integrated approach. The hardware and software architecture, as well as the mechanical design of the prototype is addressed in the work. The performed experiments aim to show the ability of the system to acquire high resolution and textured 3D meshes. The accuracy of the system is proven by comparing the digitized body model obtained or scanned with inbody and with a laser scanner system. The presented results of the paper are intended to show how inbody might be rapidly integrated and efficiently exploited in many healthcare applications.

1.1 State-of-the-art

In this section we present an overview of the state of the art in 3D body scanning technologies and applications for the medical field.

Currently, the most diffused scanning methods for healthcare are based on laser, depth sensors or pattern projection technologies. Laser scanning uses light sources to project on the human body one or more tiny tripes, which are detected by light sensors. The human body surface is digitized using triangulation. To generate multiple light stripes from a single laser beam, optical systems made of mirrors are used. A laser scanning unit is made up of the laser source, the optical system and the light sensors. The unit is moved across the human body to digitize its surface. The kind of movement, the number of employed units and the scanning time can vary according to the body part to measure. RGB-D cameras refer to systems composed by one red–green–blue (RGB) camera, one infra-red (IR) camera and one IR projector. These systems digitize the surfaces by recording the pattern projected onto the subject. Pattern projection allows the digitization of surfaces by projecting a light pattern, usually in the form of horizontal or vertical stripes, on the subject to scan. A light sensor acquires the scene. Each stripe is singularly measured by triangulation. The interferences of multiple light sources make difficult the instaneity of the process.

One promising technology in scanning systems which guarantees the instantaneity of the process is the passive photogrammetry. This technology allows, with the same system, to obtain informations about the geometry and the texture of the human body by capturing multiple synchronized photos of the patient from different point of view.

Table 1 3D healthcare scanning application taxonomy
Full size table

The basic idea behind the 3D photogrammetry is the geometry of stereo vision, called epipolar geometry. Figure 2 illustrates the epipolar geometry of a pair of stereo cameras, using the pin-hole model. When two cameras view a 3D scene from two distinct positions, indicated by the reference frames \(Ox_Ly_Lz_L\) and \(Ox_Ry_Rz_R\), there are a number of geometric relations between the 3D point P and its projections onto the 2D planes \(OX_LY_L\) and \(OX_RY_R\) that lead to constraints between the image points, which are indicated by the blue dots. These relations are derived based on the assumption that the cameras can be approximated by the pinhole model, so that the conversion from 3D to 2D can be referred to as a perspective projection. When the relative position of the two cameras, indicated by the homogeneous matrix \({{T}}_{LR}\), is known, these geometric relations, referred to as epipolar constraints, are described by the fundamental matrix, which relates corresponding points in stereo images. A comprehensive reference on multiple view geometry in computer vision can be found in [2].

Fig. 2
figure 2

Epipolar geometry of a pair of stereo cameras—pinhole model

Full size image

In the recent years, body scanning has been introduced in the healthcare domains with different applications. The main objectives in the medical field is to measure (size, shape, surface area, volume) and visualize (head, chest, whole body) a replicated model of the patient [1]. The measurement of human body model is useful in applications where the doctors are interested only to the geometry of the body, while the visualization finds applications wherein the geometry and the texture play the same relevant role. From a medical perspective, one can divide the healthcare applications into four groups: epidemiology, diagnosis, treatment and monitoring, as indicated in Table 1.

  • Epidemiology is the study and analysis of the patterns, causes, and effects of health and disease conditions in defined populations. Body scanning helps in monitoring the population through anthropometric surveys and in screening individual subjects.

  • Diagnosis is the process of determining which disease or condition explains a person’s symptoms and signs. Body scanning plays an important role in diagnosis by detecting deformities and by analyzing the skin of the patient.

  • Treatment is the plan to ensure that the appropriate medical care is provided to patients, based on their disabilities or illnesses. Body scanning can provides body measurements for burn treatment and for calculating drug and chemotherapy dosages. The digital reconstruction of the patients’ body acts in the design and fabrication of custom-fit orthoses and prostheses. Craniofacial surgery may also benefits from digitization of the patients’ head.

  • Monitoring is the observation of a disease, condition or one or several medical parameters over time. Being noninvasive, body scanning helps in monitoring body morphology, whether due to exercise, nutrition, or diet programs administered as part of clinical treatment or through attendance at a gym or diet club.

Several documented applications use 3D scanning technologies in the healthcare domains, such as custom-made shoes [3], lower-limb prostheses [4], nasal prostheses from laser scanning [5] as well as from structured fringe projection scanning [6]. An overview of applications of 3D scanning in prosthetics and orthotics clinical practice can be found in [7]. The first attempt to use photogrammetric scanning in medicine can be found in [8], while a photogrammetric approach using standard smartphones camera to digitize sockets for prosthetics can be found in [9].

In the current work we introduce the design and development of a novel photogrammetric scanning system which offers interesting properties in its use in healthcare scenarios by presenting its applicability in the treatment domain. We describe the interactive design methodology used for its development, as well as novel interactive medical procedures which can be easily integrated in real scenario through its implementation.

1.2 Main contributions

The main contributions of the present work are listed as follows:

  • Interactive product design methodology for complex integrated systems: the inbody system development requires multiple submodules to be assembled and work together to achieve the desired objective. We show how we map the system into the design requirements, as well as the technical specifications which lead to the final product. By using a modular system architecture, we can do iterative modifications of the system at the hardware, software and mechanical level, if the initial system requirements are not satisfied.

  • Using stereo photogrammetry technology for human body digitization: the use of photogrammetry has been limited in the past by low camera resolutions and high computational and processing time required. Nowadays, the use of this technology shows results, in terms of accuracy and timing, comparable with others. Thus it needs to be revisited and wider applied.

  • Development of a scanning system for healthcare applications: our case study examine the use of scanning systems in medical applications. We show how the system can be integrated in a real enlarged manufacturing process.

  • Interactive approach for the digital manufacturing of prostheses and orthoses: the instant acquisition and fast reconstruction allows to rapidly reconstruct the patient’s body shape from the different positions indicated by the practitioners. This could be important for many healthcare applications, as for the design and fabrication of spinal orthoses. The generated 3D models can be used in a virtual and interactive environment to extract measurements, to visualize the texture, to design medical products. The fast generation (and eventually, re-generation) of accurate and textured human body models might allow the development of interactive approaches in different medical fields, in the contexts of epidemiology, diagnosis, treatment and monitoring of many disfunctions and diseases.

2 Requirements and methodology

The considered instant body scanner is a modular collection of a series of components which permit the system to work as a whole. In this section we highlight the requirements which have led to the system design. The system requirements have been identified with the help of doctors and experts working in the orthopedic and rehabilitation domains. The main goal is to ensure an instant data acquisition, which might allow a correct scanning of patients with mobility dysfunctions. The secondary objective is to guarantee a sufficient accuracy of the replicated models: such accuracy has been identified in being below 10 mm. The 3D scanning in healthcare does not requires the high accuracy of the industrial sector. Indeed, in the clinical practice we deal with real patients: even the breathing may alterate the shape of the body. Thus, the feasibility of a non-invasive and fast scanning procedure mainly motivated our work.

Fig. 3
figure 3

The mapping from system requirements to design requirements for the main three components of inbody: the software for 3D reconstruction, the mechanical structure, the management and control unit

Full size image

The instantaneity and the accuracy requirement might be satisfied by using the digital photogrammetry technology and selecting the appropriate type of image sensor, as well as its number and position in 3D space. A modular hard- and software architecture of the system might allow to scan in different settngs and easily improve the system with novel capabilities, if needed.

The three main actors of inbody have been identified by three sub-systems:

  • mechanical structure;

  • control and management unit;

  • 3D reconstruction software.

The mechanical structure is the essential part that guarantees the correct positioning of the camera sensors in the 3D space. One of the major requirement has been identified in the optimization of cameras disposition in the 3D space for covering an ideal cylindrical area which will have contained the patient. From the mechanical point of view, we develop a modular infrastructure made up by multiple submodules which comply with a circular disposition. The control and management unit has the objective to control and manage all the hardware components: it is in charge of synchronize the commands given by a human operator to permit synchronous data acquisition. It also acts as a human–machine interface for the system management. The 3D reconstruction software is a unit which assure the generation of 3D point clouds and meshes from the multiple acquired photos.

The design requirements for each sub-system have been obtained by mapping the system requirements in a 1:1 fashion, as we can see in Fig. 3, We carry out the product development using an interactive approach. Indeed, in the development of complex systems, the decisions taken at the different design levels are strictly dependent. The mechanical, the hardware and the software design, which are inherently coupled, have been decoupled using a mapping which provides independent design requirements for each component of the scanner. The interactive methodology, which guides the design process through the mapping of Fig. 3 and lead to a modular system architecture, allows to perform iterations at the mechanical, hard- and software levels, if the initial system requirements are not satisfied in the final product. In the next sections, we describe into details each component of inbody and we point up the existing interactions.

3 Hard- and software architecture

In this section we extensively describe the hardware and software developed and integrated into the system. The hardware architecture of inbody has been built around its main component, the Omnivision OV5647, a 1/4” color CMOS 5-megapixel image sensor.Footnote 2 It provides multiple resolution raw images to a BCM2835 processor mounted on standard micro-controllers through a dedicated MIPI second generation camera serial interface (CSI-2) connector. All the systems composed by one camera and one micro-controller are connected over an ethernet network, and they are managed by a remote management controller. A control software has been developed to manage the hardware in the system. It articulates on the client multiple-server architecture [10], which allows to design softwares for networked systems. The client is represented by the remote management controller, while each server is represented by the micro-controller, as it provides a series of services. These services are wrapped in threads, for instance: remote shooting, pictures’ transfer, turning off. The key element of the software architecture is to provide an instant capture of the subject to scan: this is guaranteed by the synchronization of the remote shooting threads on each server. This means that multiple photos are captured in the same temporal instant, when a request is generated from the client-side. As for the remote shooting, also the pictures’ transfer and turning off services are synchronized. Indeed, a multi-threading programming approach has been followed to synchronize all threads [11]. Figure 4 shows the inbody hardware and software architecture, based on the client–server paradigm. The software has been developed using Java programming.

Fig. 4
figure 4

inbody network setup and its software architecture

Full size image

In the case of multiple cameras, the suface reconstruction of a subject involves two steps: cloud point generation, using, for example, the scale invariant feature transform (SIFT) algorithm proposed by Lowe [12], and the 3D mesh generation, using for example the Poisson reconstruction algorithm [13].

4 System optimization and mechanical design

In this section we deal with the mechanical design of the scanner, which has the primary objective of insuring a reliable and robust positioning for the image sensors. The design specifications have been obtained through a system optimization regarding the number and space arrangement of the image sensors.

Table 2 Design of experiments (DoE) to obtain the optimal configuration for the number and space arrangement of the image sensors
Full size table
Table 3 Design of experiments (DoE) results
Full size table

4.1 Constrained optimization

The system optimization has the objective to find the number and the space arrangement of the image sensors. For this objective, we planned a design of experiment procedure [14] using one pillar prototype which is able to move around a static object, a vertical tube of diameter 60 mm and length 1750 mm, for its 3D reconstruction. We choose this object since we plan to design a scanner which is able to digitize an ideal cylindrical area which will have contained the patient. We select three factors which we vary for each experiment: the layout, the number of pillars, and the number of cameras for each pillar. We setup a \(2^{3-1}\) fractional plan by changing the three factors in the two levels as indicated in Table 2. We use fixed values for the radius of the circunference (1200 mm) and for the semi-axes of the ellipses (1000 and 1400 mm): these values have been obtained by the cone opening angles of the selected camera in its vertical and horizontal planes. The levels on the number of the pillars have been selected considering that an accurate 3D reconstruction requires at least 60% overlap between two adiacent 2D images [15]. Table 3 reports the results of the first experiments in terms of average and standard deviation values with respect to the exact model of the tube. The configuration which guarantees the best accuracy is obviously the one with the highest number of pillars and cameras. However, for economical reasons, we select the configuration with a lower number of pillars, preserving the higher value for the number of cameras per pillar since they allow smaller standard deviations. For this reason, our optimization has been constrained by economics. The final solution has the following configuration: circular ring, 17 pillars, 7 cameras for pillars. As we can see in the experimental section of the paper, this configuration allows to satisfy the initial requirement for scanning human bodies.

Fig. 5
figure 5

Product design and development phases which leveraged the prototype manufacturing

Full size image

4.2 Product design and development

The previous statistical approach has lead to the design specifications for the mechanical system. The design of the mechanical structure able to position in the space the image sensors has followed the classical product design and development phasis [16, 17], as we can see in Fig. 5. We started by dividing the design in sub-components and generating some conceptual alternatives. We came up with three concepts: open scanner, semi-open scanner and closed scanner. In order to select the best design solution, we use Eligere, a distributed software platform for group decision making in engineering design [18,19,20]. This software is based on fuzzy AHP, a multiple criteria decision making method wherein the set of alternatives relies in a discrete space. The evaluation criteria for the three alternatives have been: simplicity; maintainability; aesthetic design. The optimal design has been selected to be the semi-open solution. The subsequent phase of engineering optimized the optimal solution from the mechanical and manufacturing point of view. The prototype that has been built in shown in Fig. 1.

Fig. 6
figure 6

3D medical scans obtained using inbody, in both configurations half-body and full-body. For the same patient, we show also the textured model

Full size image
Fig. 7
figure 7

3D whole-body textured reconstruction of a person using inbody

Full size image
Fig. 8
figure 8

A qualitative picture of the interactive digital manufacturing process of prostheses and orthoses using inbody

Full size image

5 Experimental validation

In this section we test and validate the instant scanning system in one of the most promising applications for 3D scanning in healthcare, the data acquisition and processing of anatomical structures for the design and fabrication of customized medical products. Figure 6 shows the 3D inbody scans which can be used for this particular purpose, while Fig. 7 shows a whole-body textured reconstruction of a person. Figure 8 illustrates the overall digital manufacturing process of prostetics and orthotics using inbody.

In the following, we describe the experimental setup used for the interactive digital manufacturing process of prostheses and orthoses, and how inbody behaves in this domain. We analyze how the main system requirements (instant capture and high accuracy) are satisfied in the digitization of human bodies for healthcare applications.

5.1 Experimental setup

The experimental setup is composed by

  • One inbody as system to digitize the surface of human bodies;

  • One prosthetics and orthotics computer aided design (P&O CAD) software for the design of the assistive device;

  • One robotic cell constituted by one 6−axis KUKA KR 30-3,Footnote 3 one motor spindle with automatic tool changer, and one rotating plate, for the fabrication of the assistive device.

5.2 Instant capture validation

We test the instant capture capability of inbody by using the currentTimeMillis function available for Java programming. The synchronization of the capture thread allows the acquisition of the human body in 50 ms.

5.3 Accuracy validation

We verify the accuracy of the system by scanning the torso of a static mannequin using inbody and a laser scanner, Polhemus FastSCAN SCORPION,Footnote 4 which has a certified absolute accuracy of 0.75 mm and practical accuracy of 0.13 mm. Figure 9 shows the comparison between the two scans. The average 3D deviations, which compute the minimum euclidean distance between two points from the two scans, are +0.90 mm for the positive side and −1.11 mm for the negative side, with a standard deviation of 1.27 mm. The laser scan is assumed as reference. The laser scan has 13,150 faces, while the inbody scan has a resolution of 68,750 faces in this experiment. Although an average deviation around 1 mm is a good achievement for a photogrammetric reconstruction, we have to consider that this data consider also the maximum and minimum errors, which occur in the border part of the models. In fact, if we do not consider the borders and we consider the central part of the model, the average deviations result in the order of tenths of millimeters. Table 4 reports the average computational time for the generation of the human body models.

Fig. 9
figure 9

inbody 3D reconstruction of a static mannequin compared with a laser 3D reconstruction. Errors in (mm)

Full size image

5.4 Discussion

inbody is as a photogrammetric scanner that has been developed using an integrated and interactive approach. It is easily implementable in a real manufacturing process, allowing interactive medical procedures. Indeed, for its validation, we integrate this system in the computer aided design / computer aided manufacturing (CAD/CAM) process of prosthetics and orthotics. Its instant capture allows a non-invasive data acquisition, while the surface reconstruction of the body surfaces requires an average time variable between three and six minutes. This processing time depends on the machine wherein the reconstruction software runs and on the scanning settings (half-body, full-body).

In our experiments, the inbody laptop client is represented by an Intel ®  Core ™  i7-4910MQ CPU (quad-core 2.50 GHz, Turbo 3.50 GHz), 32 Gb RAM 1600MHz DDR3L, NVIDIA ®Quadro ®K2100M w/2GB GDDR5 VGA machine.

The accuracy of the system is 0.21 ± 1.27 mm. This accuracy makes the use of the system attractive in most of the healthcare applications.

The photogrammetric scanner presented in this work, besides guarantee an instant capture and a good accuracy, it is capable of an high fidelity color reconstruction. This feature founds in the spreading of digital medical records in the next future the main application. The textured 3D models might be an useful monitoring mean for numerous cutaneous disease.

In healthcare, the correct measurement and shape reconstruction are the main uses of a body scanner. However, the capability to reconstruct the texture might open also other new interesting scenarios. In fact, in combination with a color additive manufacturing machine, it is possible to address the needing of biomorphic medical products, for istance, the orthosis or the prosthesis fabricated based on the patient’s skin.

In sum, the main features of this body scanning system make it attractive in developing interactive approaches for many medical domains, which require measurement, visualization, use and modification of accurate and textured human body models, digitized using a fast and noninvasive procedure.

Table 4 Average computational time required for the generation of the body surface models
Full size table

6 Conclusions

A novel body scanner is presented to measure and visualize anatomical structures for epidemiology, diagnosis, treatment and monitoring applications. It allows an instant data acquisition and a relatevely fast computational time for 3D models generation, whose accuracy suits for the healthcare domains. The ability to reconstruct the texture might open new interesting applications beyond the current scenarios, such as the development of biomorphic custom-made medical devices or the spreading of digital medical records of the patients.

The scanner has been developed using an integrated approach, where the mechanical design inherently interacts with the hard- and software design. The main features of the scanner might leverages interactive approaches in many medical domains which require to measure and visualize human bodies.

The availability in medical centers of such an instant system for results profitable for: (i) the patients, in their clinical experience, thanks to the minimally invasive procedure of acquisition; (ii) the medical practitioners, in having a system which results simple to use and to manage. The system might be used in research centers as well, since its modularity allows developers a simple extension with new capabilities. The authors believe in the importance to develop minimally invasive procedures when humans have to interact with the technologies, in particular in the medical field.