Reverse Engineering

Abstract

One of the most time-consuming aspects of creating 3D virtual models is the generation of geometric models of objects, in particular if the virtual model is derived (digitized) from a physical version of the object. A variety of commercially available technologies can be used to digitize objects at the molecular scale but also multi-storey buildings or even planets and stars. The process of 3D digitizing basically consists of a sensing phase followed by a rebuild phase. The sensing phase collects or captures raw data and generates initial geometry data, usually as a 2D boundary object, or a 3D point cloud. Sensing technologies are based on tracking, imaging, and range finding or their combination. The rebuild phase is internal processing of data into conventional 3D CAD and animation geometry data, such as NURBS and polygon sets. Finally, in most cases, the digitized objects must be refined by using the CAD software to gain CAD models of optimal quality which are needed in the downstream processes. Leading CAD software packages include special modules for such tasks. Many commercial vendors offer sensors, software and/or complete integrated systems. Reverse engineering focuses not only on the reconstruction of the shape and fit, but also on the reconstruction of physical properties of materials and manufacturing processes. Reverse engineering methods are applied in many different areas, ranging from mechanical engineering, architecture, cultural heritage preservation, terrain capture, astronomy, entertainment industry to medicine and dentistry.

1 Introduction

Engineering is the process of designing, manufacturing, assembling, and maintaining products and systems. There are two types of engineering: forward engineering and reverse engineering. Forward engineering (FE), or engineering design, is a process of creating a new part or a complete product, applying imagination, creativity and originality. Reverse engineering (RE) is a process of duplicating an existing part, assembly, or product, generally without any technical documentation. Chikofsky and Cross [1] defined RE as “the process of analysing a subject system to—identify the system’s components and their interrelationships and—create representations of the system in another form or at a higher level of abstraction”. Wang [2] described RE as “a process of measuring, analysing, and testing to reconstruct the mirror image of an object or retrieve a past event. It is a technology of reinvention, a road map leading to reconstruction and reproduction. It is also the art of applied science for preservation of the design intent of the original part”. Pham and Hieu [3] defined RE “as a process of analysing an object or existing system (hardware and software) to identify its components and their interrelationships and to investigate how it works to redesign or produce a copy without access to the design from which it was originally produced”. The focus of this chapter is on the acquisition of the shape of an industrial product.

Many modern machines were invented with inspiration from nature, or reinvented through reverse engineering based on what was observed in nature. The airplane is one of the most noticeable examples. One of the widely cited reverse engineering examples in the military is the Soviet Tupolev Tu-4 bomber. During World War II, three battle-damaged U.S. B-29 Superfortress bombers made emergency landings in the Soviet Union. Although most airplanes can be distinguished from one another by their respective characteristics, the similarity between the general characteristics of the B-29 and the Tu-4 bomber (Fig. 12.1) has led to a conclusion that the Tupolev Tu-4 was a replica of the B-29 [2].

Fig. 12.1

Boeing B-29 superfortress bomber (left) and tupolev Tu-4 bomber (right)

Nowadays, a three-dimensional geometric model is widely used in engineering. It is an intermediate representation shared among the participants in the product creation process and it contains the semantic information needed in the singular steps (e.g. simulation parameters, materials, tolerances and manufacturing entities). If the subject is an existing product whose digital model is unavailable, it is necessary to first rebuild it with appropriate reverse engineering techniques. For many years, reverse engineering methods have found wide application in manufacturing, industrial design and reverse innovative design (RID).

Answers to the questions why RE is needed and what is its role in Concurrent Engineering can be found in Sect. 12.2, accompanied with related studies. The RE process, data acquisition process and the most common non-contact scanning methods are described in Sect. 12.3. Importance of material characteristics, durability and life limitations are given in Sect. 12.4. Tools for RE, especially software tools for data processing, are presented in Sect. 12.5. Section 12.6 presents applications of the RE methods in different areas: automotive industry, railways, machinery, architecture, archaeology and medicine. Ethical and legal issues considering RE are discussed in Sect. 12.7, followed by brief conclusions and an outlook.

2 Background and Related Work

RE allows the duplication of an existing part by capturing its physical dimensions, features and material properties. The challenge for RE is to reproduce this part with an equivalent or preferably better functionality at lower costs. RE usually offers a good value for money only if the items to be reverse engineered reflect high development costs or will be reproduced in large quantities. But even if it is not cost effective, RE is sometimes the best choice for engineers who have to carry out a task of producing, for example, a part which is indispensable and crucial to the system. Furthermore, RE allows the study into an unknown or malfunctioning system to be carried out in order to enhance its efficiency and reliability. As to why the use of RE might be the best solution for accomplishing an engineering task, Raja [4, 5] and other authors quote the following:

• The original part or its design data are no longer available, but a customer needs the product for repair.

• Creating data to refurbish or manufacture a part for which there are no CAD data, or for which the data have become obsolete or lost.

• Inspection and Quality Control—Comparing a fabricated part to its CAD description or to a standard item.

• Some bad features of a product need to be improved, e.g. excessive wear.

• Analysing the good and bad features of competitors’ products.

• Exploring new avenues to improve product performance and features.

• Creating 3D data from an individual model or sculpture for animation in games and movies, or to create, scale, or reproduce artwork.

• Fitting clothing or footwear to individuals and determining the anthropometry of a population.

• Architectural and construction documentation and measurement.

• Generating data to create dental or surgical prosthetics, tissue engineered body parts, or for surgical planning.

• Documentation and reproduction of crime scenes.

2.1 Reverse Engineering in Concurrent Engineering

Although reverse engineering techniques have been applied in very different areas, they are applied in a similar manner. RE not only helps to rapidly reproduce a competitive product, but can also be used to study the behaviour of an existing system. In the forward development process, Müller et al. [6] proposed to apply simultaneously reverse engineering and forward engineering to achieve a more complete understanding of the system. In this way, RE reduces the number of faults in the development process and speeds up significantly the development of our own new products. Performances, accuracy and speed of various RE techniques depend crucially on the applied scanning method (Table 12.1).

Table 12.1 Three RE types based on the end-use applications and technical requirements

2.2 Related Work

The area of reverse engineering is developing in several directions: development of new methods and technologies, improvement of tools for data acquisition (hardware) and data processing (software) in terms of the accuracy and acceleration of the process itself, and usage of RE methods in new applications. Several important papers deal with the development in each of these areas.

A comprehensive overview of the state of the art of 3D sensing techniques and devices, their applications in a wide range of measurement problems in industry, cultural heritage, medicine and forensics can be found in [8]. The authors present an overview of techniques and sensors for the optical 3D measurement of surfaces and evaluate different approaches to highlight which method can be used and what are its main applications. They also give an insight into the results achieved in the mentioned applications. The overview of systems proposed in this paper yields a number of conclusive remarks. The first one concerns the cost of the equipment for 3D acquisition. The second remark concerns the fact that the use of 3D acquisition is not a trivial task: the systems are still rather complex to use and need skilled personnel to operate them. The third remark, which is crucial from the metrologist’s viewpoint, is the need for norms to guarantee the traceability of 3D measurements to recognised standards. As 3D acquisition is rather new, these norms are not yet mature. The final remark concerns the fact that 3D systems, in many complex metrological issues, may not represent the solution to the problem when used alone. The concurrent use of combinations of contact and non-contact systems, including 3D systems, may be required for a full metrological solution. The most common RE methods are described in Sect. 12.3, and applications of the methods in different areas in Sect. 12.6.

An analysis of a digitization system on the basis of its accuracy, effectiveness and the quality of distribution of points and triangular meshes in the field of reverse engineering can be found in [9]. The actual experimentation with simple or complex objects and different materials yields results that, in some cases, refute the effectiveness of those systems. In order to help in choosing a digitization system on the basis of its accuracy and the quality of distribution of points and triangular meshes, the authors compared a few digitization techniques. It is shown that measurements on real calibrated pieces and with different materials give greater uncertainties than those given by the manufacturers. According to the authors, the quality of the scanning system has been divided into: the accuracy of digitization, the digitization of the piece, the distribution of points, the roughness of the mesh, the mesh of edges and holes without meshing.

In [10], the authors evaluate recent advances in data acquisition and processing, and provide an overview from a manufacturing perspective. Success of generating a virtual representation of a physical object from a dataset of point clouds relies on reliable algorithms and tools. Whereas 3D scanners have become powerful, the performances of the corresponding software tools are perceived as unsatisfactory. Commercial 3D modelling tools lack the ability to deal with large amounts of data. End users often wish to automatically process a wide range of objects, possibly from a variety of data capture devices with different characteristics, to produce models in a variety of representations and accuracies. Many effective methodologies have been developed to solve various problems involved in data acquisition and processing. But without a sound background knowledge of mathematics and computer science, it is often hard to understand theoretical fundamentals of the methodologies. The authors give an overview of software tools for data processing through data filtering, data registration and integration, feature detection, 3D reconstruction, surface simplification and segmentation. Also, the authors list available 3D scanners, their manufacturers and stand-alone 3D data processing software tools. Tools, especially software tools for data processing, are presented in detail in Sect. 12.5.

Surface reconstruction from point clouds is fundamental in many applications. A brief overview of surface reconstruction methods and a literature review are available in [10]. In the topic of surface reconstruction and speeding up the process of data processing, there is still a lot of room for improvement. Many researchers are involved in this specific area. In [11], the authors proposed a surface meshing method capable of dealing with a great variety of surfaces such as those which are closed or not, orientable or not, uniformly sampled or not, with non-manifold intersections or without. The current implementation of the proposed method is roughly as fast as other recent popular methods. A new high-performance method for triangular mesh generation based on a mesh-growing approach is proposed in [12]. The performance of the proposed method has been compared with the performance of reference method with applications of the mesh-growing approaches to some benchmark point clouds and artificially noised test cases. The results show that the proposed method is competitive in terms of tessellation rate, quality of the generated triangles and produced defectiveness. In [13], the authors proposed a robust algorithm to surface reconstruction. The robustness of the proposed method and some other methods is demonstrated on several examples with noise and invalid data and with dense point clouds. In spite of limited application with respect to the size of the input point cloud, both in running times and memory use, the proposed method is proved to be a versatile tool for surface reconstruction.

Application of reverse engineering methods to new areas is shown on the example of new design methodology called RID proposed in [14]. RID methodology comprises advanced design methodologies that facilitate the acquisition of design knowledge and creative ideas for later reuse. RID is an integrated digital design methodology incorporating digitizing, modelling with shape and product definition parameters, CAE analysis-based product optimization and rapid prototyping (RP). Figure 12.2 shows a comparison between the workflows of conventional RE and RID. The core of RID is the feature based parametric solid model constructed from scanned data, for analytically shaped models as well as models with freeform shapes. For analytically shaped models, features with natural definition parameters will be extracted with remaining freeform shapes fitted. Since freeform shapes stand at the centre of RE historically, the ability to generate parametric models with product definition parameters from models with freeform shapes is essential.

Fig. 12.2

Workflow of conventional RE versus RID [14]

In [15], the authors present a new technique for reconstructing a single shape and its deformation (non-rigid motion) from a temporal sequence of point clouds from real-time 3D scanner data. In addition, the authors give a brief overview of related work in reconstructing correspondences of time variant geometry, review the related work in the area of deformation modelling and compare it to the proposed technique. The authors apply the technique to several benchmark data sets, increasing significantly the complexity of the data that can be handled in comparison to the previous work, while at the same time improving the reconstruction quality.

Review and classification of methods for the acquisition of surface geometry or volumetric descriptions of objects or phenomena with complex optical characteristics (transparent, specular, etc.) can be found in [16]. While the 3D acquisition of opaque surfaces is a well-studied problem, transparent, refractive, specular and potentially dynamic scenes pose difficult problems for acquisition systems. The acquisition of digital models of such objects is far from being a solved problem. This report is providing a reference for and an introduction to the field of the transparent and the specular object reconstruction. Figure 12.3 shows an example of transparent object scanning.

Fig. 12.3

Fluorescent immersion range scanning. A photograph of an acrylic glass object (left), a direct volume rendering of a recovered voxel model (middle, left), a cut-away iso-surface view (middle, right) and a realistic rendering (right) [16]

3 Methods of Reverse Engineering

When the shape of an object should be reconstructed without access to its design, a RE process has to be applied. The selection of RE technology depends on various factors: size of the object, its complexity, its material (hard or soft), its finish (shiny or dull), its geometry (internal or external), the required accuracy, etc.

3.1 Reverse Engineering Process

The RE process consists of three phases: object scanning, point processing and generation of 3D geometrical model (Fig. 12.4). The first phase, i.e. data acquisition, is a crucial part of RE. In this phase RE hardware is used to collect geometric data that represent a physical object. The outputs are point clouds or 2-D cross-sectional images that define the scanned object geometry. RE data produced by RE hardware are transformed into a 3D geometric model by using the RE software. The final outputs of this data processing are a polygon mesh or a NURBS mesh (non-uniform rational B-splines mesh). Polygon models, usually in the STL, VRML, or DXF format, are used for RP, laser milling, 3D graphics, simulations and animations. NURBS surfaces or solids are used in computer-aided design, manufacturing and engineering applications (CAD-CAM-CAE).

• 1. 3D Scanning: In recent years, an enormous number of various scanning technologies for capturing the shape of a physical object have been made available at reasonable prices. The selection of appropriate tools and techniques has become very challenging. The starting point for all of them is the acquisition of a set of X-Y-Z coordinates in space, called point clouds, in one of the convenient output formats. The clouds are then processed in the second phase of RE process to provide an applicable output for the re-creation of the scanned object.

• 2. Data Processing: This phase involves importing the point cloud data, reducing the noise in the data collected and reducing the number of points [4]. These tasks are performed using a range of predefined filters. It is extremely important that the users have very good understanding of the filter algorithms so that they know which filter is the most appropriate for each task. This phase also allows us to merge multiple scan data sets. Sometimes, it is necessary to take multiple scans of the part to ensure that all required features have been scanned. This involves rotating the part; hence each scan datum becomes very crucial. Multiple scan planning has a direct impact on the point processing phase. Good datum planning for multiple scanning will reduce the effort required in the point processing phase and also avoid introduction of errors from merging multiple scan data. A wide range of commercial software is available for point processing. The output of the point processing phase is a point cloud data set in the most convenient format.

• 3. Generation of 3D Model: The generation of 3D surface or solid CAD models from point data is probably the most complex activity within RE because potent surface fitting algorithms are required to generate surfaces that accurately represent the three-dimensional information described within the point cloud data sets. Most CAD systems are not designed to display and process large amounts of point data; as a result, new RE modules or discrete software packages are generally needed for point processing [4]. The RE software allows the user also to compare the two different data sets. This process is very useful for inspections of manufactured parts. In such cases, the designed CAD model is imported by appropriate software and overlaid with the scanned point cloud data set of the machined part.

Fig. 12.4

The generic process of reverse engineering. Adapted from [4]

3.2 Data Acquisition

The taxonomy of hardware used for collecting geometric data of an explored object is shown in Fig. 12.5. There are two main non-destructive technologies for RE data acquisition: contact and non-contact.

Fig. 12.5

A taxonomy of scanning hardware according to [3, 8, 17]

When contact technology is used for scanning, tactile measurement machines (Fig. 12.6) have to touch the object of interest to measure its geometry [4]. In such a case either the object or the measurement probe could be damaged. Some commercial coordinate measuring machine (CMM) systems claim to be non-contact devices, but they still require a measurement probe to be quite close to the point of measurement, just not touching it. On the other hand, non-contact optical systems make measurements at some standoff distance. Furthermore, if the temperature of the surface is too hot or too cold, the heat transfer could damage the measurement probe. With a touch probe, a CMM (or a user) must carefully select a measurement path that properly covers the surfaces of an object but that avoids wedging the probe into tight spaces. The CMM must use a path that covers the object and yet obeys the physical constraints imposed by the interaction between the object and the probe. In practice, however, due to their disadvantages, optical 3D scanning systems are less used than CMMs. One of the most significant features is accuracy. Ultrahigh accuracy CMMs work in the 1–2 μm range, and more moderate CMMs (in terms of cost) in the 10–20 μm range. Computer vision methods cannot compete—as of yet—with these levels where most systems operate in the submillimeter range of accuracy. The trend, however, indicates that CMMs have plateaued. Only a few microns difference in accuracy can result in more than a €100 K increase in cost. On the other hand, computer vision research indicates that greater accuracy is yet to come with more precise lasers and higher resolution imaging sensors.

Fig. 12.6

Multi-axis tactile measurement machines: Spin Arm M, 7-axis articulated measurement arm and Crysta Apex, 3-axis CNC CMM (both from Mitutoyo)

Contact scanners employ contact probes that automatically follow the contours of a physical surface (Fig. 12.7). In the current marketplace, contact probe scanning devices are based on CMM technologies, with a tolerance range of +0.01 to 0.02 mm. However, depending on the size of the part scanned, contact methods can be slow because each point is generated sequentially at the tip of the probe. Tactile device probes must deflect to register a point; hence, a degree of contact pressure is maintained during the scanning process. This contact pressure limits the use of contact devices because soft, tactile materials such as rubber cannot be easily or accurately scanned.

Fig. 12.7

Contact versus non-contact sensing: the scanning probe cannot touch the sharp inside corner C, and the laser beam missed vertical walls parallel to laser axis [4]

Contact methods use sensing devices with mechanical arms, CMMs and computer numerical control (CNC) machines, to digitize a surface. There are two types of data collection techniques employed in contact methods:

• 1. Point-to-point sensing with touch-trigger probes and

• 2. Analogue sensing with scanning probes.

In the point-to-point sensing technique, a touch-trigger probe is used that is installed on a CMM or on an articulated mechanical arm to gather the coordinate points of a surface [18]. A manually operated, articulated mechanical arm with a touch-trigger probe allows multiple degrees of freedom (DOF) of movement to collect the measurement points (Fig. 12.6, left). A CMM with a touch-trigger probe can be programmed to follow planned paths along a surface. A CMM provides more accurate measurement data compared to the articulated arm. However, the limitation of using a CMM is the lack of number of DOF so that a CMM cannot be used to digitize complex surfaces in the same way as an articulated arm. In analogue sensing, a scanning probe installed on a CMM or CNC machine is used (Fig. 12.6, right). The scanning probe provides a continuous deflection output that can be combined with the machine position to derive the location of the surface. When scanning, the probe stylus tip contacts the feature and then moves continuously along the surface, gathering data as it moves. Therefore, throughout the measurement, it is necessary to keep the deflection of the probe stylus within the measurement range of the probe. The scanning speed in analogue sensing is up to three times faster than in point-to-point sensing. The more advanced CMM systems allow operators to upload a CAD model of the object and then the CMM uses this model for the path planning strategy. The CMM will analyse the CAD model to identify critical points and regions.

Non-contact Scanners: A variety of non-contact scanning technologies available on the market capture data with no physical part contact [4]. Non-contact devices use lasers, optics and charge-coupled device (CCD) sensors to capture point data. Although these devices capture large amounts of data in a relatively short period of time, there are a number of issues related to this scanning technology:

• The typical tolerance of non-contact scanning is within ±0.025 to 0.2 mm.

• Some non-contact systems have problems generating data describing surfaces, which are parallel to the axis of the laser beam (Fig. 12.7).

• Non-contact devices employ light within the data capture process. This creates problems when the light impinges on shiny surfaces, and hence some surfaces must be prepared with a temporary coating of fine powder before scanning.

These issues restrict the use of remote sensing devices to areas in engineering where the accuracy of the information generated is secondary to the speed of data capture. However, as research and laser development in optical technology continue, the accuracy of the commercially available non-contact scanning device is beginning to improve (Table 12.2).

Table 12.2 Advantages and disadvantages of the contact and non-contact techniques [7]

3.3 Optical Scanning Methods

There is a remarkable variety of 3D optical techniques, and their classification, as given in Fig. 12.5, is not unique. In this section, some of the more prominent approaches are briefly explained and compared (Table 12.3).

Table 12.3 Comparison of optical range imaging techniques [8]

Active optical devices are based on an emitter, which produces some sort of structured illumination on the object to be scanned, and a sensor, which is typically a CCD camera and acquires images of the distorted pattern reflected by the object surface [17]. In most cases the depth information is reconstructed by triangulation (Fig. 12.8), given the known relative positions of the emitter-sensor pair. The emitter can produce coherent light (e.g. a laser-beam) or incoherent light; in both cases, a given light pattern (point-wise, stripe-wise or a more complex pattern) is projected on the object surface. Different technologies have been adopted to produce the structured light pattern: laser emitters, custom white light projectors (which can filter light by means of a glass slide with a stripe pattern engraved via photo-lithography), low cost photographic slide projectors and finally digital video projectors.

Fig. 12.8

A scheme of a typical optical scanner, where the 3D positions of the sampled points are computed by triangulation given the sampled point projection B(C) on the sensor plane and the known relative position/orientation of the emitter and the sensor [17]

Passive methods reconstruct a 3D model of an object by analysing the images to determine coordinate data [3]. It is similar to (active) structured-light methods in its use of imaging frames for 3D reconstruction; however, in passive methods, there is no projection of light sources onto the object for data acquisition. The typical passive methods are shape from shading and shape from stereo.

Laser triangulation method is a technique which uses the law of sine to find the coordinates and distance of an unknown point by forming a triangle with it and two known reference points [2]. In Fig. 12.8, A and B are the two reference locations given by the camera and the sensor locations, and C is the location of the object point of interest. The distance from A to B can be measured as c, and the angles α and β can also be measured. Following the law of sine the distances a and b can be calculated. The coordinates of A and B are known, then the coordinate of C can also be calculated. The same principle is employed in various other scanning methods.

Time-Of-Flight (TOF) method uses the radar time-of-flight principle [8]. The emitter unit generates a laser pulse, which impinges onto the target surface (Fig. 12.9). A receiver detects the reflected pulse, and suitable electronics measures the roundtrip travel time of the returning signal and its intensity. Reflective markers must be put on the target surfaces. The measurement resolutions vary with the range. For large measuring ranges (15–100 m), time-of flight sensors give excellent results. On the other side, for smaller objects, about 1 m in size, attaining 1 part per 1,000 accuracy with time-of-flight radar requires very high speed timing circuitry, because the time differences are extremely short. In many applications, the technique is range-limited by allowable power levels of laser radiation, determined by laser safety considerations. Additionally, time-of-flight sensors face difficulties with shiny surfaces, which reflect little back-scattered light energy except when oriented perpendicularly to the line of sight.

Fig. 12.9

Time-Of-Flight system (TOF) (left) measures the time required for a laser pulse to travel to and return from an object. Continuous wave system (right) is a variation on the TOF method: distance is computed by comparing the phase shift between an emitted wavelength and the received light [18]

Structured-light systems project a predetermined pattern of light onto the object, at a known angle. An image of the resulting pattern, reflected by the surface, is captured, analysed and the coordinates of the data point on the surface are calculated by using a triangulation method. The light pattern can be (i) a single point; (ii) a sheet of light (line); and (iii) a strip, grid, or more complex coded light (Fig. 12.10). The CCD camera, the object and the light source form the triangulation geometry (Fig. 12.11). The accuracy of these methods is primarily a function of the camera resolution and secondarily of geometric dimensions and illumination precision. System geometry and illumination are not as critical. Thus, structured-light systems offer a more practical solution than passive stereographic systems in achieving the accuracy necessary for an RE system.

Fig. 12.10

Different light patterns used in structured-light techniques [3]

The most commonly used pattern is a sheet of light, generated by fanning out a light beam. To improve the capturing process, the light pattern containing multiple strips is projected onto the surface of an object. The strips must be coded to enable the recording without ambiguity. Structured-light systems have the following strong advantages compared to laser systems: (i) the data acquisition is very fast (up to millions of points per second), (ii) colour texture information is available, (iii) structured-light systems do not use a laser and because of that they have the advantage of being inherently eye-safe. These features have resulted in favouring structured-light systems for digitizing images of human beings.

Fig. 12.11

A simple approach to recover the shape of an object: shape from contours (or silhouettes). The silhouette and the point of observation for each view form a cone containing the object. The intersection of multiple cones is an estimate of object shape. Shape from contour techniques, however, fail at recovering object concavities [19]

Moiré interferometry is used to measure tiny deformations of solid bodies, caused by mechanical forces, temperature changes, or other environmental changes [20]. It has been applied for studies of composite materials, polycrystalline materials, layered materials, piezoelectric materials, fracture mechanics, biomechanics, structural elements and structural joints. It is practiced extensively in the microelectronics industry to measure thermally induced deformation of electronic packages. Moiré interferometry combines the simplicity of geometrical moiré with the high sensitivity of optical interferometry, measuring in-plane displacements (Fig. 12.12). It is characterised by a list of excellent qualities. Moiré interferometry has a proven record of applications in engineering and science.

Fig. 12.12

Schematic diagram of four-beam moiré interferometry [20]

3.4 Other Non-contact Scanning Methods

Computer Tomography (CT) allows three-dimensional visualization of the internals of an object. It provides a large series of 2D X-ray cross-sectional images taken around a single rotational axis [3]. By projecting a thin X-ray or Y-ray beam through one plane of an object from many different angles and measuring the amount of radiation that passes through the object along various lines of sight, a cross-sectional image for the scanned surface is reconstructed (Fig. 12.13). CT is widely used for medical applications; however, it has been extended and adapted to a wide variety of industrial and 3D modelling tasks. Today, high-resolution X-ray CT and micro CT scanners can resolve details as small as a few tens of microns, even when imaging objects are made of high-density materials. It is applicable to a wide range of materials, including rock, bone, ceramic, metal and soft tissue.

Fig. 12.13

Working principle of a CT scanner [3] and CT scanner for medical applications

Magnetic Resonance Imaging (MRI) is a state-of-the-art imaging technology that uses magnetic fields and radio waves (Fig. 12.14) to create high-quality, cross-sectional images of the body without using radiation [3]. When hydrogen protons in the human body are placed in a strong magnetic field, by sending in (and stopping) electromagnetic radio-frequency pulses, the protons emit signals. These signals are collected and processed to construct cross-sectional images. Compared to CT, MRI gives superior quality images of soft tissues such as organs, muscle, cartilage, ligaments and tendons in many parts of the body.

Fig. 12.14

Magnetic resonance imaging (MRI): scanner and gradient magnets [21]

CT and MRI are powerful techniques for medical imaging and reverse engineering applications; however, they are the most expensive techniques in terms of both hardware and software for data processing (Table 12.4).

Table 12.4 Comparison of performances of various non-contact scanning technologies. Data extracted from [3]

4 Material Characteristics, Durability and Life Limitation

Material characteristics are the cornerstone for material identification and performance evaluation of a part made by using reverse engineering [2]. The determination of relevant material characteristics and their equivalency requires a comprehensive understanding of the material and the functionality of the part that was made of this material. To convincingly argue which of the mechanical, metallurgical and physical properties are the most relevant material properties that need to be evaluated in a reverse engineering project, the engineer has to carry out at least the following elaboration:

• 1. Property criticality: Explain how critical this relevant property is to the part’s design functionality.

• 2. Risk assessment: Explain how this relevant property will affect the part performance, and what the potential consequence will be if this material property fails to meet the design value.

• 3. Performance assurance: Explain what tests are required to show the equivalency to the original material.

Mechanical properties include ultimate tensile strength, yield strength, ductility, fatigue endurance, creep resistance and stress rupture strength. Metallurgical properties refer to the physical and chemical characteristics of metallic elements and alloys, such as the alloy microstructure and chemical composition. Physical properties usually refer to the inherent characteristics of a material (i.e. density, melting temperature, heat transfer coefficient, specific heat, electrical conductivity, etc.).

Many mechanical components have life limits in their service due to the deterioration of their durability over time. Although it is more technically challenging to reverse engineer a life-limited part, market demands and higher profit margins provide strong incentives for their reproduction using reverse engineering. The life cycle of a part is determined either by the total load cycles the part has experienced or by the total time period the part has been placed in service. The reverse engineered parts are expected to maintain the same level of safety attributable to the integrity of materials and machine functionality. A mechanical component usually fails due to excessive elastic deformation, excessive plastic deformation, fracture, environmental effects or a combination of these factors. The prevention of part failure requires full knowledge of material characteristics, loading condition and service environment. A thorough understanding of the part design functionality and operation is critical for reproducing an equivalent mechanical component using reverse engineering.

Fatigue is a dynamic and time-dependent phenomenon. When a component is subject to alternating stresses repeatedly, it fails at a much lower stress than the material yield strength due to fatigue. Most mechanical failures are related to dynamic loading; therefore, the safety assessment in fatigue life plays a critical role in reverse engineering.

The performance of a reverse engineered part compared to its original equipment manufacturer (OEM) counterpart is vitally critical to the success of a reverse engineering project. The performance of these parts is usually evaluated based on three primary criteria: engineering functionality, marketability and safety. From an engineering functionality perspective, part performance is judged based on its structural integrity and system compatibility [2].

5 Tools for Reverse Engineering

Although the domain of RE is very broad, as mentioned in Sect. 12.3.1, a conventional RE process involves the following three steps [14]:

• 1. 3D scanning of physical projects, typically generating a point cloud.

• 2. Data processing such as noisy data removal, registration, sampling, smoothing, topology repair and hole-filling.

• 3. Surface reconstruction from mesh or point cloud by direct surface fitting or surface reconstruction through curves such as section curves and feature lines.

Data acquisition and data processing include both hardware and software tools and the results of RE are usually surfaces that need to be imported into a 3D CAD software. A hardware system acquires point clouds or volumetric data by using established mechanisms or phenomena for interacting with the surface or volume of an object of interest. There are many types of 3D scanning or data acquisition systems available, as shown in Sect. 12.3.2, which differ in their characteristics such as accuracy, speed, working volume, environmental operating constraints, reliability, cost, etc.

A software system processes raw point clouds or volumetric data and transfers them into a virtual representation of the object such as surfaces and features. One of the critical tasks of vision-based manufacturing applications is to generate a virtual representation and its success relies on reliable algorithms and tools. Processing of raw scanned data or data cleaning is very important since curves and reconstructed surfaces are based on the mesh model. Data processing and surface reconstruction is the centre piece of a RE process. The interpretation of raw data to a required computer model is a complicated process, and it involves the following typical issues [10]:

Data Filtering: Raw data include noise, distorted and invalid data caused by the hardware system and/or the environment. The acquired data must be filtered to eliminate invalid data. Point data can be invalid due to many reasons, often caused by reflectivity of surface elements, objects in the background, moving objects, atmospheric effects, bright objects, etc. The elimination process often has to be done interactively since no automatic method can foresee all possible causes. The acquired data also may be filtered to reduce the level of noise caused by the precision of the data acquisition system or to reduce the number of points in a dense area.

Data Registration and Integration: Registration and integration are needed for two different purposes, first, the combination of several point clouds taken from different observation points and second, the referencing of the object in a global coordinate system. A vision device can capture the surface facing the device in the field of view. Therefore, multiple views are needed to acquire data over the entire surface, and the data from different views have to be integrated. The registration is used to determine the transformation of data from two different views so that data can be integrated under the same coordinate system. Integration is the process of creating a single surface representation from the sample points of two or more range images.

Surface (3D) Reconstruction: Surface reconstruction from point clouds is fundamental in many applications. Using the raw point clouds or volumetric data acquired from an unknown surface, an approximation of the surface can be constructed and used to compare it with CAD models or for surface-based automated programming. Reconstruction methods can be classified into two types: the computational geometry approach focuses on the piecewise-linear interpolation of unorganised points and defines the surface as a carefully chosen sub-set of the Delaunay triangulation in a Cartesian coordinate system, and the computer graphics approach focuses on the visual quality of the resulting model without constraining the surface to interpolate the sampled points.

Surface (Data) Simplification and Smoothing: A compact approximation of a shape can reduce memory requirements and accelerate data processing. It can also accelerate computations involving shape information, such as finite element (FE) analysis, collision detection, visibility testing, shape recognition and display. Simplification is useful to make storage, transmission, computation and display more efficient.

Data Segmentation: Segmentation involves the partitioning of a given image into a number of homogeneous segments in such a way that the union of any two neighboring segments yields a heterogeneous segment. Segmentation refers to a process for extracting the selected regions of interest from the rest of data using automated or manual techniques. Data filtering and segmentation are two different aspects of the same problem. A good filtering process should distinguish between a set of significant regions and the border between them. Such a filtering process assumes, implicitly, that the segmentation is known.

Feature Detection: Feature detection is used to recover a high-level geometric description from the lower-level geometric representation of a part. Defects or some basic elements on a surface can be dealt with as features. Examples of such features include size, position, contour measurement via edge detection and linking, as well as texture measurements on regions. Feature detection is used to identify defects with certain features or validate if the acquired data fit a specific feature.

Data Comparison: A reference model is usually available for data comparison. Data comparison calculates the derivations or differences between the physical model and the reference model. It can be applied to inspection, surface control, or CAD model comparison. For example, (i) in feature detection, point clouds can be used to measure geometric elements such as planes, cylinders, circles, spheres and boundaries; and (ii) in monitoring and control, as-designed and as-built models are compared so that the deviation (average error), tolerance and distribution can be evaluated.

Many methodologies have been developed to solve various issues involved in data processing and some of them are mentioned in [10]. Available software tools for data processing include an extensive collection of modules for different purposes ranging from scanner control to 3D modelling.

In recent years, the RE process has not only been used for scanning and converting data into a 3D surface, but also into solid parts. RP is often used in industry due to its capability of creating 3D parts with complex geometries. To fabricate a part by using RP processes directly from a representation in the form of point cloud data, the necessity of integrating data processing in RE and RP is comprehensively established. One of the methods for the direct generation of RP models from arbitrarily scattered cloud data can be found in [22].

6 Use Cases

Application of the RE methods is widespread in various fields of human activity. Some of the applications are illustrated by the cases to be discussed next. The RE methods are often integrated with other methods, thus representing a multidisciplinary approach to problem solving.

6.1 Automotive

In the automotive industry, the RE methods are for instance used in the following areas: redesign, reconstruction and dimensional control. The following two cases are similar examples of vehicle reconstruction with different final uses. In order to make the required design changes, a 3D model of original vehicle parts had to be created and measured. The surface of such complex objects and specific points on the object can be digitised using various optical measuring systems. In the following tasks, presented in [23], two measuring systems were used: TRITOP (an optical 3D CMM) and ATOS (an optical 3D scanner).

One of the tasks in the development of a racing vehicle for the Dakar Rally was to fit the driver and co-driver’s doors, used in a serial production of SUVs, onto the space frame of the racing vehicle to be developed. The development of completely new doors, with high demands placed on water and dust proof sealing, only for this specific application would be too expensive and too complicated (Fig. 12.15).

Fig. 12.15

The ATOS measurement (above, left), the 3D models made up of the STL files (above, right), a 3D model of the racing vehicle for the Croatian Dakar Rally Team (down) [23]

One of the fields of the automotive industry where the RE methods can be widely used is the adaptation of serial production vehicles to the needs of persons with disabilities. In this case, the idea was to develop a rear door opening mechanism which would perform a translatory movement of the door, so that a person with disabilities could put their wheelchair more easily behind the driver’s seat (Fig. 12.16).

Fig. 12.16

The TRITOP/ATOS measurement (left), a CAD model of the new rear door opening mechanism (right) [23]

The obtained 3D models represent a good basis for the design interventions and various analyses and simulation tasks. The use of optical measurement systems results in shorter overall development time.

6.2 Railway

Welded swivel-trucks (Fig. 12.17) are used in many European trains. Each swivel-truck needs to be measured and delivered with a measuring protocol. The measurement of these swivel-trucks, a task that a few years ago could have only been performed using tactile 3D CMMs or measuring arms, can be carried out easily and efficiently with a portable TRITROP photogrammetry system.

Fig. 12.17

Finished swivel-truck (left), a frame detail with the reference points and marked features (middle), the measuring process (right) [24]

With TRITOP and an automated evaluation routine, one person is able to perform the measurement of such a swivel-truck and create a measurement report (Fig. 12.18) within forty minutes.

Fig. 12.18

Example of a measurement report. The deviations from the CAD data are displayed and evaluated (e.g. passed, failed, colour markings) [24]

According to [24], this time is required to apply the reference points, mark the characteristic features, place the scale bars, record about fifty images and transfer them to a laptop, carry out an automatic evaluation (the definition of the marker lines, the alignment of the measurement data with the nominal data, the calculation of the deviations) and to prepare and print out a measurement report.

6.3 Machinery

The case presented in [25] covers the measurement of a large iron casting for a wind turbine gearbox with an optical LED-based triangulation system (Fig. 12.19). As the castings undergo a final machining process, it is important to know the amount of excess material. The presence of sufficient excess material will be precisely determined by the alignment of the part on the machine. By measuring the castings before machining, it is possible to determine in advance the best suited alignment and detect the castings that deviate too much to fabricate good parts, so that at least the machining costs can be saved.

Fig. 12.19

Camera-based triangulation system in the back, the casting in the front (left), coloured dots indicating the deviation from the CAD file of the casting (middle), a colour plot of the casting scan compared with the CAD file of the machined part (right) [25]

The results show that a systematic inspection of castings is an added value of the production process. By measuring the part, it is possible to determine the most suitable alignment for machining, so that the presence of sufficient excess material is guaranteed over the whole part. It is also possible to detect in advance bad castings, so that machining costs can be saved.

The RE methods are applied to various sizes of machine elements, from the production of replacement gaskets, presented in [26] to the retrofit of turbines presented in [27]. The main reasons for the turbine retrofit are extending turbine life time, improving reliability and operational flexibility, decreasing specific heat consumption and improvement of inner thermodynamic effectiveness. Typical steps which include RE methods in a retrofit process are shown in Fig. 12.20.

Fig. 12.20

Retrofit of a turbine [27]

6.4 Architecture/Archaeology

In architecture, building reconstructions, city planning and similar projects a 3D data capturing of smaller scenes and large areas is needed. In such cases remote sensing is required. The most popular methods used are: static terrestrial laser scanning, terrestrial cinematic laser scanning from ground vehicles and airborne laser scanning from aircraft. All of them have their limitations. For projects that include a rapid and cost effective 3D data capturing of larger street sections, especially if they include tunnels (Fig. 12.21), terrestrial cinematic laser scanning could be the best solution.

Fig. 12.21

A tunnel profile measured by the terrestrial cinematic 360° four laser scanning system StreetMapper [28]

Documentation of architectural and archaeological sites and monuments is an activity that requires the capturing of information from different sources. Experience has shown that it is possible to provide the necessary information with the required accuracy and completeness only by the integration of multisource data. A parallel use of geodetic and surveying measurements and photogrammetric data acquisition with imagery and terrestrial laser scans has proven to be an ideal combination, especially in large and complex monuments.

Fig. 12.22

View of the obelisk tomb and the Bab As-Siq triclinium in Petra/Jordan and a 3D view of the four-point clouds collected [29]

A successful 3D modelling of the Obelisk Tomb was achieved by integrating laser scanning technology with photogrammetry (Fig. 12.22). The approach combines the 3D models (developed from the range-based data) with multiple high resolution external images to yield photorealistic 3D models. The subpixel accuracy in the 2D-3D co-registration process between the images and the model is fundamental to matching accurately the texture from the imagery to the final 3D model without distortion. The presented approach is not only suitable for yielding high quality perspective views and photorealistic 3D models but it is also suitable for making amazing reality-based movies.

6.5 Medical Applications

Medical Reverse Engineering (MRE) is aimed at using the RE technology to reconstruct 3D models of the anatomical structures and biomedical objects for the design and manufacturing of medical products as well as Biomedical Engineering research and development. Different concepts and methodologies are provided to understand fundamentally the MRE processes and workflow. According to [7, 30], the key MRE applications are personalised implants for bone reconstruction, dental implants and simulations, surgical tools, medical training, vision science and optometry, orthopaedics, ergonomics, orthosis, prosthesis and tissue engineering. In addition, the RE methods are today indispensable in the Virtual Reality Surgical Planning (VRSP) process, whose flowchart is shown in Fig. 12.23.

Fig. 12.23

Flowchart of the virtual reality surgical planning process [31]

6.5.1 Dentistry

In the case presented in [32], laser scanning was used to evaluate, by indirect methods, the accuracy of computer-designed surgical guides in the oral implant supported rehabilitation of partially or completely edentulous patients. Five implant supported rehabilitations for a total of twenty-three implants were carried out by computer-designed surgical guides, performed with the master model developed by muco-compressive and muco-static impressions. For all the cases, the VRSP process, starting from the 3D models obtained by the dental scan CT data, was performed. The implants were inserted in the pre-surgical casts in the positions defined in the process of virtual planning. These positions were acquired by three-dimensional optical laser scanning and compared with the laser scans of the intraoral impressions taken post-operatively. A comparison between the post-surgical implant replica positions and the positions in the pre-operative cast, made for five patients, shows the standard deviations within the range, which are absolutely negligible in the surgery (Fig. 12.24).

Fig. 12.24

The master model is oriented and drilled so that the implant analogues can be placed in it (left), a CAD model of the post-surgical cast (middle), CAD model comparison (right) [32]

The results of this research demonstrate an accurate transfer of the implant replica position by virtual implant insertion into both a pre-operative and a post-operative cast, obtained from impressioning. In previous studies, the evaluation of the implant positions has required a post-surgical CT scan. With the indirect methods, using laser scanning techniques, this extra radiation exposure of the patient can be eliminated.

In the case presented in [33], the author’s first objective was to assess the strain magnitude and distribution within the 3D trabecular bone structure around an osseointegrated dental implant loaded axially. The second objective was to investigate the relative micromotions between the implant and the surrounding bone. In order to reach these objectives, a μCT-based FE model of an oral implant implanted into a Berkshire pig mandible was developed along with a robust software methodology. The FE mesh of the 3D trabecular bone architecture was generated from the segmentation of μCT scans. The implant was meshed independently from its CAD file obtained from the manufacturer. The meshes of the implant and the bone sample were registered together in an integrated software environment (Fig. 12.25). A series of non-linear contact FE analyses, considering an axial load applied to the top of the implant in combination with three sets of mechanical properties for the trabecular bone tissue, was devised (Fig. 12.26).

Fig. 12.25

Bone-implant complex (left), a 3D STL surface representation of the segmented μCT bone-implant complex (middle), the application of load (red arrow) to the FE model of the bone-implant complex and the enforcement of boundary conditions (blue surfaces) (right) [33]

Fig. 12.26

Threshold plot showing the strain magnitude (equivalent micro strains) distribution within the bone micro architecture (left), an open view of the implant-bone complex showing the local displacements (micromotions of the bone with respect to the implant) of the trabecular architecture (right) [33]

The high level of resolution in the FE mesh of a novel μCT-based 3D FE model of the trabecular bone structure provided a new insight into the complex bone strain distribution pattern and showed that the calculated level of strain and micromotions in response to an axial load is in some qualitative and quantitative agreement with published experimental data, thus confirming the usefulness of μCT-based FE models in dental mechanics.

6.5.2 Surgery

The VRSP process is used in the cases of bone fracture with a complex geometry, presented in [34]. The production of a copy of the fracture or a deformity in a bone can be one of the important applications of the integration between two modern computer-based technologies, reverse engineering and RP (Fig. 12.27). This case presents the use of medical CT/MRI scanning, three-dimensional reconstruction, anatomical modelling, computer-aided design, RP and computer-aided implantation in treating a complex fracture of acetabulums, calcaneum and medial condyle of femur (Hoffa’s fracture). This methodology reduces the surgical time, lowers the requirement of an anesthetic dosage and decreases the intraoperative blood loss.

Fig. 12.27

Preoperative X-rays—a Judet obturator view (left), the stereolothographic image of the fractured acetabulum (middle, left), a rapid prototyping (RP) model of the fractured acetabulum (middle, right), a postoperative Judet view of the acetabulum (right) [34]

6.5.3 Prosthesis Development

In [35] the authors proposed a new 3D design paradigm for the development of custom-fit soft sockets for lower limb prostheses. The new paradigm is centred on a digital model of the selected part of the human body and it is completely based on a computer-aided modelling and simulation of the two interfacing parts: the stump and the socket (Fig. 12.28). In such a context, different issues related to the human body are considered: acquisition of the stump morphology, generation of a complete virtual model including both the external shape (skin) and geometry of the internal parts (muscles and bones) and mechanical characterisation of the stump, simulation of the socket-stump interaction, realisation of the physical prototype.

Fig. 12.28

Reverse Engineering equipment and techniques for morphology acquisition (up, left), digital models of the stumps of four amputees (up, right), the generation of the 3D model of the socket (down, left), the FEM simulation of the socket wearibility (down, right) [35]

According to the authors, the proposed approach yields a better quality of the final product, a shorter involvement of the amputee implying a lower psychological impact, a limited use of physical prototypes and a shorter development time.

7 Ethical and Legal Issues

Intellectual Property Rights (IPRs) have become the key issue of the global innovation policy (see Chap. 18). They are generally protected by utility patents, design patents and copyright, but their strength varies from country to country. The Agreement on Trade Related Aspects of Intellectual Property Rights (TRIPS), signed in 1994 as a founding element of the World Trade Organisation (WTO), represents the most important attempt to establish a global harmonisation of the Intellectual Property protection. TRIPS, although an international treaty that obligates member states of the WTO to protect trade secrets, neither requires nor sanctions a reverse engineering privilege [36]. The World Intellectual Property Organisation (WIPO) assists developing countries in the implementation of TRIPS [37]. On the European level, the European Union Directive 2009/24/EC obligates Member States to protect computer programs by copyright, by analogy to the protection of literary works.

A standard legal definition of reverse engineering accepted by the U.S. Supreme Court (1974) is that it is a process of “starting with the known product and working backwards to divine the process which aided in its development or manufacture” [2]. The Supreme Court (1989) underlined the importance of reverse engineering, characterising it as an “essential part of innovation”.

A prohibition on reverse engineering would seem to have two beneficial effects [36]: It increases incentives to introduce innovative products on the market, and it avoids wasteful expenditures on reverse engineering. However, reverse engineering has beneficial effects that must also be considered: it can create competition in the marketplace, leading to lower prices and it can spur second comers to introduce additional innovations into that market (Table 12.5).

Table 12.5 Social calculus of reverse engineering in manufacturing sector [36]

The role of RE is very well expressed by Samuelson and Scotchmer [36] who said that reverse engineering is fundamentally directed towards discovery and learning. Engineers learn about state-of-the-art engineering not just by reading printed publications, going to technical conferences and working on projects for their firms, but they also learn by reverse engineering others’ products. Learning about what has been done before often leads to new products and advances in know-how. RE may be a slower and more expensive way of obtaining information to percolate through a technical community than patenting or publication, but it is nonetheless an effective source of information.

Analysing international activities in the field of IPRs over the last several decades, Archibugi and Filippetti [38] argue that the importance of TRIPS in the process of generation and diffusion of knowledge and innovation has been overestimated by both their supporters and their detractors. Although the main knowledge is today concentrated in the Western world, according to his opinion “TRIPS alone will not lead to an increase in the technology gap between western and emerging countries”. Giving the summary of the current intellectual property global rule Henry and Stiglitz [39] claim that this rule may obstruct both innovation and dissemination and suggest reforms to foster the global dissemination of innovation and sustainable development.

8 Conclusions and Outlook

Engineering design is the process of devising a part (component), device, system, or process, focusing on engineering intuition, creativity and originality. On the other hand, reverse engineering is the process of discovering the technological principles of a part, device, system, or process through the analysis of their structure, function and operation. RE focuses on the recreation (reinvention) of the original parts, system, or process and includes alternative engineering solutions. In recent years, reverse engineering has become a standard practice for mechanical engineers who need to replicate or repair a worn part, or control quality of a produced part. Nowadays, reverse engineering has also become a practice often used by various experts in different areas. Consequently, demand for RE tools has become increasingly important and has led to the development of tools that are now commercially available. Data acquisition tools (hardware) and data processing techniques (software) are evolving rapidly. Tools and techniques are developing in terms of accuracy, acceleration of the process and use of RE methods in new applications.

In this chapter, basic information on RE methods has been presented. Particular emphasis has been placed on reviewing, classifying and comparing the most common RE methods and their applications in various fields of human activity. Further, a short review of some aspects of RE (data acquisition and processing, surface reconstruction, etc.) is included. The chapter also considers several important papers dealing with each of these aspects. The presented applications of RE highlight the wide range of tasks that can be solved by using RE methods.

So far, RE applications have been introduced in many areas, and typical applications include product development and manufacture (CAD-CAM-CAE), RP, quality control and inspection of mechanical parts, 3D graphics and animations, 3D art modelling (sculpturing), topography, architectural, archaeology and cultural heritage documentation applications, as well as biomechanical and medical applications. The applications also show that the RE methods are often used together with other methods which results in a multidisciplinary approach to problem solving. This approach requires the interaction and collaboration of various experts from different areas, including reverse engineering and RP, design and manufacturing, material sciences, biomedical engineering, medicine, etc. All experts, with different and complementary advantages and limitations, can improve RE activities, and such an approach is particularly effective when dealing with complex tasks.

Today, a wide range of tools for reverse engineering is available. It is often difficult to select the most suited tool or system for a specific task. All systems have their own particular strengths and weaknesses. When selecting a RE system, three main technical specifications should be kept in mind: task requirements, part restrictions and environmental restrictions. However, the use of data acquisition tools is not a trivial task: the use of the systems is still rather complex and skilled professionals are required to operate them. Also, data processing in most of the RE applications require high skills of image processing as well as design and geometrical modelling.