Introduction

Defect identification and mitigation is an important avenue of research in additive manufacturing (AM), more commonly referred to as 3D printing. Defects can cause a 3D print to fail, and this failure results in lost time and materials. Such failures are not necessarily costly for a small, desktop 3D printer that fabricates parts with inexpensive plastic. However, the cost is amplified for large-scale additive manufacturing. In large-scale AM, prints routinely take several dozen hours and involve hundreds of pounds of material. An example of large-scale AM systems is shown in Fig. 1.

Fig. 1
figure 1

Example large-scale AM. The left image shows a polymer-based system, while the right image shows a metal-based system

Full size image

To combat defects, researchers at Oak Ridge National Laboratory (ORNL) investigated automated methods to generate a height map and adjust the height on a layer-by-layer basis. To support a generalized solution for any arbitrary geometry, alterations were made to both slicing and control software. Additionally, image manipulation libraries were used to compare an object under construction to the geometrically accurate CAD model, or ideal model, of that object. Lastly, sensing equipment in the form of a laser profilometer and thermal camera were installed to gather data on an object under construction. When combined, these changes allowed more robust control of an object’s shape during the printing process.

Related Work

Because of the importance of defect identification and mitigation, numerous other avenues of research have explored this topic. Multiple overviews exist, of which Everton et al. and Calignano et al. are only two examples.1,2 In these works, the authors have provided an overview of AM technology as well as methodologies for detecting defects.

Specific examples for detecting defects are also many and varied, and include object design, polymer, and metal-based system implementation, and mobile and machine-learning solutions. This paper will touch on each of these topics. Defect identification may begin with the design of the object. Work by Zhu et al. investigated algorithms designed to assess the deviation of completed parts to their CAD counterparts based on stereolithography (STL) representations.3

Several authors have investigated methodologies for inspecting objects at a microscopic surface level to identify defects. Such examples include work by Book and Sangid.4 In this work, the authors investigated the microstructure of the material during deposition to assess suitability for in situ processes.

Defect identification via camera and laser scanner analysis has also been investigated in a polymer-based system. Both Baumann and Roller and Straub investigated camera-based visual inspection of the build volume using a Makerbot system.5,6 These cameras were pointed toward the build volume, and images of the object under construction were compared to identify various types of errors. Laser scanner analysis has also been investigated by Faes et al.7 In this work, the authors investigated the use of a 2D laser scanner directed at the extrusion location to identify build errors.

Numerous in situ and defect identification studies have also been conducted on metal-based systems. In work by Kleszczynski et al. and zur Jacobsmuhlen et al., visual inspection of various aspects of a metal-based AM process was investigated.8,9 The work focused on process stability to reduce errors during the printing process.

Defect identification has also been carried out using mobile devices and machine-learning techniques. In work by Hurd et al., the authors used a smartphone and image analysis to capture pictures of the object under construction and determine if an error had occurred.10 Similarly, work by Ferguson investigated the use of neural networks for image analysis to describe errors during the build process.11

The work reflected in this paper draws and builds upon the work mentioned above. Like previous work, we take an image analysis approach to identify and mitigate defects. However, unlike previous work, we mounted equipment in several locations. The equipment is mobile, which allows us to capture a much larger build volume, as a single sensor cannot cover the build space. Further, such large-scale manufacturing introduces unique challenges when dealing with image analysis as a single camera view cannot capture the entire object. These issues are described in detail in the next section.

Implementation

Defect identification and mitigation is made possible through both hardware and software modifications to the 3D printer. These improvements can be implemented on a variety of additive manufacturing systems; they are not limited to the systems discussed in this work. In this case, hardware includes sensor equipment in the form of a laser profilometer and thermal camera. The thermal camera’s contributions to this algorithm are still in development and will not be the focus of this discussion. We hope to provide additional details in the future.

This equipment is mounted orthogonal to the build surface, and an example of such equipment is shown in Fig. 2, in which the equipment is an example of sensors attached to the BAAM (Big Area Additive Manufacturing) system. BAAM is a gantry-style pellet-fed polymer AM machine; however, similar mounting was accomplished using a robotic arm setup on a separate system. Importantly, this equipment is directly mounted to either the gantry or a robotic arm. This allows the equipment to be moved to appropriate locations for data collection, thereby increasing the area of sensor coverage.

Fig. 2
figure 2

Example sensing equipment. The left image shows a Keyence laser profilometer, while the right image shows a FLIR thermal camera

Full size image

The software is composed of multiple components. At a minimum, the Human–Machine Interaction (HMI) code that controls the 3D printer must be present and capable of altering printer behavior. Pathing and ideal versions of the object must also be generated as part of the slicing process. Lastly, code must be developed that controls data collection and analysis and returns results to the HMI code.

Necessary changes to the slicing software will be discussed first. ORNL developed large-scale AM software inhouse. The software is called ORNL Slicer. To perform this research, ORNL Slicer was altered to automatically generate appropriate toolpaths for scanning an object when a layer’s construction is complete. Typically, when constructing an object, the object is divided into multiple layers. Each layer is constructed in sequence to build an object, and each layer contains numerous pieces of geometric information. One piece of geometric information considered is an axis-aligned bounding box (AABB), which represents the smallest, axis-aligned rectangle that can encompass a particular layer. Using an AABB and knowing the width of the laser beam, a raster path can be calculated across the layer. This raster path is a serpentine path that represents the quickest route to cover the entire object with the minimum number of scanning passes.

The ORNL Slicer software was also altered to compare real-world data from the object under construction to the ideal version of the object in the slicing software to calculate deviation. The ideal version of an object is simply a 2D image representation of each layer of the object. To create the ideal version of an object, the polygons that constitute a layer were combined with the laser profilometer’s scan line resolution and step size. The scan line resolution is hardware-specific and describes the dots per inch (dpi) or data density of a single scan. The scan line resolution also determines the resolution of the resulting image in the same axis that the laser profilometer is mounted. The step size is a user-specified value that describes the desired distance between each scan. The step size determines the resultant image’s resolution in the orthogonal axis to which the laser profilometer is mounted. Combining this profilometer information with the AABB and polygon information allows a 2D image with the appropriate aspect ratio to be constructed to represent the layer. This image creation and all subsequent image manipulation was carried out with the OpenCV framework.12

With the pathing and ideal representations constructed, data collection is ready to commence. Once a layer is deposited, the scan path, as calculated by the slicer, is traversed. The laser profilometer is triggered to collect data at the appropriate location. In our case, the laser profilometer has multiple trigger methods. Both a timed trigger that elapses on a recurrent basis as well as an encoder have been used. Once the scan information is collected, analysis occurs in several discrete steps that include image cropping and reconstruction, noise reduction, and height map generation.

First, scans are cropped, and a single image is created. This method resembles the process of creating one panoramic image from multiple smaller images. Each scan pass is placed in the appropriate general region of the overall image. The scan pass’s alignment is then adjusted, and each scan pass is overlaid in turn. Once a single image has been reconstructed, noise reduction must be performed. When collecting data, the laser profilometer may not collect a value for an individual pixel, or the data may be corrupted by reflections. Reflections play a more significant role with metal materials than polymer. To deal with these poor data measurements, a simple neighborhood average was adopted to identify and remove inconsistent measurements. Finally, once erroneous measurements have been dealt with, the image can be transformed into a height map adjusted for what the expected nominal height should be at various locations for the layer.

With the height map calculated, the HMI code that controls the 3D printer can adjust subsequent layers based on previous heights. This is done through a simple interpolation. The HMI code calculates the closest point in the height map matrix to the current position. Then, based on the matrix’s value, the amount of material being deposited can either be increased or decreased as necessary. Using such a process, several objects were constructed.

Results and Discussion

Using the system described above, we constructed several objects. A demonstrator object was constructed to highlight several different geometries. The demonstrator object was constructed using a metal-based directed energy deposition AM process. The profilometer was mounted directly to the robotic arm and encased in a copper shield to protect the sensor from stray energy bursts. An example of the object’s path planning is shown Fig. 3. This path was generated by the slicing software and represents the construction of a single layer.

Fig. 3
figure 3

Demonstrator path planning

Full size image

Once this layer was constructed, scans were taken of the object. To scan the entire object, multiple passes were necessary. This is a common occurrence of building large objects as the objects are typically much larger than the beam width of the laser profilometer. An example of a raw scan and the individual passes is shown in Fig. 4.

Fig. 4
figure 4

Raw scan with individual scan passes

Full size image

With the raw scan collected, the processing steps detailed in the Implementation section of this paper were performed. Once complete, a height map was generated that describes the height as an offset of the ideal nominal height for a particular layer. An example of the height map for the demonstrator part is shown in Fig. 5 and a zoomed-in sub-region is shown in Fig. 6.

Fig. 5
figure 5

Height map of complete layer

Full size image
Fig. 6
figure 6

Zoomed-in height map

Full size image

Figure 5 is an example of the height map for a single layer. For this object, the targeted layer height was 1.1 mm per layer. The mean, median, and standard deviation for this height map is − 3.8 mm, − 1.5 mm, and 5.3 mm, respectively. So, on average, the generated height map shows the height of the object to be low by several layer heights. Such a measurement suggests that there is an issue with data collection. This data collection does not account for a common issue seen in metal-based deposition.

In metal-based depositions, the convex nature of the bead and the highly reflective nature of the metal combine to create data gathering issues for the laser profilometer. This is just one example of several types of issues that can be encountered when using optics to scan metal.13 Essentially, when the profilometer strikes specific angles of the bead, the light is reflected off the object and never returned to the profilometer, which leads to erroneous data. These erroneous data are primarily around the edges of beads at specific angles of the beam to the laser profilometer. This issue is not seen to the same degree in polymer-based systems in which, while the plastic has a similar convex shape during deposition, at a microscopic level, it is still porous. As a result, the odds of an infrared beam returning to the collector of the profilometer is higher compared to the smooth surface of the metal. To illustrate the reflection issue with metal deposition, an example is shown in Fig. 6.

As shown in Fig. 6, the height accuracy is less than one layer height’s distance for a majority of the scan. If you calculate with a small inset on each bead, in this case, 1 mm, your statistics improve greatly. In this example, the mean, median, and standard deviation are -1.07 mm, -0.142 mm, and 0.989 mm, respectively. So, the generated height map shows the height of the object to be low by less than a single layer’s height as compared to multiple layer’s height earlier. Ultimately, the center of the bead has the most accurate data measurement and is the most important location for the data to be correct.

As the next layer is deposited on top of the scanned area, an interpolation will be calculated. This interpolation will evaluate the closest point in the height map to the machine’s current position. This current position is the center of the bead as the center defines the pathing that is generated by the slicing software. To illustrate this, Fig. 7 shows the zoomed-in height map with the path for the next layer overlaid. As shown in this image, the paths for the next layer align over the middle of the height map where the data are most accurate.

Fig. 7
figure 7

Zoomed-in height map with path planning overlay for next layer

Full size image

Profilometer accuracy will need to be addressed in future iterations of the system. To address data accuracy, several methodologies can be investigated. Due to the degrees of freedom offered by a robotic arm setup, multiple-angled scans present themselves as an obvious choice. Using multiple-angled scans, the geometry of the beads and object can be used to adjust data gathering, which will result in better quality data. Alternative mounting solutions would be necessary for a gantry style system. Initial investigation into such a method has yielded positive results, with the data from multiple-angled scans appearing to be complementary, which results in a much higher quality scan. However, further work is required, and a full description is outside the scope of this paper.

Despite some issues with profilometer performance, overall, the current system performs well. Height control is robust enough to enable the creation of complex geometries while adjusting for height deviations in each layer of the object under construction. The resulting demonstration piece using this system is shown as constructed in Fig. 8.

Fig. 8
figure 8

Final construction

Full size image

Conclusions and Future Work

As demonstrated, the current system collects data in situ to generate a height map of each layer of an object under construction. This height map is then used to adjust subsequent layers to maintain an appropriate height. Despite promising performance from the first iteration, many avenues of improvement exist for this system.

We intend to pursue additional research to address data accuracy and overall system robustness. We plan on investigating the use of additional sensors such as thermal cameras as well as conducting multiple scans for each layer. Additional scans will enable a more sophisticated correlation of data collected to the object under construction. Such height information could also be used for alterations to the slicing software that would allow true dynamic path re-planning. Lastly, this information can be related to thermal information that will be gathered using a thermal camera.