Introduction

This article describes the data collection and analysis procedures executed in support of the Air Force Research Laboratory (AFRL) Additive Manufacturing Modeling Challenge (AMMC) series, Challenge 2. The overall challenge series was introduced in March 2018 [1] and is described both in an overview in the present issue [2], as well as in individual articles for each challenge problem [3,4,5,6] as well as the present work.

Challenge 2 was specifically aimed at interrogating process-to-structure links at the microscale. Challenge participants were asked to make predictions of several geometric aspects of as-deposited metal laser powder bed fusion (LPBF) tracks produced by relatively simple laser scan paths. The data to support this challenge is divided into two components: one set of data describing single-track deposits was collected and provided for potential challenge participations to use in model calibration activities, and a second set describing multi-track and multilayer deposits was withheld for validation purposes. The methods to collect and analyze both of these sets are described in the following work along with basic observations and conclusions.

As noted in many previous studies of LPBF, track dimensions are affected by a number of process parameters to include but not limited to: laser power P, scan velocity v, and laser spot size [7]. Additionally, deposit dimensions at a particular location are influenced by the temperature of the material onto which they are deposited, often referred to as preheat, and as such by previous energy input and transport processes. The laser scan path, the location within the component, and the overall component geometry/size therefore influence location-specific deposit geometry. The design of experiments for this challenge explicitly included changes in processing parameters, measurement location, and laser scan geometry, whereas the calibration data only explicitly incorporates changes in P and v.

Deposit geometry was chosen as the measured response for three primary reasons. First, it is directly relevant to the formation of defects such as lack of fusion porosity [8]. Insufficient deposit overlap both within a layer and from layer to layer leads to void formation which ultimately has a deleterious impact on mechanical performance metrics such as fatigue life [9, 10]. Models that can accurately assess deposit geometry could be used, for instance, to quickly screen for pathological combinations of scan path, component geometry, and process parameters. Second, deposit geometry is a rudimentary indicator of the nature of the full spatiotemporal temperature field \(T\left(\boldsymbol{r},t\right)\). The dimensions of the molten region observed in the post-build condition are essentially a record of the maximum extent of the solidus temperature Ts. It is ultimately desirable to assess the full \(T\left(\boldsymbol{r},t\right)\) as it drives many aspects of both the initial microstructure (e.g., crystallographic texture, grain morphology, solid-state transformations in some alloys), as well as macroscopic behaviors such as part distortion. Agreement on the dimensions of the molten region is taken as an initial indicator of simulation accuracy, though it is clearly not sufficient to guarantee general accuracy of the full \(T\left(\boldsymbol{r},t\right)\). Third, as will be shown in the following, deposit geometry can be reliably and repeatedly measured in the post-build state with widely available methods that do not require modification of the AM system. We do not discount the value of in situ monitoring data for qualitative indicators of processing behavior, and also acknowledge that there are advanced tools and techniques capable of quantitative, real-time measurements of the temperature field [11,12,13]. However, well-calibrated measurements up to the melting temperature are still quite challenging and not widely available at this time. Also, thermal imaging techniques are limited to assessing temperature on the external surfaces of the material only, and techniques employing embedded thermocouples produce measurements at individual points which are often remotely located [14].

The remainder of this work is organized as follows: in the Methods section, we describe the general responses to be measured and the techniques employed for material fabrication and characterization. The single-track calibration item data collection is covered, including both cross-sectional and top-down measurements, followed by the validation items including both multi-track and vertical wall specimens. Several observations from these raw data are described in the Discussion section, and finally the Conclusions section summarizes key points.

Methods

We use several coordinate systems to describe the specimens throughout this work. A global coordinate system described in the machine reference frame is denoted by directions (X, Y, Z). This system is generally consistent with that described in ISO/ASTM 52921 [15], with the exception that in the present case the origin is at the front left corner of the build plate, not the center. The Z direction is orthogonal to the build plate and pointed vertically upward, X is parallel to the front of the machine with + X pointed to the right as viewed from the front of the machine. Finally, Y is orthogonal to X and Z and oriented such that it forms a right handed coordinate system. We also employ individual specimen-centered coordinate systems for convenience, and denote these with a primed notation (X’, Y’, Z’). In general, Z’ is parallel to Z, and X’ and Y’ are rotated 10° counterclockwise (positive sense by the right-hand-rule) about the Z’ axis from the global X and Y directions. The local origin for each specimen-centered system X’,Y’ = (0,0) is coincident with the scan-vector beginning or end point that has the lowest Y value in the machine coordinate system for that specimen. While there are several such specimen-centric coordinate systems across the work, the one in use at any given time is generally obvious and our notation does not explicitly distinguish between them for brevity.

Material Fabrication

All material for Challenge 2 was fabricated on a stock EOS M280 LPBF system in good working order, and came from one build that was part of the overall AFRL AMMC campaign. This build employed commercially available IN625 powder feed stock; the powder chemistry and particle size distributions are described in the supplemental information. A soft recoater brush and conventional mild steel build plate (250 mm × 250 mm × ~ 30 mm) were utilized, along with a layer thickness of 40 μm. The beam has a nominally Gaussian intensity distribution in the build plane with 4σ diameter of 0.1 mm as reported by the system manufacturer, though this was not independently verified on the specific machine used. The build-preheat temperature was nominally 80 °C, though this quantity was not independently measured. All material interrogated for this challenge was examined in the as-deposited state with no further heat treatments applied.

The full build geometry is shown in Fig. 1 and is defined in the STL file format in the calibration data package. The actual scan vectors for all challenge items are described in the CLI file format, which are also included in the data package. We note that there were a multitude of other items printed in this build, but these are generally widely separated from Challenge 2 items. Three types of geometries were used in Challenge 2 referred to here as single-tracks, multi-tracks, and vertical walls. The single-track items were used for calibration purposes, whereas the multi-tracks and vertical walls were the basis for all challenge questions and the associated validation portion of the challenge.

Fig. 1
figure 1

Top-down a and oblique view b of the build geometry. The individual items utilized in Challenge 2 are shown in color; gray items are substrate pads or items not utilized as part of Challenge 2

Full size image

The 11 total single-tracks are denoted by Specimen IDs B10 to B20, and are produced using a single laser pass each. The specific processing conditions including laser power P, scan speed v were varied across the specimens as listed in Table 1. These parallel passes are a total of 20 mm in length and separated from each other by a distance of 3 mm, which is much larger than the typical melt-pool geometry produced for the range of power and scan speed used. The nominal processing time for each path is on the order of 10 to 20 ms; an estimate of a characteristic thermal diffusion distance for IN625 over this time is on the order 0.5 mm, and this extends to only 1.7 mm for the full duration of all 11 passes. These tracks are essentially thermally non-interacting via conduction through the substrate block on the time scale of the deposition.

Table 1 Processing conditions and experimentally observed width and depth values for all single-track items
Full size table

Multi-track items were fabricated using a series of laser scan tracks intended to replicate a typical snaking or raster scan strategy that is the basis of space filling algorithms utilized by many commercial LPBF systems shown in Fig. 2. These tracks are generally spaced close enough that they at least partially overlap, remelting previously deposited material. Unlike a conventional build however, these items consist of scans in only a single layer, which is again deposited on the top of substrate pads. The individual scan vectors for the items in question alternate in direction between parallel and antiparallel to the + X’ direction, and are always perpendicular to Y’. Vectors are scanned successively beginning with the vector with the lowest Y’ value and working toward the most positive Y’. The first vector processed (track n = 1) is processed with the beam progressing in the + X’, and thus the second would progress along − X’, with subsequent vectors alternating in a similar fashion. When the beam reaches the end of a scan vector, there is an approximately 0.5 ms period during which the laser beam is off (i.e., no energy delivered to the material) while the beam moves to the beginning of the next scan vector. This estimate is based on previous experience with sky writing behavior on this system, but no direct measurement of this quantity could be made on the unmodified printing system. Multi-track specimens include items with IDs B26, B27, B31, B34, B35, and B38, and the specific processing parameters used for these items are listed in Table 2. Note that Item B35 is a composite item consisting of two sub-domains processed in sequence, whereas all other multi-track items consist of a single domain and are therefore considered “simple.”

Fig. 2
figure 2

Renderings of the scan vectors for multi-track items used in Challenge 2. Color represents the order of vectors within the multi-track item with purple being first and yellow last. The red circle is the local coordinate system origin, X’ = 0, Y’ = 0

Full size image
Table 2 Processing parameters and geometric details for all multi-track items
Full size table

Vertical wall items are similar to the single-track items in that each is composed of a single laser pass within the layer. However, unlike the single-track items, additional identical vectors are processed in subsequent layers. These are positioned directly on top of the first vector and repeated for 9 additional layers, for a total of 10 layers. These vectors are each 5 mm in length, and are all processed with the beam progressing in the + X’ direction. There were 5 items produced, B21–B25, though only B21 and B25 were included in the challenge question. The specific processing parameters utilized for all 5 features are shown in Table 2, and a pair of oblique secondary electron (SE) images including all items in Fig. 3a and a close-up of item B21 are shown in Fig. 3b.

Fig. 3
figure 3

a Oblique view of vertical wall features on a substrate pad. B21 is closest and B25 furthest. b Side view of item B21; this vertical wall was scanned from left to right as oriented in this view on each of 10 successive layers. Walls are nominally 5 mm in length from start to finish

Full size image

All three types of items described above were deposited on top of substrate pads that were, in-turn, built by the LPBF process. All single tracks were deposited onto one substrate pad, all vertical walls were deposited onto a second substrate pad, and all multi-track specimens were deposited onto individual substrate pads. These substrate pads were 5 mm in height, or 125 processing layers, were built directly onto the build plate, and generally extend within the plane by at least 3 mm beyond the extent of the challenge items. The last three layers of the pads were processed with “top-skin” parameters, a parameter set commonly used to leave a relatively flat, final surface in preparation for the deposition of the calibration and validation items in layer 126. The final processing of the substrate blocks occurs at the beginning of layer 125. The calibration and challenge items are processed at the end of layer 126, at an absolute nominal height of 5.04 mm. Specimens B21 and B25 continue to layer 135. Layer processing times are approximately 85 s to 90 s up to layer 122, then 275 s for layers 123–125, 39 s on layer 126, and finally 27 s thereafter. A full description of layer timings is provided in the data package. A nominal powder spreading operation was performed at the beginning of layer 126 before the items were printed. After the build was completed, a wire electro-discharge machining (EDM) operation was utilized to remove the substrate pads from the build plate, with no additional heat treatment.

Measurement Response Descriptions

For single-track specimens, there are 4 measurable quantities extracted from cross-sectional images shown schematically in Fig. 4a. The maximum or Feret width of the track Wm measured parallel to the substrate plane, the width of the track where it intersects the substrate W, the maximum height of the track H above the substrate and parallel to the Z’ direction, and the depth D of the track below the substrate, again parallel to the Z’ direction. These quantities are shown in Fig. 4a. Additionally, top-down images of single tracks were also used as a separate measure of the maximum apparent width, Wt. Occasionally, spatter or partially melted particles are encountered, and these are also described in Fig. 4a. When the exterior contact angle is less than 90° the particle is considered only partially incorporated, and is not counted toward the maximum extent of either width or height, as is the case for the item labeled 1 in Fig. 4a. However, as for the item labeled 2 in the same schematic where the exterior contact angle is greater than 90° the item is counted as part of the track and could contribute to either the width or height measurements.

Fig. 4
figure 4

Schematics of a single-track and b multi-track cross-sectional observable quantities. Tracks are viewed along the − X’ direction, the track in question is depicted in green, and substrate material is shown in gray. For multi-tracks in b the track in question is numbered n, with previously deposited tracks denoted n − 2 and n − 1, and subsequently deposited tracks n + 1 and n + 2. Note that a depicts a single-track case where the maximum track width Wm occurs above the substrate. In cases where the maximum width occurs at the track–substrate intersection, Wm = W

Full size image

There are again 4 measurable quantities extracted for each track within cross-sectional images of the multi-track objects. These are defined differently than those of the single tracks and are shown in Fig. 4b. The un-remelted width Wu is the distance measured parallel to the Y’ direction extending from the lowest Y’ for any part of track n to the lowest Y’ value along the interface between track n and the next subsequent track it intersects with (i.e., typically n + 1 but sometimes n + 2 in case of extensive remelting). In cases where there is no overlap with a subsequent track, we report the distance from the lowest to the highest Y’ value for track n. Note that in either case, these extrema in Y’ do not necessarily occur at the same Z’ value. The half-width at maximum depth Wd is the distance measured parallel to the Y’ direction from the lowest portion of track n in the Z’ direction to the lowest Y’ value of track n. The total depth Dtot is the distance measured along the Z’ direction from lowest to highest points of track n. In the case of multiple local maxima in the Z’ direction, we use the value with Y’ value closest to the Y’ of the absolute minima. Finally, Dr is the distance measured along the Z’ direction from the lowest point of track n in Z’ to the intersection of melt-pool boundaries for track n with track > n (i.e., n + 1 or n + 2). If the track in question has no intersection with an adjacent track in the direction toward n + 1, then Dr is equivalent to Dtot.

These quantities were chosen as they are generally simple to identify from cross-sectional views and capture several important aspects of track-to-track overlap. Additionally, they do not refer to a global datum in the Z’ direction which can be difficult to define on as-printed substrates. When the process is operating in a quasi-steady-state Wu is expected to approach the hatch spacing h, which is the nominal scan-vector to scan-vector spacing along the Y’ direction. Wd is taken as an indicator of the overall track width prior to any remelting from subsequent tracks. Dtot is an indicator of the total penetration depth plus deposited material, and Dr is an indicator of the degree to which track-to-track overlap is maintained below the substrate plane. Insufficient Wd for a given Wu or hatch spacing could lead to gaps between tracks. Values of Dtot insufficiently large in comparison to the layer thickness would lead to lack of fusion between subsequent layers, and a large value of Dr could leave gaps between tracks even though the tracks overlap at the top surface. While these are simple metrics, previous work has shown that such geometric arguments can be effective indicators of parameter and scan-vector pathologies [16].

Single-track Measurements

Data provided to participants as part of the calibration data package included the measurements of the width and depth of the single-track deposits described in the Measurement Response Descriptions section. Each individual single track was first imaged using Back Scatter Electron (BSE) Scanning Electron Microscopy (SEM) along the − Z’ direction. A series of image tiles arranged along the length of every track were collected and then combined into a single montage using Fiji’s [17] Grid/Collection Stitch filter, an example is shown in Fig. 5a.

Fig. 5
figure 5

Examples of a top-down BSE and b cross-sectional optical etched microscopy images used to quantify single-track dimensions

Full size image

These montages were independently examined by two members of the AFRL team, and the width of each track was recorded at 20 locations separated by approximately 200 μm each. These locations were within the central 10 mm of the track, and did not capture transient behavior near the beginning or end which was typically isolated to within the first or last 1 mm. Occasionally, adhered powder particles or other features characteristic of dynamic motion of molten material were encountered at the measurement locations. Such features were not included as part of the width if they formed an exterior contact angle less than 90°. Both independent observers were in close agreement; the average of the absolute difference across all 11 tracks was 2.6 μm on track widths that ranged from 80 to 130 μm. The full populations of each measurement quantity collected by each observer were compared using Welch’s t test; no statistically significant differences between observers were indicated at the level p = 0.01. Table 1 gives average and standard deviations for each track.

In order to observe the depth of penetration into the substrate pad, the tracks were sectioned and mounted in order to view a Y’Z’ type plane (i.e., a plane orthogonal to the single-track processing vectors). These sections were etched and optical microscopy (OM) images were collected, an example of which is shown in Fig. 5b. Additional material was then removed using a modified Robo-Met serial sectioning system [18], and the etching and imaging process for each subsequent section was repeated. In total, a series of 10 images was collected for each track, with each plane separated from the previous by approximately 100 μm along the X’ direction. In each of these sections, 4 quantities described in the previous section were manually collected independently by two observers. Table 1 contains the average and standard deviation of the observations for all tracks. The population of measurements collected by each observer was again compared for each measurement quantity using Welch’s t test as with the top-down images; no statistically significant differences were indicated at the level p = 0.01. Note that the depth to width ratios are largely consistent with conduction mode, with no evidence of significant keyhole or keyhole porosity [19, 20].

Validation Data: Multi-tracks

The multi-track specimens were first imaged in the as-printed state from a top-down perspective using BSE mode in the SEM as shown in Fig. 6. The viewing orientation was essentially parallel to the − Z’ direction, and an image montage was collected. In these images, the system-specific coordinate direction + X’ is pointed horizontally to the right, and + Y’ is pointed vertically upward.

Fig. 6
figure 6

Top-down BSE images of specimen B27 in the a as-received condition, b as-received conditions annotated with planned FIB fiducial markings and EDM cut locations in white lines, and c after actual FIB marking and EDM cutting operations. In c separate images of the left and right half of the item were collected and then aligned to the image in (a) using Linear Stack Alignment with SIFT in Fiji. The local coordinate system X’, Y’ is explicitly denoted only in panel (b) for clarity

Full size image

The challenge question measurement locations were originally defined with respect to each specimen-centered coordinate system, with its origin at the beginning of the first laser scan track (X’ = 0, Y’ = 0). The first measurement plane at X’ = 100 μm in particular must be located to within ± 10 μm of the nominal position as significant systematic transients in track geometry are anticipated in this region; the quasi-steady region near the specimen mid-line only requires positional accuracy on the order of the track width, ± 100 μm. Because of the dynamic nature of the melting and solidification process and subsequently irregular track shapes, the origin cannot be directly determined from the top-down view with sufficient accuracy using the one end of the tracks alone. Instead, the position of the scan origin was identified by superposing a rectangle with the nominal dimensions of the scan path (e.g., 3 mm × 10 mm for B27) onto the top-down image, and then centering this on the as-deposited tracks. The lower left corner of this rectangle was taken as the specimen-specific coordinate system origin, and all measurement plane positions were measured from this datum. In this manner, the origin is determined without introducing bias from distortions in the turnaround regions, as we expect both the turnarounds in both the low and high X’ coordinates to be symmetrically distorted about the specimen mid-plane. In subsequent work, we have included additional independent fiducial marks to make identification of the laser scan coordinate system simpler and more objective.

Once the system-specific coordinate system origin was identified, the X’ positions of the measurement planes were located, and a fiducial marking strategy was devised such that the position of subsequent cross-sectional Y’Z’ planes could be clearly linked back to the top-down views. Because track geometry could vary rapidly with X’, particularly near the transient turnaround regions, it is necessary to verify that the sections used to characterize these geometries were within an acceptable tolerance of the intended X’ where modeling results were requested. Two sets of marks were added to the specimen. The first set included a series of triangular notches removed from the edges of the substrate pads using a wire-EDM cut with the wire parallel to the Z’ direction. These gross marks cause the apparent width of the substrate pad measured along the Y’ direction to vary as a function of the X’ position as viewed in a Y’Z’ cross section and quickly afford an estimate of the X’ position.

The second set of marks consists of a series of arrowhead features added on the top surface of the substrate pad adjacent to the multi-track region using a focused ion beam (FIB). These marks are located at each X’ location where a cross-sectional view was to be collected, and placed on both sides (+ Y’ and − Y’) of the multi-track region. After the FIB operation was complete, each specimen was separated into two pieces with a wire-EDM cut along a line parallel to the Y’ direction. The top surface of each of these pieces was again imaged in top-down BSE as described above, and these two new image montages were then registered back to the original top-down image of the whole specimen in Fiji using the Linear Stack Alignment with SIFT [21]. This process allows for verification of the actual fiducial marking positions in the original coordinate system. Drift during the FIB operation resulted in positions of the trenches being different than the original planned locations and in some cases additional marks had to be added in a second FIB session. Post-FIB imaging and the registration process documented the actual mark locations in the X’Y’ coordinate system, so these were still sufficient to determine the section plane position to the required accuracy.

After all top-down fiducial marking and imaging operations were complete, each of the portions of the original item was mounted such that the X’ direction is normal to the polishing plane. Polishing generally proceeded in both the + X’ and − X’ directions beginning from the wire-EDM cut, and was executed in a Robo-Met automated serial sectioning system [18]. For imaging planes where X’ < X’EDM the section is viewed along − X’ direction, with + Y’ pointed to the right in the image, and + Z’ is vertically upward. The first track deposited appears on the left side of the image, and the final track deposited appears closest to the right edge. For imaging planes where X’ > X’EDM, the section is viewed along the + X’ direction, and thus + Y’ is oriented horizontally to the left in the image, and + Z’ is vertically upward. Here, the first track deposited appears on the right side of the image, and the final track deposited appears closest to the left edge. These orders are important to ensure the sequence as described in Fig. 4b is honored.

When viewed in the Y’Z’ cross-sectional planes, the FIB marks appear as “V” shaped trenches, with one or more pairs of trenches located on both sides of the multi-track region. The bottom (most negative Z’) of each trench can easily be identified, and the distance between the trench pairs along the Y’ direction can be directly measured. This distance can then be correlated with the mark location as documented in the top-down view, and thus converted into an absolute X’Y’ location. This is done for each pair of visible trenches, and the sectioning plane position is then identified by a line connecting the X’Y’ locations from trench pairs on either side of the multi-track deposit. Both the difference in the absolute X’ position of the sectioning plane from the nominal position (ΔX’) and any apparent rotation of the nominally Y’Z’ about Z’ are determined and documented in Table 3.

Table 3 Multi-track section nominal positions X’, as well as observed angular tilt about Z’ and section plane displacement ΔX’ to the nearest half-micron
Full size table

Track geometry is most transient for the sectioning planes within 1.5 mm from the turnaround regions. For these sections, fiducial trench measurements indicate the mean absolute deviation of X’ value from the target, ΔX’, is within 3.0 μm ± 2.0 μm of the intended X’ value. Tilt of the Y’Z’ planes about the Z’ axis also contributes to uncertainty in X’, and the trench position indicates that the mean of the absolution angular deviation was 0.17° ± 0.09°, which results in a total deviation in X’ across the 3 mm wide multi-track region of 8.1 μm (i.e., ± 4.05 μm about the mean X’ value). Assuming a similar angular uncertainty in the tilt about the Y’ axis, this would correspond to 0.5 μm deviation over the more limited range of 100 μm to 200 μm in the Z’ direction. Finally, an additional contribution to uncertainty in X’ is non-planarity of the polished surface. Polished surfaces are not perfectly planar and exhibit some degree of “doming,” wherein the center of the mount/specimen is slightly taller than the edges. A scanning white light interferometer was employed to measure the profile of the polished surface. Slight doming was observed with maximum variation from the center to the edge of the metallic portion of the mount of 2.5 μm. In a smaller window including only the multi-track region this span was reduced to 0.4 μm. Taking the square root of the sum of the squares of these contributions suggests a total sectioning plane position uncertainty along the X’ axis on the order of 5 μm, within the targeted value 10 μm. Note that for X’ planes greater than 1.5 mm from the turnarounds do exhibit larger uncertainty. However, these uncertainties are still less than 100 μm which was deemed sufficient given the significantly reduced impact of X’ uncertainty on track dimensions for the quasi-steady regions.

In general, serial polishing progressed until the apparent substrate pad width viewed along the Y’ direction was close to the expected width based on the coarse EDM fiducial marks. At this point, the distance between the FIB trenches was measured as described above and the absolute X’ position determined. Sectioning progressed until such measurements indicated that the sectioning plane was sufficiently close to the desired location. When the nominal section position was achieved within the required accuracy as indicated by FIB trench spacing, the serial polishing operation was paused. Final polish before etching consisted of 1 min using a 1 μm diamond on an Allied Final Red polishing cloth. The specimen was removed from the system and manually swiped with 4 or 5 passes of an etch consisting of 60 mL HCl, 18 mL H2O, and 3 g CuCl2, and then optically imaged.

The resulting image montages were manually examined and key points were identified and annotated using Fiji. Each key point is assigned an integer index, and the absolute position of these points in the image coordinate system was exported for subsequent analysis. The list of key points begins with the edge of the specimen which has lowest Y’ value, and proceeds with identification of FIB trench locations on the low Y’ side of the multi-track, the edges of the multi-track deposit itself, then FIB trenches on the higher Y’ side of the multi-track, and finally the edge of the substrate pad with the highest Y’ value. Then, four key points on each track in the multi-track region are identified. For track number n, these begin with the topmost (maximum Z’) visible portion of the track, which is labeled “Top,” proceeds to the edge of the track toward the direction of the previously deposited track n − 1 (minimum Y’, labeled “Left” for imaging planes viewed toward − X’, and “Right” in imaging planes viewed toward + X’), then to the lowest (minimum Z’) visible portion of the track (“Bottom”), and finally with the point of intersection with the next track deposited n + 1 (“Intersection”). If track n and n + 1 do not intersect, the point labeled “Intersection” represents the location on track n with the maximum Y’ value. This process proceeds for each track in order of increasing track number, which is also increasing Y’.

Once all key points are identified, the reduced quantities of interest from Fig. 4b are computed for each track, and then averaged as described in the original challenge description. The first three and last three tracks are not included, and the average and standard deviation are computed for the remaining tracks after they are divided into subgroups for each unique combination of specimen ID, measurement plane, and even or odd track number. These values are finally reported in Table 4.

Table 4 Average and standard deviation of multi-track experimental measurements for all conditions by measurement location and even or odd track type
Full size table

In total, across the 6 multi-track specimens 23 imaging planes were examined: 16 as part of the challenge, and 7 additional non-challenge locations. A total of 498 challenge tracks and 219 additional tracks were characterized through identification of 2868 key points. One track from one item (B35) could not be clearly identified due to etching irregularity, but this particular track was not contained in a challenge plane and so no further attempt was made to quantify it.

Validation Data: Vertical Walls

The vertical wall specimens were also imaged via BSE in the SEM before being mounted and sectioned to document the general specimen condition and configuration. The substrate pad holding all five vertical walls was then mounted and polished in the Robo-Met system along the + X’ direction, with the sectioning plane parallel to Y’Z’. The sectioning progressed with a step size of approximately 10 μm, and the full set of walls and the substrate pad were imaged with optical microscopy at each step. Sections are numbered in the sequence in which they were collected, and a total of 544 sections spread across the full 5 mm wall length were collected. The raw images are oriented such that + Y’ is approximately oriented vertically upward, and + Z’ is approximately horizontal and toward the right. The actual coordinate system is rotated from this simple description by − 2.5° about the X’ direction, but the exact orientation was determined as part of the data reduction process described below. In this gross orientation, item B21 is located closest to the bottom of the image, and item ID increases toward the top in numerical order finishing with item B25 at the top of the image.

These raw images were assessed to determine the desired quantities including maximum wall height and cross-sectional area as a function of X’. A fixed region of interest measuring 3 mm in Y’ was cropped around each wall. A datum line was fit to the substrate-mount interface using all points in the interface more than 250 μm from the nominal wall centerline. The slope of this line was used to rotate the image such that the mount-substrate interface was horizontal; bi-cubic interpolation was employed and the rotation angle for each section was recorded. The intercept was also used to identify the substrate datum plane from which the wall height would be measured. Next, Otsu’s method was used to define an intensity threshold to optimally separate the mount from the metal [22]. This threshold was applied to create a binary mask, and then any remaining particles not attached to the main wall (within the image itself), and any holes completely within the metal were filled if their respective equivalent diameters were less than or greater than 68 μm, respectively.

The total wall height in the section is then identified as the distance between the aforementioned datum and the pixel with the maximum Z’ coordinate. To determine the cross-sectional area, the metal pixels within the wall must be separated from those in the mount. All metal pixels with Z’ values above the datum plane and contiguous with the pixel identified as the top are initially considered part of the wall. Occasionally, there are “wings” or excessive fillets where the substrate and wall meet. To exclude these from the wall, the subset of pixels in the metal within the height ranging from 15 to 100% of the total height above the datum was then considered. The standard deviation of the wall width and its centerline position in the Y’ direction are computed using these pixels. Then, any pixels in the lower 15% of the wall height that fall more than 3 standard deviations from the centerline were excluded from the wall and considered to be part of the substrate. All pixels remaining within the wall are then counted toward the wall area for the section in question.

Figure 7a shows an example of a raw cross section image and Fig. 7b shows the corresponding processed version. The pixels in Fig. 7b are set to black in the area identified as the mount, gray in the substrate pad, and white in the region identified as the wall. The solid blue lines indicate the substrate datum plane and the top of the wall, and the dashed blue line indicates the location 15% of the way up from the substrate toward the top of the wall. The dashed red line indicates the wall centerline, and the solid red lines indicate the extent of the horizontal range used to remove any excessive fillets. All images for each wall have also been combined into a fly-through video to afford a rapid human review.

Fig. 7
figure 7

Example cross sections of the vertical wall specimens. a is an example of a raw images, and b shows the processed and annotated version. Only white pixels in b contribute to the areas and heights listed in Table 5. Vertical and horizontal axes are recorded in terms of pixel indices. Each square pixel measures 0.566 μm

Full size image

This process is repeated for all sections and all walls. The substrate tilt angle generally varies smoothly from section to section, and falls within the range of − 1.5° to − 3.0°. Occasionally though, this value fell well outside this range. Several such images were manually examined and these all exhibited either a scratch or some imaging artifact which obscured or altered apparent substrate/mount interface. A threshold of 2.5 times the standard deviation of the substrate angle tilt across the full population of section images was sufficient to identify these anomalous images. This removed less than 10 of the more than 500 images for each wall, so no further attempt was made to analyze these images. It is likely these could be included using a more complex or even a manual segmentation approach, but this was deemed unnecessary given the large population of sections available. After all images are processed and pathological sections are identified and removed, the appropriate subsets were utilized to compute the average and standard deviation of the height and cross-sectional area within three zones as described in the original challenge description: 0 < X’ ≤ 0.5 mm, 0.5 mm < X’ ≤ 4.5 mm, and X’ > 4.5 mm, which are then recorded in Table 5.

Table 5 Summary measures of the vertical wall items for each measurement zone
Full size table

Discussion

While the primary intent of this article is to document the data collection and post-processing procedures to enhance the utility of the raw data, there are several basic observations that can be made from both the calibration and validation data sets. In several cases, these corroborate expectations based on prior knowledge of the LPBF process and enhance confidence in the overall experimental campaign and subsequent data analysis. There are also several unanticipated findings which are worth noting as they suggest areas of improvement for both specimen fabrication and experimental data collection and analysis procedures for future validation efforts.

Single-tracks

First, we compare the mean values of the single-track width measurements as determined from top-down BSE to the maximum widths observed in the cross sections. These measurement modes typically differed by 1 to 4% relative to the average of the means of each method. Furthermore, a Welch’s t test at the level p = 0.01 for each track indicates that the means of these two measurement methods do not show statistically significant differences. In general, at least for tracks exhibiting generally regular morphology, top-down measurements, which are easier to perform, are sufficient to characterize track width if maximum width is sought. Figure 8 shows the result of both measurement methods as a function of the track linear energy density (LED) defined as the ratio of process power to scan velocity:

$$ {{LED}} = \frac{P}{v}. $$
(1)
Fig. 8
figure 8

Track width as viewed in top-down and cross-sectional measurements as a function of LED for all single-track conditions. Solid symbols indicate the mean of all observations for a track, and error bars denote ± 1 standard deviation of the population

Full size image

The Pearson correlation coefficient (PCC) for both measures of mean width as a function of LED is above 0.98, indicating a strong positive correlation. Considering all data points individually, not just the mean values, the PCCs drop to 0.716 and 0.769 for top-down and cross-sectional views, respectively, indicating that there are width fluctuations not directly explained by LED alone, but that LED is still highly correlated with track width.

Next, we consider relationships between the 4 response quantities determined from the single-track cross-sectional measurements. As it has been established that the maximum cross-sectional width Wm is consistent with top-down views Wt, and furthermore that those quantities are both strongly correlated with LED, we compare the other three cross-sectional quantities to the maximum cross-sectional width. A statistical comparison indicates that the mean Wm is correlated with the mean width at the track–substrate intersection W and mean depth below the substrate D with PCC = 0.985 and PCC = 0.950, respectively. However, the height above the substrate H was considerably less strongly correlated with both the maximum. Visual inspection of a plot of height above the substrate vs. LED suggests that these quantities are effectively independent, and indeed considering all measured heights as a function of LED yields PCC = 0.135. These relationships are largely consistent with expectations for the conduction melting mode, higher LED leads to wider and deeper tracks, but track height above the substrate is largely dictated by layer height plus fluctuations driven by phenomena not directly accounted for by LED (e.g., powder density fluctuations, fluid-flow phenomenon, etc.). Track morphology was generally regular over the present range of processing conditions again suggesting that the P,v values used yield largely conduction mode behavior. More extreme parameter combinations have been observed to induce more complex phenomena (e.g., extreme balling, unstable keyhole mode), and it is likely that the relationships described above differ more significantly when moving outside the present P,v ranges.

To assess stability of the processing environment, replicates of single tracks were included in the single-track set at the nominal P,v combination (B10, B11, and B18). Welch’s t test at the level p = 0.01 indicated that each pairwise combination of these three tracks was statistically indistinguishable across all measured quantities from the cross section and top-down measurements. This is taken as a positive indicator of process stability, however we note that the single tracks were produced both physically and temporally close to one another, and characterization of tracks produced over a larger spatial and temporal range (perhaps via top-down width) are needed to quantify longer range stability.

Multi-tracks

The multi-track specimens utilized a significantly reduced range of P and v values, with most specimens processed at the nominal value, however significant systematic variability in dimensions was introduced via the scan path geometry. The strongest drivers of track dimension changes were the location of the cross section examined. This is most easily quantified through the measurement plane’s value of X’. For all of the simple multi-track items (e.g., all items except B35), as X’ → 0 mm the measurement plane location approaches the leftmost turnaround regions in the specimen. As X’ increases from 0, the plane moves further from this transient region, and the maximum reported value of X’ is located at the scan-vector midpoint, equidistant from the two turnarounds (again, with the exception of specimen B35). Recall that for the first track processed, track n = 1, the laser begins at X’ = 0 mm, and moves toward + X’. Track directions alternate thereafter, and thus all tracks with odd n are processed toward + X’, and tracks with even n toward − X’. At the vector midpoint, the time elapsed since the beam was most recently at the same X’ value on the preceding track is equivalent for both odd and even tracks. However, as X’ → 0 mm, this symmetry is broken, with odd tracks having a lower time since the previous pass, and even tracks taking on a larger value.

Taking the condition for B27 with vector length of 10 mm, processing velocity 1230 mm/s and the estimated turn delay time of 0.5 ms, the time between passes for odd tracks varies from a minimum of 0.66 ms at X’ = 0.1 mm, up to 8.63 ms at X’ = 5.0 mm. For even tracks this time is 25 times larger (16.60 ms) at X’ = 0.1 mm, but again is equivalent to the odd timing at X’ = 5.0 mm. Taking a typical thermal diffusion coefficient for IN625 and computing a diffusion distance over these time-between-pass ranges, at X’ = 0.1 mm the odd tracks have a distance of only 103 μm, approximately one hatch spacing, whereas the even tracks correspond to 514 μm. At the mid-plane, the diffusion lengths are equivalent at 371 μm. This analysis suggests that near the turnaround region there will be a considerable difference in the degree to which previously deposited energy will influence the temperature distribution, and furthermore that the odd tracks are likely to be strongly affected, which has been previously reported [26, 27].

Fig. 9 shows the measured track dimensions for all of the simple multi-track objects, where the values have been split by Specimen ID as well as odd or even nature. The averaging process is as described in the Methods section wherein the first and last three tracks are not considered. One initial basic observation is that, as expected, values of Wu approaches the hatch spacing at the specimen midpoint, albeit with nontrivial fluctuations. Finally, typically values of Dtot generally approach approximately three times the nominal layer height at the specimen midpoint.

Fig. 9
figure 9

Average track dimensions for all of the simple multi-track specimens separated by even and odd track type plotted as a function of X’. Solid symbols indicate the mean of all observations for a condition, and error bars denote ± 1 standard deviation

Full size image

We first note that at measurement positions X’ = 0.100 mm and 0.250 mm there is marked asymmetry in behavior between even and odd tracks, with odd tracks generally being much larger in all measured quantities. As X’ increases toward the vector centerline, all measures of track dimension within a specimen ID tend to converge, with most of the change occurring in the first 1.5 mm. Also, the impact of position on the odd track dimension is stronger than that for the corresponding even track. This behavior is generally consistent with the preceding preheat argument; odd tracks experience considerably more preheating near the turnaround region, but the asymmetry between even and odd tracks decreases at the vector midpoint. For a fixed energy input, increased preheat will result in an overall larger track, which is consistent with increases in Wd, Wu, Dtot. Note that our definition of overlap depth Dr with the previously deposited track, so larger odd tracks would generally drive down the value Dr tabulated for even tracks, fully consistent with the behavior in Fig. 9d.

Qualitative examination of the raw data, both top-down and cross-sectional images, reveals that multi-track items exhibited an extensive molten region in the turnaround region compared to the specimen midpoint, suggesting that material at the termination of track n was still molten when scanning track n + 1 was initiated. In some cases, track n + 1 completely overlapped track n. Several of the challenge participants indicated that their models suggested this would be the case, and indeed a forthcoming analysis using an AFRL developed process model [26] also indicates that liquid melted by track n is still present when the laser begins processing track n + 1. The particular magnitude of the turning delay associated with the skywriting process was neither carefully measured nor controlled in these experiments, but this parameter could strongly influence the degree of preheat present at the initiation of track n + 1. Since such transient regions can be prone to several defect formation mechanisms, understanding and controlling the details of the turnaround procedure is highly desirable.

While the strongest driver of track dimension is position and direction of the scan vector, processing parameters also exhibit effects on the measured track dimensions in the multi-track items. Specimens B31 and B34 use the same P, v, and overall multi-track geometry as the nominal item B27. However, B31 employs a hatch spacing that is 25% narrower, whereas in B34 the vectors are 25% further apart compared to the nominal. The track dimensions from this subset of multi-track items are shown in Fig. 10. At a given measurement position X’, specimen B31 generally exhibited the lowest values of both Wu and Dr, consistent with a relatively narrow hatch spacing leading to enhanced track overlap, both in the Y’ and Z’ directions. Notably, specimen B34 has very high values of Dr indicating poor penetration of the melt overlap between adjacent tracks. In fact, both the top-down BSE images and the cross-sectional images frequently show gaps between the tracks as shown in Fig. 11; this is not particularly surprising as the hatch spacing employed was slightly larger than the typical mean track width for the corresponding single-track conditions.

Fig. 10
figure 10

Observed average track dimensions for selected multi-track items as a function of hatch spacing split for even and odd tracks at the section X’ = 5000 µm. Solid symbols indicate the mean of all observations for a condition, and error bars denote ± 1 standard deviation

Full size image
Fig. 11
figure 11

Selected region of the top surface of B34. Gaps between the tracks are apparent, and the underlying substrate surface is visible

Full size image

Item B38 applied the nominal hatch spacing, but employed a linear energy density P/v that was 25% above the nominal condition, primarily due to a decreased scan velocity. Fig. 12 shows the dimensions of item and B38 at LED = 0.304 J mm−1 compared to item B27 which was processed at the nominal value of LED = 0.244 J mm−1, both measured at the multi-track centerline. These two items also have slightly different vector lengths with B27 = 10 mm, and B35 = 15 mm, but the change in LED is expected to be the primary driver for differences observed near the track centerline. The width Wd showed noticeably larger values for B38 with the increased LED, though Dtot and Dr were similar across the change in conditions. As with single tracks, increasing LED drove an increase in molten width. Interestingly, even at the track centerline, asymmetries are apparent between even and odd tracks for Wu and especially in Dr.

Fig. 12
figure 12

Observed average track dimensions for selected multi-track items as a function of LED for even and odd tracks at the multi-track centerline (X’ = 5000 µm for B27 with LED = 0.244 J/mm, and X’ = 7500 µm for B38 with LED = 0.304 J/mm). Solid symbols indicate the mean of all observations for a condition, and error bars denote ± 1 standard deviation

Full size image

Another interesting and generally unexpected finding was the presence of several keyhole features in a subset of the multi-track items. These were only observed in the X’ = 100 μm sections, and specifically only for the final track in the multi-track sequence. They manifest as an irregularly deep final track, in some cases with visible trapped porosity, and are also apparent as a deep fissure in the corresponding top-down BSE image. Of the 5 simple multi-track items, 1 exhibited a completely nominal final track, 2 exhibited a clear top-surface depression, and the remaining two showed extensive keyhole features with trapped porosity. Significant remelting occurs near this turnaround region, and in principle, this could lead to refilling of a keyhole feature developed on the end of a previously molten track. However, none of the tracks which were subsequently covered by remelted material exhibited evidence of primary melting nearly as deep as the final track keyhole features, suggesting that the keyhole behavior was indeed isolated to the final track only. As the last 3 tracks in each deposit were not considered for the challenge questions, these features have no bearing on the average values in the validation key, but clearly suggest that a different and unexpected processing procedure was unintentionally employed at the termination of the final vector, perhaps a reduction in velocity or even an extended delay with a stationary beam (Fig. 13).

Fig. 13
figure 13

Images showing top-surface depressions or keyhole pores on the end of the final vector of several multi-track conditions in the X’ = 100 μm sections for a B26, b B27, c B31, d B35, and e B38. Image fields of view are all 400 µm wide

Full size image

Vertical Walls

Fig. 14 shows the measured cross-sectional areas for all 5 vertical walls as a function of position along the wall. Recall that the scan direction proceeds in + X’ direction in each of 10 successive layers. There is a clear transient in at the beginning of the wall where X’ ≲ 500 µm evidenced by increased height at the beginning of the tracks also visible in Fig. 3a and b. This height transient is significantly taller than a single-layer thickness. This build was fabricated with a compliant recoater which did not appear to damage the wall in this region, at least for the limited number of layers used for these walls. With only 10 layers, it is unclear if this build-up eventually approaches a sustainable steady-state value or if it would grow uncontrollably and cause more significant problems. We note that this startup transient is exacerbated by the fact that the scan vector started in the same location on the wall every layer, which is not a common scan path design feature, and also that such transients have been observed in directed energy deposition AM processes [28]. Fig. 14 also shows a local decrease in area for specimen B25 near X’ = 2500 µm. Closer inspection of individual section images in this region indicates a through-thickness hole near the wall-substrate interface. This feature was only observed for B25, which was also the lowest LED setting used. Finally note that though there is some noise, in general B24 exhibits the largest cross-sectional area at any given X’, B25 exhibits the lowest value, and B21, B22, and B23 cluster in the middle.

Fig. 14
figure 14

Cross-sectional area of all vertical walls as a function of position along wall. Vertical gray lines indicate the boundaries between the beginning (X’ < 0.5 mm), middle (0.5 mm ≤ X’ ≤ 4.5 mm), and end regions (4.5 mm < X’)

Full size image

This trend with LED can be seen more clearly in Fig. 15 which shows the averages of the area and heights within the middle region (500 µm ≤ X’ ≤ 4500 µm) for all 5 walls. Fig. 15a indicates that cross-sectional area increases with increasing LED, whereas Fig. 15b shows that wall height is not statistically significantly different across the range of LED investigated. This implies that increasing LED drives increases in the average wall width which mimics the trends observed for single tracks described in Table 1 and Fig. 8. These trends carryover to single-track multilayer deposits. In fact, considering the average areas and heights in Table 5 and Fig. 15, the effective average widths range from 91 µm to 134 µm, closely matching the range of single-track widths reported in Table 1 built over the same LED span.

Fig. 15
figure 15

Average cross-sectional area and height as a function of LED changes for vertical wall specimens for the middle region where 500 µm ≤ X’ ≤ 4500 µm. Average area increases monotonically with LED, but height is essentially constant. Solid symbols indicate the mean of all observations for a condition, and error bars denote ± 1 standard deviation

Full size image

Conclusions

Several key findings are enumerated below. These findings indicate that the experimental approaches described above achieved the original primary objectives, but also highlight areas of improvement for future LPBF validation experiments:

  1. 1.

    Basic metallographic preparation procedures along with carefully designed fiducial marking strategies were sufficient to characterize the geometry of the molten regions for both quasi-steady and more importantly transient regions of single and multi-track deposits with positional uncertainty on the order of 10 µm or less. Additional fiducial marking on the top of substrate pads directly using the AM system’s laser would allow for objective alignment and reduce sectioning plane uncertainty.

  2. 2.

    Track geometry in both single- and multi-track configurations indicate that these experiments were operated largely in conduction mode, though with some notable keyhole mode observations in transient regions.

  3. 3.

    There was good agreement between top-down and cross-sectional measurements of the maximum width of single tracks, and links between various measures of width and depth were observed for single-track deposits over the range of P and v studied

  4. 4.

    Transferability of single-track single-layer geometry to both multi-track single-layer, and single-track-multilayer conditions were observed for locations where the track shape is expected to be in quasi-steady state.

  5. 5.

    The range of process parameter variation, multi-track geometry configurations, and section plane locations was sufficient to capture transient track geometry, though these are certainly not exhaustive given the breadth of LPBF scan path generation schemes and complex geometries

  6. 6.

    Systematic transient track geometries were observed in regions where the beam trajectory was not steady, including multi-track turnarounds and single-track vector start and stop points. In multi-track configurations, these transients were consistent with preheat arguments, but such arguments cannot explain the transients observed in single-track deposits; other work indicates that both fluid flow and laser–material interaction are important driving forces in these regions [29] and also contribute to variation in the multi-track configuration.

  7. 7.

    Several aspects of machine behavior that were held constant but not directly measured in this campaign could still be significant and it is desirable to directly quantify these in future validation experiments. Some examples include the overall build-preheat temperature, the laser spot size, and energy intensity distribution more generally, as well as the details of the timing of the energy source motion (e.g., skywriting delays). Unexpected end-of-track keyhole behavior and in some cases keyhole porosity on the final vector of the multi-track items is suggestive of a deviation between the planned and actual behaviors, though this did not impact the validation measurements.