Introduction

Cellulose nanocrystals (CNCs) are cellulose based nanomaterials extracted from different biological sources, typically as a product of strong acid hydrolysis process. In general, CNCs have a spindle-like particle morphology (4 to 20 nm in diameter, 50 to 500 nm in length, with tapered ends), but they exhibit varying particle morphologies and size distributions depending on the hydrolysis process parameters and cellulose source (Foster et al. 2018; Moon et al. 2011). When working with CNCs, it is critical to have a comprehensive and accurate characterization of the particle morphology, size, and surface characteristics. Measurement techniques for length, width and height have recently been extensively developed, including interlaboratory comparisons of dimensions obtained using transmission electron microscopy (TEM) (Meija et al. 2020) and atomic force microscopy (AFM) (Bushell et al. 2021), as well as an ISO technical specification (ISO/TS 23151). The current state-of-the-art image analysis to measure CNC length, width and particle size distribution is based on manual measurements, which are subject to variability and error due to the subjectivity in the identification of individual CNCs (as opposed to agglomerated CNCs) and are time consuming and can suffer from analyst fatigue. There is a need to develop automated approaches for CNC particle size measurements to improve the consistency of CNC selection and to reduce the analysis time. Commercial or open source [e.g., plugins for ImageJ, such as FibrilJ (Sokolov et al. 2017)] semi-automated image analysis programs for TEM and AFM image analysis of nanosized particles have been inconsistent when applied to cellulose nanomaterials due to difficulties in correctly identifying single particles. Additionally, recent machine learning algorithms by Wang et al. (2021) have demonstrated great capability for TEM image analysis of metallic nanomaterials, in particle identification, morphology differentiation, particle classification, and analysis. The images analyzed had low noise and the nanoparticles had high edge contrast, facilitating object identification. Applying such a program for CNC analysis may prove problematic as it is not an idealized particle system and it is difficult to image, as described below.

Quantitative measurement of CNC morphology is challenging because of three factors: CNC material issues, imaging issues, and image analysis issues. Material issues are related to variability in the shape of CNCs (e.g., not perfect rectangles or ellipses), broad size distributions, and strong propensity for aggregation. Imaging issues are related to limitations from the preparation of well-dispersed, low density CNC distribution on the imaging substrate, and measurement technique or equipment such as, high level of noise, low feature or edge resolution, low contrast between CNCs and background, and for AFM, tip convolution effects. The extent of these variabilities can be decreased by optimizing and controlling CNC dispersion, sample deposition on (and selection of) the imaging substrate, and the staining method, as summarized in recent literature (da Silva et al. 2020; Foster et al. 2018; Kaushik et al. 2015; Ogawa and Putaux 2019; Stinson-Bagby et al. 2018). It should be noted that despite these detailed studies of the effect of various parameters, the conclusions are in many cases qualitative, rather than based on a statistical analysis of data obtained by analysis of a sufficiently large number of particles. Image analysis issues are related to variabilities associated with analyst bias/subjectivity and inconsistency during image analysis, both strongly related to the experience level of the analyst, and they can result in unreliable measurement results (Meija et al. 2020).

The impact of TEM imaging conditions and analyst subjectivity on the variability of CNC size distributions was recently assessed by Meija et al. (2020) in an interlaboratory comparison (ILC) study with ten participating research groups from around the world. This ILC study focused on the effect of differences in imaging conditions (e.g., different TEM machines and imaging settings) and image analysis (e.g., particle selection and measurements) on the particle size distributions of a reference CNC material (CNCD-1 2016). The TEM samples were prepared by a single laboratory, minimizing any variability associated with sample preparation. After a rigorous comparison, the study reported that the particle selection and sample heterogeneity (e.g., agglomerates versus individual CNCs, uneven staining/contrast) are primary reasons for differences in the measured length and width size distributions.

In an effort to address these issues in CNC particle size measurement, Yucel et al. (2021) developed a semi-automated image analysis framework (Standardized Morphology Analysis for Research and Technology—SMART) that can detect and measure the dimensions of individual CNCs in TEM and AFM images. As is the case for manual image analysis of CNCs, it is critical for SMART image analysis to have high quality TEM images (e.g., low noise, high edge resolution and contrast), with well-dispersed CNC particles. The SMART approach was developed and compared critically against the TEM and AFM image data sets (e.g., 434 TEM images, 66 AFM images) and image analysis completed by conventional manual approaches reported by Jakubek et al. (2018). The CNC particle size measurements and distributions as measured by SMART were consistent with those measured from the current state-of-the-art manual approaches used by Jakubek et al. but with a much faster throughput (e.g., measurements of less than 1.5 s/CNC, compared to ~ 30 s/CNC for manual approaches). The utility of the SMART approach is that it can expeditiously process high-throughput image data while being minimally impacted by human error and variability. Our first SMART paper (Yucel et al. 2021) used images from a single laboratory, while this current study uses images from several laboratories to assess the generality of the method to data collected in different laboratories, and with different instruments, etc.

In this study, the SMART approach was used in the analysis of TEM images from four laboratories that participated in the ILC study on CNC particle size measurement (Meija et al. 2020). The following aspects were explored using SMART: effects of image contrast and heterogeneity on CNC identification (e.g., noise, contrast, CNC density), image analysis issues (e.g., SMART versus manual approaches), assessment of representative measurements for length and width, and image analysis of agglomerated CNCs. The results obtained from the SMART approach were compared critically against the results obtained from the conventional manual approaches used within the ILC study.

Methodology

TEM sample preparation and imaging

For quantifying CNC particle morphology high quality images are needed, which is contingent on having a homogeneous dispersion of CNC particles with minimal CNC-CNC contact or agglomeration, a uniform contrast across the image, and high edge contrast between CNC particles and the supporting substrate. The TEM images used in this current study were from the ILC study by Meija et al. (2020), which reported on the optimized methods for CNC dispersion, TEM sample preparation and image acquisition. Overall, the image quality was high for all laboratories, a representative example of which is shown in Fig. 1a. The ILC used the CNC certified reference material (CNCD-1) produced by the National Research Council, Canada (CNCD-1 2016), and all TEM samples were prepared as previously reported by depositing 0.02 wt% CNC aqueous suspension onto a plasma-exposed carbon-coated copper grid and then staining with uranyl acetate to improve contrast. The TEM samples were prepared at a central laboratory and sent to each participating laboratory for imaging and CNC particle size analysis.

Fig. 1
figure 1

Image quality assessment illustrating contrast and noise assessment using a TEM image from Lab2. a Original (grayscale) TEM image with pixel values from 0 to 255. Background pixels have darker and lower intensity values, while CNC pixels have lighter and higher intensity values (see part c). b Segmented (binary) image for which pixels are either 0 (background) or 1 (CNCs). c Histograms of the grayscale intensity values for background (blue bars) and CNC pixels (orange bars). The black vertical lines show the mean values (120 for background and 150 for CNC pixels), and ∆ (30) is the difference between the means. Noise is the standard deviation of the blue bar histogram (background intensity scatter)

Full size image

From the ten laboratories that participated in the ILC study, four data sets (Lab1, Lab2, Lab6, and Lab7) were selected for SMART analysis (see Table 1), based on the relatively large variation in their reported CNC particle size measurements and the imaging parameters used. Note that for Lab1, TEM images were taken at two different magnifications resulting in different pixel size, but the image analysis results were combined in the ILC report since no statistical difference between the two was detected. The 8-bit grayscale TEM images were exported as TIF files from the microscope software and analyzed by either the manual approach (using Image J) or SMART.

Table 1 Imaging parameters and measurement summaries for each laboratory in the ILC study report (Meija et al. 2020)
Full size table

SMART semi-automated image analysis framework

The identification of CNC particles and their size measurements were completed using the recently developed SMART framework that employs a combination of automated and semi-automated image processing algorithms. All analysis/calculations reported in this manuscript were completed on a desktop computer with an i7 processor. A brief description is given here, but for complete details see Yucel et al. (2021). The SMART framework uses a multi-step process that includes: (i) pre-processing to remove noise and improve contrast in the image, and segmentation to identify CNC edges, (ii) CNC classification to identify individual CNCs for measurement, and (iii) digital measurements and analysis tools for CNCs. Pre-processing is critical for TEM images of CNCs that are typically noisy and have low contrast and low edge sharpness. Pre-processing used a sequence of image processing algorithms that included: contrast enhancement, smoothing (for noise removal), and sharpening. Segmentation was used to process the grayscale image into a binary image of only 0’s (background) and 1’s (CNCs), which separates the objects being imaged from the substrate background and facilitates SMART in identifying CNC edges.

For pre-processing, choosing the proper filter levels is important for detecting the CNC edges. Image processing filter size (N by N neighborhood) and pixel size are inversely proportional; for further details see Yucel et al. (2021). Filter size for smoothing and sharpening operations (i.e., choosing the level 1–5 filter sizes) was determined after an initial analysis of a 10-image data set for each laboratory. Noisy image data sets (all but Lab2) were processed with level 5 for smoothing and level 5 for sharpening, while the less noisy data set (Lab2) was processed with level 1 for smoothing and level 5 for sharpening. The filter size for segmentation is automatically calculated based on the pixel size information. The time to determine pre-processing filter levels ranges between 5 and 20 min for each laboratory depending on the quality of TEM images.

Image quality assessment

Image quality influences how effective SMART is at the identification and measurement of CNCs. Images were analyzed qualitatively (e.g., CNC density, and individual CNC selection) and quantitatively (e.g., CNC groupings, image noise and contrast). To quantify image noise and contrast, SMART first performs image segmentation so that each pixel in the image is either assigned as CNC or background. Each pixel has a grayscale density value between 0 and 255, and the segmentation process, which was explained in our initial study in detail (Yucel et al. 2021), assigns 0’s for background pixels and 1’s for CNC pixels (Fig. 1b). The original grayscale values of background pixels (black pixels in Fig. 1b) are analyzed (the histogram of these pixel values is shown with blue bars in Fig. 1c), and the standard deviation of these values is used to parameterize the background noise of the image. If the background region consists of pixels with a large variation in grayscale intensity values (i.e., high standard deviation), it appears as a noisy and grainy background that challenges the detection of CNC edges. The original grayscale values of CNC pixels (white pixels in Fig. 1b) are also analyzed (the histogram of these pixel values is shown with orange bars in Fig. 1c). The difference between the mean of CNC pixel values and the mean of background pixel values is used to parameterize the contrast of the image. This difference is represented as ∆ in Fig. 1c where black vertical lines show the means of both histograms. In the 10-image date set study, the influence of image noise, contrast and CNC agglomeration on individual CNC identification and measurement was investigated.

CNC grouping identification

The ideal well separated and uniformly distributed CNC configuration on a TEM grid is difficult to achieve. In this work, CNCs were categorized into three groups: border CNCs, isolated CNCs (e.g., no contact with other CNCs), and agglomerated CNCs (e.g., multiple CNCs touching, overlapping, parallel stacked, etc.). Border CNCs are objects that touch one of the edges of the image and are identified using the built-in MATLAB function imclearborder (MATLAB 2020). Isolated and agglomerated CNCs were identified using a 2-step process: coarse grain selection and a fine grain selection. The coarse grain selection is based on encapsulating the object within an ellipse and calculating the major and minor axes lengths (Fig. 2a, b). By selecting a unique combination of ranges for aspect ratio (i.e., the ratio of the length of the major axis over the length of the minor axis) and size limitation (e.g., minimum and maximum lengths of major and minor axis), isolated CNCs can be distinguished from agglomerates. The parameters used in this study for isolated CNCs are typical of those produced from wood pulps: aspect ratio greater than 2.5, major axis length between 15 and 250 nm, and minor axis length of less than 15 nm. These parameters were selected based on prior reports in the literature (Jakubek et al. 2018).

Fig. 2
figure 2

Schematic showing coarse grain selection: a an isolated CNC and b overlapping CNCs, with their corresponding elliptical construct (black line), minor axis (red line), and major axis (blue line) overlaid. The fine grain selection is based on cord length analysis. c The segmented CNC image horizontally rotated, vertical and horizontal cords are measured. The three red lines represent three chords that are perpendicular to the long axis of the CNC and representing particle width. d The width profile (red thin line) consists of the collection of vertical chord lengths along the long axis of the CNC particle. The black points mark the range used to estimate the width of the CNC. The red dashed line is average value of the black points

Full size image

The fine grain selection used chord length analysis to further assess the width of an object, to identify if there are parallel stacked or “stepped” CNCs (Figs. 3e, S1). Chord length analysis of each object in the segmented image starts with the rotation of the object so that the major axis lies horizontally (as in Fig. 2c), then measures the lengths of vertical chords (e.g. red lines in Fig. 2c) which represent the width profile of the object (Fig. 2d). This same procedure is repeated to obtain the length profile. A specific width ratio, the ratio of the maximum width over the average width (e.g., wratio = wmax/wave), is defined (wratio > 1.5) and used to remove parallel stacked CNCs. Figure 3 demonstrates an example of the fine grain selection by comparing an approximately elliptical-shaped CNC and a stepped object that could be parallel stacked CNCs.

Fig. 3
figure 3

Width-based CNC selection using vertical chord profiles. a Starting Lab6 TEM image. b and e SMART encapsulation of CNC objects (green line). c and f segmented and rotated CNCs. d and g width profiles. While an isolated CNC (b, c, and d) has a smooth width profile, parallel stacked CNCs (e, f, and g) have a “stepped” structure and as a result was removed from being considered as an isolated CNC. h CNC grouping and area percent of isolated CNCs (red objects – 12%), agglomerated CNCs (yellow objects – 57%), and border CNCs (blue objects  – 31%)

Full size image

The CNC length and width are obtained by averaging the chord values (black data points in Fig. 3d). The selection of the averaging regions for length and width was empirically based on comparisons between SMART and manual measurements as described in detail (Yucel et al. 2021). The range over which averaging occurred was based on maximum chord minus 10% of the range of all chords for length and 50% for width. Averaging over the selected longest cords removes artifacts associated with noisy background pixels becoming “attached” to the periphery of the CNC and artificially increasing the corresponding chord length. This attachment effect is more dominant for width chords, as the size of the attachment can be a significant fraction of the width of a CNC. As a result, the averaging range for width measurements was increased to 50%. This averaging approach reduces artifacts resulting from edge effect attachments on the measurement of CNC length and width.

It is possible to assess CNC agglomerates by selecting intermediate sized objects (e.g., greater in length or width and lower in aspect ratio than isolated CNCs). To illustrate this a CNC grouping defined by aspect ratio higher than 4 and a minor axis length between 10 and 20 nm was used. This grouping would typically be several CNCs bundled together either as parallel stacked, linear aligned or a combination of these two (Fig. S1). The resulting length and width of such objects were estimated by fitting the SMART identified periphery with an oval shape using MATLAB’s regionprops (MATLAB 2020). The estimated agglomerated object length and width were the maximum and minimum ellipse axis, respectively (Fig. S1). A word of caution: the agglomerates observed for CNCs deposited on the TEM grid may not represent the CNC configurations within a given CNC suspension. However, if such agglomerates are confirmed to exist in the original suspension, they may affect the performance of the suspension and should be characterized.

Area percent

Area analysis is an additional metric that SMART can employ to characterize the CNC configuration within TEM images. The area percent of individual, agglomerated, and border CNCs with respect to the total image area or total CNC area can be calculated. Each area value is obtained by counting the pixels of each color-coded group (e.g., area of isolated CNCs is the number of red pixels, Fig. 3h). While the total CNC area is the total number of all CNC pixels (i.e., sum of red, yellow, and light blue pixels, Fig. 3h), the total image area is the total number of pixels in an image (e.g., images are 2048 × 2048 pixels). Calculating these area percentages, SMART can provide additional details on the configuration, size and area percent of each group. As shown in Fig. 3h, the area covered by isolated CNCs is a low fraction of the total area covered by CNCs.

Results and discussion

This study used SMART to analyze TEM images from the ILC study by Meija et al. on CNC particle size distributions measured by TEM (Meija et al. 2020). The advantages of this ILC data set are that a well characterized CNC certified reference material was used, all TEM samples were prepared by the organizing team using optimized techniques and sent to the participating laboratories, and all laboratories followed detailed guidelines for TEM imaging, the identification of isolated CNCs and measurements of their length and width; all these factors help to minimize measurement artifacts. This allowed the ILC to assess variations in laboratory-to-laboratory measurements associated with differences in TEM equipment, image acquisition settings, resolution, and CNC selection. It is critical for both manual and SMART image analysis of CNCs to have high quality TEM images (e.g., low noise, good contrast between CNCs and the background region, distinct CNC edges, and a low density of CNCs homogeneously distributed on the imaging substrate), requiring special care in TEM sample preparation and imaging as explained in references (Meija et al. 2020; Yucel et al. 2021). The SMART analysis was completed in two phases: a detailed 10-image data set study (e.g., 10 TEM images from each laboratory), and a more general analysis of the complete TEM image data set.

10 TEM image data set analysis

Detailed direct image-to-image comparisons between laboratories and between image analysis approaches (SMART versus manual) was completed using a small subset of TEM images. This was necessary to analyze and confirm how SMART identifies and selects different objects within a given TEM image. For each laboratory, 10 TEM images were randomly selected from their data set, and SMART image analysis was completed, and comparisons were made to assess: (i) image quality differences between laboratories, (ii) CNC identification differences between SMART versus manual approaches, and (iii) differences in measured dimensions between SMART versus manual approaches for commonly identified CNCs. For this 10-image data set study we have separated Lab1 into Lab1-a and Lab1-b because TEM images were taken at two different magnifications resulting in different pixel size and this separation allowed SMART to assess these data sets separately.

Image quality assessment

Image quality strongly influences the identification of CNCs and their size measurements, affecting both SMART and manual image analysis approaches. Thus, it is important to parameterize the features of image quality that directly affect image analysis so that these factors can be considered. The ILC study reported that imaging resolution, contrast, and analyst bias all contributed to variation in measurement. The study suggested that a higher resolution (0.2 nm/pixel rather than the recommended 0.3 nm/pixel) would give a more accurate estimate of CNC width but was not a factor in the accuracy of the estimate of CNC length. In the current study, the image quality from each laboratory was sufficient for SMART analysis. However, as shown in Fig. S2, there was some variation in image quality between the laboratories and an attempt was made to assess these differences and understand how they affect the SMART identification and size measurement of CNCs.

Image quality was parameterized using noise, contrast, pixel size, and CNC density within TEM images. SMART assessed noise, contrast, and CNC area percentages as described in the methods section. The results are summarized in Table 2 show a low level of variation between the laboratories. Images with lower noise, higher contrast, smaller pixel size (e.g., higher magnification), and lower CNC density are expected to facilitate image analysis and improve the capability of SMART and manual approaches to correctly identify and measure individual CNCs (Fig. 4). CNC density was qualitatively assessed by relating the CNC area percent (e.g., total CNC area divided by total image area) to the relative percentages of isolated CNCs versus agglomerated and border CNCs. The total area percent of CNCs within the images for each laboratory was less than ~ 12%, and represented a combination of isolated, agglomerated and border CNCs with a reasonable low level of CNC density. In general, a lower CNC area fraction with a higher percentage of isolated CNCs indicates a lower CNC density. Examples of very low and high CNC density levels are shown in Fig. S3. The pixel size was inversely related to noise, with smaller pixel size associated with images with greater noise, but was not related to contrast, CNC total area or CNC density. Pixel size was partially accounted for in the SMART analysis during image pre-processing operations with the selection of filtering operations (e.g., smoothing, sharpening, and segmentation) as mentioned in the methods section and described in greater detail in Yucel et al. (2021).

Table 2 Image parameters for each laboratory (10-image dataset)
Full size table
Fig. 4
figure 4

TEM images showing differences in image quality and the effect on individual CNC identification. a and b are images with mid-level contrast, low noise, and a lower CNC density (Lab2), while c and d are images with low contrast, high noise, and higher CNC density (Lab7). Manual identification of individual CNCs from the ILC study are shown in parts a and c, in which the superimposed blue ovals highlight the identified CNCs. SMART identified individual CNCs are shown in parts b and d, where the green outline is the SMART identified object perimeter. The high noise, low contrast, and CNC agglomeration in TEM images increases SMART misidentification of CNC fragments as individual CNCs (part d). Misidentified CNCs are labeled with*

Full size image

Interestingly, even with optimized TEM sample preparation and imaging parameters there was still variation within TEM images, both intra- and inter-laboratory, with respect to noise, contrast, and CNC dispersion. One advantage of SMART image quality assessment is that it is based on individual images via parameterizing the noise, contrast, and CNC density within TEM images and can also assess the effect of pixel size. This gives a more robust and quantitative assessment of the image quality and potential influence on image analysis.

CNC identification assessment

A detailed side-by-side comparison between SMART and manual analysis of each image was used to assess the capability of SMART to identify individual CNCs and measure their dimensions. Based on visual inspection of the TEM images, three aspects were considered and grouped as: (i) “shared” identification, where the same CNC was identified in both SMART and manual, (ii) “unshared” identification, where CNCs were identified only in SMART or manual but not both, and (iii) “misidentification” where objects were incorrectly identified as individual CNCs. The total number of CNCs measured by both analysis methods and the number of CNCs for the three groups are summarized in Fig. 5. By taking the number of identified objects in each group and dividing by the total number of objects identified for each laboratory, we can assess the percentages. The shared identification group accounted for approximately half of all the CNCs identified, except for Lab1-b, which is much lower. Interestingly, the observation that the shared identification accounts for only approximately half of the identified CNCs indicates that SMART analyzes features within the images differently than the manual approach. Likewise, this also leads to higher percentages of the unshared identification groups, where the range was between 14 and 35% for SMART (e.g., manual did not identify these), and between 18 and 58% for manual (e.g., SMART did not identify these). Combining the shared and unshared identification groups gives the percentage of “correctly” identified CNCs. For SMART Lab1-a (78%), Lab1-b (58%), Lab2 (78%), Lab6 (72%), and Lab7 (59%), while for manual Lab1-a (76%), Lab1-b (87%), Lab2 (73%), Lab6 (81%), and Lab7 (70%). There was fairly good agreement between SMART and manual approaches except for Lab1-b, and Lab7, which revealed a higher level of misidentification for SMART.

Fig. 5
figure 5

Comparison of SMART and manual analysis for CNC identification from the 10-image data set study: 1 Shared identification (correctly identified by both methods) 2 Unshared identification (correctly identified by one method but not the other) 3 Misidentified (incorrect identification)

Full size image

The level of misidentification, as identified by one of co-authors, was within the range of 22% to 41% for SMART, and 13% to 30% for manual depending on the given laboratory data set. Some level of misidentification is to be expected, but it should be noted that our visual assessment of “misidentification” is subjective, and thus should be considered a qualitative assessment. Regardless, the observation that both SMART and manual approaches have misidentified objects as individual CNCs shows that neither approach is perfect.

Misidentification by SMART was primarily based on identifying fragments of agglomerated CNCs as isolated CNCs as shown in Figs. 4d, S4 and seemed to be exacerbated in images with higher noise, lower contrast, smaller pixel size and higher CNC density. These four parameters act concurrently, making it challenging to observe trends as to the impact of each parameter on the level of misidentification for these five data sets (Table 2). The highest misidentification for SMART was 41% for Lab7 and Lab1-b, and for manual it was 30% for Lab7. The Lab7 images had the highest noise, lowest contrast, smallest pixel size, and a higher CNC density, while Lab1-b images had high noise, high contrast, small pixel size, and the highest CNC density (e.g., a total CNC area of 12.1%, of which 96% was either agglomerated or border CNCs, see Figs. S2b, S3b). This combination for Lab1-b was challenging for SMART, but less so for the manual approach in which only 13% of CNCs were misidentified. Interestingly, when comparing Lab1-a and Lab1-b, SMART analysis had fewer misidentifications for Lab1-a, despite having the same level of noise, lower contrast, and larger pixel size. The key difference was in the CNC distribution within the images, in which Lab1-a had a lower CNC density as compared to Lab1-b (e.g., Lab1-a had a lower CNC area fraction consisting of a lower percentage of agglomerated and border CNCs as compared to Lab1-b). Lower CNC density within images seems to facilitate SMART analysis. In support of this, the highest shared CNC identification for both SMART and manual was for Lab1-a and Lab6, both of which have a lower CNC density within the images. In general, higher CNC area fraction and a larger number of agglomerated CNC configurations are extremely challenging for SMART, since the localized contrast variations cause SMART to misidentify agglomerate fragments as isolated CNCs (Figs. 4d, S4).

Misidentification by manual image analysis was primarily based on identifying multiple CNCs aligned end-to-end or parallel stacked CNCs as single particles. For example, as shown in Fig. 4c (lower right), there are three objects identified as individual CNCs, however, the step-like features of these objects suggests that they may be parallel stacked CNC agglomerates. Note that this assessment is subjective as CNC identification is tricky at best, which is why the analyst’s experience and ability to consistently apply particle selection guidelines are critical for manual analysis. The ILC study had initially tested the manual image analysis protocol by sending a single small image set to multiple labs for analysis. There was significant variation in the number of particles counted and the mean lengths and widths. Although the protocol was further optimized prior to data acquisition and image analysis by each participant, the final results indicated that analyst subjectivity was still an important contributor to variation between laboratories.

CNC measurement assessment

Considering the commonly identified CNCs by both methods (e.g., the shared identification group) length and width measurements of each CNC were compared point-by-point in Figs. 6 and S5 where the x-axis in Fig. 6c represents the manual measurement values and the y-axis represents SMART measurement values. If SMART and manual measure the same length or width for a given crystal, the data point would fall directly on the black line. In general, the length measurements from the two methods are in good agreement (i.e., the majority of the points lie on the black line, SMART = manual). There were a few cases where SMART measurements were lower than those from manual, which is indicative of SMART “trimming” off the ends of lower contrast CNCs as described in Yucel et al. (2021). The width measurements have some difference between the two approaches; thinner CNCs (e.g., less than 6 nm) are larger in SMART measurements, while thicker CNCs (e.g., greater than 7 nm) have smaller SMART measurements than for manual. Such differences are not surprising and can be partially associated with SMART calculating an average width for each CNC while only a single measurement was made for manual measurements. The SMART averaging was used to help negate the effects of noisy background pixels that inadvertently add to the CNC edge and make the CNCs appear wider than they should be. With the variability as to the extent of noisy pixels that may increase the size, SMART averaging may slightly overestimate or underestimate the CNC width. Additionally, there may be some variability in the manual measurements of width. For example, the CNC widths are so small that the manual selection of the particle edge using a cross-section perpendicular to the long axis of the particle is very difficult. In addition, placing the line on the particle edge versus just outside the edge would introduce differences in width measurement. The level of analyst fatigue completing such measurements can be high. The average length and width measures are reported in Table S1; for the shared CNC group the difference between SMART and manual for each laboratory was typically less than 5%, except for the Lab6 width measurement which was about 15% larger for the manual method. These results indicate that the SMART and manual approaches give very similar values for the same particle. When considering all CNCs (e.g., shared, unshared, and misidentified) the mean values of length for each lab change by up to 20%, but less than 10% for width. The inclusion of the unshared and misidentified objects can increase the differences between SMART and manual length measurements, substantially, as was the case for Lab1-a (45%), and Lab1-b (31%), and is indicative of the effect of differences in object selection between SMART and manual approaches. Despite the level of unshared and possibly misidentified objects, the overall agreement between the two methods suggests that SMART is “as good as a human”.

Fig. 6
figure 6

Side-by-side image analysis comparison between SMART and manual methods for Lab1-a. a Manual identification and measurements. Three CNCs were identified and measured (blue ellipses superimposed only as a highlight), b SMART identified four CNCs (green outline is SMART identified CNC perimeter), three of which were identical to manual measurement, c one-to-one comparison of measured length and width for each of the 19 commonly identified CNCs in the 10-image data set for Lab1-a

Full size image

Complete TEM image data set analysis

The complete TEM image data sets for each laboratory were analyzed by SMART. This larger image data set is more representative of that used for typical CNC particle morphology characterization, where the number of measured CNCs is more likely to reach the critical threshold necessary to obtain representative values for particle length and width. The image data sets for Lab1-a and Lab1-b were combined as the ILC study confirmed that the manual measurements were not statistically different. From the complete TEM image data set the following five aspects were studied: (i) Comparing laboratories with SMART, (ii) SMART versus manual assessment, (iii) representative measurement assessment, (iv) area fraction assessment, and (v) CNC agglomeration assessment.

Comparing laboratories with SMART

Comparison of the SMART length–width 2D histograms for each laboratory (Fig. 7) shows a similar profile shape, length and width ranges and the location of a higher intensity probability zone. The histogram profile appears to have an “oval” shape with one end positioned at low width and length and the other end position at high width and length, which indicates that wider CNCs have longer lengths. There is also a smaller zone of higher probability, centered near a length of 60 nm to 70 nm and a width of 6 nm to 7 nm. This zone of higher probability is similar to the results reported by Jakubek et al. (2018) for manual analysis of TEM images of the same CNC certified reference material (CNCD-1) that is used in this current study (CNCD-1 2016). In Table 3, the SMART mean length (92.4 nm, 98.4 nm, 89.2 nm, 77.9 nm), mean width (7.9 nm, 8.1 nm, 7.0 nm, 7.6 nm), and mean aspect ratio (11.7, 12.1, 12.9, 10.2) for each laboratory (Lab1, Lab2, Lab6, Lab7, respectively) were similar and variations in the means between laboratories was well within the range of standard deviation for the 4-laboratory mean. This consistency in measurements suggests that the variations in image contrast, noise, pixel size and CNC density (Table 2) from the different laboratories did not have a significant effect. However, there may be some limited discrepancy for Lab7 as it has the lowest mean length and aspect ratio, which may be attributed to the combination of the highest noise, lowest contrast, smallest pixel size, and high CNC density within TEM images (Table 2, Fig. 4c, d), which caused SMART to misidentify fragments as single CNCs and subsequently shift the measurements to lower lengths and aspect ratios.

Fig. 7
figure 7

2D-probability histograms for CNC length and width measurements for SMART and manual analysis. Each small box within the plots represents an interval of 10 nm for length and 0.5 nm for width. Less than 2% of the data lies outside of these plots (e.g., 1.8% of Lab1 – manual data set has CNCs with length > 250 nm). Note that probability times total number of CNCs (Table 3) for a given laboratory gives the count for each box

Full size image
Table 3 SMART versus manual analysis summary for each lab. (Complete image data set)
Full size table

SMART versus manual assessment

The number of TEM images for the four laboratories ranged between 115 and 244 (Table 1), from which the SMART and manual approaches identified similar numbers of CNCs (e.g., 500 to 600 CNCs) except for Lab6 for which the manual approach identified 1179 CNCs (Table 3). The average number of CNCs analyzed per image for laboratories 1,2,6,7 was similar (SMART: 2.9, 3.9, 2.3, 5.1; manual: 3.4, 4.6, 4.8, 4.5, respectively). The analysis time for SMART was less than 15 min for each laboratory (Table 3). The analysis time for the manual approach was not reported by ILC participants, so an estimate of 30 s/CNC was used here, which includes the analyst’s time to identify isolated CNCs, measure their dimensions, and record and plot the data. While the shorter analysis time is an important advantage of SMART, consistency in particle selection and measurement is essential.

In general, there was reasonable agreement in CNC particle size measurements between SMART and manual approaches as assessed by considering the mean lengths and widths, (Table 3), the length and width distributions (Fig. S6), and the length–width 2D histograms (Fig. 7). Both approaches measured similar means for length (77 nm to 99 nm, except for the manual—Lab1 of 116.6 nm), width (6.9 nm to 8.1 nm) and aspect ratios (10.2 to 15.7). Interestingly, the average mean for the four laboratories is remarkably similar for SMART and manual approaches (length 89.5 nm, 93.9 nm, width 7.7 nm, 7.5 nm, and aspect ratio 11.7, 13.1, respectively). The measurement distributions for both SMART and manual were similar. Length and aspect ratio measurement had asymmetrical distributions with the peak shifted to lower values and a tail to higher values (Fig. S6). The width measurement distribution was nearly symmetrical for both SMART and manual approaches, with SMART having a narrower distribution (Figs. 7, S6). The width distributions may be narrower for SMART (than manual) due to the filter parameters used, though as demonstrated in the fitting parameter study in Yucel et al., there was only a maximum of 10% error in width measurement when comparing the smallest and largest filter parameters (Yucel et al. 2021).

In general, the length–width 2-D measurement histograms show a good level of overlap between the SMART and manual approaches (Fig. 7, top vs. bottom row, respectively). As described in the prior section the SMART histogram profile is oval shaped, has a higher probability zone and is consistent between the four laboratories. In contrast, the manual profile appears to be more circular in shape, with a less well-defined zone of higher probability, and is less consistent between the four laboratories. Manual analysis of Lab1 appears to be more of an outlier having a wider spread in length and width. Interestingly, since SMART shows lower laboratory-laboratory variation as compared to manual, it suggests that by using SMART the variability of analyst bias of manual approach, which is different at each laboratory, is removed and thus improves the consistency in CNC size measurement between the laboratories. This outcome agrees with the conclusions from ILC study that generated the data used in the current study (Meija et al. 2020). The authors of the ILC paper stated that ‘analyst bias/subjectivity and sample heterogeneity are the main sources of ILC variability. The subjectivity in choice of analyzable CNCs can in principle be reduced by use of automated image analysis methods that are currently being developed.’

The SMART and manual analysis results for the four laboratories can also be compared to the overall consensus values obtained from the ILC study. The latter are based on manual analysis of ten laboratory data sets and development of a final consensus values using a data pooling approach. This gave mean values of 95.8 nm and 7.65 nm for length and width, respectively, with distribution widths (1 standard deviation) of 39.0 nm and 2.20 nm. The manual analysis for the four data sets used here gave means of 94.0 nm and 7.53 nm for CNC length and width in good agreement with the consensus values. The SMART analysis means of 89.5 nm and 7.65 nm for length and width are also in good agreement with the ILC consensus values, particularly for width. Although the length values from SMART and manual analysis differ by ~ 5 nm, the standard deviations are sufficiently large that one cannot distinguish whether this is a statistically meaningful difference. Nevertheless, the level of agreement for a limited number of data sets is impressive and indicates the utility of the SMART approach for dramatically reducing the analysis time compared to a manual approach.

Representative measurement assessment

The SMART analysis can simultaneously calculate cumulative means (length and width) for every measured CNC and give feedback to the analyst to help assess when measurements are representative (e.g., reach a stable or steady-state value). By tracking the percentage change from the overall mean for each cumulative mean value it may be possible to assess when a representative measurement has been achieved. The cumulative mean length and width and the percent change versus number of CNCs measured for each of the four laboratories are shown in Fig. S7. During analysis as more measurements are added the cumulative means and the percent change curves will oscillate and generally taper off to a constant value. This is what was observed for each laboratory, indicating that a sufficient number of CNCs were measured to obtain a representative measurement. The horizontal dashed lines represent the mean length or width values for the entire data set, which are listed in Table 3. For Lab1, 536 CNCs were measured and gave a mean length and width of 92.4 nm and 7.9 nm, respectively (Fig. 8a). The percent change curves can highlight the scale of deviation from the data set mean value and can give more confidence when assessing if a representative measurement has been achieved. For example, if 300 CNCs were measured (Fig. S8), the corresponding mean length (96.8 nm) and width (7.9 nm) differ from the means obtained after 536 CNC measurements by only 4.7% and 0%, respectively (Fig. 8b). Thus, with the additional CNC measurements from 300 to 536 the change in mean length and width were small, which further confirm that the overall means are representative measurements. Interestingly, for this example, width measurements were stable with near zero percent differences after only 200 measurements, demonstrating that a stable measurement was achieved more readily for width than length. When assessing how many measurements to perform, the analyst needs to determine what percent difference (or error) is acceptable for their measurement purposes. Additionally, if SMART is run concurrently with the imaging experiments, it can be used to help guide the analyst when an adequate number of images has been collected. This is not practical with manual analysis of particle size as used in the ILC study.

Fig. 8
figure 8

SMART assessment of the critical number of analyzed CNCs to reach a representative measurement for Lab1. a Cumulative mean length and width versus number of CNCs measured, where the dashed lines show the calculated mean length and width value for the entire TEM data set (e.g., 536 CNCs measured). b Percent change in cumulative mean length and width versus number of measured CNCs

Full size image

Area fraction assessment

The area fractions of isolated and agglomerated CNCs provided by SMART can be used to provide a quantitative assessment of the CNC density within TEM images. Correct identification and measurements of the CNCs depend on how well dispersed the CNCs are on the TEM grid. The dispersion level varies significantly both within and between samples, as illustrated in Fig. S3 which shows very low (Lab6) and high (Lab1-b) CNC densities. Assessment of the CNC density was performed by relating the total CNC area to the total image area as well as to the specific CNC grouping areas (i.e., isolated, agglomerated, and border). Figure 9 shows the area percentages for the three CNC groupings for each laboratory. Each stacked bar represents the area percent of isolated CNCs (red), CNC agglomerates (yellow), and border CNCs (blue). The total CNC area percent over total image area for each laboratory ranged between ~ 6 and  ~ 14%, with the corresponding area percent ranges for isolated CNC (0.3% to 1.4%), agglomerated CNCs (4.0% to 9.3%), and border CNCs (2.1% to 3.4%) as summarized in Table S2 and Fig. 9. The relevance of this area analysis is that it gives a quick quantitative indicator as to the CNC density within the image. Isolated CNCs account for 3.7% to 10.1% of the total area covered by CNCs within the images, demonstrating that only a small fraction of the CNCs seen within a TEM are used to assess CNC particle morphology. This is currently standard practice and reflects the challenges with obtaining well-dispersed CNC samples.

Fig. 9
figure 9

Area fractions of each CNC group (i.e., isolated, agglomerated, and border) over total image area for each laboratory. The values are the average of the entire image data set for each lab analyzed by SMART

Full size image

CNC agglomeration assessment

To provide additional morphology information, SMART can analyze the agglomerated CNCs, which compose 59% to 66% of the CNCs within the TEM images. CNC agglomeration can take many forms, including multiple CNCs touching, overlapping, linearly linked, parallel stacked, etc. SMART was used to identify two subsets of linearly and parallel stacked agglomerate morphologies (Fig. S1). These objects were identified by two approaches: (i) “stepped” objects were the objects removed by width-based selection from the standard SMART analysis (i.e., aspect ratio greater than 2.5, major axis length between 15 and 250 nm, minor axis length of less than 15 nm), and a width ratio greater than 1.5 as defined by the maximum width over the average width, and (ii) “grouped” objects were defined by having an aspect ratio greater than 4 and minor axis length between 10 and 20 nm. Other agglomerate morphologies could also be analyzed, requiring only small adjustments to object selection criteria. Visual inspection of features that SMART identified as “stepped” and “grouped” (Fig. S9) confirmed that object selection was consistent with the “stepped” and “grouped” selection criteria. The stepped CNC objects were predominately isolated objects that had a rougher edge profile than the isolated CNC selection, a result consistent with parallel stacked CNCs. Additionally, some objects were identified where the roughness was caused by interaction of background noise with the CNC edge. The grouped CNC objects were typically longer, wider, and had a rougher edge profile than the isolated or stepped CNC selections. These objects appeared to be multiple CNCs; many were isolated objects, but some were fragments from larger agglomerates. The selection of the stepped and grouped CNC objects also appeared to be influenced by image quality. The length and width measurements of the stepped and grouped CNC objects were based on the maximum and minimum ellipse axis used to encapsulate the object. As shown in Fig. S9, the ellipse simplifies the object, while still capturing the average length and width of the object.

A summary of the size measurement of isolated CNCs, stepped CNCs and grouped CNCs is given in Fig. 10. For all four laboratories, the mean length measurements of the stepped CNCs were similar to the isolated CNCs. In contrast, the grouped CNCs were longer and had a broader distribution, which was expected based on the object selection criteria. The width measurements of the stepped and grouped CNCs were larger than that of the isolated CNCs. The length–width 2D histograms for isolated, stepped, and grouped CNC plots for Lab1 are given in Fig. S10, and show pictorially how the stepped objects are wider and how the grouped objects are much wider and longer than the isolated CNCs. The inclusion of the stepped and grouped objects increased the number of objects (e.g., for Lab1, 706 stepped, 1416 grouped) and area percentage of the cellulose within the TEM image to be analyzed. The area percent for the stepped and grouped objects for each lab was: Lab1 (0.5%, 0.9%), Lab2 (0.9%, 1.1%), Lab6 (0.4%, 1.1%), and Lab7 (0.7%, 1.7%), respectively. When combined, this stepped and grouped area percent accounts for approximately a quarter of the agglomerated CNCs within the TEM images, which could give greater insight into the types of object morphologies that make up a given CNC suspension. These results should be considered cautiously as it is unclear if such agglomerates are artifacts from the TEM sample preparation or are hard agglomerates that existed after CNC preparation in suspension. The relevance here is that hard CNC agglomerates are likely to influence the properties of suspensions (e.g., rheology, self-assembly, etc.) and their utilization in various applications. With SMART we have a tool to facilitate investigation of such questions.

Fig. 10
figure 10

Size measurements of isolated CNCs, stepped CNCs and grouped CNCs. Object length and widths are estimated by the major and minor axis length of the oval encapsulation of the given object, respectively. Diamond and star markers represent mean values for each group, with filled symbols for the isolated CNCs. Upper and lower edges of the vertical rectangles correspond to the 75th and 25th quartiles for each group

Full size image

Applicability of SMART to different CNC morphologies

The spindle or rod-like morphology of CNCs extracted from wood or plant sources remains fairly consistent between species, with process conditions, and if produced in the laboratory or industrially, as shown in the following papers (Delepierre et al. 2021; Kaushik et al. 2015; Reid et al. 2017). Provided that the objects are not branched, or agglomerated, and the TEM images are of high quality with a low CNC density, SMART can accommodate deviations from the “ideal” spindle shape by adjusting the object identification parameters listed in section—Methodology/CNC Grouping Identification. For cases where the CNCs are in tight agglomerate bundles, such as in Gicquel et al. (2019), SMART will exclude almost all features as being agglomerates. Even though SMART cannot identify the individual CNCs within the agglomerates, it is possible to analyze the agglomerates by adjusting the object identification parameters. The proof of principle of this was demonstrated in the current study by adjusting parameters to isolate larger length objects. SMART can then provide additional analysis of the fraction of isolated CNC particles versus agglomerated CNC bundles.

Considerably different morphologies are observed for algal, bacterial and tunicate CNCs, which are wider and much longer than the wood based CNCs. The much larger sizes can be accommodated/identified by adjusting SMARTs aspect ratio or minimum, maximum length criteria. However, these objects may have kinks, curvature, overlap with other particles, or be agglomerated (Dunlop et al. 2020; Sacui et al. 2014), which will be problematic for the SMART analysis. Incidentally, there are other types of image analysis programs that may be better suited for fiber-like geometries, as summarized by Usov and Mezzenga (2015), who developed the analysis software FiberApp and demonstrated that it can be used to measure length, width, height, curvature, and kinking for various fiber systems including cellulose nanofibrils.

Conclusion

A semi-automatic image analysis program, SMART, analyzed TEM images from four laboratories that participated in a recent ILC study (Meija et al. 2020) on CNC particle size measurement. The SMART results were compared to the manual analysis of the same images by ILC participants. It was critical for SMART analysis to use high quality TEM images (e.g., low noise, good contrast between CNCs and the background region, distinct CNC edges, and a low density of CNCs homogeneously distributed on the imaging substrate); SMART was ineffective at identifying individual CNCs in poor quality images. The SMART analysis was completed in two phases: a detailed image-to-image study of 10 TEM images from each laboratory, and a more general analysis of all TEM images from each laboratory.

In the detailed image-to-image study, comparisons in CNC identification between SMART and manual approaches were classified into three categories: (i) “shared” identification, where the same CNC was identified in both SMART and manual, (ii)”unshared” identification, where CNCs were identified only in SMART or manual but not both, and iii) “misidentification” where objects were incorrectly identified as individual CNCs. In general, 50% of the CNCs identified by SMART and manual approaches were the same CNCs (e.g., shared CNCs), for which the percentage of “correctly” identified CNCs (e.g., shared and unshared CNCs) by SMART was 58% to 78%, and for manual was 70% to 87%. The inclusion of the misidentified objects (22–42% for SMART, and 13–30% for manual) in CNC size measurements was the primary cause of deviations between SMART and manual image analysis results. SMART was able to parameterize image quality, quantifying noise and contrast and qualitatively assessing CNC density within TEM images. In general, the noise, contrast level and CNC density for the images were compatible with use of the SMART approach for identifying isolated CNCs and measuring their dimensions. However, it was observed that images with a combination of higher noise, lower contrast, smaller pixel size, and higher CNC density, resulted in increased SMART misidentification of CNC agglomerate fragments as individual CNC particles (e.g., Lab1-b and Lab7).

The SMART analysis of all TEM images was a large data set in which the number of images analyzed and number of CNCs measured was as follows: Lab1 (185, 536), Lab2 (115, 449), Lab6 (244, 580), and Lab7 (125, 633), respectively. In general, there was overall good overlap in SMART and manual image analysis of CNC length and width. SMART had narrower distributions in width and contained a smaller zone of higher probability in length–width 2D histograms, centered near the length of 60 nm to 70 nm and width of 6 nm to 7 nm, which was consistent across all four labs. SMART also showed an expected trend that wider CNCs have longer lengths, which was less apparent with the manual measurements. Laboratory-laboratory variation was lower for SMART as compared to manual, suggesting that by using SMART the variability of analyst bias of manual approach was reduced. By plotting the cumulative mean length and width and the percentage change from the overall mean versus number of CNCs measured, the SMART analysis provides a mechanism to assess the likelihood of reaching a representative measurement for CNC particle size. SMART area analysis of CNCs within the TEM images found that less than 10% of the total area covered by CNCs was due to isolated particles, indicating that the majority of CNCs within a given TEM image are not characterized. As a demonstration to assess more of the CNC material within the TEM images, a function was added to SMART to analyze a small subset of linearly aligned and parallel stacked CNC agglomerates. However, as it is unclear if such agglomerates are artifacts from the TEM sample preparation or are hard agglomerates that existed in the starting CNC suspension, the implications of such results should be considered cautiously.

After the initial optimization of processing parameters, the SMART image analysis time was less than 15 min for each laboratory (having between 115 to 244 images). This analysis included object identification (individual, agglomerate, and border CNCs), object measurements (length, width, aspect ratio, area), and all plotting of data (e.g., 1-D histogram, 2-D histograms, cumulative means and percent change). The utility of the SMART approach for dramatically reducing the analysis time compared to a manual approach is demonstrated by the encouraging level of agreement in CNC particle size measurements from the SMART and manual approaches. The level of agreement in CNC size measurement between SMART and manual approaches, the lower laboratory-laboratory variation, the lower analyst bias and fatigue, the additional measurement capability, and the increased speed of image analysis highlights an opportunity with SMART to improve the standardization of CNC size characterization. A standard approach for image analysis will also facilitate a more in-depth exploration of the effect that CNC properties, sample deposition for imaging and microscope parameters have on the reliability of CNC size measurements. The SMART code and tutorial is now available for the public to use for free at Github™ (https://github.com/seyucel/CNC-SMART).