Keywords

1 Introduction

Several spectral imaging modalities have been adopted for the analysis of fine works of art as well as polychrome archeological art objects on the macroscale. At present, the modalities that have been more widely explored fall into two classes: those that provide elemental information and those that provide molecular information about the artists’ materials used. In all cases the information obtained is in the form of a three-dimensional image cube, consisting of two spatial dimensions representing the surface of the work of art and the third containing the spectral information. Thus, regardless of the spectral imaging modality or instrument used, the data can be considered as a large number of independent spectral measurements collected at regularly spaced spatial intervals over the surface of a work of art. These data sets are typically referred to as image cubes which are explored in order to find spatial pixels having similar spectral features that can be related to a material with a specific chemical composition. If X-ray fluorescence is measured, then the chemical elements can be identified, and if reflectance or luminescence is measured, then electronic transitions or vibrational transitions (reflectance only) can be used to identify artists’ materials (pigments or binders). In this way all of these spectral imaging modalities can be considered forms of imaging spectroscopy.

This chapter focuses on diffuse reflectance imaging spectroscopy which has been used to provide a variety of information including identifying many of the artists’ materials used and mapping their distribution across the surface of the work of art (e.g. paintings). The identification of artists’ materials (pigments and paint binders) by reflectance imaging spectroscopy relies on the presence of absorption features in the spectra from electronic and vibrational transitions. Thus, in principle the optimal spectral range extends from the near-ultraviolet through the mid-infrared, where visible is defined as 400–750 nm, near-infrared (NIR) 750–3000 nm, and mid-infrared (mid-IR) as 3–25 μm.

2 Definition of Diffuse Reflectance Imaging Spectroscopy

Diffuse reflectance spectroscopy is used to provide information about absorption features associated with electronic and vibrational transitions of molecules and thus is analogous to absorption or transmission spectroscopy. However, unlike transmission spectroscopy, it involves the collection of light that is backscattered from the object. As such, the spectral signature encodes not only absorption features from electronic and vibrational transitions but also scattering characteristics of the sample as well. While this makes the interpretation of the spectra more complex, it does allow for the study of absorption features of opaque materials.

The formal definition of diffuse reflectance is the ratio of irradiance of light reflected back to the detector to that of the irradiance onto the surface of the art object as a function of wavelength. Thus, the measured reflectance spectrum is the ratio of the spectral irradiances and will include contributions from the air/surface interface and vary with illumination and collection geometry. In practice for cultural heritage analysis, these specular contributions are not of interest and the diffuse reflectance spectrum is calculated by taking the ratio of irradiance of the illumination light off of the object of interest to that of a nearly Lambertian reflectance standard referred to as the white standard due to its ability to reflect approximately 99% of the illumination light. The implication of this practical simplification is that differences in the surface roughness between the white standard and the object can result in differences in the absolute reflectance values. Thus, reflectance spectra measured in this way are referred to as “Apparent Reflectance” to note this difference (Schaepman-Strub et al. 2006).

3 Use of Diffuse Reflectance Imaging Spectroscopy in Cultural Heritage Science

The central goal of reflectance imaging spectroscopy is the identification and mapping of artists’ materials (pigments and paint binders) in situ (Cucci et al. 2016). The primary identification of the material is done using the spectral features (electronic and vibrational transitions) in the reflectance spectra and the mapping is done by identifying the spatial pixels that have similar spectral features to the material of interest. As discussed in the reflectance image cube processing section, this is done several ways. The most common way is to classify areas that have similar reflectance spectra and identify the reflectance spectra of the representative classes (i.e. characteristic or endmember spectra) as a specific artist material by mathematically comparing the endmember spectra with those from a spectral library or visually identifying the key spectral features.

A wide variety of paintings have been studied using reflectance imaging spectroscopy to map and identify artists’ materials. These have included archaeological paintings from the ancient world (Delaney et al. 2017; Alfeld et al. 2018a, b), Byzantine paintings (Radpour 2019) and illuminated manuscripts from Italy (Ricciardi et al. 2013; Mounier and Daniel 2015; Cucci et al. 2018; Kleynhans et al. 2020a, b). Other Italian Renaissance paintings include those of Cosmé Tura (Dooley et al. 2014) and Bellini (Dooley et al. 2019) and Dutch painters such as Vermeer (Delaney et al. 2020). Late nineteenth and twentieth-century artists such as Van Gogh (Zhao et al. 2008; Dooley et al. 2020), Munch (Monico et al. 2020), Picasso (Delaney et al. 2010, 2016), and Pollock (Dooley et al. 2017) have been studied not only for pigments but in the latter case for paint binders as well.

Another use, especially when reflectance imaging spectroscopy is conducted in the NIR is to improve on the results obtained from traditional infrared reflectography (Delaney et al. 2016). Often NIR reflectance imaging spectroscopy provides more detailed information about preparatory sketches and changes in the painted composition than infrared reflectography. The optimal visualization of preparatory sketches or compositional paint changes relies on imaging in a spectral region where the contrast is greatest between the background and the spatial features of interest. This optimal spectral window differs depending on the optical properties of the pigments in the paint layers and the underdrawing material as well as the reflectance properties of the ground layer they are on. Having a large number of spectral bands across this optimal spectral window allows for application of image processing tools that may increase the separation of overlapping features painted with different materials. For example, the use of principal component analysis, although more complicated than constructing false-color images (selection of three spectral bands placed in the RGB display channels), can often help a user better separate the drawing from partially penetrated paint. Or in the case of changes in the painted composition or the re-use of a canvas, the false-color images of transform images (i.e. the output images after performing PCA on an image cube) can provide clear images of earlier painted compositions. A variety of examples exist showing these added benefits from Picasso’s famous re-used canvases of his Blue Period paintings to Italian Renaissance paintings such as The Feast of the Gods by Bellini whose landscape was subsequently altered twice, including by Titian (Bull 1993; Dooley et al. 2019).

Visible reflectance imaging spectroscopy has also been explored to improve on color accuracy of digital captures, as a tool to monitor color changes and produce simulations of paintings if an aged varnish was removed or if a faded pigment was restored. The large number of spectral bands allows a more accurate calculation of the color coordinates compared to a three-color channel camera since the functions of the standard observer can be used rather than the approximation provided by the three-color filters on the camera. The ability to monitor changes in color of an art object with time translates into looking for changes in the reflectance spectra but requires ensuring repeatable illumination conditions and image cube registration. The visible reflectance image cube allows for the possibility to estimate an average transmission cube for an aged varnish as well as scattering terms in order to simulate the visual effects of removing an aged varnish (Trumpy et al. 2015; Kirchner et al. 2018a). Such a data set allows the virtual replacement of areas of faded pigments with reflectance spectra obtained from Kubelka-Munk modeling of pigment concentrations determined by other means such as X-ray fluorescence (Kirchner et al. 2018b).

4 Spectral Range of Hyperspectral Reflectance Cameras Used in Cultural Heritage Science

Practically, the useful spectral range of reflectance spectra is set in part by the artists’ pigments most often encountered and the types of hyperspectral reflectance imaging cameras or scanning single pixel spectrometers available. The electronic transitions associated with white pigments such as zinc white (384 nm) and both forms of titanium white (anatase, 372 nm and rutile, 406 nm) set the lower wavelength limit to 350 nm or just into the near-UV (Bacci et al. 2007). The higher wavelength limit relates more to development of sensors used in remote sensing of the earth, where the thermal radiation of the earth dominates over the solar reflected radiation at wavelengths greater than about 3000 nm. Thus, in remote sensing of the earth (where the light source is the sun) hyperspectral cameras operating from 400 nm to 2500 or 3000 nm rely on reflected radiation and above 3000 nm they rely on thermal emitted radiation from the objects. In cultural heritage applications, most hyperspectral imaging experiments performed in the spectral region >3000 nm have relied on reflected radiation off of the art object from a thermal source (Rosi et al. 2013,;Legrand et al. 2014) and one study collected thermal emitted radiation from paintings (Gabrieli et al. 2019). In summary, the most used spectral ranges in cultural heritage hyperspectral imaging have been in the visible to ‘optical near-infrared’ (abbreviated VNIR) from 400 to 1000 nm, in the NIR region from ~1000 to 2500 nm, and in the mid-IR region at wavelengths longer than 3000 nm.

5 Instrumentation and Experimental Procedures for Reflectance Imaging Spectroscopic Studies (350–2500 nm)

A variety of spectral optical systems have been used to construct diffuse reflectance image cubes, such as single pixel and line scanners built around a dispersive spectrometer or Fourier transform imaging spectrometer that images areas. Camera systems that employ a large number of spectral filters or individual spectral lamps have also been used. The ultimate choice about which system is best for analysis of a collection of cultural heritage objects depends on several factors, including the range of artist materials expected, the spatial resolution desired, and object size. In cases where a small number of pigments are expected, a limited number of spectral bands may be all that is needed to spectrally separate and map them. The use of spectral filters or spectral lamps with an imaging array may suffice. For cases where a large number of pigments are expected to have similar spectral features, higher spectral resolution over a large spectral range is often required and an imaging spectrometer is needed. The simplest system is a single pixel 2-D scanner in which the collected light is directed to one or more spectrometers. Such systems often provide the widest spectral range, highest spectral resolution, and highest signal-to-noise, but have the lowest spatial sampling and scanning speed (spatial pixels per second). For example, single pixel scanners built around a commercial fiber optics spectroradiometer that spans the spectral range of 350–2500 nm have been found useful for laboratory measurements (Delaney et al. 2018a, b) and field work (Radpour 2019). The development of available line-scanning imaging spectrometers for the remote sensing community and more recently for industrial materials testing (pharmaceutical, manufacturing, food industry) has resulted in several commercial imaging spectrometers which subsequently have been adapted for the study of cultural heritage objects. Compared to single pixel scanners, they have scan rates that are orders of magnitude faster with similar spectral resolution (Fig. 4.1). Such line-scanning systems can provide spatial sampling as fine as ~200 μm2 and allow collecting image cubes of paintings as large as several square meters in less than one day. These hyperspectral cameras are typically built to operate in either the VNIR or the NIR spectral regions, although cameras that span the full range are becoming available.

Fig. 4.1
figure 1

Schematic operation of a line-scanning hyperspectral reflectance camera to acquire a 3-D reflectance image cube. A map of pigments and an image product showing the preparatory sketch can be obtained. (Cucci et al. 2016)

Full size image

The image collection geometry most commonly used for diffuse reflectance imaging spectroscopy has the hyperspectral reflectance camera oriented normal to the paintings surface with two illumination sources at 45 degrees to the normal.

Larger angles of the illumination sources are often used for paintings having a low average reflectance and glossy varnish in order to minimize glints from specular reflections from the lights. The illumination sources are often lamps with tungsten halogen bulbs whose vertical position coincides with the center of the slit of the spectrometer. Tungsten halogen bulbs powered with an external power supply can provide a stable light intensity. Alternatively, two fiber optic line-lights oriented parallel to the slit can be used to only illuminate a narrow vertical strip on the painting. Newer LED line lights designed for reflectance imaging spectroscopy are becoming available. In any illumination system used to study light-sensitive cultural heritage objects it is important to reduce the intensity of light (e.g. UV and thermal) outside the spectral region of interest and keep the illumination at acceptable levels as per conservation guidelines.

Data collection with single pixel or line scanners involves translating the scanner or the painting in order to collect the raw image cube, followed by collection of a ‘dark’ image cube (using a lens cap to prevent light from entering the hyperspectral camera) and finally the image cube of the white standard in order to calibrate the image cube of the artwork to apparent reflectance. This calibration set is typically incorporated into the collection software for commercial hyperspectral reflectance scanning cameras.

The key performance metrics of hyperspectral reflectance cameras includes both spatial and spectral parameters and it is important to include some of this information in publications. The spatial sampling is the projected size of a pixel at the painting and should be reported. Since the primary analysis of the reflectance image cube is performed in the spectral domain, the spectral sampling, the spectral resolution, and the signal-to-noise ratio are important metrics to measure. Spectral sampling is the number of nanometers a spectral pixel spans and should also be reported, but knowledge of the spectral resolution, which relates to the spectral response function (full width at half height in nanometers of an atomic emission line), is also useful. Note that the spectral sampling and resolution (at a given signal-to-noise ratio) that is deemed necessary to discriminate among materials depends upon the spectral position and width of their characteristic electronic and vibrational transitions. In materials that have similar spectral features that are shifted by only a few nanometers, sufficient spectral sampling and resolution is required to be able to confidently separate materials in the presence of noise.

Typically, specific electronic transitions in the visible and vibrational transitions in the NIR set the spectral sampling/resolution requirements for a hyperspectral camera. For example, the ability to distinguish among electronic transitions in the visible for cobalt pigments, iron earth pigments such as jarosite, and red lake pigments such as kermes and madder lake, are often important. This requires a spectral sampling of ~2–3 nm and a resolution of ~5–6 nm. To separate the NIR overtone due to hydroxyl stretching that occurs in azurite, gypsum and various clays, spectral sampling of about 3–6 nm and spectral resolution of 10–12 nm are generally required. Being able to measure the shift in NIR wavelength of the CH2 combination band associated with different lipidic binders (egg yolk, drying oil, beeswax) requires spectral sampling of 4 nm or less. The signal-to-noise can be given in percent reflectance and a root mean square noise of < 0.5% reflectance (measured from a 2% ‘black’ reflectance standard) is preferred.

Low signal-to-noise, i.e. high random noise in the reflectance spectrum, is the most common problem with hyperspectral reflectance image cubes of paintings. Care needs to be taken to set the exposure time and light level such that the intensity of the light reflected off the white standard (99% diffuse reflectance reference) totals 80% of the dynamic range of the detector (i.e. utilizes 80% of the well capacity of a pixel in the detector). Assuming the noise is random, the noise can be reduced by spatially averaging pixels in the image cube (but at a corresponding loss of the spatial sampling). Alternatively, noise can be reduced by spectral averaging although this decreases the spectral sampling. As a rule of thumb, the optimal spectral pixel aggregation should be set to match the width of the spectral slit. Most commercially produced VNIR hyperspectral cameras offer the user the ability to change the spectral sampling since they often contain focal plane arrays with pixel sizes that are smaller than the slit width of the spectrometer.

The intensity and duration of the illumination light used in collecting the reflectance image cube of cultural heritage objects, paintings, and works on paper needs to be addressed before any scanning is undertaken. Most commercial hyperspectral reflectance cameras were optimized for imaging outdoors where the natural light levels are 10 to 100x that of museum lighting conditions. Thus, care needs to be taken to ensure the light levels and their duration is adequate for the work of art being examined. A number of factors needs to be considered, including keeping the total accumulative exposure time as short as possible to minimize fading. This is often done by calculating the light exposure to be received using the experimental conditions in terms of the equivalent number of hours or days under normal gallery lighting conditions. Another factor of the illumination conditions to consider is ensuring the rise in temperature of the object, especially black paints, does not exceed acceptable conservation levels. The American Institute for Conservation provides practical guidelines for imaging works of art that can be used as a starting point.

6 Image Processing and Exploitation of Reflectance Image Cubes (350–2500 nm)

The analysis of reflectance image cubes is a multi-step process that depends just as much on the questions being asked as the skill level of the person analyzing the image cube. At one extreme, the analyst might only be interested in looking for a specific material whose unique spectral features they know. At the other extreme, the user wants to make labeled maps of all the artists’ materials present such as pigments. Currently this can be done in two different ways. One way involves a two-step process that initially classifies the reflectance image cubes into a set of reflectance spectra, called endmembers, which have characteristic spectral features that represent specific regions of the image cube. Next, the pigments and binders represented by each endmember spectrum are assigned by using the spectral features of each endmember, along with information obtained by other site-specific methods (i.e. X-ray fluorescence, fiber optics reflectance spectroscopy, Raman spectroscopy). The other way is a one-step process that creates labeled maps of artists’ materials directly without having to find spectral endmembers. This one-step process is of great utility to the cultural heritage community, and development of such algorithms is still an active area of research.

In the case where the analyst is looking for a few specific materials whose unique spectral features are known, analysis can be performed by using a reference spectrum pulled from a spectral library, such as the one maintained by the United States Geological Survey (USGS). Next, an algorithm is used to identify the spatial pixels in the image cube that best match the library reference spectrum. Many matching algorithms exist for this purpose, such as Spectral Angle Mapper (Dooley et al. 2014). Since these libraries are typically of pure materials that are optically thick, they do not often work well for matching to reflectance image cubes of paintings, where the pigments are in paint layers that are not optically thick beyond the visible region and are applied on a preparatory ground layer having its own spectral properties. Because of the differences encountered between spectra from the collected image cube and a reference library, often the analyst tries to find a representative spectrum from the image cube that can serve as the reference spectrum. Successful results can usually be achieved by restricting the spectral range and removing the spectral continuum in order to isolate the spectral feature of interest.

When the user wants to make labeled maps of all the artists’ materials present, either a two-step or a one-step process can be used. In the two-step process, the first step involves finding the spectral endmembers and separating the reflectance spectra in the image cube into classes. This can be achieved manually by visual inspection of the spectra in the image cube, but use of a statistical analysis algorithm is often a better approach. One of the more widely used algorithms to find endmembers in cultural heritage science is the ENVI spectral hourglass wizard (ENVI-SHW, Harris L3 Corp) which is rooted in convex geometry (Delaney et al. 2010). The ENVI-SHW algorithm uses principal component analysis (PCA) to reduce the number of spectral bands (dimensions) of the reflectance image cube, which can number in the 100’s, down to between 10 and 30 principal component (PC) dimensions. Next, the Pixel Purity Index (PPI) algorithm is used. PPI uses a limited set of random projections (10’s of thousands instead of millions) to find spectra that are the most spectrally diverse. The most diverse spectra are displayed as points in a cloud of n-dimensional space, where n is the number of PC dimensions, using the n-D visualizer tool. In principle, the spectral endmembers will be the spectra that reside at the vertices or corners of the point cloud. Although the ENVI-SHW can automatically identify endmembers, usually the best results are obtained when an experienced user refines the automatic selection or manually selects some endmembers that may have been previously missed. As a result, the method is only semi-automatic but through the interactive clustering process, the analyst can gain insight into the spectral data set.

In recent years, researchers have been testing existing remote sensing algorithms that automatically find spectral endmembers on reflectance image cubes of cultural heritage objects. Algorithms tested include ones based on k-means clustering (Rohani et al. 2016), and a combination of BH t-SNE (Barnes-Hutt t-Distributed Stochastic Neighbour Embedding) and DBSCAN (Density-based spatial clustering of applications with noise) (Grabowski et al. 2018). Also, algorithms based on a convex hull, such as MaxD (Maximum Distance), are able to find the majority of spectral endmembers previously identified with ENVI-SHW (Kleynhans et al. 2020a). Once the endmembers are found, classification maps, defined by the endmember spectra, can be made. Commonly used classification algorithms include Spectral Angle Mapper (SAM) (Delaney et al. 2010), Maximum Likelihood (ML) (Balas et al. 2018) and Spectral Correlation Mapper (SCM) (Deborah et al. 2014). These algorithms compare each spectrum in the image cube with the endmembers to find matches. The classification map for a given endmember shows the spatial pixels whose spectra match those of the endmember. The second step of the two-step process involves the identification of the artists’ materials represented by the endmember spectra. The artists’ materials are determined by identifying the reflectance spectral features of each endmember, along with information obtained by other site-specific methods (i.e. X-ray fluorescence, fiber optics reflectance spectroscopy, Raman spectroscopy).

The second analysis approach creates the labeled material maps directly without having to find spectral endmembers in the image cube. This is the most challenging approach, but potentially of the most use to the cultural heritage community. The reason why it is so challenging is because, unlike most research questions encountered in remote sensing, the materials encountered in cultural heritage (like paint layers) are not optically thick over the full spectral range. The paint also consists of pigments that are intimately mixed, so their reflectance spectrum is not the weighted linear sum of their component spectra. One method that has been used to try and automatically identify materials includes attempts to linearize the intimate mixing problem by using an approximation to the radiative transfer equation for simple mixtures of optically thick paints (Zhao et al. 2008). The unknown spectrum from the image cube is linearly fit using a set of weighted absorption and scattering coefficients from a database, which limits its effectiveness because it requires a priori knowledge about what pigments and pigment mixtures will be encountered. It has been found to work well for mock paintings, but it has not been found yet to work in widespread applications and more than one mixture might produce acceptable results. An interesting approach to address these limitations has been the use of a neural network to preselect the pigments represented in the database (Rohani et al. 2018).

Another one-step approach for creating labeled material maps has restricted the number of pigments expected to be encountered to pigments found in particular schools of painting where the paint methods are similar in terms of the pigments and pigment mixtures used. A 1-D convolutional neural network was trained using four illuminations from the Laudario of Sant’Agnese c. 1340 attributed to two painters (Kleynhans et al. 2020a, b). The network was trained on large regions of these paintings whose pigment composition had been previously identified and mapped. These regions included areas with a range of pigment mixtures and areas with the same pigment having different optical thicknesses. The neural network was found successful in directly classifying and identifying the pigments in another painting from the Laudario, as well as another painting from the same time period. These and other studies suggest a direct material mapping from reflectance image cubes is possible, and could be especially useful when information from other spectral imaging modalities is incorporated as well.

7 The Mid-IR Instrumentation, Experimental Procedures, and Image Processing (4000 cm−1 to 650 cm−1, 2.5 to 15.4 μm)

In remote sensing, mid-IR hyperspectral cameras and analysis tools are well developed, but they are still an active area of research in the application to studying works of art. This spectral region is rich with absorption features from numerous functional groups which is what makes it attractive. However, the difficulty in obtaining affordable mid-IR hyperspectral cameras that can operate over the mid-IR spectral range, defined here as 4000–650 cm−1, has limited most of the research to single pixel, FTIR spectrometer scanning systems, operating in reflectance mode, as first demonstrated by Legrand et al. 2014. Given their short air path length (<1 cm), these systems can collect spectra over the full mid-IR spectral range. However, they suffer from low spatial resolution/sampling (1–2 mm) and slow scan speeds (about 1 pixel per second versus 1000’s of pixels per second for the VNIR and NIR hyperspectral reflectance cameras) even compared to XRF scanners. Still, results from these mid-IR scanners have shown the promise of mid-IR imaging spectroscopy by providing low resolution maps of many of the pigments (Legrand et al. 2014) as well as paint binders and paint fillers (Gabrieli et al. 2019) in unvarnished paintings such as illuminated manuscripts.

Efforts to increase the spatial pixel collection rates have focused on adapting mid-IR hyperspectral cameras that were optimized for remote sensing of the earth. Unlike the single pixel scanners discussed above, these cameras were designed to operate in one of two atmospheric spectral windows in the mid-IR, namely 3–5.5 μm (3333–1818 cm−1) or 7.7 to ~14 μm (1298 to ~714 cm−1). Using an imaging FTIR camera (1440–900 cm−1), and operating in reflectance mode, Rosi et al. captured details from a Burri painting, Sestante 10, and identified and mapped organic (acrylic and alkyd binders) and inorganic (silicates and sulfates) compounds (Rosi et al. 2013). Gabrieli et al. used a line scanning imaging spectrometer (1240–760 cm−1) in emissive mode and demonstrated the ability to map paint binders (oil, acrylic and alkyds), pigments, and fillers in a mock painting as well as in Edward Steichen’s Study for “Le Tournesol” (Gabrieli et al. 2018).

With each mid-IR collection mode, reflectance or emissive, the origin of the spectral radiance collected to make the spectral image is different. In emissive mode, the spectral radiance collected is dominated by the thermal emission from the painting, whereas in reflectance mode, the collected radiance is dominated by the radiance that is reflected off the painting from an external thermal source. Reflectance mode is common in laboratory measurements and requires some care to minimize the heating of the painting during the data cube collection. The conversion of the measured spectral data cube to apparent reflectance is done using a gold reflectance standard. Emissive mode in a laboratory environment emulates how mid-IR spectral imaging is done in remote sensing, namely the spectral radiance emitted by the object dominates over the downwelling spectral radiance from the sky that is reflected off of the object since the sky temperature is at a lower temperature than the object on the earth. To conduct emissive mode mid-IR imaging spectroscopy in a laboratory, a ‘box’ is placed between the painting and the camera to reduce the spectral radiance from the room which is at the same temperature as the painting under study. The ‘box’ is configured to be a source of radiance from a blackbody at a lower temperature than the room. Note that the measured spectra from each spatial pixel in the image cube is given by the product of the emissivity of the paint and the blackbody function, both of which vary with wavelength. Thus, in order to recover the emissivity spectrum at each spatial pixel, the collected image cube of the painting is first converted to units of spectral radiance, and then each spectrum is fit with a blackbody function. One of the advantages of emissive mode mid-IR imaging over reflectance mode mid-IR imaging is the signal strength, i.e. the spectral radiance from the signal of interest. This is because the emissivity of the paints is close to one and by Kirchhoff’s law the reflectance is one minus the emissivity. Nevertheless, each mid-IR imaging mode has its strengths and weaknesses.

The data processing of the mid-IR reflectance or emission image cubes can be done using the same procedure used for the visible and NIR image cubes including the use of automatic and semi-automatic algorithms to find spectral endmembers. However, since most of the users are familiar with the spectra of many functional groups in the mid-IR, maps are most often made by limiting the spectral range used to that of the spectral feature of interest. Then an endmember spectrum is selected from the cube and a mapping algorithm such as spectral angle mapper is used to make the map. The identification of spectral features from functional groups for both the reflectance and the emissive spectra is often not as straightforward as absorptions seen in transmission FTIR spectra due to contributions from specular reflection by the pigment particles that leads to spectral distortion in the bands, a phenomenon that is well known (Miliani et al. 2012).

8 The Future of Reflectance Imaging Spectroscopy

The increased interest in macroscale spectral imaging modalities by conservation scientists, conservators, curators, and art historians can be expected to further invigorate the field. This is precisely because the image products are accessible to this wide audience. The desire for more spectral image cubes of paintings is not just for the new results they are giving but also because they help to confirm, and expand on, earlier findings obtained from detailed chemical analysis of microsamples. The rapid adoption of reflectance imaging spectroscopy in the pharmaceutical, manufacturing, and food industry has resulted in the lowering of the cost of VNIR hyperspectral cameras for the cultural heritage science community. The increased availability and improved performance of NIR and mid-IR cameras is also decreasing the barrier to purchasing hyperspectral cameras in these spectral ranges as well. Finally, collection of reflectance image cubes from the near-UV through the mid-IR promises to offer a more robust ability to classify and identify artists’ materials. New algorithms, including some used in machine learning, offer more automated processing of the reflectance image cubes. Finally, the fusion of reflectance imaging spectroscopy with the results from molecular and elemental fluorescence can be expected to provide a more complete understanding of artists’ working methods and artists’ materials.

9 Case Studies

A few case studies are provided that highlight the variety of results which can be obtained using reflectance imaging spectroscopy.

9.1 Identification and Mapping of Artists’ Materials: Pigments

A central goal of reflectance imaging spectroscopy is the identification of classes of reflectance spectra that can be used to classify regions of the reflectance image cube into regions that have similar reflectance spectra. In this case study, the VNIR reflectance image cube of an illuminated manuscript cutting, Christ and the Virgin Enthroned with Forty Saints, from the Laudario of Sant’Agnese, attributed to Master of the Dominican Effigies c. 1340, was collected and analyzed. The reflectance image cube was analyzed using the ENVI Spectral Hourglass Wizard (ENVI, Harris L3 Corp). In the analysis, 16 spectral endmembers were obtained by manual clustering (Kleynhans et al. 2020a, b). These endmembers defined the classes used to construct the map. A combination of spectral features from the reflectance endmembers and results from site-specific X-ray fluorescence and fiber optics reflectance spectroscopy (350–2500 nm) were used to assign the pigments to each class (Fig. 4.2).

Fig. 4.2
figure 2

(Left) Christ and the Virgin Enthroned with Forty Saints, Master of the Dominican Effigies c. 1340, National Gallery of Art, Washington. (Middle) Class map of the reflectance spectral endmembers (right) with their assigned pigments. (Kleynhans et al. 2020a)

Full size image

9.2 Identification and Mapping of Artists’ Materials: Paint Binders

Reflectance imaging spectroscopy in the NIR spectral region 2000–2400 nm offers the ability to separate and classify drying oils, egg yolk, wax and paint binders containing carbohydrates and proteins, depending on the substrate. However, mid-IR imaging spectroscopy (reflectance or emissive mode) offers more functional groups to better map and identify these binders, but currently at the expense of spatial resolution and time. In this example, the identification and mapping of paint binders is demonstrated in Lorenzo Monaco’s Praying Prophet, a cutting from a fifteenth-century illuminated choir book commissioned by the Camaldolese monks of Santa Maria degli Angeli in Florence. NIR and mid-IR reflectance imaging spectroscopy were used to map spectral features associated with the binding media, including the NIR CH2 lipidic feature at 2309 nm which has been associated with egg yolk tempera binder (Gabrieli et al. 2019). This assignment was confirmed by the map of the mid-IR carbonyl group and lipidic components of egg yolk tempera. The marginalia were found to possess mid-IR spectral absorptions associated with polysaccharides and not egg tempera (Fig. 4.3).

Fig. 4.3
figure 3

(a) Color image of Lorenzo Monaco’s Praying Prophet, 1410/1413, Rosenwald Collection, National Gallery of Art, Washington. (b) Map of the C─H lipidic spectral feature associated with egg yolk (c). (d) Map of carbonyl group (C═O) spectral features (e) associated with the proteic (amide I) and lipidic component (5797 nm or 1725 cm−1) in egg yolk. (f) Map of ultramarine and polysaccharide spectral features (g) associated with the blue portions of the initial E and blue areas of the marginalia. (Gabrieli et al. 2019)

Full size image

9.3 The Earlier Composition of Fragonard’s Young Girl Reading

A previously collected X-ray radiograph of Jean-Honoré Fragonard’s painting Young Girl Reading, c. 1769, showed another figure below the girl, although the features of this person were not clear. The painting also has a pentimento in the background above the girl’s head that suggested the presence of a feather associated with the prior figure’s head. This earlier figure has a face that is oriented towards the viewer, which was more in keeping with a series of portraits that Fragonard painted known as the Fantasy Figures. The underlying portrait was tenuously associated with the Fantasy Figures based on the similarity of the composition, but its association was confirmed when a Fragonard drawing of quickly executed thumbnail sketches of many paintings in this series was discovered in 2012 (Jackall et al. 2015). The first sketch on this drawing depicts a woman holding a book and looking towards the viewer. The combination of NIR (1000–2500 nm) reflectance imaging spectroscopy and XRF fluorescence imaging spectroscopy confirmed the prior figure was a woman and revealed new details, including the presence of a black ribbon tied around her neck and a feather in her hair with red beads. The appearance of the underlying woman not only matches the quick sketch in the Fragonard drawing, but also fits in with the style of the other Fantasy Figures (Jackall et al. 2015). The combination of the image products from the two imaging modalities allowed for a simulation to be made of the prior composition (Fig. 4.4).

Fig. 4.4
figure 4

(Top left) Color detail of Jean-Honoré Fragonard’s Young Girl Reading, c.1769, National Gallery of Art, Washington. Top right: False-color diffuse reflectance near-infrared image (1000, 1300, 2100 nm) (J. Delaney and K. Dooley). Bottom right: XRF image obtained from the mercury L-alpha line, likely associated with vermilion. Bottom left: Image simulation of figure underlying Young Girl Reading extrapolated from technical images (B. Goodman and D. Doorly). (Jackall et al. 2015)

Full size image