1 Introduction: light, shadow and surface characteristics

The interplay of changing light directions and shades plays a central role for an examiner to visually explore and understand the characteristics of a surface. A researcher or conservator–restorer is trained in the first place to apply this with the naked eye using white (VIS) light. Since several decades ago, multispectral (MS) imaging (ultraviolet (UV), infrared (IR) and near-IR) has proven its value for cultural heritage studies, especially since one of its main advantages lies in the non-invasive/non-destructive character of this approach [1, 2]. In technical art research, conservation science and archaeology, applying these different spectra enables a precise examination and identification of colours and pigments, the visualisation of underlying features, and the identification of varying materials or surface conditions. A shortcoming of these approaches, however, is the absence of geometrical information on the surface in these datasets and the user-friendliness of their acquisition and viewer systems.

The multispectral Portable Light Dome system (MS PLD) combines the technology of the (white light) PLD system [3–5] with a multispectral component (Fig. 1a). The MS PLD consists of a hemispherical structure, with an overhead camera with lens and LEDs emitting in five different parts of the electromagnetic spectrum, namely UV, three visible and IR light, distributed regularly in the dome’s inside surface [6]. The overhead machine vision camera has a monochrome 35-mm CCD sensor with a resolution of 6576 × 4384 pixels and a spectral response between 350 and 1000 nm. Thanks to the electronic shutter, vibration-free captures can be made, necessary for accurate registration and mapping of the images. The lens has five calcium fluorite elements, allowing true apochromatic performance between 315 and 1100 nm, the part of the electromagnetic spectrum required for the intended studies. For a similar set-up without the multispectral component, reference can be made to the use of the so-called RTI domes for several documentation projects [7, 8]. For the presented system, the main technical specifications can be summarised as follows: (a) camera: Allied Vision Prosilica GX 6600; (b) lens: CoastalOpt UV–VIS–IR 60 mm Apo Macro lens; and (c) light sources: LED EnginLZ1 product family (UV: 365 nm, blue (B): 465 nm, green (G): 523 nm, red (R): 623 nm, IR: 850 nm).

Fig. 1
figure 1

a MS PLD recording a medieval manuscript (© RICH and ESAT–VISICS, KU Leuven) and b MS PLD image [shaded filter (sides) vs. R/G/B false colour filter (central part)] of Fayum mummy portrait E.4857 (© RMAH, Brussels and ESAT–VISICS, KU Leuven)

Full size image

The dome extracts genuine 3D and shading information based on the principles of photometric stereo. The newly by VISICS (ESAT/PSI) introduced custom-made interactive multispectral module offers the ability to search for the most optimal visualisation conditions while shifting between calculations based on the recordings with the different spectra. All visualisation modes computed in this PLD viewer (v7.0.03b) are based on one single recording procedure providing the dataset of differently lit images, taking only a few minutes and executed as well by a custom-made PLD acquisition tool (v7.0.03b). They enable the detection of the smallest surface characteristics, the generation of 2D+ models, and the lifelike or visually enhanced interaction with the recorded artefact. This leads to a user-friendly and cost-effective methodology, applicable to a wide variety of heritage objects.

The MS PLD has been used by the RICH and EES projects for the research on inter alia medieval manuscript illuminations, bookbindings, (ancient) coins, cuneiform tablets, mummy portraits, painted ceramics and (fragile) ancient Egyptian objects [4, 6, 9]. The MS images not only provide the opportunity to study the underdrawings, faded pigments and inscriptions—often invisible to the naked eye—they also allow the researcher to study the characteristics of the object, such as the relief created by the layers of paint (Fig. 1b) and the material underneath. For every pixel, the system can register and map both the reflective characteristics in the five MS spectra and the surface geometry.

All these abilities have been clearly presented in the recordings of the so-called Fayum mummy portraits of the RMAH collection, i.e. portraits painted on wooden boards that were attached to Egyptian mummies of the Roman period (late first century BCE to mid-third century CE) [9]. By adjusting the position of the lights, for example, one can reconstruct the brush strokes and study the cracks in the wooden boards. This is especially demonstrated in the ‘shaded’ representation (Fig. 1b), where such images clearly demonstrate the potential of the system for technical art research.

2 Pigment identification and reflection maps

2.1 MS PLD imaging

The main concept behind the MS PLD system is understanding how the different light spectra have reflected the scanned surface from a number of different angles. Based on that, the system creates interactive 2D images that visualise the surface as it appears under different light spectra and on which its geometric characteristics can be examined; all to be obtained in a recording session of only a few minutes, executable by non-technical trained researchers due to a streamlined pipeline of actions in the PLD acquisition tool (v7.0.03b). In addition to merely documenting objects for conservation and study purposes, new areas of research can, moreover, be explored by mining these same datasets. The RICH and EES projects are currently testing the system’s potential for the study of pigments and materials. One of the first possibilities encountered during these tests was the detection of modern pigments on ancient artefacts. Although they might appear similar in white light, it is common knowledge modern pigments used during the restoration of ancient objects will reflect light differently as the composition of their materials will differ from the original (ancient) materials. In the MS viewer interface of the PLD, this can be visualised instantly by combining the five electromagnetic spectra of the MS PLD into false colour representations.

The heavily restored ancient Egyptian jar E.2458 (Fig. 2a) at the RMAH proved to be an excellent case study to demonstrate this application. The modern pigments clearly react differently in the false colour images. In the presented case, the restored pieces are also recognisable with the naked eye and it therefore seems that the MS PLD images mainly confirm the expected result. However, they also show that the modern restoration paint overlaps the adjacent, original pigment (Fig. 2b–d), providing important information for conservators. Similar results have been obtained on decorated objects where the restored parts cannot be discerned from the original with the naked eye.Footnote 1 The MS PLD can thus be regarded as a valuable aid in conservation and restoration sciences, especially since the interactive consultation of these datasets, switching from one view mode to the other (e.g. Fig. 2e, f), can be performed in real time.

Fig. 2
figure 2

Ancient Egyptian decorated jar E.2458 (c. 1200 BCE): a conventional digital photograph in white light; b MS PLD false colour: IR/R/G; c MS PLD false colour: IR/G/B; d MS PLD: IR; and e MS PLD—R/G/B; f MS PLD—shaded: surface characteristics (© RMAH Brussels)

Full size image

2.2 Reflection maps and histograms

In addition to false colour representations, the local reflectance on a surface can also be shown as a reflection map. For every pixel in the interactive PLD images, the reflectance of that local surface structure is observed for each of the individual LED lights. The reflection map representing the layout of the interior of the dome shows the intensity of that reflectance. This observation can be computed for both the white light PLD and the MS PLD results. Obviously, exactly these observations are used by the software to determine the local surface normal and the corresponding reflectance characteristics. Thus, the reflection map can be used in its own right to investigate the nature of the local material properties.

The maps are observations or observed intensities (I) for a given pixel related to the angle between the incoming light (L) and the normal (N) at that position (Fig. 3). In its most fundamental form, this relationship is given by the bidirectional reflectance distribution function (BRDF). In order to calculate the surface normal and material characteristics, most materials can be described by a lower-dimensional parametric formulation. Photometric stereo has been introduced initially by assuming a Lambertian lighting model: I = ka × L × N, where a is the albedo. In this context, the principle has been extended using a Phong-like model that also allows including specular components. I = ka(L × N) + ks(R × V), where ks a specular reflection constant, R the direction of the perfectly reflected ray and V the viewpoint of the observer.

Fig. 3
figure 3

Model of observed reflections (© ESAT–VISICS, KU Leuven)

Full size image

In Fig. 4, this approach is illustrated with a white light PLD recording explored further on in the paper. The reflectance of the metallic nature of the gold medallion is clearly shown by the specular peak it creates towards only those LEDs perpendicular to the inspected pixel (bright response), whereas the diffuse nature of the blue field (a more uniformly spread and dimed brightness) demonstrates a close to perfect Lambertian response.

Fig. 4
figure 4

White light PLD reflection maps and histograms. Left golden medallion CBIB W827 (top) and detail of frontispiece Anjou Bible (bottom); middle reflection maps of one particular pixel; right histograms of a selected area (© RICH and ESAT–VISICS, KU Leuven)

Full size image

For the multispectral dome, the IR, R, G, B and UV channels are all displayed in one and the same map (Figs. 6, 7, 8, 10). In order to support the interpretation of the response, the local spectral content can also be presented within the PLD viewer interface as a histogram showing the distribution of calculated albedo responses for each of the spectral bands. In these histograms, the more the different bands are being plotted to the right, the higher their reflective response was for the selected area on the interactive PLD image (Figs. 6, 7, 9, 11). The albedo response is given by the x-axis, while the y-axis gives the probability distribution for each of the spectral components for the given area. As shown in Fig. 4 as well, for the white light dome this represents the distribution of R, G, B values of the extracted albedos in the given rectangular area. Typically, observations for a given material show a Gaussian distribution. The average response can be found where the distribution peaks.

2.3 Case study 1: blue pigments in the Anjou Bible

During the study of the miniatures of the fourteenth century Anjou Bible in 2010 and 2016, highly detailed photographs with transmitted light in combination with non-destructive analysis and MS photometric stereo provided insights into the materials and techniques used by the medieval illuminator [10, 11].Footnote 2

In the frontispiece of the Anjou Bible, king Robert of Anjou is depicted in front of a blue cloth with the Angevin coat of arms. The fleurs-de-lis are painted on top of a blue field, identified by µ-X-ray fluorescence (µ-XRF) measurements as being painted with azurite (rendering a large Cu peak in the XRF spectrum) [12]. Azurite, a basic copper carbonate [Cu3(CO3)2(OH)2], is produced by grinding and washing the mineral that occurs naturally in copper deposits in Italy, Spain and particularly Germany (‘azurium de Alemania’). During further examinations of the blue field, based on a MS PLD recording, a comparison of differently rendered false colour images revealed that the background must be composed of more than just one pigment; in the IR/G/B image, a dark blue and a purple colour can be distinguished from each other (compare Fig. 5b, c). In white light, however, this blue mixture used to paint the background has a very similar appearance (Fig. 5a). As the XRF analyses could not identify with certainty the presence of another pigment, the most likely candidate for the (in white light blue appearing) material that becomes visible in the false colour image is ultramarine (Na8–10Al6Si6O24S2–4), made from lapis lazuli and not detectable by XRF. A definite identification of this pigment should always be confirmed by applying another techniques [13–15].

Fig. 5
figure 5

Detail frontispiece Anjou Bible: a conventional digital photograph in white light conditions; b R/G/B image based on MS PLD dataset; and c false colour image, IR/G/B, based on MS PLD dataset; pigment in measure point Y has been identified as azurite by XRF (© RICH and ESAT–VISICS, KU Leuven)

Full size image

The observations in the false colour rendering are also easily corroborated by the reflection maps and the local spectral statistics on the calculated albedo responses. Figure 6 shows the reflection maps and histogram on an ultramarine and on an azurite painted area. When comparing both maps, it is easily observed that there is a substantial difference in the IR responses. When taking a closer look at the local spectral distributions, one can observe that the IR response for ultramarine is shifted towards the right (indicating a higher response), and in addition distinguish a smaller shift in UV and ratio differences in R/G/B separately. Such observations made with this MS PLD system are supported by many similar spectral distribution analyses [16, 17].

Fig. 6
figure 6

Reflection maps and histogram (screenshots from the MS PLD viewer v.7.0.02, R/G/B image in the back, both azurite and ultramarine appear as blue). Above pixel and zone X (see Fig. 5), ultramarine; below pixel and zone Y (see Fig. 5), azurite (© ESAT–VISICS, KU Leuven)

Full size image

In order to confirm the analysis, a MS PLD recording and analysis is carried out on the standard Pigments Checker v.2.1 by Cultural Heritage Science Open Source (CHSOS, https://chsopensource.org), containing a selection of 54 historical pigments. Again, the MS PLD reflection maps show a very similar behaviour (Fig. 7): besides both depicting an overall presence of blue, they show for the azurite ‘swatch’ on this Pigments Checker a higher reflectance in the green band (523 nm) and for the ultramarine ‘swatch’ a higher reflectance with the IR (850 nm) emitters. Interestingly, the ‘pure’ pigment swatches of the Pigment Checker reveal azurite has a very moderate response in IR and R with the MS PLD hardware, whereas the Bible of Anjou sample did give higher responses in IR (Fig. 6). This most probably indicates a certain degree of mixture of both the azurite and ultramarine pigments in the studied blue field. In conclusion, the reflectance information from the Anjou Bible and that of the Pigments Checker confirm the hypothesis that the blue field consists of two different materials and that their identification as azurite and ultramarine is most probable. In this case study, the MS PLD functioned as an aid in the identification of pigments that cannot be identified solely based on XRF datasets.

Fig. 7
figure 7

MS PLD reflection maps of the Pigments Checker (CHSOS); above ultramarine; below azurite (© RICH, KU Leuven)

Full size image

2.4 Case study 2: gold and metallic surfaces or paints

As a second case study, experiments have been set up to investigate the potentials of the PLD system to distinguish metallic surfaces or paints. Focus was laid in particular on the use of gold in medieval illuminations such as the Anjou Bible.

Even though it is obviously not possible to determine the carat rating or the percentage of other metals present in the alloy based on PLD images, the reflection maps of the white light and MS PLD nevertheless can assist in differentiating between gold and gold-like metals, which can be hard to distinguish. In order to gain insight in the matter, an experiment was set up using a set of 2 antique golden coins (CBIB O.I.613 and O.I.1), one inscribed golden medallion (CBIB W827), a modern 2 euro coin (Belgian) and a modern Egyptian 1 lb coin. The latter two have a ‘golden look’ but are made out of different alloys: for the 2 euro coin, the inner segment consists of a gold-coloured amalgam of 75 % Cu, 20 % Zn and 5 % Ni (three layers: nickel brass, nickel and nickel brass), and the Egyptian pound inner segment consists of an amalgam of 94 % steel, 2 % Ni and 4 % Cu plating. All surfaces were recorded in one and the same acquisition session. The analysis is carried out on the extracted albedo values of the samples, to make sure the specular reflections do not influence the results. The false colour results and two reference reflection maps are shown in Fig. 8, histograms on four of the results in Fig. 9.

Fig. 8
figure 8

False colour representations and 2 examples of spectral reflectance responses on the collection of coins (MS PLD, © ESAT–VISICS, KU Leuven)

Full size image
Fig. 9
figure 9

Reflectance histogram on the collection of coins (MS PLD, © ESAT–VISICS, KU Leuven)

Full size image

For the gold surfaces, similar reflection behaviour can be observed in the false colours and the albedo of the euro coin is immediately isolated from all other results. The histograms give an even better insight. The genuine gold pieces show an almost perfect aligned UV and B spectral distribution; the R and IR curves show a very similar behaviour. The IR response reflectance for the Egyptian pound sample is even plotted completely to the right. Further, the relative ratios between the different bands render the real gold pieces more ‘greenish’ than the (non-gold) modern coins. So again, the combination between the false colours, reflection maps and histograms gives sufficient indications to differentiate materials. When combined with known reference collections, even particular materials can be identified.

Given these results, we used these observations to determine variations in gold-coloured illuminating materials, in particular the difference in gold leaf and mosaic gold. In medieval illuminations, pure gold leaf and gold powder as a paint medium were already used in early Romanesque illumination. Another gold-coloured layer in manuscripts is based on alloying a base metal (or metals) with a noble metal as described in the fourteenth century De Arte Illuminandi [18, 19]. According to this treatise, tin and/or lead were frequently alloyed with silver, whereas copper and silver were alloyed with gold. Tin sulphide (SnS2) is described as an imitation gold. This so-called mosaic gold or porporino [20] was able to replace costly ‘true’ gold, but it did not have the full brilliance of real gold.

The monitoring tools of the PLD system can, based on the various responses in the different spectral bands, distinguish real gold foil from mosaic gold. This is demonstrated with the bas-the-page miniature of the nativity on folio 230 verso, as shown in Fig. 10. XRF measurements of these layers in the Anjou Bible indicate that the two lions and the two colossus are painted in mosaic gold, whereas other elements, such as the background of the scene and the floral elements, are decorated with pure gold foil [21]. In IR/R/G false colour mode, the real gold foil shows up as a darker greenish material (as was the case for the coins), an effect that is much lower in the areas with the mosaic gold. The specular peak is overall also higher for the foil than for the mosaic gold. As a comparison, the yellow bar in the shields below is not gold or metallic at all; thus, it shows a basically Lambertian behaviour (it is painted in orpiment, As2S3). The spectral distribution on the lion and a piece of gold foil are presented, respectively, in Fig. 11. Here also, one can observe the difference in alignment in the UV and B channels, and the ratios between the other spectral bands that show similar patterns as were observed with the coin samples.

Fig. 10
figure 10

Bas-the-page miniature of the nativity on folio 230 verso of the Anjou Bible. Left IR/R/G view, right R/G/B view, and reflectance maps. (MS PLD, © RICH and ESAT–VISICS, KU Leuven)

Full size image
Fig. 11
figure 11

Spectral statistics on the lion painted in mosaic gold (left image) and the pure gold foil background (right image). (MS PLD, © RICH and ESAT–VISICS, KU Leuven)

Full size image

3 Conclusion

The preliminary results of the case studies discussed above demonstrate that the new approaches in the (MS) PLD—a system originally developed for imaging purposes—enable expanding the application of this system into the realm of material studies. The potential of the imaging system for technical art research is obvious, and the recent addition of the reflection maps and histograms is believed to become an important tool for conservation science. These features allow a preliminary non-destructive identification of pigments and materials, thus supporting further (non-)invasive research and technical art examinations. At this moment, the number of samples used in the experiments is too moderate to allow definite conclusions. The case studies provide mainly visual guidelines that are able to distinguish different materials, but are for now based on a limited framework of reference material. We are confident that given a larger amount of samples, a more automated methodology can be developed, using state-of-the-art classification mechanisms based on, for example, SVM and CNNs. The use, testing and the further development of the (MS) PLD system is imbedded in a number of ongoing research projects. They will build on the results presented above and follow the necessary research strategies to further pioneer this new examination tool.