Keywords

1 Discovery of Infrared (IR) Region

The discovery of an invisible component beyond the red end of the solar spectrum (modern meaning-infrared (IR) region in a broad sense) is ascribed to William Herschel, a German-born British astronomer, who is famous for Herschel telescope. In 1800 one day, he investigated the effect of sunlight divided from violet to red by a prism on temperature increase. He used just sunlight, a prism, and thermometers. Figure 1.1 shows his portrait and the experimental set up he employed. He happened to find that the significant temperature increase occurred even outside of red. He thought there was a different kind of invisible radiation from visible light beyond the red end of sunlight and named this radiation “heat ray.” This was really a great discovery in science, but even he could not imagine that this is light. He was 62 years old when he discovered “heat ray.” Sixty-two years old in 1800 probably corresponds to today’s 80 years old or so. Thus, his discovery demonstrated that even very senior scientist could have intensive serendipity.

Fig. 1.1
figure 1

The portrait of William Herschel and his experimental set up in 1800

Full size image

Interestingly enough, just one year after the discovery of “heat ray,” Johann Ritter, a German scientist found another invisible component beyond the violet end of the solar spectrum based on an experiment of blackening of silver chloride. In this way, a new era of light was opened at the turning point from the eighteenth century to the nineteenth century. After the discovery of “heat ray,” many scientists investigated it. In 1835, it was confirmed that “heat ray” is invisible light which has longer wavelength than visible light. He named this light “infrared (IR).” Maxwell elucidated theoretically in 1864 that ultraviolet, visible, and IR light are all electromagnetic wave. In 1888, Hz proved it experimentally.

2 Introduction to NIR Spectroscopy

IR region is so wide energetically, ranging from 150 to 0.12 kJ mol−1 (12,500–10 cm−1). If one compares the energy of the highest wavenumber end of NIR region with that of the lowest wavenumber end of far-IR (FIR) region, one can find that the difference in the energy between the two ends is more than 1000 times. Therefore, nowadays, the IR region is divided into three regions, NIR region (12,500–4000 cm−1; 800–2500 nm), the IR region (mid-IR; 4000–400 cm−1; 2500–25,000 nm), and the far-IR region (FIR; 400–10 cm−1; 25 μm-1 mm) [1,2,3,4,5,6,7]. NIR spectroscopy is spectroscopy in the region of 12,500–4000 cm−1, where bands arising from electronic transitions as well as those due to overtones and combinations of normal vibrational modes are expected to appear [1,2,3,4,5,6,7]. Therefore, NIR spectroscopy is electronic spectroscopy as well as vibrational spectroscopy. Ultraviolet (UV)-visible (Vis) spectroscopy is electronic spectroscopy while infrared (IR) spectroscopy is vibrational spectroscopy, so that NIR spectroscopy is something special. It lies in between electronic spectroscopy region and vibrational spectroscopy region.

Figure 1.2 shows chemical structure of immobilized metal affinity chromatography (IMAC) material and NIR spectra in the region of 10,000–4000 cm−1 of 32 kinds of IMAC materials [8]. Broad features in the 10,000–7500 cm−1 region are due to the d-d transition of Ni coordination compound and bands in the 7500–4000 cm−1 region arise from overtones and combinations. The spectra in Fig. 1.2 are very interesting examples, demonstrating that in the NIR region, one can observe both bands assigned to electronic transition and those originating from vibrational transitions. Most of the electronic transitions appearing in the NIR region are the d-d transitions, charge-transfer (CT) transitions, and π-π* transitions of large, or long, conjugated systems [1, 3, 7]. NIR spectroscopy involves absorption, emission, scattering, reflection, and diffuse-reflection of light [1,2,3,4,5,6,7].

Fig. 1.2
figure 2

Reproduced from Ref. [8] with permission

Chemical structure of immobilized metal affinity chromatography (IMAC) material and NIR spectra in the region of 10,000–4000 cm−1 of 32 kinds of IMAC materials.

Full size image

NIR spectroscopy together with Raman, IR, and Terahertz/FIR spectroscopy forms “four sisters of vibrational spectroscopy.” Since NIR spectroscopy is concerned only with overtones and combination modes as a vibrational spectroscopy, it is very much unique compared with Raman, IR, and Terahertz/FIR spectroscopy.

The NIR region can be divided into three regions; Region I (800–1200 nm; 12,500–8500 cm−1), Region II (1200–1800 nm; 8500–5500 cm−1), and Region III (1800–2500 nm; 5500–4000 cm−1) [1,2,3,4,5,6,7]. The boarders of the three regions are not rigorous. In Region I, bands arising from electronic transitions and those due to higher-order overtones and various types combination modes are expected to appear. Region I shows high transparency because all the bands appearing in this region are very weak, allowing biomedical applications and applications to agricultural products. Region I is the region where CCD cameras work very well, and this region is called “window of biological materials” because of high transparency to body. It has also a few more nick names: “the short-wave NIR (SWNIR) region,” ”near NIR (NNIR) region,” or “the Herschel region.”

Region II is a region for the first and second overtones of the XH (X = C, O, N) stretching vibrations and various types of combination modes. Region III contains mainly bands attributed to the combination modes except for the second overtone of the C = O stretching vibrational mode. It shows poorer “permeability.”

NIR spectroscopy, particularly vibrational NIR spectroscopy, is spectroscopy of anharmonicity [1,2,3,4,5,6,7]. The overtones and combination modes are the so-called forbidden transitions for a harmonic potential, yielding very weak bands. Both the frequencies and intensities of NIR bands are controlled by anharmonicity. Therefore, investigations on overtones and combination modes, anharmonicity, vibrational potential, and dipole moment function regarding NIR spectroscopy are important. However, these studies have been far behind applications of NIR spectroscopy probably because until 1990s, it was difficult to obtain accurate NIR spectra and to make reliable band assignments. It is only recent that quantum chemistry has been introduced to studies of frequencies and intensities of overtones and combination bands (Chap. 5).

The fact that bands in the NIR region are weak or very weak is what makes this region unique and markedly different from the other regions [1,2,3,4,5,6,7]. The reason why the NIR region is valuable in various applications is because only the NIR region serves as a highly transmitting window to radiation thanks to anharmonicity.

3 Brief History of NIR Spectroscopy

It is uncertain when NIR spectroscopy began, but there is the report that Abney and Festing measured the spectra of some simple organic compounds in the 700–1200 nm region as well as in the Vis and IR regions. In the beginning of the twentieth century, main concerns of molecular spectroscopy were UV–Vis and IR spectroscopy. It was 1920s and 30s that systematic measurements of NIR spectra were carried out. A chance came from the development of a spectrometer by Brackett. In 1930s, spectroscopists already recognized that NIR spectra arise from overtones and combination modes [9, 10]. In 1950s, NIR spectroscopy received considerable interest for hydrogen bonding studies and studies on anharmonicity [11, 12]. The development of an innovative spectrometer, Carey 14 Applied Physics in 1954, pushed NIR spectroscopy research as well as UV–Vis studies [1]. However, the development of basic studies of NIR spectroscopy was still rather slow because even development of new spectrometers was insufficient for NIR spectroscopy to observe weak NIR bands accurately, and also systematic analysis of NIR bands was very difficult. Moreover, NIR spectroscopy could not find application fields not only in basic science but also in practical applications [1,2,3,4,5,6,7]. Until 1960s, NIR spectroscopy was a “sleeping giant” in terms of both basic science and applications.

It was not a spectroscopist but an engineer in an agricultural field who woke up the sleeping giant. He was Karl Norris (Fig. 1.3a) of the US Department of Agriculture (USDA). Norris learnt electrical engineering as well as agricultural engineering at universities, and thus, he had good background for developing spectrometers and systems with computers. Norris was involved in a research of drying grain by use of infrared technology. He happened to find that the grain had absorbances in the NIR region. He focused on the fact that NIR spectroscopy is suitable for nondestructive analysis, and he and his colleagues tried to use NIR spectroscopy for quality assessments of agricultural products. Norris proposed to use statistical methods to build calibration models from NIR data [13, 14]. First, he employed simple linear regression and then multiple regression. His idea realized the advantages of NIR spectroscopy in practical applications. Thus, Norris is called “Father of NIR spectroscopy.” Norris, Phil Williams, Fred McClure, and other engineers applied NIR spectroscopy to many applications in agriculture and then foods. Beltsville (Maryland, USA) was a place for assemblage for the bold and ambitious. Their great success partly came from the strong request of quality assessment from consumers which already started in North America from 1960s and partly from the development of spectrometers and computers.

Fig. 1.3
figure 3

Portrait photos of a Karl H. Norris, b Camille Sandorfy, and c Frans F. Jöbsis

Full size image

However, many conventional spectroscopists did not accept adamantly the eccentric idea of utilizing statistical methods such as multiple regression analysis to develop calibration models of NIR data. After rather long dispute, some traditional spectroscopists started to recognize the usefulness of the statistical methods to analyze NIR spectra. Particularly, Tomas Herschfeld played a very important role in making a bridge between the spectroscopists and agricultural engineers.

Of note is that in 1960s, there was significant progress also in the applications of NIR spectroscopy to basic studies [15,16,17]. For examples, in 1963, Bujis and Choppin [17] measured NIR spectra of pure water and investigated water structure in relation to hydrogen bonds. Late 1960s and early 70s, a few research groups including Camille Sandorfy (Fig. 1.3b) group [16] found very interesting fact concerning with the relative intensities of free and association bands of the OH and NH stretching bands compared for the fundamentals and overtones. The relative intensity of the free band is much greater for the overtones than fundamentals. One can say Sandorfy is a pioneer in basic studies of NIR spectroscopy. He is famous particularly in the research on relation between anharmonicity and hydrogen bondings.

It is also very important to know that there is another great scientist who advanced the practical application of NIR spectroscopy. He was Frans, F. Jobsis (Fig. 1.3c), who carried out the in vivo monitoring of the redox behavior of cytochrome c oxidase (or cytochrome aa3) (Chap. 20) [18]. Since his pioneering study medical application of NIR spectroscopy has shown distinctive growth as described later [19].

It is not clear when chemometrics was born, but it is clear that the use of statistical methods by Norris was one of the initiations of the development of chemometrics [1, 2, 4, 6]. Among various molecular spectroscopy, NIR spectroscopy was the first in using chemometrics. For the last half century or so, chemometrics developed NIR spectroscopy and NIR spectroscopy developed chemometrics. Nowadays, chemometrics is used in almost all kinds of spectroscopy including IR, Raman, far-infrared (FIR)/Terahertz, UV–Vis, fluorescence, and NMR spectroscopy.

In 1980s, NIR spectroscopy was used mainly for agriculture and food engineering fields, but applications to polymers and on-line analysis started in those days. After entering 1990s, application of NIR spectroscopy made remarkable progress thanks to the development of spectrometers, detectors, computers, and chemometrics. It has expanded to chemical, polymer and petroleum industries, pharmaceutical industry, biomedical sciences, environmental analysis, and even analysis of cultural resources. In the last ten years or so, development of NIR imaging and portable and handheld instruments has been a matter of big attention. Besides progresses in NIR imaging, and portable and hand-held spectrometers, those in on-line monitoring, process analysis technology (PAT), sensing for security and safety, and medical diagnosis have been particularly noted [1,2,3,4,5,6]. NIR world is stretching strongly over a huge area of science and technology.

Medical application of NIR spectroscopy is nowadays called functional NIR (fNIR) spectroscopy [20]. It uses mainly electronic NIR spectroscopy in Region 1, the region of “window of biological materials.” fNIR is applied not only to medical applications but also to brain research.

Basic studies of NIR spectroscopy such as overtones, combination modes, anharmonicity, and vibrational potential, and application of NIR spectroscopy to basic science like studies of hydrogen bondings, intermolecular interactions, and solution chemistry experienced “renaissance” in the 1990s due to rapid progress in NIR spectrometers particularly FT-NIR spectrometers and spectral analysis methods like two-dimensional correlation analysis.(Chap. 13) [1, 2, 3, 5, 7] Quantum chemical calculations have realized simulations of NIR spectra not only of simple compounds but also of rather complicated molecules such as long chain fatty acids, caffeine, nucleic acid bases, and rosemaric acid (Chap. 5). They also enable one to make band assignments of NIR spectra [20]. It is noted that quantum chemical calculations are useful for both basic studies and applications of NIR spectroscopy.

NIR spectroscopy is expanding markedly to a variety of fields such as astronomy, security and safety sensing, forensic science, building site, paleocultural property science and brain science.