Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Infrared (IR) imaging systems are widely used in a variety of applications like remote sensing, surveillance, medical, fire and mine detection, etc. Largely, Infrared imaging systems are based on the Infra Red Focal Plane Array (IRFPA) [1], which consists of an array of infrared detector elements aligned at focal plane of the imaging system [2, 3]. Recently, there has been an increasing research in IR detector technologies that resulted in realization of large detector formats like 640 × 512, 1024 × 768, etc., having smaller pitch and better thermal sensitivity. The performance of IR detector is strongly affected by several degrading factors such as the lens diameter causing blurred image, detector’s photo response resulting in intensity loss, under sampling because of limited active area of each detector (limited pixel size), Poisson (shot) noise and the additive Johnson noise generated by the electrons. One of the most challenging and degrading effect is caused by random spatial and temporal photo response non-uniformity of photodetectors. Since each individual detector in the array has a different photo response under identical irradiance, due to mismatch of fabrication process, it results in fixed-pattern noise (FPN) or non-uniformity [4] superimposed on the true image. These fluctuations between pixels leads to degradations such as 1/f noise associated with detectors, corresponding readout input devices and the nonlinear dependence of the detector gain. This non-uniformity changes slowly in time with change in the FPA temperature, bias voltages, and scene irradiance. These changes reflect in the acquired image in the form of a slowly varying pattern superimposed on the image resulting in reduced resolving capability. NUC techniques normally assume a linear model for the detectors, characterizing thus the non-uniformity response problem as a gain (responsivity) and offset (detector-to-detector dark current) estimation problem.

2 Non-Uniformity Correction (NUC): Concepts

The non-uniformity arises due to number of factors, prominent among which are the large variation in responsivity (gain) and detector-to detector dark current (offset). The magnitude of the offset and gain variation depends on the two things: (i) the active material of IRFPA and (ii) the technology used to fabricate the FPA detector. The responsivity variations is the least (~1 %) in case of PtSi Schottky barriers, but may be quite large (~10 %) in case of MCT-based detectors.

Each detector element is associated with a fixed offset which is different for the different elements which is known as Fixed-Pattern Noise (FPN) offset. Gain of each detector element is not ideal (Gain = 1). It differs from pixel to pixel, due to which there will be variation in output of the particular element. These variation needs to be compensated. Non-uniformity between pixels values is represented by equation of line shown in Fig. 1.

Fig. 1
figure 1

Offset, gain, and full-scale errors

Full size image

Gain of detector element is represented as slope of the line (i.e., m 1) passing through origin (assuming zero FPN offset) thus for m 1 = 1 no correction is required to the detector output. For gain other than unity the detector output will have to be multiplied by the reciprocal of the gain value for that element to get correct pixel value. Similarly FPN offset in the detector output represented as b, i.e., offset of the line with reference to origin on y-axis. Hence, to get the correct pixel value, the FPN offset will be subtracted through the detector output. Thus, the NUC includes both offset and gain compensation.

Several NUC techniques have been tried out to overcome the problem of fixed-pattern noise in IR detector array. Keeping vast application area of infrared imaging system, a continuous development is in progress to improve the NUC techniques. Mainly there are two types of NUC techniques: (i) Calibration method-based [5] NUC techniques (ii) Scene-based [6–8] NUC techniques.

2.1 Calibration-Based Techniques

To correct for non-uniformities, simplest and most accurate methods is calibration-based method. Single point correction (SPC), two-point correction (TPC) and multiple point correction (MPC) methods are common method which fall under calibration-based techniques. Parameters like gain and the offset are estimated by exposing FPA to a uniform IR radiation source at one or more temperatures. The response of the individual pixels is recorded simultaneously to calculate gain and the offset coefficients. In case of TPC and MPC, two and more temperatures are used, respectively, to compute gain and offset coefficient. These coefficients are stored in suitable format and then used to compensate for the non-uniformity using associated sensor on-board electronics. The performance of the present method is optimal when the detector response varies linearly and is time invariant between the calibration temperatures.

2.2 Scene-Based Non-uniformity Compensation

The scene-based non-uniformities compensation [9] uses different image processing algorithms by exploiting change in the actual scene-related features or the motion in order to compute coefficients of scene temperature per detector. The true image from the fixed-pattern noise-affected scene is generated by compensating these scene-based coefficients. Statistically, the temperature diversity provides a reference point which is common to all detectors. The detectors response can be normalized for the non-uniformity based upon this reference point calculation. These algorithms are difficult to implement in real time and they do not provide the required radiometric accuracy. Since the scene-based NUC algorithms normally use motion as one of the criteria for separating the true image from the FPN, these algorithms usually leave artifacts in the image due to presence of non-moving objects, which are required to be corrected algorithmically.

3 The Mathematical Model of Calibration-Based Techniques

In order to provide completeness, calibration-based NUC been presented. Computational complexity and implementation strategy of these models have also been presented.

3.1 Single Point Correction (SPC)

The SPC method is used to correct the offset of every pixel in the IRFPA. This is performed by placing a uniform IR radiation source in front of the imager lens. Using one point correction, the fixed-pattern noise will be minimum at the reference temperature and with perfect correction [3]; there will be no fixed-pattern noise at reference temperature (Fig. 2). The residual FPN is produced due to the different spectral response of detectors along with the truncation errors in the normalization algorithm. Fixed-pattern noise tends to increase as the background temperature deviates from the reference calibration temperature. This increase depends upon how far the detector responsivity curves deviates from linearity. This method is used to update an existing NUC and can be performed in the field environment easily.

Fig. 2
figure 2

FPN after single point correction

Full size image

3.2 Two-Point Correction (TPC)

The most common and widely used method to correct for non-uniformity of IRFPAs is the TPC method. In two-point correction method [10, 11], the spatial noise will be minimum at two reference intensities and increases for other intensities. There is a curve known as W-curve (Fig. 3) where, in the region between two references, the spatial noise is less compared to the spatial noise outside two references. This method uses two uniform IR sources at two different temperatures (T1 and T2) to estimate the gain and offset of each detector element to compensate the non-uniformity.

Fig. 3
figure 3

FPN after two-point correction

Full size image

3.3 Multiple Point Correction (MPC)

To cater for wide operating temperature ranges, multi point correction technique [12] is most suitable method for non-uniformity correction. MPC also known as piecewise-linear correction method is an extension of the two-point method where, a number of different temperature points are used to divide the nonlinear response curve into several piecewise linear sectors to correct for non-uniformity. The fixed-pattern noise may be additive or multiplicative, for arrays with dark currents, the noise powers are additive and arrays with different responsivities produce multiplicative noise. The response of detector element in an FPA is nonlinear in nature, but it is modeled as a linear response having a multiplicative gain and an additive offset.

A two-point non-uniformity correction assumes that the value of each pixel can be corrected by multiplying it by gain and adding an offset to it. The measured signal Y ij for (ij)th detector element in the FPA at given time t can be expressed as:

$$Y_{ij} \left( t \right) = \alpha_{ij} \left( t \right).X_{ij} \left( t \right) + \beta_{ij} \left( t \right)$$
(1)

where, α ij (t) and β ij (t) are the gain and offset of the (ij)th detector element, and X ij (t) is real irradiance received by the detector element.

From Eq (1), the real incident radiation (irradiance) is given by

$$Xij(t) = \frac{Yij(t) - \beta ij(t)}{\alpha ij(t)}$$
(2)

Now to perform 2 point calibration, IR imaging system captures the images corresponding to lower and higher temperature from uniform radiation source (blackbody).

Defining α ij (t) [11]

$$\alpha ij(t) = \frac{T_{2ij} - T_{1ij}}{M_{2} - M_{1}}$$
(3)
$$\upbeta_{\text{ij}} \left( {\text{t}} \right) = {\text{M}}_{ 1} -\upalpha_{\text{ij}} \left( {\text{t}} \right).{\text{T}}_{{ 1 {\text{ij}}}}$$
(4)
$$M = \frac{1}{m.n}\sum\limits_{i = 1}^{m} {\sum\limits_{j = 1}^{n} {T_{ij}} }$$
(5)

where T 1ij (t) & T 2ij (t) are (ij)th detector element intensities at lower and higher temperatures at time t [4]. M 1 and M 2 are mean intensities (mean signal output) of the all detector elements in one frame (76,800 values in 320 × 256 FPA) at lower and higher temperatures. Corrected output of (ij)th detector element can be obtained from Eq. (1) using values of α ij (t) and β ij (t) calculated from Eqs. (3) and (4).

In staring IR focal plane arrays, each detector will have different gain and offset coefficient and this variation produces fixed-pattern noise. Figure 4 shows signal outputs of different detectors for same input intensities.

Fig. 4
figure 4

Effect of fixed-pattern noise before correction

Full size image

Figure 5 illustrates the normalized output after correction at two points Fixed-pattern noise will be minimum at two reference temperatures T 1 and T 2; it increases for any other reference temperature. If all detectors had linear responsivities, then all the curves would coincide (as shown in Fig. 5), spatial noise is minimum between T 1 and T 2. The residual spatial noise is present at temperature T.

Fig. 5
figure 5

Effect of fixed-pattern noise after correction

Full size image

4 Application to LWIR Imaging System

The two-point NUC scheme is implemented under the present scope of work and tested on LWIR cooled Imaging system based on 320 × 256 MCT-based IR focal plane array.

Design parameters of LWIR Imaging System

  • Spectral band: 8–12 μm (LWIR)

  • Detector: 320 × 240 MCT IRFPA (cooled)

  • F-number: 2

  • Aperture Dia: 130 mm

  • Spatial Resolution: 115 μrad

  • Video output: CCIR-B, 50 Hz

The LWIR imaging system electronics board generates detector interface signals, performs two-point non-uniformity correction, image processing tasks and finally generates CCIR-B compatible video output. Two sets of image data are cap re 10°C and higher temperature 35 °C, respectively, with integration time of 20 μs. Twelve image frames at each lower and higher temperature are acquired to correct the temporal noise and used to compute gain and offset coefficient. These gain and offset coefficients are used to correct the uncorrected image data. Figure 6a shows the raw IR image and Fig. 6b shows the image after NUC. Figure 7a, b shows the 3-Dimensional representation (histogram) of image data before and after NUC. Figure 8a, b illustrates the IR image from LWIR imaging system before and after two-point NUC. To determine the effectiveness of given NUC method, the Image quality measurements which are performed on the corrections (after NUC) are crucial. This method is offline calibration at factory level, thus, different tables for different temperature ranges are stored in Look up Tables (LUTs) and user can select the desired table as per field environmental conditions.

Fig. 6
figure 6

Image frame a before and b after two-point NUC

Full size image
Fig. 7
figure 7

3-Dimensional plot a raw data and b data after NUC

Full size image
Fig. 8
figure 8

Image frame a before and b after two-point NUC

Full size image

Figure 9a, b illustrates the result of another IR image before and after two-point NUC.

Fig. 9
figure 9

Image frame a before and b after two-point NUC

Full size image

In the present work, Residual Fixed-Pattern Noise (RFPN) [13, 14] is used to measure the NUC capability of the proposed method.

$${\text{RFPN}} = \frac{\text{Standard Deviation (Output level)}}{\text{Mean (Output level)}}$$
(6)
$${\text{RFPN}} = \frac{SD}{M} = \frac{1}{m.n}\sqrt {\sum\limits_{i = 1}^{m} {\sum\limits_{j = 1}^{n} {\left( {xij - yij} \right)^{2} } } }$$
(7)

where x ij is corrected image, y ij is reference two-point calibrated image, and (m. n) is total number of pixels in the image frame.

$$\begin{aligned} {\text{After}}\,{\text{NUC}}\,{\text{correction}},\,{\text{Measured}}\,{\text{Standard}}\,{\text{Deviation}}\,(\sigma) & = 6 4. 1 7 8 4\\ {\text{Mean}}\;\left( {\text{M}} \right) & = 1 4 5 2\\ \end{aligned}$$

So, measured Residual Fixed-Pattern Noise \(RFPN = \frac{\sigma }{M} =\) 0.0442

5 FPGA-Based Hardware Implementation

The NUC correction implemented in present work is based upon classical two-point NUC correction algorithms. FPGA-based Architecture of prototype hardware is shown in Fig. 10. IR detector having array of size 320 × 240 producing 14-bit digital data is exposed to a uniform IR source, i.e., blackbody at lower and higher temperature. Raw video digital data at different temperatures is stored in the SRAM through a serial link, this data is used to calculate the offset and gain coefficients. This is a time consuming and complex task to perform, where around 76,800 pixels with 14 bit data are processed. Since, these values are only valid for a given ambient temperature, the gain and offsets coefficients are stored in two flash memories having the capacity of at least one image frame. The implementation is carried out in an FPGA (Xilinx XC5VLX110)-based electronics board [15, 16]. Flash memory controller detector interfaces are designed in RTL using VHDL [17] and processing module to apply these coefficients on incoming data has also been designed in VHDL (employing Xilinx ISE tool). The data path and control path is designed to exploit the parallism within the processing block to achieve real-time performance. Resource utilization of the targeted device is given in Table 1.

Fig. 10
figure 10

FPGA-based hardware implementation of two-point NUC

Full size image
Table 1 Device utilization summary
Full size table

6 Conclusion

In the present paper, an approach namely blackbody calibration to perform NUC using two-point method is implemented on LWIR 320 × 240 IRFPA-based imaging system. There is significant reduction in FPN in the output images obtained after NUC. The experimental results confirm that the two-point calibration-based methods using uniform IR radiation source are effective and efficient for real-time correction of fixed-pattern noise visible due to non-uniformity. The real-time implementation of FPGA-based hardware architecture and realization using the 2-point NUC algorithm is also given for LWIR imaging system.