Keywords

1 Introduction

Contemporary thermal imaging systems [11] employ highly sensitive, larger format Infra Red (IR) focal plane arrays (FPA) which consist of a number of photo detectors [13] in a particular geometry at the focal plane [12] of an optical system. Mismatches in the photo response of the individual photo detectors and parameter variations cause unavoidable non-uniformities resulting in the superposition of a fixed pattern noise on the image. Spatial non-uniformities in the photo response of the individual detecting element can lead to unusable images in their raw state. Contemporary NUC techniques can be broadly divided in two primary categories [7]:

  1. 1.

    Reference based correction techniques [2, 5, 7]

  2. 2.

    Scene based techniques [1, 3, 4]

Two-point NUC is one of the most popular reference based correction technique, where in radiometric response of system is captured at two distinct temperatures/integration times keeping a uniform source as scene. However, two point NUC is limited by variations in performance over the entire operating range causing the residual non uniformity (RNU) to vary more on moving farther from the calibrated points. In the present approach, the radiometric responses of the thermal imaging system are captured at various integration times points within the entire operating range, instead of the usual two integration times. In place of linear interpolation [9] between these two points a linear approximation for complete range is explored with error minimization [8].

Rest of the paper is organized as follows. Related work model is presented in Sect. 2. In Sect. 3, methodology of proposed approach is elaborated. Section 4, includes results and analysis. A comparison of the proposed approach with the existing and very popular two-point NUC is also given in this section. Authors have made an attempt to conclude the work in Sect. 5.

2 Related Work

Hardie et al. [1] have proposed a simple, relatively less complex, scene based NUC algorithm that deals with relatively low levels of non-uniformities. This method exploits global motion between frames in a sequence to trace the true scene value along a motion of trajectory of pixels. Assuming the gain and biases of the detector elements to be uncorrelated along the trajectory, an average of these pixel values is calculated for different scene values for each detector. The observed pixel values and corresponding estimates of true scene values form the points used in line fitting.

Scribner et al. [3] have suggested a scene based NUC method, using neural network approach that has the ability of adapting sensor parameters over time on a frame by frame basis.

Torres et al. [4] have developed an enhanced adaptive scene based NUC method based on Scribner’s adaptive NUC technique. This technique is improved by the addition of optimization techniques like momentum, regularization and adaptive learning rate.

Kumar [2] has proposed an Infrared staring sensor model and mathematically analyzed output of any pixel in terms its unique non-uniformities. Apart from this an overview of processing algorithms for correcting the sensor non-uniformities based upon calibration as well as scene based methods to correct gain and offset parameters has been presented. Hardware implementation architecture of both types of algorithms has also been discussed in a comprehensive manner.

Khare et al. [5] have discussed NUC correction technique for additive and multiplicative parameters and their implementation in reconfigurable hardware for Long Wave Infrared (LWIR) imaging systems.

Kumar et al. [7] have discussed calibration based two point NUC correction technique and its implementation in reconfigurable hardware for Mid Wave Infrared (MWIR) imaging systems based upon 320X256 detecting element based IRFPA.

Kay [8] describes least square approaches as an attempt to minimize squared difference between given data and assumed model or noiseless data and mentions that as no probabilistic assumptions have been made about data hence method is equally valid for gaussian and non gaussian noises.

3 Methodology

If \(y_{1_{ij}},y_{2_{ij}},y_{3_{ij}},.......y_{m_{ij}}......y_{n_{ij}} \) are n radiometric responses of any \((i,j)_{th}\) detecting element of FPA of MWIR imaging system with M \(\times \) N InSb detecting elements, collected by capturing a uniform scene at n distinct integration times \( t_{1},t_{2},t_{3},.....t_{m}......t_{n} \) respectively, then:

  1. 1.

    Let the response of any \((i,j)_{th}\) detecting element of MWIR FPA captured at any particular integration time \(t_{m}\), may be expressed in the form of a linear equation as following:

    $$\begin{aligned} \begin{aligned} {y_{m_{ij}}}=a_{ij}{x_{m_{ij}}} + b_{ij} \end{aligned} \end{aligned}$$
    (1)

    where \(a_{ij}\) and \(b_{ij}\) are the gain and offset non-uniformities associated with the \((i,j)_{th}\) detecting element of FPA, respectively. Assuming uniform distribution of the incoming photon flux over complete FPA, value of \({x_{m_{ij}}}\) at any particular integration time \(t_{m}\) may be defined as mean value of radiometric response of all detecting elements at that integration time.

    $$\begin{aligned} x_{m}={x_{m_{ij}}} ={\frac{1}{M\times N}\left( \sum _{i=1}^{\MakeUppercase {M}} \sum _{j=1}^{\MakeUppercase {N}}{x_{m_{ij}}} \right) } \end{aligned}$$
    (2)

    Using Eq. 2, Eq. 1 may have an alternate representation:

    $$\begin{aligned} \begin{aligned} {y_{m_{ij}}}=a_{ij}{x_{m}} + b_{ij} \end{aligned} \end{aligned}$$
    (3)
  2. 2.

    Following is the matrix representation of radiometric responses of \((i,j)_{th}\) detecting element of FPA at n distinct integration times:

    $$\begin{aligned} \begin{aligned} \left[ \begin{matrix} y_{1_{ij}}\\ y_{2_{ij}}\\ y_{3_{ij}}\\ \vdots \\ y_{m_{ij}}\\ \vdots \\ y_{n_{ij}} \end{matrix} \right] = \left[ \begin{matrix} x_{1} &{} 1\\ x_{2} &{} 1 \\ x_{3} &{} 1 \\ \vdots &{} \vdots \\ x_{m} &{} 1\\ \vdots &{} \vdots \\ x_{n} &{} 1 \end{matrix} \right] *\left[ \begin{matrix} a_{ij} \\ b_{ij} \end{matrix} \right] \end{aligned} \end{aligned}$$
    (4)
  3. 3.

    Let

    $$\begin{aligned} \begin{aligned} \varUpsilon _{ij} = \left[ \begin{matrix} y_{1_{ij}}\\ y_{2_{ij}}\\ y_{3_{ij}}\\ \vdots \\ y_{m_{ij}}\\ \vdots \\ y_{n_{ij}} \end{matrix} \right] , \varPhi = \left[ \begin{matrix} x_{1} &{} 1\\ x_{2} &{} 1 \\ x_{3} &{} 1 \\ \vdots &{} \vdots \\ x_{m} &{} 1\\ \vdots &{} \vdots \\ x_{n} &{} 1 \end{matrix} \right] , \varTheta _{ij}=\left[ \begin{matrix} a_{ij} \\ b_{ij} \end{matrix} \right] \end{aligned} \end{aligned}$$
    (5)

    then Eq. 4 may be represented as following:

    $$\begin{aligned} \begin{aligned} \varUpsilon _{ij} = \varPhi *\varTheta _{ij} \end{aligned} \end{aligned}$$
    (6)
  4. 4.

    Least Square estimate [8] \(\hat{\varTheta _{ij}}\) of vector parameter \(\varTheta _{ij}\) of Eq. 6 is following:

    $$\begin{aligned} \begin{aligned} \hat{\varTheta _{ij}}={(\varPhi ^{T}\varPhi )}^{-1}\varPhi ^{T}\varUpsilon _{ij} \end{aligned} \end{aligned}$$
    (7)

    In this way the gain and the offset values for all other detecting elements of FPA may be estimated.

  5. 5.

    After NUC correction Eq. 1 can be expressed as following:

    $$\begin{aligned} \begin{aligned} {x_{m_{ij}}}= a'_{ij} {y_{m_{ij}} + b'_{ij}} \end{aligned} \end{aligned}$$
    (8)

    where

    $$\begin{aligned} \begin{aligned} a'_{ij} =\frac{1}{a_{ij}} ; \end{aligned} \end{aligned}$$
    (9)

    and

    $$\begin{aligned} \begin{aligned} b'_{ij} =-\frac{a_{ij}}{b_{ij}} \end{aligned} \end{aligned}$$
    (10)

    Estimated gain and offset values are utilized for calculation of \(a'_{ij}\) and \(b'_{ij}\). These values are stored in non volatile memory in the form of a table. During the image formation process Eq. 8 is implemented in real time to generate non-uniformity corrected video sequences.

  6. 6.

    Apart from doing non-uniformity correction present algorithm also suggests a mechanism for identification of defective pixels [7, 10]. Suppose any set \( I_m =\{ y_{m_{11}},y_{m_{12}}.......y_{m_{ij}}...........y_{m_{MN}}\} \) represents radiometric responses captured from FPA with M \(\times \) N detecting elements at any fixed integration time \(t_m\) with spatial mean \({\mu }\) and standard deviation \({\sigma }\) then let \(\forall i \in \{ 1,2,3,4 ....M\}\) and \(\forall j \in \{ 1,2,3,4 ....N\}\); if \(\exists \) any \(y_{m_{ij}}\)

    $$\begin{aligned} \begin{aligned} : \mu -3 \sigma> y_{m_{ij}} > \mu + 3 \sigma \end{aligned} \end{aligned}$$
    (11)

    A set of bad pixels \( B_m \) can be formulated with all such \(y_{m_{ij}}\)s identified above at that integration time \(t_m\).

4 Results and Analysis

Responses of a 640X512 InSb detecting elements based MWIR imaging system are captured in the range of 1 ms to 19 ms at a fixed interval of 1 ms. A minimum error linear approximation to these responses is explored with help of least square estimation for each detecting element. Residual Non Uniformity (RNU) \(\mathfrak {R}\) defined as following is considered as performance measure:

$$\begin{aligned} \mathfrak {R}=\frac{\sigma }{\mu } \end{aligned}$$
(12)

where \({\sigma }\) is the standard deviation of any considered frame of size \(M\times N\) defined as following:

$$\begin{aligned} \sigma =\sqrt{\frac{1}{M\times N}\left( \sum _{i=1}^{\MakeUppercase {M}} \sum _{j=1}^{\MakeUppercase {N}}\left( x_{i,j} -\mu \right) ^{2} \right) } \end{aligned}$$
(13)

and \({\mu }\) is spatial mean of considered frame.

Two point NUC [5] is performed by considering responses of imaging system at 4 ms and 16 ms integration times. Values of \(\mathfrak {R}\) as in Eq. 12 are calculated at different integration times and plotted as in Fig. 1(a). INT-LSR-NUC can be declared as conqueror for most of the considered integration time ranges in the plot except few areas in vicinity to points chosen for two-point NUC. As two-point NUC is curve fitting based approach, hence this type of behavior of \(\mathfrak {R}\) is trivial. Similarly in Fig. 1(b) plots for spread of gray levels with respect to integration time variation are shown. It has been assumed that most of the signal of relevance lies within \(\mu \pm 3\sigma \) band and spread of this has been shown for both type of NUCs. One can easily conclude that spread of \(\mu \pm 3\sigma \) band is lesser in case of INT-LSR-NUC as these frames are more uniform than frames after two point-NUC.

Fig. 1.
figure 1

(a) *Comparison of RNUs after two-point NUC and INT-LSR-NUC at various temperatures (b) *Comparison of spread of graylevels (\(\mu \pm 3\sigma \) band) of the scene after two-point NUC and INT-LSR-NUC at integration times from 1 ms–19 ms (*for a uniform radiation source as scene)

Full size image
Fig. 2.
figure 2

(a) Uncorrected response of FPA (standard deviation \(\sigma _{raw} = 751.83 \) and RNU \(\mathfrak {R}_{raw} = 0.0467 \)) (b) Response of same FPA after two-point NUC (standard deviation \(\sigma _{tp} = 427.09 \) and RNU \(\mathfrak {R}_{tp} = 0.026 \)) (c) Response of same FPA after INT-LSR-NUC (standard deviation \(\sigma _{lsr} = 154.8 \) and RNU \(\mathfrak {R}_{lsr} = 0.0096 \)) Peaks are representing locations of defective detecting elements.

Full size image

Figure 2 is representing response of FPA with an uniform source at 10 ms integration time as input scene; one can observe that there are a-lot of peaks in response at various places. By virtue of Eq. 11, when response at any location of FPA crosses \(\mu \pm 3\sigma \) band, detecting element of that location can be declared as defective element. A proper defective pixel replacement mechanism is invoked to get rid of this defect. Authors have detected 37 defective detecting elements in present work using above criterion. In Fig. 2(b) and (c) response is becoming more uniform as two point and INT-LSR-NUC has been performed respectively.

5 Conclusion

In this approach rather than simple two point line fitting, a linear approximation in responses is explored and a line with minimum error values is chosen. Two-point NUC only considers two points and makes error values zero there without worrying about RNU minimization at other points. In the present approach there is no guarantee that error will be zero at any point but it will assign an optimal value at each considered point. Fortunately least square estimation is a very old and well established algorithm. This makes implementation aspect of present model very simple. The results are promising and since the model is extremely simple, it can be applied for real time system realization.