1 Introduction

The international competition and increasing requirements for high quality and low cost has increased the unpredictability of surroundings creating an urgent need for implementing new technologies and utilising existing commercial technologies as a vital approach for industrial survival. Condition monitoring of manufacturing operations is an important strategy to be implemented. It offers a flexible, effective and economical tool to improve the entire performance of manufacturing systems through: better design; enhanced health and safety standards; the minimisation of unproductive time of staff; improved quality and reliability; minimum environmental pollution; the improved availability of machine tools; better customer satisfaction; maximum profits and the optimised quality of the manufactured products [1]. Productivity can also be improved by including the necessary inspection and quality control processes within the production stage. What is needed is an automated process condition monitoring system that predicts failures before they cause damage or breakdown [2]. Condition-monitoring systems should be able to track process faults, which can offer the highest potential for avoiding unproductive down-time and maintain the highest quality of the manufactured products.

Many different types of sensors and signal processing methods are now commercially available for monitoring manufacturing processes [3]. Many ideas have been presented and numerous approaches have been proposed for condition monitoring. Manufacturing processes, in general, and machining processes, in particular, are difficult to monitor due to the high combinations of operating conditions and faults. To fully understand and attempt to control the behaviour of machine tools and the manufacturing processes, effective condition monitoring systems should be developed which guarantee the reliability of the system operations and the quality of products [4]. Multiple sensors have been beneficially implemented in complex manufacturing condition monitoring systems to obtain comprehensive information about the process [5]. The utilisation of different sensors involves integration and fusion of the sensory signals to extract the key features from the data by removing any existing redundancy.

Many different types of sensors are now commercially available coupled with signal processing methods. Sensors are key elements of a successful monitoring system [3]. Sensors far as force, acoustic emission (AE), vibrations and the conventional visible spectrum camera [5,6] and infrared cameras for monitoring the heat patterns or temperature of machine or process have also been used in Laboratory environments [7]. However, high sensitivity infrared cameras are exceptionally expensive for use in a real production environment. This paper investigates the use of a new autonomous low-cost, low-resolution infrared imager for monitoring manufacturing processes [8]. Grinding, drilling, welding and soldering processes are monitored to distinguish between "normal" and "faulty" states using the acquired infrared images combined with a novelty detection algorithm.

2 Infrared thermography

Objects transfer heat by three means: conduction, convection and radiation. Conduction is the transfer of heat through solid objects. Convection is the transfer of heat through the movement of a fluid such as air and radiation is the transfer of heat energy via electromagnetic radiation emitted by the object. The radiation emitted by the object includes infra-red radiation which can be detected by infra-red imagers. Infrared radiation is emitted by every object at a temperature above absolute zero (−273°C). The amount of infra-red energy emitted by the object is partly a function of the temperature of the object. The infra-red energy emitted increases as temperature increases. Infra-red radiation consists of electromagnetic waves of a length between 0.7 μm and 1000 μm. However, the available infra-red imagers in the market normally work between 0.7 μm and 20 μm. For the calibration of thermal imagers a black body is used. A blackbody is defined as being a perfect absorber as well as a perfect emitter. Also, a blackbody does not reflect infrared radiation from other objects in the surrounding and its temperature is proportional to the infrared radiation emitted for it. However, actual objects in real life do not always behave as a black body. In order to describe the capability of a surface to emit energy compared with a blackbody, the emissivity value (ε) is defined as the ratio of the thermal radiation emitted by a surface at a given temperature and that of the blackbody and for the same spectral and directional conditions [9]. Therefore, Emissivity of a blackbody equals 1 and Emissivity of any other type of surface is less than 1 and greater than or equal to 0.

An Infra-red thermal imager senses infra-red radiation which is proportional to the temperature of an object. Based on the Stefan-Boltzmann Law [10]:, the radiation emitted by a surface is proportional to the fourth power of the absolute temperature of the surface, see Eq. 1, (i.e. the hotter the object the more infra-red energy is produced).

$${W = \varepsilon kT^{4} }$$
(1)

Where

W=:

Radiated Energy

ε=:

Emissivity

k=:

Boltzman's constant (5.67×10−8 W/m2.K4)

T=:

Temperature (K)

The imager converts the radiated energy into an electric voltage which can be calibrated. A computer system then can be used to read the data and display the images or it can be utilised for image processing and interpretation. The imager used in this research is a pyroelectric imager. In the pyroelectric detectors, the radiation received by the element causes a change in the charge present on the device electrode. They have the advantage of not requiring cooling. However, they require a constantly changing image signal and so require a chopper.

The imager used in this paper are of a newly developed generation of imagers type IRISYS IXS9009 [11], see Fig. 1a. It is housed in a die-cast box that contains the imaging optics, detector, electronics, optical modulator (chopper) and rechargeable battery. It operates through an RS232 serial connection to a PC. A software was developed using Matlab to communicate directly with the imager and display the images with different resolutions and colour-maps and enables the user to save the required images into the PC for on-line or off-line analysis. When compared with a high resolution imager, as shown in Fig. 1b, it has been found that the low resolution imager gives similar thermal information to the high resolution one. However, the low-resolution images costs about 7% of the total cost of a typical high-resolution infrared imager and is about 20% of its size. Moreover, the low-resolution imager is specially designed for embedded system where the data can be directly streamed through an RS232 connection to a computer for on-line monitoring and decision-making. The IRISYS thermal imager is a low-resolution imager (16×16 pixels). In order to improve the human interpretation of the images, a 2D image interpolation process is used. Figure 2 presents the original (16×16) thermal image of a grinding process and the same image after bicubic interpolation to achieve a 128×128 resolution. For bicubic interpolation, the output pixel value is the weighted average of pixels in the nearest 4-by-4 neighbourhood. The bicubic interpolation is only used to ease human interpretation of the acquired images. However, it is not needed for automated recognition because it does not add any more information to the data.

Fig. 1.
figure 1

The low-cost, low-resolution infrared imager set-up (a) and a comparison between a low resolution and high resolution infrared imager (b)

Full size image
Fig. 2.
figure 2

The interpolation of the low-resolution thermal images of a grinding process

Full size image

Let P(y)=a 0+a 1 y+a 2 y 2+a 3 y 3 be a third degree polynomial. The Lagrange polynomial interpolation is given by:

$${P{\left( y \right)} = {\sum\limits_{i = 0}^3 {f_{i} L_{i} {\left( y \right)}} }}$$
(2)

where,

y :

is the point at which interpolation takes place.

P(y):

represents the interpolated value.

f i :

are the known values on the grid at points yi.

L i (y):

are the Lagrange polynomials such as

$$ {L_{i} {\left( y \right)} = {\prod\nolimits_{k \ne i,k = 0}^3 {{\left( {y - y_{k} } \right)}/{\left( {y_{i} - y_{k} } \right)}} }} $$
(3)

The clear advantage of using low-resolution images over a single pixel sensor is because that a 2D image has much more information in terms of the general shape of the objects, the thermal distribution, the size (i.e. of the number of pixels) of the thermal effect and calibration of the specific area against the changing temperature of the environment.

3 Novelty detection

Novelty detection is used in this work as a self-learning approach to characterise the "normal" state of the process. Novelty detection [12] is a classification technique that recognises presented data to be novel (i.e. new) or non-novel (i.e. normal). The advantage of novelty detection comes from its ability to differentiate between its normal training data and new data which it has not be seen before. Different types of novelty detection algorithms and applications have been reported. Reference [13] discussed the extreme value theory and its application far a novelty detection in biomedical data processing. In [14], generalised radial basis functions neural networks are used to form a Bayesian classifier which is capable of detecting novel data. Reference [15] uses the novelty detection approach to diagnose failure in structures. The results of applying novelty detection show that it has the potential to be applied successfully in many applications.

The training data for the novelty detection algorithm consists of only one class (i.e. the normal one) which is often much easier to obtain. Since a degree of overlap is normally expected between different classes, classification problems have a probabilistic nature [16]. Novelty detection involves estimating the probability-densityfunction (PDF) of a normal class from the training data and then estimating the probability that a new set of data belongs to the same class. The classification decision in novelty detection is based on Bayes' theorem as shown in Eq. 4).

$${P{\left( {C_{i} |x} \right)} = {{p{\left( {x|C_{i} } \right)}.P{\left( {C_{i} } \right)}} \over {p{\left( x \right)}}}}$$
(4)

Where

\({P{\left( {\left. {C_{i} } \right|x} \right)}:}\) :

The Posterior Probability, the probability that a given vector, x, belongs to class C i .

P(C i )::

The Prior Probability, the probability that a future input, x,belongs to a class, C i , based on the ratio of training examples that belong to the same class.

\({p{\left( {\left. x \right|C_{i} } \right)}:}\) :

The Class-Conditional Probability Density, the probability of obtaining an input vector from a given class based on estimating the PDF of a class.

p(x)::

Unconditional Probability Density, probability density of x regardless of the which class it belongs to.

The Unconditional Probability Density should also satisfy the following equation:

$${p{\left( x \right)} = {\sum\limits_{i = 1}^k {p{\left( {x|C_{i} } \right)}.P{\left( {C_{i} } \right)}} }}$$
(5)

Where

0≤P(C i )≤1 and

$${{\sum\limits_{i = 1}^k {P{\left( {C_{i} } \right)}} } = 1}$$

The accuracy of a novelty detection classification is dependent on the accuracy of the modelled density functions [12]. Three main methods are normally used to model the PDF: Parametric methods [17], Non-Parametric methods [18] and Semi-Parametric method [19]. The parametric methods assume sufficient statistical information about the training data set which is not normally available. In non-parametric methods no assumptions are made regarding the underlying density functions and they depend on the training data to find the probability density function for a new input. Reference [16] classifies such methods as being Kernel based techniques and K-Nearest Neighbour techniques. The K-nearest neighbour method depends on the probability that a number K data points of a vector fall within a specific volume. The Kernel-based technique calculates the volume by defining width parameters for a number of known probability distribution functions (Kernels) to provide a general model for the training set. However, non-parametric methods require long computations for every input vector. Semi-Parametric density estimation is used in this paper for novelty detection because it combines the advantage of both Parametric and Non-Parametric techniques and does not require extensive computational effort. Semi parametric methods use fewer Kernels. A Gaussian Mixture Model (GMM) is used in this paper to estimate the PDF. Unlike non-parametric methods the training data are used only during the process of construction of the density model and are not needed for the calculation of the PDF for new vectors.

The probability density estimation of GMM is obtained by Bayes' theorem, similar to Eq. 5, as follows [16]:

$${p{\left( x \right)} = {\sum\limits_{j = 1}^M {p{\left( {x|j} \right)}.p{\left( j \right)}} }}$$
(6)

Where

$${0 \le p{\left( j \right)} \le 1}$$
M:

is the number of components in the mixture model

p(j):

is the Prior probability of selecting the jth kernel function

\({p{\left( {x|j} \right)}}\) :

is the conditional density of x on the jth kernel.

For a Gaussian Mixture Model, the following equation is derived from Eq. 6 [16]:

$${p{\left( x \right)} = {\sum\limits_{j = 1}^M {\phi _{j} {\left( x \right)}.\alpha _{j} } }}$$
(7)

Where

ϕ j :

is the response of the jth Gaussian component

α j :

is the mixing coefficient (priors) of ϕ j

When the probability distribution function is calculated. A threshold value can be implemented to define the borders between a novel vector and a normal data set [12]. Figure 3 explains the methodology for the novelty detection which is used in this work to detect faulty conditions.

Fig. 3.
figure 3

The GMM and the way it is used to detect novelty in infrared images

Full size image

Novelty detection software NETLAB [20] is incorporated with Matlab programs to investigate the 16×16 pixels infrared images obtained during the experimental work. More details regarding the novelty detection can be found in [16, 17, 18, 19, 20].

4 Grinding process

In a grinding processes, temperature rise is an important concern because it can affect the surface properties and reduce surface quality. Excessive heat could also cause distortions by differential thermal expansion and contraction. Excessive heat could also cause burning to the surface which could also damage surface quality [21]. The efficiency of the cooling process in grinding is therefore important in order to increase the reliability of the process and improve the surface quality of the machined parts. The application of a video camera for monitoring the grinding process based on the sparks generated by the process it has been reported in previous research [6]. Some researchers, see [22], used thermal modelling to estimate the heat generated by a grinding process and to prevent thermal damage. Other researcher utilised on-line monitoring of an infrared temperature measurement device [23] that measures the temperature of only one point. This could reduce the flexibility and the reliability of the monitoring process by monitoring a signal rather than a complete image. Other methods such as force [6], acoustic emission [24], laser triangulation [25] and mircomegnitic techniques [26] have also been used. This paper investigates the applicability of a low-cost infrared imager for monitoring grinding processes. The advantage of using an infrared imager is the capability of monitoring the heat of the wheel as well as the spark intensity, see Fig. 2. The imager is used to distinguish between normal grinding conditions using an efficient cooling process and for an insufficient cooling process that reduced the quality of the machined surface.

5 Drilling process

In drilling, machining occurs inside the work piece. Chips exit for the same drilled hole resulting in high friction and producing heat. The use of cooling fluid in the drilling process is important in order to improve drill life. High friction decreases drill life and reduces the surface quality of holes. Efficient monitoring of drilling temperature is a key element in monitoring tool life and the drilling conditions particularly tool wear, chip jamming and coolant efficiency. Several types of sensors have been used in monitoring drilling operations such as thrust force, torque, spindle power [27], vibration [28] and acoustic emission [29]. In this paper an infrared imager is used to monitor a drilling process to attempt to differentiate between a healthy process (fresh drill) and a process which produces excessive heat due to tool wear.

6 Welding process

The infrared imager is used to monitor a gas metal-arc welding (GMAW) process, formerly known as MIG welding (Metal Inert Gas). The weld area is shielded by an effectively inert atmosphere of an inert gas to reduce and prevent oxidation. It is a common process in the metal-fabrication industry particularly in robotic applications. This paper investigates the application of an infrared imager to monitor the health of the welding process. The paper investigates a "normal" welding process and compares it with a faulty process using a low flow of the inert gas. Some techniques such as sensing arc length in welding have been suggested [30]. Infrared and laser sensing [7] have been also used for monitoring arc welding operations. A laser system has been used for geometry profile measurement and an infrared image has been used for monitoring thermal fields. However, the infrared camera used is a high resolution type which includes a nitrogen cooled detector and which costs 20 times the cost of the imager used in this research work. As previously mentioned, infrared cameras are expensive and difficult to embed into manufacturing processes.

7 Soldering process of electronic components

Soldering is an important process in the electronic industry. Electronic devices should be reliably soldered to their printed circuit boards in order for the electronic devices to operate for a long period of time and withstand vibration and temperature deviation. This becomes more important for portable devices such as mobile phones and laptop computers as well as most of the military equipment. However, some electronic components could be sensitive to soldering temperature. In other cases it is important to monitor the heat pattern produced by soldering of the electronic devices in order to ensure a uniform and strong soldering and avoid weak or dry soldering which could cause faults in the electric devices. It is also important when disassembling electronic circuits for repair to ensure that other parts of the circuit are not affected. Thermocouples have been suggested for monitoring soldering processes [31], however, a thermocouple is capable only of measuring temperature of a single point and they also need to be physically attached to the components. High-resolution infrared thermography has been used [32] to model electronic soldering. In this paper, a low-cost infrared imager is used to study the heat generated from a soldering process in order to evaluate the imager capability for automated monitoring of soldering electronic components.

8 Experimental work

Figure 4 presents the experimental work for the four manufacturing processes. The manufacturing processes selected in this test were optimised to produce the normal quality. A typical industrial fault was then introduced, based on suggestions by industrial technicians, in order to evaluate the ability of the imager to detect the faulty state. The grinding process, shown in Fig. 4a, was performed on mild steel using a surface grinder with a A60-K5-V10 wheel of 300 mm diameter and 25 mm width. Feed rate was 10 m/min, cutting speed 30 m/sec, and depth of cut 10 μm. The test was performed with a sufficient flow of coolant (soluble 1:25 dilution with water) and with an ineffective flow of coolant (70% of the initial flow rate). The drilling process, see Fig. 4b, was performed on mild steel work-pieces using a 10 mm twist drill. The drill surface speed was 20 m/min and feed rate was 0.3 m/rev. Two drills were used to test the applicability of the infra-red imager: a fresh drill and a worn one which had flank wear of 0.3 μm. The gas metal-arc welding (GMAW) was performed using the set-up shown in Fig. 4c. The imager was located in a position to monitor a manual welding process with "normal" and "faulty" welding based on the heat distribution of the arc. A DC current of 50 A and a nominal voltage of 25 volt were used. Steel sheets of 1 mm thickness were welded with a gas shield of (Ar-2%O2). A consumable steel electrode is used for spray transfer welding. The flow rate of the shielding gas was optimised for the process (15 L/min) and then it was reduce to (9 L/min) to produce a less effective welding joint and to test the thermal imager. For the soldering process, see Fig. 4d, a 50 watts (max) soldering iron was used to solder electronic components into a printed circuit board. The soldering temperature is controlled to produce optimum soldered joints. The soldering process was performed manually in this test and faulty joints were introduced using excessive heat, to study the capability of the infrared imager to detect change in heat patterns.

Fig. 4.
figure 4

The experimental work for the four tested manufacturing processes: grinding (a), drilling (b), welding (c) and soldering (d)

Full size image

9 Qualitative results

In this section bicubic interpolation of some infrared imagers are presented with visual images as examples to evaluate the capability of the infrared imager. Two-dimensional surfaces of the infrared data are also used to evaluate the capabilities of the imager. The qualitative results depend on human interpretation of data to compare normal and faulty states. In the following section, the automated results found using novelty detection is discussed.

9.1 Grinding process

Figure 5 shows a comparison between a normal grinding processes using sufficient coolant and a faulty process one which caused the grinding wheel to get hotter and caused the formation of grinding sparks during the operation.

Fig. 5.
figure 5

Results of a faulty and normal grinding process

Full size image

By analysing the thermal images, see Figs. 5 and 6, it has been found that the faulty process produces much heat on the contact surface between the wheel and the work-piece. The generated sparks contribute to a wider hot area when compared with normal process. By averaging the 256 temperature pixels of the wheel as shown in Fig. 6, it has been found that the faulty process generated 40% extra average heat when compared with the normal grinding process. Figure 6 presents the two infrared images data combined together as once surface to ease the comparison and interpretation.

Fig. 6.
figure 6

A 3D surface comparing the heat patterns of a normal and faulty grinding process

Full size image

9.2 Drilling process

Figure 7 presents the visual image and the associated infrared images of a drilling process using a normal drill and a worn one. As shown in Fig. 7, the worn drill produces more heat than a fresh one.

Fig. 7.
figure 7

Infrared images of a fresh and a worn twist drill

Full size image

Figure 8 presents a comparison between the normalised temperature of the normal and faulty drilling process. It has been found that the normal drill produces much less heat (about 35%) than the worn one.

Fig. 8.
figure 8

A 3D surface comparing the heat patterns of the fresh and worn drills

Full size image

9.3 Arc welding process

It has been found that changing the shielding gas can have a considerable effect on the nature of metal transfer from the electrode to the work-piece and therefore affect the quality of the welded joint, see Fig. 9.

Fig. 9.
figure 9

The welded joints of a normal and a faulty arc welding process

Full size image

It has been found that changing the shielding gases can have substantial effect on heat transfer which is detected by the infrared imager as shown in Fig. 10. The normal weld produces more concentrated heat patterns while the faulty one, which lacks the shielding gas has a wider and higher heat patterns which could be due to the increase in oxidation of the metal during the welding process. The hot spots area of the infrared image of the faulty welding process, see Fig. 10, is found to be 500% hotter than the hot area of the normal process (i.e. number of hot pixels).

Fig. 10.
figure 10

An Infrared images and the associated 3D surfaces of a normal and faulty welding process

Full size image

9.4 The soldering process

Figure 11 presents the visual and infrared images of thesoldering process of a transistor using an optimum normal heat level and another one with twice the heat power.

Fig. 11.
figure 11

Visual and infrared images of normal and faulty soldering processes of a transistor

Full size image

By studying the thermal images it can be concluded that the infrared imager was capable of detecting the difference in temperature for small objects and for study of the thermal distribution. The overheated soldering processes has caused damage to the joints of the other soldered components on the same printed circuit board in addition to the possibility of damaging the soldered transistor.

10 Quantitative results using novelty detection

The advantage of using low-cost infrared imager is in the application of embedded and automated systems for on-line detection of faults in manufacturing processes. The novelty detection NETLAB software is used with modification to analyse the acquired signals. The response of the Gaussian kernels ϕ j is defined by a covariance matrix and a centre. A single variance parameter for each Gaussian component is calculated using a different number of centres (i.e kernels) in the mixture model in order to select the most suitable structure based on the speed and the accuracy of results. Two normal infrared images are used to define the "normal" conditions. Twelve samples of every process (six healthy and six faulty) are used to test the novelty detection algorithm. Since the probability values obtained from NETLAB algorithms are found to be too small, logarithmic values are used to simplify the presentation of data. The results of the tested infrared images for 10 kernels of the four manufacturing processes are shown in Fig. 12. The x-axis presents the test number and y axis is the logarithmic probability that a specific infrared image belongs to "normal" or "novel" group. The threshold value can be selected to achieve the required Type-I and Type-II error.

Fig. 12.
figure 12

Novelty detection results: grinding process (a), drilling (b), welding (c) and soldering (d)

Full size image

11 Discussion

It has been found that the approach is most successful with the grinding process that has the maximum difference between the normal and novel data. The drilling and soldering processes show similar success while the welding process was the least successful in categorising normal and novel images. This is could be due to the characteristic of the welding process, see Fig. 10, where the difference might not be clear enough or more complex image processing is needed to analyse the infrared images. However, by using a suitable threshold level, the novel conditions can be detected with specific Type-I and Type-II errors.

Although the application of IR technology has been found successful in the presented work, there are some limitations on the technology which should be considered:

  1. 1.

    The monitored faults need to be the result of a change in heat pattern or infrared radiation. For example, the detection of catastrophic breakage of a drilling cutter would not be possible using the change in heat patterns.

  2. 2.

    The speed of the currant infrared imager is limited to about 8 infrared images/second. Since the pyroelectric technology uses a chopper, some fast events could occur when the chopper is over the infrared sensor. This makes it difficult for the infrared imager to detect events that takes less than 0.12 of a second.

  3. 3.

    The change of the Emissivity value of the work-piece/cutter of the process could change the temperature readings of the infrared imager. The Emissivity could change as a result of change in colour, material or surface texture.

  4. 4.

    The blockage of the view during the monitoring process could cause faulty results as in any visual systems.

  5. 5.

    The location of the infrared imager is important. The reflection of infrared radiation from other hot objects in the area could cause significant noise on the infrared imager.

The success of the novelty detection is dependent on the infrared data. Any significant noise in the infrared images can increase the overlap between the probability distribution of the novel and normal data (see Fig. 3). This means that the selection of a threshold value would be critical. Depending on the threshold value, the system would either give unnecessary fault alarms or would categorise more novel data as being normal (i.e. increase Type-I or Type II errors).

12 Conclusions and further work

In this paper, a new low-cost infrared imager is evaluated for monitoring faults in manufacturing processes combined with novelty detection algorithm to automate the monitoring process. Grinding, drilling, welding and soldering processes were tested. The low-cost infra-red imager, which has a low resolution of 16×16 pixels is found useful in detecting the difference between a normal and novel conditions. The approach has been found most successful for grinding, drilling and soldering processes. The welding process could have high Type I or type II error based on the selected threshold value. The reduced cost of the evaluated system, in comparison to the more expensive infrared imagers available in the market, allows the monitoring system to be embedded into machines and manufacturing processes for an on-line monitoring system for manufacturing processes. Future work will include the implementation of the low-cost infrared imager for monitoring manufacturing processes in real production environments. The newly developed sensor combined with the novelty detection will be embedded into production machinery for on-line monitoring of a manufacturing process.