Keywords

1 Introduction

The control itself does not create added value because it does not affect (change) the assessed product. However, it is one of the auxiliary processes that, thanks to collecting information about the finished product, allows gaining knowledge which elements can be improved and corrected [1, 2]. In this way, it indirectly allows receiving the added value that is significant both from a process and customer perspective. From quality control in the production process, the key element is the quality of workmanship, which determines the compliance of the finished product with its original design.

It is currently believed that quality is noticed by the customer only when it “does not appear” in the product. Therefore, nowadays, it is required that the methods and techniques of quality control and also appraisers competencies [3] are as accurate as possible, and at the same time, do not extend the production time of a single end product. This requirement may be fulfilled by Automatic Optical Inspection (AOI). It involves replacing the employee’s subjective visual assessment with control using advanced vision systems and computer software. Depending on the standard used in the inspection, AOI based on image analysis and AOI using algorithm analysis can be distinguished.

The paper aims to develop and describe the parameters of an optical inspection station on the Smart Factory automatic production line. The optical inspection software was developed for a product consisting of blocks. The developed software consists of two modules used for two cameras. The starting point for achieving the set objective was to perform a concise analysis of literature on quality control and to become familiar with the functioning of software provided by the hardware producer.

2 Literature Review

Today, machine vision and optical inspection applications are frequently used in many different areas such as automotive, food, and electronics. They are more and more popular as they allow contactless control. In contactless measurement systems, products are usually controlled by one or more specialized devices in the form of vision systems (cameras) [4]. Automated Optical Inspection is an advanced form of visual inspection, i.e., it consists of replacing the employee’s subjective visual assessment with inspection using cameras and computer equipment with dedicated software [5, 6].

The use of automatic optical inspection is part of the automation of the production process [7,8,9]. The vision system automatically carries out activities performed traditionally by an employee. The human task is to supervise the operation of devices as well as to create and implement appropriate control programs using modules (subprograms) supplied by the software producer [5, 10]. This approach is consistent with modern I4.0 industry assumptions [11,12,13].

Vision systems allow performing [14, 15]: object geometry checks; object dimensions and structure control; object color assessment; text control; code reading; control of the presence and position of object elements.

3 Research Methodology

The research object was an OMRON automatic optical inspection station. The research aimed to determine the parameters of the station using the finished product (Fig. 1), which consisted of blocks of different shapes and colors. The diversity of colors and shapes of individual elements allowed for the assessment of the correctness of the optical inspection algorithm presented in the next part of the work.

The OMRON automatic optical inspection station is located in the Smart Factory laboratory housed in the building of the Faculty of Mechanical Engineering at Poznan University of Technology. The laboratory belongs to the Institute of Materials Technology. The elements of the optical inspection station are (Fig. 2):

  • Computer (FH-1050-10 box type controller; Intel® Celeron® dual-core processor allowing for up to four cameras to be connected);

  • Two cameras 3Z4S-LEVS-0814H1 (each with 8 mm lens and aperture from F/1.4 to F/16);

  • Two LED light sources (light intensity can be adjusted in the range from 1 to 400 units for both sources simultaneously or by selecting the appropriate channel for each one individually);

  • Monitor;

  • Omron S8VK-C24024 power supply;

  • Leads.

Fig. 1.
figure 1

Research object.

Full size image
Fig. 2.
figure 2

Optical inspection station.

Full size image

The second aim of the research was to indicate the place of implementation of the optical inspection station in the production line in the Smart Factory laboratory (built for scientific and didactic purposes). The line, built by FlexLink, allows manufacturing finished products consisting of blocks in diverse variants. The production line consists of four loops enabling the product to be transported to subsequent stations arranged along with it. Each of the loops has crossovers enabling the product to be redirected to any transport loop. The product moves along the line on a pallet marked with an RFID tag, which allows it to be identified. There are RFID reading heads in front of each site and workstation that allows changing the direction of the route. A control panel controls the entirety. Assembly stations are independent organizational units responsible for the production of components of finished products. Each station is equipped with flow racks that allow storing containers with parts and components for assembly. Managing the production system is carried out using the 4Factory IT system, and communication between system elements is carried out via the Internet of Things (IoT).

4 Results

Vision system software includes numerous control methods, the so-called subprograms, and enables the creation and saving of own control programs [5].

The first step to properly create an optical inspection algorithm using software is to place the tested object within the range of cameras. Then, the next step is to configure the image captured by the vision system at the site. The range of the basic configuration is shown in the scheme (Fig. 3).

The optical inspection software was developed for a product consisting of blocks. The assembly of this product is the basis for work in the laboratory – this type of product is made on the production line in the Smart Factory lab. The proposed software uses modules and subprograms of the OMRON hardware and software manufacturer. The main purpose of the inspection software is to check several critical elements of the product. Yellow (dashed line) indicates critical elements tested for the correct color of the blocks, while white for elements tested for the correct shape (Fig. 4).

Fig. 3.
figure 3

Basic configuration scheme.

Full size image

The developed software consists of two modules used for two cameras with the Camera Switching option (Fig. 5). The first module is the basic module (Fig. 5, 0–4) responsible for proper image capturing and subsequent processing, i.e., it is responsible for performing basic software preparatory functions.

Fig. 4.
figure 4

Tested object.

Full size image
Fig. 5.
figure 5

Inspection software implemented in the OMRON environment.

Full size image

The second module (Fig. 5, 5–7) aims to carry out a proper optical inspection of the object. It compares and assesses the color of specific blocks using Colour Data, checks the shape of the marked critical elements of the product, and compares the captured image with the reference model pixel by pixel using Fine Matching.

Camera Image Input FH forms the basis of the entire software. After proper configuration, it captures the image of an object, ensuring adequate focus and exposure. It also allows narrowing the captured image to a minimum, which speeds up further work of the entire software. The incorrect setting of capture parameters can lead to distorting the entire test results.

Configuring both cameras is the basis for creating each inspection program using software provided by the producer. Each camera requires a separate parameter setting, including:

  • Manual setting of the aperture and image focus using the moving parts of the camera;

  • Camera settings-Camera shutter – camera shutter speed and Gain – granularity;

  • Number of lines to be read – an option that allows to reduce the captured image.

  • White Balance;

  • Calibration – allows the measurement data to be converted into real dimensions, e.g., using the three-point method (using the Specify Points option – one should mark three points selected by the user on the screen (Fig. 6), and then provide their actual position; the software will calculate the calibration parameters).

Fig. 6.
figure 6

Example of calibration using Specify Points.

Full size image

The next step in the development of optical inspection software for a selected product is the addition of the Shape Search 3 subprogram, which allows to quickly search for the contour of the tested object, along with the detection of interferences. It consists in specifying in the settings the reference contour (in the Model and Region Settings tabs) to which it will compare the products and setting a detection point, which, if possible, should be located in the center of the object.

Shape Search III is a function that allows registering the object based on its contour, taking into account interferences. It allows specifying the degree of acceptable correlation. It enables fast and accurate detection of the tested object.

Position Compensation allows for correcting the image of an object when it is turned to the reference image. In more precise terms, it compares the measured coordinates of the object and compares it with the proper coordinates, and in the case of displacement, it corrects the position of the object by the difference observed. It should be emphasized that if the object exceeds the frames set in the Camera Image Input FH, the inspection result may be negative.

Background Suppression allows additional narrowing of the tested area by “muting” the part whose brightness is lower or higher than assumed when configuring this option. Similarly to narrowing the tested area using Camera Image Input FH, this option also reduces the work time of the entire program.

A Trapezoidal Correction is an option that allows the correction of shape errors of the examined object. These errors occur when the lens captures the image. They mainly result from the fact that the cameras are not placed directly above the product in a perpendicular way to the product base. They are located on a metal structure, thanks to which they capture the image at an angle, making the object illusively resemble a trapezoid, not a rectangle or square. It is an important option for examining the correctness of the object’s shape critical features.

As it was mentioned, the second proper part of the created program is the object inspection module (Fig. 5, 5–7). The task of this module is to perform the optical inspection of a given object. It uses subprograms that allow controlling the colors and shape of individual blocks and the entire product.

The Colour Data function allows assessing the correctness of the color of the blocks. The assessment is carried out by calculating the average color of a given area (block) and comparing it to the color of the reference element. In this case, lighting and a variety of marked blocks have a significant impact on the measurement and assessment of color. If two blocks of different colors are marked as one area, then the permissible color can be averaged in such a way that the measurement result and assessment will be false.

The Search function allows searching for the desired shapes/components of an object in the previously marked area of interest. It allows determining if an element of a certain shape is in the right place. The inspection result depends on the degree of correlation specified by the user.

Fine matching is a subprogram that compares the reference image with the image captured by the camera pixel by pixel. The assessment of the object’s correctness is carried out by comparing half of the surface of the object’s elements that differ from the reference pattern to the allowable size of the different fields. It is worth noting that the subprogram is very sensitive to shifts in the position of the examined object.

Table 1 presents the parameters and their values that were adopted (based on own empirical research in the Smart Factory laboratory) and set during the configuration of subprograms, i.e., functions of the developed inspection software for the OMRON inspection station.

In order to estimate the time necessary to be reserved on the production line for quality control, many measurements of the duration of a single inspection were made. Forty measurements of the duration of optical inspection were made depending on different light intensity (Fig. 7), and basic statistics were calculated for the duration of inspection (average value, median, standard deviation, and the maximum and minimum values were determined).

The measurement results, presented in Fig. 7, show that the duration of inspection is not affected by the change in light intensity. The longest inspection time was 518 ms, the shortest 460. It is worth noticing that the times are significantly exceeding the average value, i.e., 476.14 ms were obtained after prolonged program inactivity. The standard deviation is 6.37 ms, and compared to the average value is very small. The most common measurement time is 473 ms, and the median value is 475 ms.

By implementing the OMRON station in the Smart Factory production line, it can be assumed that approx. 476 ms should be spent on performing an inspection in the program. It should be kept in mind that the measurements presented in the tables do not include the time during which the production line was stopped.

Table 1. Parameters of subprograms of the developed inspection software.
Full size table
Fig. 7.
figure 7

Measurement results – duration of optical inspection depending on light intensity.

Full size image

The line for scientific and didactic purposes in the Smart Factory laboratory was built by FlexLink. The line allows the manufacturing of finished products consisting of blocks in diverse variants. The production line (Fig. 8) consists of four loops enabling the product to be transported to subsequent stations arranged along with it.

Each of the loops in Fig. 8 has crossovers enabling the product to be redirected to any transport loop. The product moves along the line on a pallet marked with an RFID tag, which allows it to be identified. There are RFID reading heads in front of each site and workstations that allow changing the direction of the route. A control panel controls the entirety. Assembly stations are independent organizational units responsible for the production of components of finished products. Each station is equipped with flow racks which allow to store containers with parts and components for assembly. Managing the production system is carried out using the 4Factory IT system, and communication between system elements is carried out via the Internet of Things (IoT).

Fig. 8.
figure 8

Smart Factory production line with the place for an inspection station (area outlined in green).

Full size image

This place was chosen due to the fact that if the station is moved to one of the workstations, there is a significant risk of creating an unwanted buffer.

Moreover, the indicated area contains an RFID reading head which allows to control and identify the product.

5 Conclusions

The aim of the work was, above all, to determine the parameters of an optical inspection station on the automatic production line in the Smart Factory laboratory. The starting point for achieving the set objective was to perform a concise analysis of literature on quality control and to become familiar with the functioning of software provided by the hardware producer. Eventually, an inspection program was created and described in the article.

The most important conclusions, useful for physically introducing an optical inspection station in the production line:

  • Inspection time for a single product is approximately 480 ms and should be extended by the time needed to stop the product. However, this should not be a problem because the production line software makes it possible to manually set the rhythm of individual operations and the speed of product movement.

  • The Fine Matching option used in the program, which analyzes the image pixel by pixel, is very susceptible to changes in the position of the examined product. This problem should be kept in mind, especially due to the malfunctioning of transfer pallets, which often rotate when moving on the production line. It is a mechanical problem that needs a solution to ensure the proper functioning of the entire line because robots are unable to assemble the finished product properly.

It is worth emphasizing that after introducing the station in the production line, the parameters adopted during the software configuration need to be checked and may need to be corrected due to the change in the environment (measurement conditions).