Keywords

1 Introduction

Technological production is growing today at a very high rate, which drives electronic equipment manufacturers into a race for the supply of electronic components at a high rate. Printed circuit boards (PCBs) are an electronic component widely used globally and represent a highly developed market [1].

Increasingly, PCB boards have a reduced size and an increase in electronic components on them one of the main problems affecting the production of PCBs is the fact that there are not many quality analysis systems in the electrical quality testing phase (circuit test systems - ICT systems) in the automotive industry [1].

This lack of analysis/quality check leads to plates being rejected or left on standby in order to be repaired at a later stage [1].

The big problem lies in the fact that PCBs are still mostly produced by hand. This leads to failures in the manufacture of ICT systems in the final product. Often these needles can suffer damage caused by the handling systems or transport systems of the same [1].

ICT systems consist of a test plate (DUT) and a set of needles that are placed on it. The needle bed consists of a test plate that can be of different sizes and has several holes. Here in these holes the receptacles and needles will be inserted. On the other side of the test plate, the set consisting of the receptacle and the needle, is coupled to a wiring system for electrical connections.

This wiring system allows electrical parameters to be measured on the PCB and a connection between software and printed circuit board [1]. Figure 1 shows each side of the test plate in detail: the needle bed and the wiring system, respectively [1].

Fig. 1.
figure 1

Test plate [1].

Full size image

The needles are manually inserted into the receptacles, which serve as a guide for the needles (see Fig. 2). This manual manufacturing process can take longer than a week depending on the type of PCB being tested. This being a repetitive process that requires operators to have a high sense of responsibility and level of concentration, mistakes can happen with the wrong choice of needles.

Fig. 2.
figure 2

Needle beds [1].

Full size image

Luís Freitas et al. (2021), defined a set of defects that occur in the manufacturing process in PCBs and that affect the productivity and quality of the final product, namely [1]:

  • Perpendicularity between the receptacle axis and the test plate [1];

  • Precision placement of the receptacle [1];

  • Defects that may be present in the needles according to their entire routing process until they are placed by the operators on the PCB (this is where this article focuses) [1];

  • The angle and force applied by the technician to the receptacles [1];

This work focuses on the process of analyzing the quality of the needles through the implementation of an artificial vision system that will allow a correct insertion of good quality needles in the receptacles, thus avoiding the rejection of PCBs.

To achieve the proposed objectives, the article is organized as follows: Sect. 2 presents a brief description of systems that exist on the market and that use vision systems to detect defects in needles. Section 3 describes how an artificial vision system works and what equipment is used in industrial applications; Sect. 4 presents the vision system that is used for needle quality analysis; Sect. 5 presents the main results, focusing on the most important aspects and also the techniques used to obtain them. Finally, the last section presents the conclusions summarizing the objectives achieved with this solution and the steps to be taken in the future.

2 State of Art

In order to present a conceptual solution to solve a problem related to the quality of needles, this chapter presents in Subsect. 2.1, a description of current systems and in which manufacturing processes are most used in the medical industry.

In Subsect. 2.2, applications with vision systems in the analysis of needle quality in the textile industry are presented.

Finally, in Subsect. 2.3, more applications in other industries with vision systems in the analysis of needle quality will be presented.

2.1 Needle’s Vision Systems in Medicine Applications

Balter et al. (2017) presented a paper on how to explained a series of motion tracking experiments using a vision system in conjunction with an ultrasound system and force detection to guide the position and orientation of the needle tip (see Fig. 3). The system enabled automated venipuncture, which provides and combines 3-D NIR and US imaging software, computer vision and image analysis, and a 9-DOF Needle Manipulator in a portable shell. The device operated by mapping the 3-D position of a selected vessel and introducing the needle into the center of the vein based on real-time image guidance and force [2].

Fig. 3.
figure 3

Needle tip positioning studies [2].

Full size image

2.2 Needle’s Vision Systems in Textile Industry Applications

Zhang, Zhouqiang et al. (2020) presented a paper on how to explained a needle detection system of a hosiery machine based on machine vision (see Fig. 4). A system was then created to detect problems with knitting needles quickly and accurately based on machine vision [3].

The system can detect problems with the knitting needle, and the needle causing the problem can be identified as the beginning of the fabric defect. In image processing, the vertical projection algorithm is introduced into the system, and the image binarization defect is improved [3,4,5].

Fig. 4.
figure 4

The principle of system operation [5]

Full size image

2.3 Needle’s Vision Systems in Other Applications

Jing Yang et al. (2020) presented a paper explaining how to a stable and accurate real-time tiny parts defect detection system and solve the problems of manual setup of conveyor speed and industrial camera parameters in defect detection for factory products. 0.8 cm darning needles were used as an experimental object.

The four types of defects on 0.8 cm darning needles, which are often used in actual production, they are crooked shapes, length size errors, terminal size errors and twist. An industrial vision camera was used in this project to detect defects (see Fig. 5) [6].

Fig. 5.
figure 5

Autonomous collection of experimental data with needles [6].

Full size image

At the industrial level, there is currently a gap in industrial PCB manufacturing systems using artificial vision systems to detect defects in needle beds, and it is important to assess whether they can be embedded in PCBs.

3 Vision Systems

3.1 Introduction

Vision systems can be broadly defined as all industrial and non-industrial applications, which through a combination of certain hardware and software components, provide operational directives or inputs to mechanisms/systems based on image capture and interpretation [7].

It is immediately possible to associate terms such as artificial vision and computer vision to these systems, sometimes even wrongly identified as the same, vision systems try to combine all the existing development in these areas in a practical way.

Thus, a vision system needs to take into account all aspects inherent to the inspection process, image capture, lighting, image processing, communication protocols, etc. Therefore, vision systems are defined as the integration engineering of mechanical, optical, electronic and software systems essential for examining objects, products, manufacturing processes and production lines, with the aim of detecting defects, improving their quality, offering more efficient manufacturing processes and production lines and insurance.

They also play a key role in the control of equipment used in manufacturing and are currently used successfully in several high-volume markets, for example, in the control and verification of the correct placement of components on printed circuit boards, plate reading and security (recognition facial, fingerprints and signatures).

The advantages that vision systems guarantee, when well designed and applied, in an industrial environment are mainly:

  • Repeatability, capable of providing production lines or manufacturing processes with consistent and reliable inspections for 24 h, 7 days a week without being affected by external constraints, such as fatigue;

  • Speed of operation, on a production line a vision system can inspect hundreds and sometimes even thousands of objects/products per minute;

  • Accuracy and high inspection detail, in some production lines and manufacturing processes there are details impossible to inspect by the human eye, vision systems when correctly designed (correct camera resolution and optical lens) are able to inspect these same details with facility;

  • Non-contact inspection, in the industry where inspection without sample contamination is indispensable, for example, pharmaceutical and food industry, vision systems present themselves as a dominant inspection method;

  • Operation in hazardous environments, vision systems can operate in extreme environments, high temperature, toxic or radiation harmful to human beings;

  • Eliminates costs associated with the wear of mechanical components of quality control systems and their maintenance time;

  • Reduces the need for human labor in production lines and manufacturing processes, increasing safety and profitability;

All these advantages are combined with the objective of improving product quality, reducing waste, increasing profitability and, finally, increasing industry profits. A well-designed industrial vision system requires robustness, reliability, high precision, mechanical stability and adequate cost.

Vision systems are made up of five main components, lens or objective, camera, lighting, image processing unit and communication interface, all of which depend on the object, product or process that will be inspected by the vision system.

3.2 Types and Classification of Vision Systems

An artificial vision system has several components, from the camera that captures an image for inspection to the processing engine itself that renders and communicates the result (see Fig. 6) [7].

The goal of good lighting is to improve and stabilize the image of the object to be analyzed. Thus, lighting correctly can be the key to success. To choose good lighting, both the material of the object to be inspected and the properties of the light source must be taken into account [7].

The functionality of a lens within the artificial vision system is to transfer a clean image of the object that we want to inspect to the sensor element (CCD or CMOS).

There are different valid technologies to capture electromagnetic energy and transform it into an image that can be treated. CCD technology is mostly used although CMOS is gaining ground day by day.

The processor is in charge of capturing the images and applying the appropriate computer treatments to obtain the relevant information in each case [8].

Fig. 6.
figure 6

Schematic diagram of machine vision system.

Full size image

Below, we make a more detailed analysis from a different types of system visions:

Vision sensors: includes different types of cameras and light sensors at all frequencies, 2D and 3D sensors [9].

Smart cameras: in addition to sensors, they include a processor capable of analyzing images and providing results to make decisions.

Advanced vision systems: These are devices, such as cobots, that integrate artificial vision systems to perform specific tasks [10].

At the level of classification of vision systems, these can be the following:

  1. a)

    light source: bright field, dark field and backlight. Each of these categories refer to the position of the light source in relation to the plane of the object to be illuminated, which results in different angles of incidence.

  2. b)

    lens characteristics: Focal Distance, Maximum aperture, Max sensor size, Distortion, Minimum focus distance, Type of Mount, Iris Adjustment and Focus Adjustment

  3. c)

    camera parameters:

    • Project in which it is inserted and future projects;

    • Technology to be used in the sensor;

    • Camera interface and communication;

    • Drivers and Software (SDK) available;

    • Working Distance (WD) and Field of View (FOV);

    • Minimum sensor resolution;

    • Lighting to which it will be subject;

    • Focus and Depth of Field;

    • Other camera features (Shutter, Color, FPS, etc.).

  4. d)

    capture capabilities: Color or Monochrome, FPS (Frames Per Second), Rolling Shutter vs Global Shutter and Environmental Working Conditions.

  5. e)

    processor characteristics: ARM type processors with interfaces for MIPI/CSI cameras, general purpose processors with x86/x64 architecture in BOX PC format and processors with architectures dedicated to intense graphics processing, such as:

    • GPU (graphics processing unit). NVIDIA and AMD platforms, multi-core architectures that facilitate parallel processing, critical to reducing latencies in image processing.

    • VPU (video processing unit)

    • GPU

    • Jetson

  6. f)

    validation algorithms: Leave-One-Out cross validation, K-Folds cross validation, Stratified K-Folds cross validation, Leave-One-Label-Out cross-validation and Random sampling with replacement cross-validation.

4 Applied Methodologies

4.1 Laboratory Acquisition System

To solve the proposed problem, an artificial vision solution for industrial application was developed. The image of the component to be inspected is taken in an environment suitable for it. The choice of lighting as well as the background color are important to obtain the best results (see Fig. 7).

Fig. 7.
figure 7

Laboratory acquisition system

Full size image

The system consists of a smart camera with a lens and filter attached to it. Also, a lighting system that the camera has is essential to obtain the best possible results. The needles are placed in a strategic position, within the camera’s field of vision. The network present in this project is the Ethernet [11, 12].

4.2 Vision System and Software

The FQ vision sensor (see Fig. 8) allows quick and easy real-time parameter adjustment. Eliminates the need to stop the machine for fine tuning and optimization of settings, resulting in zero machine downtime [13].

Fig. 8.
figure 8

FQ2 – smart camera from OMRON [13].

Full size image

The FQ vision sensor combines a high-powered processing unit and true color processing technology that allows fast inspections using color images. The same technology is used in Omron’s leading-edge vision sensor model, which is widely used in industry. This smart camera utilizes Omron’s high dynamics (HDR) processing technology, improving system dynamics up to 16 times over conventional vision sensors. The result is stable detection of objects that are highly reflectors, even when the placement of parts is not consistent.

The software used for this project was touch finder for PC (see Fig. 9) [13].

Fig. 9.
figure 9

Software touch finder for FQ2 – smart camera OMRON [13].

Full size image

4.3 Camera

The choice of the Smart Camera FQ2, for this project, was based in a comparative study of the more important existing ones on the market, presented in Table 1.

Table 1. Advantages and disadvantages of the Smart cameras chosen for the system.
Full size table

5 Analysis and Discussion of Results

5.1 Defects

A new system has been developed that can be used to inspect a variety of different types of needles and geometries fully automatically and can analyze defects in them.

This type of analysis allows a further identification of which needles do not meet the requirements to be used in the manufacture of PCB boards (see Fig. 10) [14].

Fig. 10.
figure 10

Needle reception plate with four sets of holes and four diameters [14].

Full size image

To carry out the insertion of the needle (see Fig. 11), it must not present defects for a correct insertion. For this, the needles must be arranged in a row so that the quality of the needles can be checked when passing through the vision system.

Fig. 11.
figure 11

Needle type for inspection [14].

Full size image

5.2 Needle Inspection

To carry out the inspection, the first step will be to obtain images of the needles with the smart camera in a position superior to them. For this we must adjust the camera resolution, through the image exposure time, gain and shutter speed.

After these steps, we get the following image (see Fig. 12).

Fig. 12.
figure 12

Needle image with camera adjusts.

Full size image

5.3 Techniques Used to Obtain Images

This project uses the Image filtering technique (Filter Items). We can filter the images that are taken by the Camera to facilitate measurement. This is used in the following cases:

  • Cut unnecessary funds so they are not measured;

  • To remove noise.

To stably find the edges of marks when other edges have been clearly extracted, gray color filters are used (only for sensors with color cameras). This allows converting an image that was input from a color camera into a monochrome image (see Fig. 13). In addition, we can use the technique called Background Suppression.

Fig. 13.
figure 13

Needle monochrome image (left) and needle with background removed (right).

Full size image

5.4 Obtained Results

Was used in the final results, the Edge Position Inspection. This inspection item is used to verify positions needles. For example, it can be used to see if a label or a product is attached at the correct position. In this case Places where the color changes greatly are called edges.

The positions of these needle’s edges are measured (see Fig. 14).

Fig. 14.
figure 14

Needle’s edges positions in measurement

Full size image

In this Teaching function, specifying the measurement region, Character color, Printing type and Correct string, the measurement parameters for OCR are set automatically.

Teaching means to store the region and partial image (see Fig. 15) as reference data for the measurement.

Fig. 15.
figure 15

Needle’s measurement region

Full size image

Shape Search II Inspection Item was used in this project (see Fig. 16). This function is for detecting user defined target to estimate target position and pose precisely. The correlation value indicating the degree of similarity, measurement target position, and orientation can be output. In shape search III, edge information is used as features, whereas in a normal search mode, color and texture information are used. It enables highly robust and fast detection robust to environmental variations including shadings, reflections, lightings, shape deformations, pose and noises.

Since state-of-the-art object detection algorithm is exploited in shape search II, it can provide much more reliable position and pose estimation with higher speed compared to shape search II (see Fig. 16).

Furthermore, it has much more parameter to tune to support a wider variety of applications.

Fig. 16.
figure 16

Shape Search II Inspection in needles.

Full size image

The needle search area can be expanded by adjusting the function called “Angle range”. The OK or NG Judgment is determined by correlation with the image pattern recorded for the search. Therefore, there may be an NG judgment result for a good workpiece if the correlation is low due to the angle being skewed. In this case, to get an OK judgment, increase the Angle range (see Fig. 17).

Fig. 17.
figure 17

The OK or NG Judgment in needles.

Full size image

6 Conclusions and Future Work

The implementation of artificial vision in the analysis of needle positions at the industrial level is currently low compared to other industrial sectors, such as in medicine with its use in conjunction with collaborative robots.

The strengths of this system proposal are: being innovative for this kind of industrial activity, low cost, light weight, easy portability and integration with other existing systems in companies.

The added value part of this study is the fact that, with this proposed system, companies will be able to save money, which comes from the total destruction of the PCB boards. Companies should choose this vision system, as it allows detecting problems in needle beds, and it is important to assess whether they can be embedded in PCBs, taking this process into account.

Currently, the presented technology, in this paper, leaves many promising aspects and allows the analysis of several research and development activities that have been carried out in research institutions and industrial companies.

Machine vision applications in the PCB assembly industry are not yet fully implemented. This is due to the simple fact that they are very specific manufacturing applications that have not yet been investigated and because it is an area that is still under development.

Future use of mechatronic needle routing systems is suggested for better image collection. The inspection on the board that receives the needles must also have a vision system that identifies which ones were correctly inserted and emits an image of those that must be replaced.

In conclusion, the use of artificial vision systems in the electronic PCB assembly industry is beginning to be used and in the coming years it will play an important role in the production and final quality of products, taking into account technological advances and market competitiveness.

The solution here presented makes it possible to eliminate the destruction of PCB boards due to the fact that the needles present defects, which will lead to a better quality in the production process and the consequent reduction of the current quality failures of the final product.

The implementation of this project aims to develop a new technological solution, capable of detecting problems with the needle beds, namely in obtaining the characteristics of each one, such as diameter and length, which are important parameters to consider in this process, among others.