Keywords

1 Introduction

Machine vision belongs to the automation process where input information is obtained by recording images through industrial cameras. These cameras process the image by computing devices. Subsequently, algorithms are used through which it is possible to create different design applications for product inspection.

Recently, machine vision systems have become more efficient and affordable. Cameras alleviate time-consuming and increase accuracy over a human inspection. Machine vision through cameras is applied in various industries. It is used from inspection to process operations. Machine vision has also found application in various types of robots – cartesian robots [1], scara robots [2], snake robots, etc., because of their wide range of applications [3].

The article creates an inspection workplace in the Visual Components program for BROKKEM s.r.o. The requirement for implementing a machine vision system arose to reduce the time consumption of inspection. In the literature review, an analysis of the current state of the camera systems was made.

In the practical part, a workplace concept is designed, in which camera systems from the manufacturer COGNEX were used. The cameras were selected according to the specified parameters from the company, where there was a requirement for partial automation of the workplace - inspection requirements, working distances, choices of lighting and lenses, etc. In the 3D design of the workplace, specific models under investigation are shown, made on actual machines used in the workplace.

The paper’s contribution consists of designing an automated workstation for the inspection and assembly of a specific product, according to the company's requirements.

The superstructure to the presented solution is to replace the motion sensors placed in the robotic cell with a camera system that can detect the worker’s movement through artificial intelligence (computer vision). Automated production will be suspended if a worker moves in close proximity to the industrial robot.

2 Literature Review

Computer vision, as the automatic analysis of images and video by computers, has a wide range of applications - use in the engineering industry, food industry, medical industry, etc. A specific application of machine vision is, e.g., in checking the position of parts in automotive production, checking the use of correct parts or fasteners, categorizing objects, checking printed circuit boards, checking for defects on products, finding imprints on apples, detecting chocolates on a conveyor and placing them in a box by a delta robot, automatic reading of license plates on vehicles, the ability to record multiple players on the ice (during hockey) and then give a penalty to that team, security checks at the airport, inserting virtual objects into movies, helping drivers by warning them (vibrating the steering wheel) when they are drifting out of their lane, creating 3D models from old photos, classifying plant species, etc. In the case of a trained artificial neural network for detection and categorization of objects in an image, artificial intelligence can also be used for other types of digitized image processing - control of separation processes in multifunctional devices [4,5,6], application of artificial neural networks (ANN) in the copper ore mining process to increase efficiency and ensure stability [7], cutting force determination based on integrated application of fixtures [8].

Fig. 1.
figure 1

Machine vision applications – a) installation of screws [1] b) assembly of the USB key [2].

Full size image

Some of the published projects (solved at the Department of Automated Manufacturing Systems in Žilina - KAVS) in which machine vision was used are:

  1. a)

    application of a camera system (Sony XC-56 camera) for the detection and subsequent assembly of bolts in automotive production by means of the FANUC M-20iA/20M industrial robot (installation of screws into car seats). This application is shown in Fig. 1a [9, 10]

  2. b)

    assembly of the USB key through the FANUC M-1iA parallel robot, where the detection of the individual parts of the USB key is detected by the Sony XC-56 camera Fig. 1b [11]

  3. c)

    application of machine vision for tire defect detection, integration of camera systems into serial and parallel robot control, detection of adhesive buildup on the product, visual product inspection based on deep learning methods, and more.

Due to the experience gained in solving the above-mentioned projects (in which machine vision is used), we were able to work on solving the assignment from the company BROKKEM s.r.o.

The company focuses, among other things, on the production of components for single-purpose machines. There will also have to be a complete assembly for the latest order. Figure 2 shows the product, which will be manufactured, inspected, and assembled directly on a single automated line.

Fig. 2.
figure 2

Product covered by camera inspection.

Full size image

The company had only a few requests for the design of an automated workplace and the selection of suitable cameras to inspect their products. Company requirements are the following:

  1. 1.

    The automated workstation can't be larger than 12 × 16 m

  2. 2.

    Design of the location of the assembly stations for the product

  3. 3.

    Maximum number of workers - 3 persons

  4. 4.

    Camera systems must be used to control products

  5. 5.

    Only cameras from COGNEX may be used for product inspection

3 Research Methodology

3.1 Smart Cameras

Products are often inspected through visual inspection by human workers. Visual inspection by human operators can be considered a critical element susceptible to errors resulting, e.g., from fatigue, insufficient reaction speed, stereotyped process, etc. In certain types of tasks, this human cognitive function can be replaced by a 2D or 3D camera system. The inspection of products through machine vision (contour inspection, object categorization, etc.) is becoming more and more widespread in the industry.

Smart machine vision cameras differ from standard industrial cameras in that they have storage and processing capability built into the camera housing. A standard camera takes an image and then sends it to another device for storage and analysis - a smart camera doesn’t need to do this. I/O and communication are frequently built-in to smart cameras. It allows them to operate autonomously and to interface with other automation equipment. Smart cameras don’t need a computer to operate, which makes them compact and robust, which is why they are one of the most emerging machine vision cameras in the industry [12].

Fig. 3.
figure 3

Cameras with integrated lighting from COGNEX – a) In-Sight 7000 b) guiding the robot through the camera [13].

Full size image

Some types of smart cameras have built-in lighting, so there is no need for additional lighting as with peripheral machine vision sensing devices. LED lighting is the most common type of lighting used for smart cameras. Figure 3 shows cameras with integrated lighting from COGNEX. The lighting is placed around the lens.

3.2 Lighting

Humans have been exposed to lighting that is referred to as “daylight” since the beginning of time. Various factors affect the efficiency of a person working in a particular room, such as office furniture placement, the lack/abundance of air in the room, or even the lack/abundance of light. Before designing the workspace (where the person will work), all of the above factors should be considered, and daylight should be preferred to artificial lighting (if possible). Researchers have conducted various studies where they have found that artificial lighting (fluorescent, incandescent, etc.) increased worker fatigue. The worker made more mistakes, and a longer latency time of the motion response to the light signal has been demonstrated [14].

Choosing the right type of lighting application is very important for people and industrial cameras. Lighting placed in areas where employees perform certain types of work (e.g., assembly products) can significantly increase work efficiency. Improper use of lighting in the workplace can create, for example, a strobe effect, which can cause serious injury to an employee. The lighting for industrial cameras is selected according to the specific type of task – checking microchips on a computer motherboard, scanning the contour of a gear-wheel, scanning objects with a reflective surface, etc. One of the main tasks of proper object inspection on an automated workstation was precisely to evaluate the use of a particular type of lighting in a specific product inspection.

Figure 4 graphically depicts four types of lighting commonly employed in machine vision (LED, Quartz halogen, fluorescent, and xenon strobe).

Fig. 4.
figure 4

Common vision lighting sources are compared and contrasted [15].

Full size image

3.3 Specific Information of the COGNEX Cameras Used

Cognex’s vision sensors and 2D and 3D vision systems all use machine vision technology to perform inspections, but they're built for various purposes.

Comparison of Vision Sensor, 2D vision, and 3D vision cameras in terms of application is shown in Table 1. Visual tools of used cameras in the automated line design are shown in Table 2.

Table 1. Comparison of vision sensor, 2D vision, and 3D vision cameras, in terms of application: [13].
Full size table
Table 2. Visual tools of used cameras [13].
Full size table

4 Results

In this chapter, a conceptual design of the workplace is created (Fig. 5). The products of BROKKEM s.r.o. are digitally recorded, processed, and evaluated using camera systems (machine vision). The single parts to be assembled in the assembly process are machined by CNC machines [16,17,18]. The requirement arose for the following types of product inspection – (a) checking the bearing house, (b) inspection of the pressed-in bearing, (c) gear check, (d) palletizing through QR code. For all the above-mentioned product inspections, cameras from COGNEX were chosen, for which we also designed mounts (welded profiles) for the automated line. The entire conceptual design of the workplace was created in Visual Components [19, 20]. The dimensions of the designed automated workplace are approximately 8.5 × 10 m.

In the future, we want to implement all the above-mentioned cameras on an automated line at BROKKEM s.r.o. In the phase of testing the use of camera systems, products (in addition to correct and non-conforming products) with defined defects, such as a defect on a gear wheel (damage to a tooth), will be randomly placed on the conveyor belt. The de-signed system will remain implemented in production when the desired product control success rate is achieved. If it is necessary to make the robotic workplace more efficient, it is possible to design robot-human cooperation (collaborative robotics) when assembling a gear with a shaft [21].

Fig. 5.
figure 5

Robotic workstation with human operators designed in visual components.

Full size image

4.1 Checking the Bearing House

A Cognex In-sight 7500 camera (Fig. 6b) with planar scanning was used to inspect the bearing house. This camera has integrated 4000 K white LED illumination to save the cost of external light sources. The working distance of the subject was 604 mm, from which a 12 mm/f2.3 lens was chosen. The lens features self-focusing, which makes it easy to mount the system. The sensor is a CMOS with a pixel size of 4.5 × 4.5 m per pixel, and the image is taken at a resolution of 800 × 600 pixels in black and white. The shutter speed is 500 ms. Therefore, the checking procedure will not be slowed down significantly. An 8 gigabyte SD card can be inserted into the camera to record the time of the inspection and recorded images that have not been inspected for possible time information. The power supply of the camera is 24VDC 1.5 A. The device provides IP 67 protection, which is sufficient protection in a collision with a robot. Communication with the software is through an Ethernet cable connected to the In-sight Explorer software. The camera placement design for detecting the bearing house contour is shown in Fig. 6a [13].

Fig. 6.
figure 6

Checking of the bearing house - a) design of the camera placement for detection of the bearing house contour b) Cognex In-sight 7500 camera [13].

Full size image

4.2 Checking Bearing Installation

The In-sight 2000 camera (Figure 7b) was used to inspect the correct bearing installation by the operator. This is one of the most cost-effective camera sensors available. The camera employs the edge detection capability to inspect the bearing installation. If a bearing edge is caught in the image, the conveyor moves it out of the process. The camera has modularity, so it has integrated white LED lighting to reduce costs. The lens’s working distance is 200 mm, while the image’s field of view is 100 mm. The lens was chosen with an 8 mm focal length. The device uses a CMOS sensor whose size per pixel is 1.93 µm with an output image resolution of 800 × 600 pixels. The output image is black and white, and communication with the software is done through an Ethernet cable with the In-Sight Explorer software. The supply voltage is 24 V/DC. Checking bearing installation is shown on Fig. 7a.

Fig. 7.
figure 7

Checking bearing installation – a) design of camera placement for bearing assembly inspection b) Cognex In-sight 2000 camera [13].

Full size image

4.3 Gear Wheel Inspection Using Deep Learning

An In-sight D902M series camera (Fig. 8b) was used for the repeating tooth features on the gear with the serial number on the shaft face, supported by artificial intelligence over conventional cameras used in the workplace. The choice of this camera was suitable for checking surface scratches, burrs, number of teeth, and OCR (optical character recognition) at different positions. Artificial intelligence allows these defects to be detected at different rotations compared to the convex ones that require a precise position for sensing. The working distance is 362 mm, where the field of view is 120 × 120 mm from these parameters, a lens with a C-aperture and a focal length of 12.5 mm was chosen. The camera sensor is a CMOS whose size per pixel is 3.45 µm × 3.45 µm. The camera provides an image resolution of 1900 × 1200 pixels whose image is in black and white format. The storage memory of the camera is 16 gigabytes which can be expanded with an additional SD card up to a maximum of 8 gigabytes when pairing the camera with a local network is non-limited storage space. The process memory of the camera is 3 gigabytes. A GiGE interface attached to the In-Sight ViDi software facilitates communication. The supply voltage is 24 VDCLighting is provided by STEMMER IMAGING with CSS LAK type. This type belongs to the category of dark field lighting, which is ideal for searching for defects. Gear wheel inspection is shown in Fig. 8a [13].

Fig. 8.
figure 8

Gear wheel inspection – a) design of camera placement for gear wheel inspection b) Cognex In-sight D902M camera [13].

Full size image

4.4 Palletizing Products Through QR Code

For proper distribution of products by SCARA robot, a fixed QR code reader Dataman 260X (Fig. 9b) rectangular version was used for compactness of the system. The camera works based on Cognex 2Dmax and Powergrid algorithms. These algorithms make it possible to recognize codes even on curved surfaces. The device provides an image resolution of 752 × 480 pixels, which is a sufficient resolution at a working distance of 90 mm. The reader has a built-in CMOS image sensor. The camera offers 60 frames per second which it can decode in an average of 0.45 s. From the tabulated values provided by the manufacturer for the field of view of the QR code, which is 15 × 15 mm, a lens with a focal length of 6.2 mm was chosen, which can read 105 million characters from the QR code at the above working distance. The lighting is integrated in the camera, where a polarizing filter was chosen to highlight the QR code and reduce the shine of the metal material. Communication with the software is realized by an Ethernet cable connected to the In-Sight Explorer. The camera provides IP-65 protection [13].

Fig. 9.
figure 9

QR code scanning – a) design of camera placement for QR code scanning b) Dataman 260X camera [13].

Full size image

5 Conclusions

In the practical part of the article, at the company’s request, a conceptual proposal for the use of camera systems in a robotic workplace was developed. The practical part was realized in Visual Components. This proposal consists of the correct choice of industrial cameras that have been provided by the manufacturer Cognex. According to the inspection requirements, suitable cameras were selected within the application and parameters for the given component inspection.

The automated workplace design meets all the company's requirements and is ready for practical implementation in the company.