1 Introduction

Two famous mid-sized robot soccer competitions, the Federation of International Robot-soccer Association (FIRA) and RoboCup, have certain robot behavior requirements. In these competitions, the robots must make decisions and move autonomously. To meet these requirements, self-localization has become increasingly important recently. Self-localization faces the challenges of the localization times, but also spots on the result’s preciseness. In this study, we use the FIRA competition as our design goal. The size of the FIRA RoboSot competition field is 6 m × 4 m (width × length) surrounded by a 75-cm border. The field is marked with 5-cm-wide white lines, as shown in Fig. 1a. The robot, with dimensions of 40 cm × 40 cm × 70 cm (width × length × height), is equipped with an omni-vision camera and a high-performance computer. The robot is also equipped with an omni-moving platform that has four high-power motors, a ball-holding mechanism, a ball-kicking mechanism, and FPGA controllers, as shown in Fig. 1b.

Fig. 1
figure 1

a FIRA Robosot competition field, b robot

Full size image

Some studies [1–5] have used the field geometry, such as goal posts or white lines, to calculate the position of a robot. White-line information has been used with the Monte Carlo method [2, 6, 7] and particle filters [8, 9] to localize a robot’s position. However, the computation time is greater than 10–200 ms due to the number of iterations. Chiang [3] proposed a white-line pattern match localization method. The white-line data of each image of the soccer field obtained by a robot were matched with simulated models pre-built in a database to obtain localization results. However, if the white lines on the field are incomplete or there are many points in the database that have the same compared error as that of the white-line pattern extracted from the image, localization will be incorrect. These limitations and poor reliability make robot localization with white-line patterns ineffective in real competition.

Therefore, some studies have employed encoders to localize the mobile robot [10–12]. However, encoder localization may cause errors if the wheels skid on the surface. Therefore, avoiding the effects of light on the vision system and skidding on the surface by the motion system are the goals of this study. In this study, we look for a reliable algorithm to accommodate white-line pattern match localization [3] and avoid significant error when white-line data in the field interfere with the environment. Both vision and motion can be integrated into an algorithm to support the effective localization of the robot. How to integrate vision and motion system, the fuzzy system is applied in the decision making system. The fuzzy system is a rule-based system that can rely on the practical experience and it has been applied in mobile robot to perform navigation [13, 14], target tracking [15], and trajectory tracking [16], etc. Thus, we propose a self-localization system that uses vision and motion information and a fuzzy system to search surrounding points to localize the robot in the integration process. The proposed scheme can operate within 30 ms with an accuracy of 6- to 10-cm deviation in a 6 m × 4 m field. The remainder of this paper is organized as follows. The algorithm is proposed in Sect. 2, experimental results are presented in Sect. 3, and conclusions are given in Sect. 4.

2 The Proposed Algorithm

In this section, we introduce the self-localization system that uses vision and motion information and a fuzzy system to search surrounding points to localize the robot in the integration process. The system diagram is shown in Fig. 2, the system use the image localization method to find the location of the robot. If the position error is greater than the threshold value, then the encoder localization method is employed. Both method provide the location for robot, the fuzzy system uses the moving speed of the robot to output the search range for image localization. More details of the system will be discussed in the following section.

Fig. 2
figure 2

System diagram

Full size image

2.1 White-Line Pattern Match Localization

The field in Fig. 1a is divided into a 75 × 55 grid points. On each grid in the system model, we calculate the distances between the first four white-line pixels from all 360-degree directions and the center of the grid point. For example, in Fig. 3, 360 scan lines were measured to calculate the distance (pixel) of the first four white pixels. The feature vector of this grid point is defined in (1):

$$P_{\text{M}} (X_{i} ,Y_{j} ) = P_{i,j} (D_{0} ,{\kern 1pt} {\kern 1pt} D_{1} ,{\kern 1pt} {\kern 1pt} D_{2} {\kern 1pt} , \ldots ,{\kern 1pt} D_{359} ),$$
(1)

where \(P_{\text{M}}\) is the position from the simulation system model, is the ith column and jth row of the grid point, and \(\left[ {D_{0} ,{\kern 1pt} {\kern 1pt} D_{1} ,{\kern 1pt} {\kern 1pt} D_{2} {\kern 1pt} , \ldots ,{\kern 1pt} {\kern 1pt} D_{359} } \right]\) are the vector distances of the first four white pixels from 0, 1, …, 359° to the center of the grid, respectively.

$$D_{k} = \left[ {D_{k}^{1} ,D_{k}^{2} ,D_{k}^{3} ,D_{k}^{4} } \right],\quad k = 0, \ldots ,359.$$
(2)

Here, \(D_{k}^{n} ,n = 1, \ldots ,4\) are the first, second, third, and fourth white pixels in line D k .

Fig. 3
figure 3

First four white-line pixels vector

Full size image

In the next step, we obtain an omni-directional image of the soccer field and attempt to match it to the simulated models in the database. Here, assume that the image of the field from the omni-directional camera is taken at location P O . Then, the distances between the center of the image and the first four white-line pixels in all directions are expressed as follows (3):

$$P_{O} (X,Y) = P(d_{0} ,{\kern 1pt} {\kern 1pt} d_{1} ,{\kern 1pt} {\kern 1pt} d_{2} {\kern 1pt} , \ldots ,{\kern 1pt} d_{359} ),$$
(3)

where \(d_{0}\), \(d_{1}\),\(\ldots\), \(d_{359}\) are the distances (pixel) of the first four white pixels from 0, 1, \(\cdots\), 359 degrees to the center of the robot and defined in (4).

$$d_{k} = [d_{k}^{1} ,d_{k}^{2} ,d_{k}^{3} ,d_{k}^{4} ],\quad k = 0, \ldots ,359,$$
(4)

where \(d_{k}^{n} ,n = 1, \cdots ,4\) are the first, second, third, and fourth white pixels in line d k . The omni-vision of the robot is shown in Fig. 4a, the corresponding scan lines to find the first four white pixels of the robot are shown in Fig. 4b, and the simulated system model pre-built in the database is shown in Fig. 4c. Then, the error between image positions and \(P_{\text{M}} (X_{i} ,Y_{j} )\) for all grid point \((X_{i} ,Y_{j} )\) is defined by (5).

$$\begin{aligned} E_{i,j} & = \left| {P_{i,j} \left( {D_{0} ,{\kern 1pt} D_{1} ,{\kern 1pt} {\kern 1pt} D_{2} {\kern 1pt} , \ldots ,D_{359} } \right) - P\left( {d_{0} ,d_{1} ,d_{2} {\kern 1pt} , \ldots ,d_{359} } \right)} \right| \\ & = \left( {\sum\limits_{k = 0}^{359} {\sum\limits_{n = 1}^{4} {\left| {D_{k}^{n} - d_{k}^{n} } \right|} } } \right). \\ \end{aligned}$$
(5)
Fig. 4
figure 4

a White line of the field captured by camera; b white-line pattern transferred from pixels to distance; c white-line pattern of a point in pre-built database

Full size image

The white-line data of each image of the soccer field obtained by the robot are extracted and matched to the simulated models in the database to obtain localization results. The location with minimum error E i,j between the simulated model and the true field markings is the image location result \(P_{I} \left( {X,Y} \right)\) in (6).

$$P_{I} \left( {X,Y} \right) = \hbox{min} \,E_{i,j} ,\quad i \in A, \quad j \in A,$$
(6)

where i and j are the ith column and jth row of the grid point in the search range A, respectively (Figs. 3, 4).

2.2 Encoder Feedback Circuit

In this study, we use a quad two-channel motor encoder with 500 counts per turn equipped at the back of each motor. When the motor rotates, the wheel inside the encoder turns and generates two digital pulses, as shown in Fig. 5. We employ a feedback circuit on an FPGA development board (myRio, National Instruments). The FPGA has four feedback circuits, each with a frequency divider, trigger module, the encoder’s two-channel counter, encoder direction detection, and a data register. The frequency divider reduces the myRio oscillator from 40 MHz to 100 Hz. The trigger module works on this 100-Hz clock to generate enable and clear signals to the two-channel counter and data registers in a period of 10 ms. The two-channel counters of the encoder are calculated at the end of the period.

Fig. 5
figure 5

Encoder signal chart

Full size image

2.3 Mathematical Model Review

According to the mathematical model of the four-wheel omni-moving platform, the speed of each wheel is defined by (7) and is shown in Fig. 6. Here, V i is the rotation speed of the ith wheel, r is the radius of the wheel, and ω i is angular velocity.

$$V_{i} = r\omega_{i} ,\quad i = 1, \ldots 4,$$
(7)
Fig. 6
figure 6

Wheel variables

Full size image

From Figs. 7 and 8, we decompose the speed of wheel 2, V 2, to the X-axis and Y-axis, as expressed by (8):

$$V_{2} = r\omega_{2} = - \sin (\theta_{m} )V_{x} + \cos (\theta_{m} )V_{y} + R\omega_{R} ,$$
(8)

where V x and V y represent two axes of the robot’s center to the field moving velocity, ω R is the angular velocity of the robot, θ m is the angle of the motor equipped on the robot’s base board (45° on our robot), and R is the distance between the robot’s center and the wheel. The velocities of wheels 1–4 are expressed by (9).

$$\left[ {\begin{array}{*{20}l} {V_{1} } \hfill \\ {V_{2} } \hfill \\ {V_{3} } \hfill \\ {V_{4} } \hfill \\ \end{array} } \right] = \left[ {\begin{array}{*{20}l} {r\omega_{1} } \hfill \\ {r\omega_{2} } \hfill \\ {r\omega_{3} } \hfill \\ {r\omega_{4} } \hfill \\ \end{array} } \right] = \left[ {\begin{array}{*{20}l} { - \sin (\theta_{m} )} \hfill & { - \cos (\theta_{m} )} \hfill & R \hfill \\ { - \sin (\theta_{m} )} \hfill & {\cos (\theta_{m} )} \hfill & R \hfill \\ {\sin (\theta_{m} )} \hfill & { - \cos (\theta_{m} )} \hfill & R \hfill \\ {\sin (\theta_{m} )} \hfill & {\cos (\theta_{m} )} \hfill & R \hfill \\ \end{array} } \right]\left[ {\begin{array}{*{20}l} {V_{x} } \hfill \\ {V_{y} } \hfill \\ {\omega_{R} } \hfill \\ \end{array} } \right].$$
(9)
Fig. 7
figure 7

Velocity of four-wheel omni-moving platform

Full size image
Fig. 8
figure 8

Vector analysis of wheel 2

Full size image

2.4 Localization Calculation

The FPGA circuit returns each motor encoder’s signal count C i in a sample period T seconds. Thus, the rotation distance of each wheel D i can be calculated by (10) using the ratio of count C i to the resolution of the encoder res. Then, it multiplies the circumference \(2\pi r\) and time interval T.

$$D_{i} (T) = \frac{1}{N} \times \frac{{C_{i} }}{{{\text{res}} \times 0.01}} \times (2\pi rT),\quad i = 1, \ldots ,4,$$
(10)

where N is the reduction ratio defined in the motor specifications. This constant shows the difference between the motor feedback and the real wheel rotation speed. To obtain the direction of the robot, we use a gyroscope to calculate the angle difference between times T j and T j−1. Therefore, the angular velocity vector distance can be represented by the arc length in (11).

$$d = \theta_{g} \times \frac{\pi }{180} \times \frac{R}{2},$$
(11)

where d is the angular velocity-distance or arc length we wish to obtain and θ g is the difference between the current and previous gyroscope reported yaw, as shown in Fig. 9.

Fig. 9
figure 9

Yaw rotation of the robot

Full size image

Next, we subtract the angular velocity-distance from the wheel’s distance calculated by the motor rotation speed to obtain linear velocity-distance S i in (12).

$$S_{i} (T) = D_{i} (T) - d,\quad i = 1,..,4.$$
(12)

Then, we can obtain each motor’s two axes distance using trigonometric function operations in (13). This is illustrated in Fig. 10.

Fig. 10
figure 10

Vector analysis of wheels distance

Full size image
$$\begin{aligned} S_{1x} (T) & = - \cos (\theta_{m} )S_{1} (T) \\ S_{1y} (T) & = - \sin (\theta_{m} )S_{1} (T) \\ S_{2x} (T) & = - \cos (\theta_{m} )S_{2} (T) \\ S_{2y} (T) & = - \sin (\theta_{m} )S_{2} (T) \\ S_{3x} (T) & = - \cos (\theta_{m} )S_{3} (T) \\ S_{3y} (T) & = - \sin (\theta_{m} )S_{3} (T) \\ S_{4x} (T) & = - \cos (\theta_{m} )S_{4} (T) \\ S_{4y} (T) & = - \sin (\theta_{m} )S_{4} (T) \\ \end{aligned}$$
(13)

Finally, by combining the four wheel’s two axes distances in (14), we can obtain the moving distance of the robot in sample time T for both X- and Y-axes. Thus, the total moving distance of the robot within sample duration t is summarized in (15). Here, P x and P y are the localization result, and θ e indicates the moving direction of the robot obtained by encoder localization.

$$\begin{aligned} S_{x} & = \frac{1}{2}\sum\limits_{i = 1}^{4} {S_{ix} (T)} \\ S_{y} & = \frac{1}{2}\sum\limits_{i = 1}^{4} {S_{iy} (T)} . \\ \end{aligned}$$
(14)
$$\begin{aligned} \theta_{e} & = \tan^{ - 1} \left( {\frac{{S_{y} (T_{j} )}}{{S_{x} (T_{j} )}}} \right) + \theta_{g} \\ P_{x} & = \sum\limits_{j = 0}^{t} {\left[ {\sqrt {S_{x} (T_{j} )^{2} + S_{y} (T_{j} )^{2} } \times \cos (\theta_{e} )} \right]} . \\ P_{y} & = \sum\limits_{j = 0}^{t} {\left[ {\sqrt {S_{x} (T_{j} )^{2} + S_{y} (T_{j} )^{2} } \times \sin (\theta_{e} )} \right]} \\ \end{aligned}$$
(15)

2.5 Integration Algorithm with Fuzzy System

In this section, we introduce the integration algorithm that combines white-line pattern match localization and encoder localization. Initially, the system uses white-line pattern match localization to search the pre-built database for the current robot location on the field to obtain the image localization result \(P_{I} \left( {X,Y} \right)\) that minimizes the matching error.

After obtaining the result with error E(P I (X, Y)) less than E TH (assume E TH is 20), the system will set this location as the point of origin for the encoder localization system and will accumulate the robot’s moving distance. In this process, the encoder localization system will return results for each period to the white-line pattern match localization system. If E(P I (X, Y)) is greater than E TH, the system will adjust the robot’s location given by the encoder localization system and limit the local search area in the next period (Fig. 11).

Fig. 11
figure 11

Fuzzy system block diagram

Full size image

Next, we define a fuzzy system for local search to localize the robot’s position. The fuzzy system uses the robot’s moving speed as input and the search range as output as shown in Fig. 11. The system’s membership function for moving speed is illustrated in Fig. 12, where VL, L, H, and VH denote very low, low, high, and very high, respectively. The membership function of search range A is shown in Fig. 13. From the if-then rule, we can obtain the output, search range A. The rules are as follows:

Rule1:

If v is VL, then A is VL

Rule2:

If v is L, then A is L

Rule3:

If v is H, then A is H

Rule4:

If v is VH, then A is VH

Fig. 12
figure 12

Membership function for the robot’s moving speed

Full size image
Fig. 13
figure 13

Membership function for search range A

Full size image

Hence, it is evident that there are four search ranges (0, 10, 20, and 30). The corresponding ranges and search points are plotted in Fig. 14. The black circle indicates the localization result and a red square in the database for those points marked in the white circle are the areas to calculate the white-line pattern match. The result is the location with the smallest error in the range. The computation times for each search area are summarized in Fig. 14. The fuzzy system can determine the local search area and obtain a search result in 0.1–1.2 ms. These results are used to compare the efficiency of the proposed system to other methods (Sect. 3).

Fig. 14
figure 14

Comparison range for local search and computation time

Full size image

However, the encoder’s feedback has accumulated error and the encoder localization system will reset its point of origin in 125 ms. The integration algorithm is shown in Fig. 15.

Fig. 15
figure 15

Integration algorithm flowchart, (*The white-line pattern match localization omitted as image localization)

Full size image

In Fig. 15, the system uses global white-line pattern search to find the position P IG(X, Y). If the pattern match error is less than the threshold E TH, the fuzzy system will perform a local search and obtain the result P IL(X, Y). If the search result error is greater than the threshold E TH, the system will use the localization result from the encoder, i.e., P IL(X, Y) ← P E (X, Y). Similarly, if the reset time is up and the error for image localization result is less than the threshold E TH, then the system will use the localization result from the image, i.e., P E (X, Y) ← P IL(X, Y). With the integration algorithm, the system can reduce the effect of light on the vision system and the ground effect of the field on the motion system. The fuzzy system provides a dynamic search area relative to the speed of the robot; the higher the speed, the larger the search area and vice versa. The fuzzy system can reduce the computation time from 1.2 to 0.1 ms.

Next, the time required for localization is calculated. Assume the image capture rate is approximately 60 frames per second, and the image processing and white-line pattern match times are 12.9 and 0.4 ms (20 cm search range), respectively. The timeline of the localization process is shown in Fig. 16. If the pattern match error from the image localization is greater than the threshold, then the integration localization will modify the result using encoder localization. For example, in Fig. 16, the image localization result is less than the threshold in the 1st image and 3rd image; therefore, the integration scheme retains the image localization. However, the error in the 5th image is greater than the threshold. Therefore, the position obtained by the encoder is updated by the integration algorithm.

Fig. 16
figure 16

Integration algorithm timeline

Full size image

3 Performance Analysis

We chose 19 points from the FIRA field as test points to analyze the performance of integration localization. The test points were divided into two categories, i.e., positions inside the field and special points on the field. Figures 17 and 18 indicate the locations of the test points.

Fig. 17
figure 17

Nine test points from general cases

Full size image
Fig. 18
figure 18

Ten test points for boundary cases

Full size image

We evaluated the white-line pattern match localization and encoder localization separately. Then, we evaluated the performance of the integration localization algorithm. The results for white-line pattern match localization for general cases (Fig. 17) and boundary cases (Fig. 18) are listed in Tables 1 and 2, respectively. The results for encoder localization for general cases (Fig. 17) and boundary cases (Fig. 18) are listed in Tables 3 and 4, respectively. Finally, the results for integration localization for general cases (Fig. 17) and boundary cases (Fig. 18) are listed in Tables 5 and 6, respectively.

Table 1 Test results with image localization (general cases; Fig. 17)
Full size table
Table 2 Test results with image localization (boundary cases; Fig. 18)
Full size table
Table 3 Test results with encoder localization (general cases; Fig. 17)
Full size table
Table 4 Test results with encoder localization (boundary cases; Fig. 18)
Full size table
Table 5 Test Results with integration localization (general cases; Fig. 17)
Full size table
Table 6 Test results with integration localization (boundary cases; Fig. 18)
Full size table

3.1 Image Localization

Initially, the white-line pattern match localization algorithm is used. From Tables 1 and 2, the results show that the average error is approximately 9 cm for general cases and 212 cm for boundary cases.

3.2 Encoder Localization

Next, we tested the encoder localization. For each test point, we moved the robot to the center of the field to reset the point of origin. The results are presented in Tables 3 and 4. The average error is approximately 10 cm for general cases (Fig. 17) and 18 cm for boundary cases. It is evident that encoder localization is better than image localization for test points on the field boundary (Fig. 18).

3.3 Integration Localization with Image and Encoder

In integration localization, the system uses the algorithm shown in Fig. 12 to obtain the robot’s position. As shown in Table 5, the average error of integration localization for general cases is 6 cm. The average error of integration localization in Table 6 for boundary cases is 4–22 cm, and the average error is 13 cm. Thus, it is evident that integration localization can eliminate the effect of light on the vision system and does not accumulate errors due to encoder distance calculations.

3.4 Comparison with Other Work

To demonstrate that the proposed scheme has higher accuracy and higher efficiency than other systems, we compared the proposed algorithm to adaptive Monte-Carle localization (MCL) [6, 7], localization with enhanced particle filter (EPF) [8], and pattern matching algorithms [4, 5, 9]. For fair comparison, we neglected the frame rate and image processing time, and defined the computation time as the pattern match time using the fuzzy system. In Monte-Carle localization and localization with enhanced particle filter, the system chooses two hundred sample points in the simulation. The results are shown in Tables 7 and 8. The mean error of the proposed scheme was approximately 6 cm; however, the errors for the pattern matching and MCL algorithms were 20–88 cm. Similarly, we also compared the computation time for localization of the proposed algorithm to the MCL and pattern matching algorithms. The computation time for the proposed scheme was approximately 0.4 ms; however, the computation time for the pattern matching and MCL algorithms was 1.76–195 ms. Thus, the proposed integration localization has higher accuracy and efficiency than MCL or pattern matching schemes.

Table 7 Comparison of errors of proposed scheme with other methods
Full size table
Table 8 Comparison of efficiency of proposed scheme with other methods
Full size table

4 Conclusions

In this study, we have proposed a localization algorithm that can integrate the vision and motion systems of a robot. In the vision system, white-line pattern match localization is used to determine the initial location of the robot. Then, the distance and moving direction are calculated by encoder localization. Integration localization using a fuzzy system is proposed to reduce the effect of light on the vision system and the ground effect of the field on the motion system. Our experimental results show that a more reliable and precise location can be obtained by combining vision localization and motion localization. The results indicate that performance errors are less than 10 cm with 30-ms positioning time. The main contribution of this study is real-time computation of the localization scheme and accurate results. The proposed scheme can localize the robot and help design strategies for the robot in mid-sized FIRA and RoboCup competitions.