Keywords

1 Introduction

In recent years an increasing interest in the monitoring of operational parameters of mining machines, particularly in the monitoring of load haul dump (LHD) machines [1, 2]. The data about operational parameters may get additional context by adding data about the positioning of the vehicles in the mine. The registration of machine positions allows relating the current state of the machine (for example overheating or operational overloads) with the particular mining region and infrastructure or environmental conditions in this region. Also, accurate position monitoring of the vehicles significantly improves the safety of operations and can be used for example in anti-collision systems [3] or during the rescue operations in the mine in case of emergency [4]. Finally, position data can be used for the effectiveness assessment of the machines or operators. Unfortunately, the GPS signal, which is the most common method of tracking various objects is not available in the underground mine. It forces researchers to search for alternatives, one of which is inertial navigation. Inertial navigation systems for underground mining have been considered since the mid-1990s [5, 6]. In most cases, such systems were developed for autonomous navigation of LHD vehicles [7,8,9]. Also, some additional sensors, like laser sensors [10] or RFID anchoring [11] are usually used, which makes the system more expensive and harder to install. Advanced data analytics and the Internet of things (IoT) technologies nowadays penetrate different areas of industry [12]. Underground mining is not an exception. IoT technologies were used for example for ground monitoring [13], safety applications, or personnel tracking. Although the last one is also based on inertial sensors, the results cannot be transferred directly from personnel to mining equipment, because the algorithms based on step count estimation are often used, which means that dedicated algorithms for underground vehicles should be developed.

As it was shown in our previous article [13], the gyroscope drift can be a sufficient obstacle. The trajectories estimated for a shorter time period, for example for a single hauling cycle were in good agreement with the map and with themselves, but in a longer time perspective, significant deviations are observed. In general, the impact of gyroscope drift can be reduced by using the Kalman filter [14]. But usually for navigation problems in this case signal from the magnetometer is used, which is not reliable in the case of the underground mine.

In this article, we propose a method of correction of the trajectory, obtained using the inertial sensors and onboard machine monitoring system by integrating this data with the spatial map of the mine. We use the fact that the mining corridors in the majority of modern underground mines have so-called “Manhattan” structures – the paths are straight and mostly intersect at a 90° angle. Moreover, the machine can turn only on the intersections, so when a turn is observed, the probability that the machine is at a given intersection can be calculated and proper corrections into the machine trajectory can be done.

The structure of the paper is as follows: in Sect. 2 the input data and the methods of its collection are described, in Sect. 3 the preprocessing of data is discussed, then the procedure of path calculation is presented, the path is next integrated with the map of the mine, and finally main algorithm and the correcting one are presented.

2 Input Data Description and Pre-processing

The inertial navigation system uses signals from the IMU (Inertial Measurement Unit), which, depending on the version, measures: acceleration, angular velocity (6 DoF (Degrees of Freedom) IMU), and magnetic field (9 DoF IMU). These signals can be used for example to determine Euler angles and then, in conjunction with the information about machine speed or distance travelled, to determine the path that was travelled by the machine. In this article, we everywhere use a reference frame with a vertical Z-axis (so that the X-Y plane corresponds to an even terrain surface). In order to calculate the two-dimensional route, the yaw angle is used, which stands for rotation around Z-axis. In the vast majority of applications, this angle is determined in two different ways: by integrating the signal from the Z-axis of a gyroscope or using X and Y axes of the magnetometer. The yaw rotation angle determined from the gyroscope has greater accuracy in a short term perspective, but it drastically decreases over time due to the gyroscope drift phenomena. The yaw rotation angle determined by the means of a magnetometer and accelerometer is characterized by lower accuracy at the start, however, it stays at a relatively constant over time. Both of these methods are usually combined using a sensor fusion algorithm such as a Complementary filter or Kalman filter. Additionally, it is possible to attach to this fusion, measurements from other systems, such as GPS. Thanks to this application it is possible to create a system characterized by a fairly high level of accuracy (especially when using GPS).

However, the situation is different when we consider more difficult environments such as underground mines. In this situation, the GPS signal cannot be used. The use of a magnetometer is also very difficult due to the multiple sources of interference. However, by using the gyroscope itself, we put the system at risk of a significant error that after time will make subsequent measurements useless. To prevent this, readings should be combined with other information sources such as mine maps, for example.

Data available from the machine includes readings from the IMU 9 DoF sensor (NGIMU) and the SYNAPSA – the on-board measuring system that acquires the most important operational parameters of the vehicle (driving speed, engine speed, fuel consumption, etc.). In experiments, the IMU sensor was mounted on the articulated joint of the machine, and to protect against environmental threats it was additionally surrounded by a 1 cm thick stainless-steel housing. Although it prevents the sensor from the mechanical impact, it damps the magnetic field significantly. However, it should be highlighted that even without the housing, the magnetometer signal is useless in these conditions. That is why only signals from accelerometers and gyroscopes can be used, mainly we use a Z-axis gyroscope. Additionally, data about speed and currently selected gear from the SYNAPSA system was used. As a source of additional information not connected to the IMU or the machine itself, a mine map and additional key information such as the machine chamber location were used. In addition RFID tags technology can be used to support localization algorithms (some mines have already implemented such infrastructure in the most crucial regions).

Data from the gyroscope (Z-axis) are sampled with 50 Hz. The IMU itself calibrates the sensors and additionally, they are processed before saving using the AHRS fusion algorithms. Data from the SYNAPSA recorder include two variables: SPEED and SELGEAR. The SPEED variable contains information about the current machine speed stored as an integer. However, this variable is always positive, so it is impossible to get information about the direction of this speed. This information can be obtained using the SELGEAR variable describing how the machine's active gear and direction changed over time. If the SELGEAR variable is negative, then it means that the machine is going backward, thus the SPEED value is reversed. Despite the fact that the recorded SPEED variable is not burdened with larger errors, it is an integer and its decimal part is irretrievably lost in the registration process, which can be a source of some errors.

Data from the SYNAPSA recorder are sampled with a frequency of 1 Hz. Finally, the data from the machine recorder and IMU should be merged. The data from the IMU in described experiments is sampled with 50 Hz frequency, thus the data from SYNAPSA is interpolated using the forward filling method. The fragment of each used signal is shown on Fig. 1.

Fig. 1.
figure 1

Variables used for localization with the use of IMU sensor.

Full size image

An additional source of information is the current map of the mining department where the experiments with the IMU sensor were carried out. The map has all possible paths marked in the mine, ventilation ducts (and therefore roads that are currently in use), and additional information, such as excavations or machine chambers. After being loaded, the map of the underground mine department must be converted into grayscale and then converted into a binary image using thresholding operation. Finally, some image compression is applied in order to reduce the image size (Fig. 2).

Fig. 2.
figure 2

The map of the mining department, where the experiments were performed.

Full size image

3 Path Calculation and Its Integration with the Map

To calculate the trajectory distance the vehicle has travelled, first one needs to calculate the Yaw angle φ in any time \(\tau \) given. This angle describes (looking at the vehicle from above) the turns of the machine (left, right). To calculate this value, the easiest way, it can be calculated is to integrate the gyroscope signal over time, what can be described by the following formula (1):

$$ \varphi \left( \tau \right) = \mathop \smallint \limits_{0}^{\tau } \omega_{Z} \left( \tau \right){ }d\tau , $$
(1)

where \({\omega }_{Z}\) describes signal from Z-axis of gyroscope. With yaw angle one can use formulas below (2,3) to calculate path coordinates with the additional use of speed \(v\).

$$X\left( \tau \right) = \int_{0}^{\tau } {\left( {v\left( \tau \right) \cdot \frac{{10}}{{36}} \cdot \cos \left( {{\rm{\varphi }}\left( \tau \right) \cdot \frac{\pi }{{180}}} \right)} \right){\rm{~}}d\tau } ,$$
(2)
$$ Y(\tau ) = \int_{0}^{\tau } {\left( {\nu (\tau ) \cdot \frac{10}{{36}} \cdot \sin \left( {\rm{\varphi (}\tau \rm{)} \cdot \frac{\pi }{180}} \right)} \right)d\tau .} $$
(3)

Due to the discrete character of collected data, in this paper, the above integrals were calculated numerically using trapezoidal approximation.

Unfortunately, the gyroscope readings are affected by drift. Drift is a constantly changing error resulting from integrating two components: a low-frequency variable called bias (offset) and a high frequency one called random walk (integrated noise). This phenomenon can be seen on the road fragments shown in Fig. 3. In these sections, the machine traveled along a constant route from the excavation face to the grid, so the signal should be a straight line between two points with small exceptions for maneuvers and the machine should come back to the starting point, but some deviations from such behaviour are observed. That is why some correction of the trajectory should be applied.

Fig. 3.
figure 3

The pass of the haul truck from the mining face to the damping point and back. The deviations of the trajectory caused by gyroscope drift are visible.

Full size image

Such correction may be based on the fact, that underground mines have a structure of a network of connected tunnels. The machines can only move in the tunnels and cannot leave the road like a car. There is also the issue of closed or collapsed tunnels as well as a continuous change of excavations and thus also used roads. Finally, due to hard conditions in the underground mine, the machines use only ventilated tunnels, thus their possible routes are limited.

Fig. 4.
figure 4

A crossroad object example.

Full size image

In order to connect the path signal with the map, we propose to look at the map as a series of connected points, where each point is one crossroad. This way one can define each of the intersections as an object from one class. This object proposed parameters are: name, position x, position y, and list with other objects connected to the current one. Such a situation is shown in Fig. 4. In this way knowing the initial starting point of the signal, all of the directions of motion will be reduced to only a few directions that are described by points connected to the analysed object. In most cases, modelling connection map will require manual creation of objects and can be hindered because many of the maps are stored on paper or in other forms unsuitable for that kind of needs. Nevertheless, such an effort is most likely one time. After the first creation, in case of any change in the mine road network, the change will most likely include only a few objects. New points can be added in a simple way by creating new instances of a class and adding that instance name to the connected object list parameter of connected intersections. Old points can be removed by doing a similar action.

Additionally around each crossroads, two decision boundaries were established. These boundaries define districts named Proposed and Visited. While the vehicle approaches specific crossroad, two situations shown in Fig. 5 may arise. Depending on the situation, that occurs, one of two algorithms of path estimation described in the next section will be used.

Fig. 5.
figure 5

Crossroad with decision boundaries.

Full size image

Path signal requires additional processing in order to match the coordinates from the map. Depending on the map scale most likely there will be a need to scale path signal along both axes. Path signal calculated in the way proposed above will also always start around point (0,0), so in order to match the map, the signal needs to be moved to the starting coordinates on the map. To automate this process, machine chamber information can be used. The machine will always start work in the machine chamber, whose name and position are known, and then it will move to the excavation area.

4 Path Correction

As it was mentioned before, two algorithms for machine path correction were developed. First of all, switching between the algorithms should be considered. For computational reasons, the best method would be to operate a less complicated main algorithm, and if necessary, switch to a more advanced auxiliary algorithm. When the information about the current position of the vehicle is obtained, the auxiliary algorithm can be turned off and the whole process can again be based on the main algorithm.

When the vehicle leaves the starting point, the distances from the vehicle to all connected crossroads are being tracked. The distance is calculated from the coordinates of the machine, obtained by the use of Euclidean distance. Based on this distance we can tell if the machine crosses some of the decision borders, described in Fig. 5. If the machine first crosses the proposed border, and crosses the visited border, then the corresponding point is set as the current location of the vehicle, and the algorithm starts from the beginning.

But the second situation, described in Fig. 5, may occur: the machine can cross the proposed border, but then instead of crossing the visited border, it goes out of the proposed region. In this case, the auxiliary algorithm is used.

The operation of the auxiliary algorithm is based on a simple method of analysing the connection diagram and calculating the distance. The auxiliary algorithm analyses the connection grid and labels each of the points. The basic (starting) point is the Last Visited point. Each point connected to it (i.e. those points that were the objectives of the main algorithm) becomes a Parent. Then, from each Parent, a list of points, which is connected to it and have no assigned label yet, get the Child label. Then the algorithm from the position of Children tracks the signal, and the Parent whose Child approaches the signal the most is selected for the Last Visited. Then the algorithm returns to the point where it started and switches the process back to the main algorithm. This process is shown in Fig. 6. By approaching and moving away from Parent 2, the signal enabled the auxiliary algorithm. The algorithm sorted the point according to labels and began to measure the distance between Children and path signal. If to choose at this point Parent 2 would be the last visited point because one of his Children (Child 2–1) has the shortest distance to the road signal.

Fig. 6.
figure 6

Demonstration of the auxiliary algorithm.

Full size image

Assistance algorithm implementations can be further developed for probability calculation. For example, after triggering the algorithm, it analyses the connections of the nearest points and on their basis creates all possible paths (branches) that the signal could take. Then, based on the signal readings, the probability of taking is calculated for each of the possible branches, and points are selected on its basis.

5 Signal Correction

All the algorithms above describe a way to correct the trajectory of the machine, but not the gyroscope readings. Without it, even the best algorithms will have difficulty with detecting the path correctly when the gyroscope drift reaches a significant level. That is why based on this data, one can estimate the degree of gyroscope error at each visited point, which should reduce the drift enough for the correct path detection.

There are several ways to approach the problem of road correction. Simple curve snapping is one of them. When visiting a point, the road signal is analysed and its sample that is closest to the point just visited is selected. This sample serves as the signal intersection. The part that has already been analysed (from the beginning of the signal to the selected sample) remains unchanged. The non-analysed part, however, is snapped to the position of the point. More precisely, the entire signal is subjected to the same transformation that is created so that the first sample of the new signal moves to the position of the visited point.

Re-estimation of the Yaw angle is a completely different approach. Knowing the previously visited intersections as well as the initial location, it is possible to create a road from the beginning of the signal to the currently analysed point. From this, it is also possible to determine the angle of Yaw, which will not have bias and random walk errors. This angle can be used to repair the angle resulting from gyro integration. At each turn of the machine. Yaw angle expresses the sum of the angles by which the machine has rotated over time, this means that, for example, when the machine turns twice to the right, the Yaw angle will be 180 at the end. In order to repair the real signal, at the moment of turning, its angle Yaw should be reduced to that obtained by determining from the visited points. The path must then be recalculated using the integration method from the last visited point. The final result of the algorithm is presented in Fig. 7.

Fig. 7.
figure 7

Adequate road plotted on the basis of detected intersections.

Full size image

Correction of the road signal completes the whole algorithm. For clarity, it was also decided to present it in the form of a block diagram visible in Fig. 8. For the sake of simplicity, the presented diagram does not include the mapping of the road based on the passed intersections (which is shown in Fig. 7).

Fig. 8.
figure 8

Algorithm for localization of mining vehicles with the use of inertial sensors.

Full size image

6 Summary

Tracking mining assets during the mining process is crucial for a number of reasons: machine location, material handling process assessment, and diagnosis, production optimization, anti-collision systems, as well as the development of inspection robots and autonomous vehicles. From the point of view of the underground mine, the available localization methods are difficult to implement, taking into account the local specific mine conditions, the problem of gyro drift, and the interference of the magnetometer signal. In this article, the authors proposed a simple method based mainly on the gyro signal and the topology of the road infrastructure in a mine with a room-and-pillar mining system. The article describes in detail the assumptions of the algorithm and all partial procedures. Finally, an example of the use of the algorithm on real data has been presented.