Keywords

1 Introduction

Truck crane hoisting operation is a general but essential operation on construction sites, which supplies a convenient solution for lifting cargos thanks to its mobility. However, it is troubled by low autonomy, requiring experienced manual guidance and driving. In recent years, some efforts have been made to increase the level of automation (LoA) of cranes, but most of them are limited in automated decisions and planning, typical examples are the research on lifting path algorithms [1,2,3,4] and automated collision checking [5, 6]. However, accurately executing the planning or adopting the suggestions provided by computers in time is challenging for workers in a real construction environment. Consequently, there is an urgent need for more automated and intelligent lifting equipment to address the issues.

Robotics is believed as an impressive technology to increase productivity in the construction field. Relative research and applications in the construction industry have been available since the 1980s, and construction robots have made significant progress in automation for single tasks (e.g., welding, bricklaying, and painting) over the past four decades. However, researchers pay less attention to construction robots for lifting equipment and few to mobile crane robots. Rosenfeld et al. claimed that the automation of cranes would benefit the productivity of the industry in the early 1980s [7], however the absence of automated lifting equipment is still a shortcoming that hinders the efficiency and safety improvement of construction automation.

This research aims to develop a robotic truck crane system (RTCS) that provides traditional truck cranes with automatic operation abilities. The rest of this paper is structured as follows. First, the literature review is made in Sect. 2, and then the architecture of the proposed system is presented in Sect. 3. Further, the implementation of the system is described in Sect. 4. Additionally, an experiment was conducted, and its result is discussed in Sect. 5. Finally, this research is concluded in Sect. 6.

2 Literature Review

According to the Oxford dictionary definition, a robot is a machine capable of automatically carrying out a complex series of actions, especially one programmable by a computer. Correspondingly, robotic cranes should be capable of automatically taking actions involved in hoisting tasks, such as monitoring, path planning and execution. A more rigorous automation strategy for cranes should be founded on the comprehension of current research and practice based on the different LoA. Sheridan et al. classified the LoA into ten levels [8], from the extreme of completely manual to fully autonomy. To further investigate the LoA, Parasuraman et al. proposed that the automation can be further decomposed into four aspects: information acquisition, information analysis, decision and action [9]. Similarly, the automation for cranes can be investigated in the above aspects.

Among the relative researches, automated information acquisition represents the earliest endeavor of enhancing the LoA of cranes. For instance, Lee et al. utilized a wireless controller with a display linked to a tower crane camera to allow the operator to observe without being in the driver cab [10]. Another approach is to provide the operator with supplemental information about cranes. For example, Price et al. developed a crane monitoring system (CMS) to inform the operator of the crane load, rotation angle and lifting height [11]; Zhang et al. used ultra wideband (UWB) technology to visualize the position of the hook relative to the building [12]. In general, automation related to information acquisition has been well established and has considerable application in practice.

Automated information analysis and decisions have recently been a focus of crane-related research. They are expected to lower the requirement of human experience, enhancing safety and productivity. Collision checking is a familiar form of automated information analysis based on predicting the movement of machinery. For instance, Tak et al. have developed a simulation system with spatio-temporal site analysis functions to predict potential conflicts [13]. Likewisely, Dutta et al. developed a decision support system (DSS) to stop the crane before a collision occurs [14]. Regarding automated decisions, lifting path planning represents the most advanced part of relative research. Many algorithms are designed and developed to support finding a feasible lifting path [1,2,3,4], but it is worth noting that most are only tested in simulated environments without being applied on a real crane.

While automation in the rest aspects is explored adequately, only a few studies have devoted to how to enhance the automation of action implementation of the crane, the lack of which has become a major barrier to developing robotic cranes. Among the limited relevant studies, most of them are also based on simulation environments, seldom involving real hardware. For example, Andonovski et al. developed an autonomic tower crane robot system with a simulated crane robot [16]. Another example is that Chen et al. used a 6-DOF robotic arm in a simulation environment to simulate the motion of a mobile crane [17]. The simulation systems in these studies generally do not concern the actual hardware of cranes. Therefore, the performance of the systems implied in this studies might decrease when they are applied to the real scenes.

In summary, current robotic crane-related research seldom considers automated action implementation ability. Thus, RTCS is proposed in this research to fill the research gap. It addresses automated action implementation while realizing automated information acquisition, information analysis, and decision as predecessor functions.

3 System Architecture

RTCS consists of four layers: hardware layer, control layer, service layer, and application layer, as shown in Fig. 1. The lowest level, hardware layer, contains sensors and actuators. Encoders, IMUs, and cameras are main sensors responsible for sensing the working conditions and surrounding environment of the crane in real-time. Actuators consist of motors, relays and switches, which control the movement of the crane by signals received. Data from sensors and signals for actuators are transmitted through hardware interface, which is provided by hardware layer as a bridge to communicate with the upper layer.

The control layer is designed for reading sensing data and sending control signals. The control layer processes sensing data from the hardware-specific form into standard form, the ROS msg [18]. Consequently, the system promises to update new features more conveniently with a unified data form. Meanwhile, the control layer is responsible for interpreting the command messages delivered by the service layer into individual commands and then sending them to each actuator.

The service layer is built on top of the control layer to accept service requests, the latter comprise inquiring for robot information and commanding, mainly raised by applications. For debugging and testing use, the layer also accepts input from users directly. Functioning as a server, the server layer maintains the information of the crane and supplies a standardized service interface to the upper applications, hence reducing the coupling of the system.

The application layer consists of applications interacting with users frequently. Users are encouraged to customize applications regarding to pratical requirements, such as monitoring, advising and planning. Rviz, a visualization tool merged with ROS, is operated as the monitoring application for RTCS. It provides users with critical information about the robot system and a straightforward 3D visualization. For lifting path planning, an open-sourced motion planning library, MoveIt, supplies an interactive interface for planning lifting path and sending movement commands. Apart from automatic controlling, a remote control application is furthermore developed for testing and emergency operation.

Fig. 1.
figure 1

The system architecture of RTCS

Full size image

4 Implementation of System

This section describes how to apply the above system architecture to a physical truck crane. For safety and economy considerations, a scaled model of a truck crane with a ratio of 1:6 is used instead of a real one. Despite its size, the model has a similar structure and mechanism to a real truck crane. Therefore, the implementation method illustrated can be migrated to a real truck crane theoretically. To deploy the system, the kinematics of the truck crane is first analyzed. Then, the details of the four system levels are described.

4.1 Kinematics Analysis of the Truck Crane

With appropriate simplification, the crane can be treated as a robotic arm combining rigid joints and links. More specifically, the rotation, deformation and swing of the rope are excluded from consideration. Thus, the wire rope is deemed a joint capable of parallel telescoping. As a result, without considering the moving wheel, the crane would be simplified to consist of 5 joints: rotate joint (rotary), raise joint (rotary), telescope joint (prismatic), pulley joint (rotary), and lift joint (prismatic), and the hook is viewed as the end effector, as illustrated in Fig. 2a. These joints are linked by six links: base, driver cab, the main boom, the boom with extension, pully, and the hook. There are two primary differences between a real truck crane and the simplified model: a real truck crane uses a hydraulic rod to raise the main boom, but it is treated as if the boom were driven directly by the raise joint in the model. Another refers to the boom with extension that consists of multiple sections, but it is treated as one joined section after simplifying. These simplifications help describe the configuration of the crane more concisely by reducing the number of joints in the model. Figure 2b presents the correspondence between the simplified model and the physical crane.

Fig. 2.
figure 2

The simplified model of the truck crane

Full size image

According to the kinematics model, the Denavit-Hartenberg (DH) parameter is measured according to the crane model, as shown in Table 1. Where, the offset, joint angle, length and twist between parent and child links are represented by \({\text{di}}\), \({\theta i}\), \({\text{a}}_i\) and \({\alpha i}\), respectively.

Table 1. Denavit-Hartenberg (DH) parameter of the crane model
Full size table

Accordingly, the position of the end effector can be acquired as Eq. (1):

$$ {\text{x}}_{{\text{end}}} = T_0^1 T_1^2 T_2^3 T_3^4 T_4^5 x_{base} $$
(1)

where, \(x_{base}\) Is the position of the base link, \(T_{i - 1}^i\) is transform matrix of the \({\text{i}}^{{\text{th}}}\) joint, expressed as Eq. (2):

$$ T_{i - 1}^i = \left[ {\begin{array}{*{20}c} {\cos \theta_i } & { - \cos \alpha_i \sin \theta_i } & {\sin \alpha_i \sin \theta_i } & {a_i \cos \theta_i } \\ {\sin \theta_i } & {\cos \alpha_i \cos \theta_i } & { - \sin \alpha_i \cos \theta_i } & {a_i \sin \theta_i } \\ 0 & {\sin \alpha_i } & {\cos \alpha_i } & {d_i } \\ 0 & 0 & 0 & 1 \\ \end{array} } \right] $$
(2)

4.2 Hardware Layer

4.2.1 Actuator

The actuators for cranes and general robots vary significantly due to several differences in their working conditions. First, cranes operate in a much harsher environment, frequently accompanied by vibration and dust. Second, the workload of cranes is much heavier than typical industrial robotic arms. For example, the biggest robotic arm of ABB, IRB8700, has a maximum loading capacity of 800 kg. However, it is standard for cranes to lift cargo of tons. Therefore, severe working conditions and requirements propel the crane to adopt actuators with simpler mechanisms and higher robustness. Most cranes employ motors without servo functions, which is common in most industrial robots. For this reason, the crane model used in this research also equips non-servo motors. More specifically, the crane has four motors that control the slewing, telescoping, raising of the boom and lifting of the hook respectively, and relay and switches are used to control the power and running directions of the motors.

4.2.2 Sensor

The sensors are applied in a non-intrusive method without changing the original mechanical and electrical structure of the crane. The kinematic analysis shows that the configuration of the crane can be presented by the states of five joints, namely, the configuration is a five-dimensional vector, shown as Eq. (3).

$$ {\text{q}} = \left[ {\theta_{{\text{rotate}}} ,\theta_{{\text{raise}}} ,{\text{d}}_{{\text{extend}}} ,\theta_{{\text{pulley}}} ,{\text{d}}_{{\text{lift}}} } \right]^{\text{T}} $$
(3)

where, \({\uptheta }_{{\text{rotate}}}\), \({\uptheta }_{{\text{raise}}}\), \({\text{d}}_{{\text{extend}}}\), \({\text{d}}_{{\text{lift}}}\) are controlled by motors. However, \({\uptheta }_{{\text{pulley}}}\) is not constrained since the pulley at the end of the boom is powerless. Luckily, it inverses with \({\uptheta }_{{\text{raise}}}\) since the wire rope is considered perpendicular to the horizontal plane due to gravity. Thereby, four sensors are sufficient to obtain the configuration.

There have been matured studies exploring how to monitor the working condition of cranes. Regarding \({\text{d}}_{{\text{extend}}}\) and \({\text{d}}_{{\text{lift}}}\), available distance sensor includes encoders [11], laser [19], and UWB [5]. In particular, encoders are the preferred choice because of their low cost and high accuracy. Meanwhile, the inertial measurement unit (IMU) is used to measure the \({\uptheta }_{rotate}\) and \({\uptheta }_{raise}\). While the IMU is fixed on the boom with its X-axis pointing to the front, the measured yaw angle and the pitch angle will be referred to \({\uptheta }_{{\text{raise}}}\) and \({\uptheta }_{{\text{rotate}}}\), respectively. The specific installation location and method for sensors are presented in Fig. 3.

Fig. 3.
figure 3

Sensors installed on the crane

Full size image

4.3 Control Layer

The control layer of RTCS is responsible for reading data from sensors and controlling actuators. It provides drivers to parse sensor signals into readable data, then reconstructs the data structure into standard ROS msg. Besides, it manages actuator controllers that control motors according to the movement command given by the upper layer. Actuator controllers are designed to access data from sensors and listen to movement commands continuously. It functions through incessant loops: as a start, it reads data from sensors to obtain the current state of the joints; then, it updates the command given by the upper layer for each actuator; finally, it decides whether to close or open the motor according to the current joint state and movement command.

Although the motors equipped by the crane are much simpler than servo motors, the control strategy is rather more complicated. Servo motors applied to the industrial robots allow quantitative control of joint position by transmitting PWM signals with specific duty cycle. However, a non-servo motor can only switch between different directions. In order to control the joints precisely, a servo function needs to be supplied by the actuator controller. Therefore, the following control strategy is designed to support the servo function, as expressed by pseudocode:

Algorithm 1.

Control strategy for the motors.

figure a

4.4 Server Layer

The server layer provides a unified interface to inquire about the system information and send movement commands. It is responsible for integrating the sensor data into structured state information and updating the latter on a public node. Nodes play the roles of maintaining data and processing access requests. For instance, the pose of the crane is maintained by a specific node. By subscribing to this node, a 5-dimensional vector representing the configuration of the crane is returned as required. Thus, the system is low coupling with high scalability due to information is maintained by seperated nodes rather than a specific application.

Simultaneously, the server layer is also in charge of commanding controllers. It has a server node that accepts the request for a new configuration of the crane, parsing the request into commands for each controller. It is worth noticing that the server node only cares about whether the requests meet the format requirements of the interface, regardless of the type of requester. Hence, RTCS supports a toggleable control mode from manual inputting to automatic running.

4.5 Application Layer

4.5.1 Monitoring

Rviz is a 3D visualizer for displaying sensor data and state information of the robot system. In RTCS, Rviz is used as a monitoring application to assists the operator in monitoring the crane’s operational status in real time. The 3D model is synchronized to the physical crane according to real-time sensor data in Rviz, shown in Fig. 4a. To reflect the state of the crane presicely, the 3D model of the crane is constructed based on the drawing in Solidwork, as presented in Fig. 4b. As described, the visual monitoring interface provides users with an intuitive method to observe the operational status of the crane, which benefits the accessibility of the system.

Fig. 4.
figure 4

The monitoring interface and the 3D model of the crane

Full size image

4.5.2 Path Planning and Executing

MoveIt is integrated with the system to support lift path planning and executing. It functions as an upper application to generate movement commands and send them to the server node. As a matured platform, MoveIt consists open motion planning library (OMPL), flexible collision library (FCL), kinematics and dynamics library (KDL) and a highly interoperable interface for movement planning. MoveIt has been applied widely in industrial robots, but this research might be the first attempt to use MoveIt on a truck crane for lift path planning.

By specifying the initial and terminal positions of the hook, as well as the shape and position of the surrounding obstacle, MoveIt automatically decodes a collision-free path based on various built-in algorithms. First, it calculates the initial and terminal configuration according to the positions of the hook by inverse kinematics. Then, collision checking and path planning algorithms cooperate to generate a series of waypoints between the configurations. Finally, the waypoints will be sent to the server node to control motors, guiding the crane to reach the terminal.

4.5.3 Remote Controlling

For testing and necessary manual intervention, a remote control application is established. The remote controlling of the crane support three kinds of input methods: via a keyboard, a joystick, or a visual interface. These control methods only vary in input tools but follow the same requirement of the server interface as described in the last section, consquently they are essentially the same regarding the control result.

5 Experiment and Discussion

In order to evaluate the performance of RTCS, a case experiment was conducted in the laboratory environment. The experiment simulated lifting cargo from the ground to the top floor of a building. We assumed that the following information was known: the position of crane, cargo and building; the shape of cargo and building. To clarify, the building model is a cylinder with a radius of 0.3 m and a height of 1 m, and the cargo is a cuboid of 0.1 m*0.1 m*0.2 m. Magnets were fixed on the cargo for connecting to the iron hook automatically. The 3D model of the environment was added to the planner application in advance, as shown in Fig. 5.

Fig. 5.
figure 5

The layout of the simulated task for the experiment

Full size image

The task consists of two steps: first, hooking the cargo; second, lifting the cargo to the target position. For the first stage, the crane moved its hook above the cargo from its initial position to hook the cargo. With the help of MoveIt, a collision-freed feasible path consisting of waypoints was planned by Rapidly-exploring Random Trees*(RRT*), as illustrated in Fig. 6a. Afterward, the path was executed by the crane. Due to measurement errors and execution errors, the hook did not reach the estimated position precisely, with a deviation of 5 cm, as shown in Fig. 6b. Despite this error, the magnet successfully connected the cargo to the hook.

Fig. 6.
figure 6

The planning and executing result in the first step

Full size image

Similarly, the second stage is composed of path planning and execution. Nevertheless, additional factors made the operation more challenging. Since the cargo was attached to the hook, its collision volume should be considered, increasing the complexity of the collision checking and taking more time for RRT* to find a collision-freed path, as shown in Fig. 7a. Worse still, longer travel distances increased the cumulative error. Compared with the first stage, the operating error of the second stage was significantly greater, reaching 15 cm, as presented in Fig. 7bc. Nevertheless, the result proved that RTCS is capable for lifting task with its error in a reasonable margin.

Fig. 7.
figure 7

The planning and executing result in the second step

Full size image

Regarding the experimental results, several discussions are demonstrated as follows. The accuracy of RTCS is affected by errors from three aspects: (1) measurement errors of the machine, cargo and environment; (2) executing errors of the actuator; (3) sensor errors. While RTCS displays acceptable mistakes, errors may scales in real scenes, consequently results in exponentially increased deviation. Moreover, RRT* and analogous algorithms for universal robots may not be suitable for the crane with particular kinematics and task-specific requirements. The fact is that some paths planned in experiments are not acceptable due to safety considerations despite being collision-free. Therefore, a new path planning method should be designed to support safe and efficient lifting.

6 Conclusion

The main goal of this research is to enhance the intelligence of the truck crane by robotic methodologies. RTCS, a robotic system for truck crane is developed to improve the autonomy of the truck crane. It is a modular designed system based on ROS, permitting high scalability, rapid implementation and iteration of functionalities. To better understand how to implement robotic technologies onto the truck crane, the kinematic and the actuating characteristic of the truck crane are analyzed. Based on this, an automatic control strategy for the crane is designed. Further, the system provides several applications to assist the operation, including a monitoring application, a lift path planner and a remote controller. In order to test the performance of RTCS, a scaled experiment was conducted, where RTCS was required to perform a typical hoisting task. In the experiment, RTCS functions as expected, presenting satisfactory accuracy in monitoring and execution. The result highlighted the potential usefulness of a robotic system for lifting equipment similar to the truck crane.

Nevertheless, non-negligible challenges still exist in applying RTCS to the practical construction environment. Firstly, it is crucial to conduct full-size experiments to examine the applicability of RTCS on actual cranes. Secondly, a more practical lift path planning method should be developed rather than universal algorithms. In addition, dynamic perception is urgently required to improve the security of the automatic lifting. Currently, the path planning function is based on a static environmental model, but dynamic obstacles are common and of great concern in the real construction scene. Besides, more functionalities are encouraged, such as integration with industrial foundation class (IFC), anti-swing controlling and safety inspection. In summary, future research will focus on filling the gap between the RTCS prototype and the real crane, increasing the system’s intelligence, and adding features to suit practical needs.