1 Introduction

In Japan, several natural disasters have occurred and will likely occur even in the future. In the case of large-scale catastrophes caused by earthquakes, heavy rain, or volcanic eruptions, civil-engineering work with construction machinery is required for restoration. However, the restoration work at a disaster site is quite dangerous for operator, and a secondary disaster may occur at the site. Therefore, to ensure the safety of the operators, unmanned construction technology has been developed and used in practical situations in Japan. As a representative example, the Tele-earthwork system [1] developed teleoperated construction machinery to excavate volcanic products and to construct mud-control dams after the eruptions of Mt. Unzen-Fugen that occurred in the 1990s. Such a system was also used for restoration work in the 2011 Great East Japan Earthquake [2].

Fig. 1
figure 1

Developed system on a teleoperated small construction machine

Full size image

Although this system worked effectively in long-term restoration work, it is difficult to make use of such dedicated machines in the initial response of natural disasters, particularly within 1 month after the disaster. This is because (1) teleoperated construction machines require much preparation before operation, (2) the number of teleoperated construction machines is small, and (3) it takes a considerable amount of time to transport the machines to the disaster site. A teleoperation technology for “general” construction machinery is required for the initial response to disasters.

As general construction machines are used for various constructions in several places in Japan, their transportation is easy, and the number of machines is large. However, teleoperation cannot be performed with general construction machines. Therefore, much research and developments have been carried out to mount hardware on the cockpit of a general-construction machine and enable its remote control [3,4,5]. With the help of the above technologies, operators can control a general construction machine by teleoperation. However, they typically use on-vehicle cameras only. It is well-known that visual information obtained from various viewpoints is effective for the teleoperation of construction machines, e.g., the image of a bird view of the machines or that from the side of the bucket. Therefore, in restoration works at Mt. Unzen-Fugen, dedicated camera-equipped vehicles and fixed cameras on external towers were used to obtain images from various viewpoints. However, it is difficult to install them for the purpose of an initial response within a month of the occurrence of a natural disaster. In addition, there are limitations to the arrangement of the cameras.

As a solution to the aforementioned issues, we have been researching and developing a system that use a multirotor micro unmanned aerial vehicle (MUAV) as an external camera carrier to obtain images from various viewpoints. The system is consisting of a tether-powered multirotor MUAV and a helipad that have a tension controlable winch to windup a tether.

By installing the tether-powered multirotor MUAV system on a general-construction machine and mounting conventional teleoperation hardware on the cockpit, it is possible to realize an instant teleoperated construction machine that can obtain images from various viewpoints by itself. Figure 1 shows our recent version of the tether-powered multirotor MUAV and helipad on a construction machine.

In this paper, we present our research on and development of the proposed system and report the results of our field tests on the actual construction machine.

2 Tether Powered Multirotor MUAV System

2.1 Proposal

The most important information for the teleoperation of a construction machine is visual information. In teleoperated restoration works at Mt. Unzen-Fugen, the operator uses not only images from the perspective of the cockpit, but also images from various perspectives obtained from dedicated camera-equipped vehicles and fixed cameras on external towers [1]. The images from the latter can compensate for the blind spot in the cockpit view, and it enables the operators to directly grasp the motion of the construction machine. In particular, the images are used to grasp a details of the distance between the operating device (e.g., bucket) and the target (e.g., volcanic rocks).

For developing images from various viewpoints, Sato et al. developed a system with multiple on-vehicle fisheye cameras [6]. It is effective for an initial response because it does not require any external camera installation in the target environment. Nevertheless, blind spots exist on the image obtained from the construction machine. Moreover, a typical construction machine has an arm and projections that cause blind spots.

Therefore, in this research, we aimed to develop a system to use a multirotor MUAV as an external camera carrier to obtain images from various viewpoints. It is now matured technologies to obtain images by MUAV from the air. However, there still exist some problems with this system, and one of these problems is its short flight time. Therefore, we have been researching and developing a system for the use of a tether-powered multirotor MUAV. The following are the advantages of the proposed MUAV:

  1. 1.

    Within the tether length, the multirotor MUAV can stay at arbitrary positions to obtain alternate viewpoints without being affected by the rough terrain.

  2. 2.

    The flight time of the tether-powered MUAV is much longer than that for a typical MUAV because a power-feeding tether is used to supply electric power from the helipad.

  3. 3.

    In case the multirotor MUAV is out of control, it flies only within the range of its tether length. Thus, the damage to the environment can be minimized.

  4. 4.

    It helps secure pinpoint landings of the multirotor MUAV by forcefully winding the tether.

There are some researches on tether-powered multirotor MUAVs [7, 8], and some have been recently used in practical applications [9]. However, the flight range of the tethered MUAV has a limitation because of its tether length. In order to allow an external camera carrier to obtain images from various viewpoints, we have developed a tethered MUAV helipad that can be mounted on a construction machine. The proposed system is effective for an initial response for disaster because the system enables long-time flight, and it requires no external camera installation to teleoperate a construction machine in the target environment.

There exist some problems in the realization of the system, and one of these problems is large vibrations in the construction machines. Therefore, we organized the problem and implemented countermeasures as described in the next subsection.

2.2 Challenges of the System

The tether-powered multirotor MUAV system has three major challenges to install it to the construction machine as we proposed.

The first challenge is the flight range. A typical objective of tether-powered multirotor MUAVs in general use is a fixed-point observation, and the MUAV is not required to move dynamically. However, for teleoperation of the construction machines, the viewpoint of the MUAV should be changed based on the task. For example, to take an image of an excavator, the MUAV had better locate at directly above of the machine for its navigation, or the MUAV had better locate at the side of the bucket when it excavates the ground. In addition, when the MUAV fly horizontal, the power-feeding tether may slack and come into contact with the environment. The greater the tension in the tether, the lesser is the looseness in the tether becomes. However, the thrust required for the MUAV’s flight increases, and the controllability of the MUAV decreases. Therefore, appropriate tension control of the tether is required.

The second challenge is a power source. A typical power-feeding tether system uses an AC power supply from the ground station. However, in the proposed system, an independent power source is required to fly the tether-powered multirotor MUAV because the helipad that works as typical ground station is mounted on the construction machine. In addition, a lightweight, thin tether is required to reduce the load on the MUAV. In this case, it requires a high-voltage supply, but the MUAV requires a voltage step-down circuit if use a high-voltage, which implies an additional payload to drive propellers’ motors. Considering the above trade-off, it is necessary to select an appropriate power supply method.

The third challenge is the vibration and inclination of the construction machine. As the power for a general construction machine is obtained from an engine, the construction machine vibrates, which also affects the equipment installed on the machine. Furthermore, the target environment is natural uneven terrain. Thus, the vibration and inclination caused by the machine navigation on such rough terrain affect the equipment of the machine. Therefore, it requires a system that is robust against vibration and inclination.

In keeping with to the above challenges, we consider that the helipad has several development factors. Therefore, in the first stage of our development, we developed a novel helipad for a tether-powered multirotor MUAV that can works on construction machine. We used a conventional airframe as the multirotor MUAV, and controlled it manually under direct visual.

3 Development of the System

The tether-powered multirotor MUAV is controlled manually based on the flight controller inside the MUAV and the radio control transmitter. The tension of the tether is controlled independently using a winch mounted on the helipad. The operators control both the systems and camera gimbals mounted on the MUAV. Figure 2 is a block diagram of the system. In this paper, we do not describe in detail the camera gimbals’ system. In the following subsections, we introduce the “power system”, “winch with controllable tether tension”, and “wireless communication”.

Fig. 2
figure 2

Block diagram of the tether powered multirotor MUAV system

Full size image

3.1 Power System

A multirotor MUAV requires an electric power source. For example, the typical MUAV developed in this study (quad-rotor MUAV with 15-in. propellers, of approximately 3.0 kg) requires 400 W for hovering, and 800 W for moving or dealing with a disturbance, in our experience. The required electric power mainly depends on the weight and the rotors’ diameter. Therefore, even if the MUAV itself is improved, it will not be possible to reduce the power consumption drastically. When such a large electric power is used, the power loss in the power-feeding tether is critical. Based on Ohm’s law, the loss due to the electric resistance in the power-feeding tether is proportional to the square of the current. Therefore, to obtain the same electric power, a high-voltage and low-current system is required to be configured for the power supplied through the tether.

On the multirotor MUAV side, we use a voltage step-down converter that allows an input between DC200 V and 420 V. A continuous output of 600 W is possible for one converter. Therefore, we use two converters in parallel to obtain an output of 1200 W. The converter was selected by the considerations of the weight and availability. The voltage step-down converter is installed on the side opposite to the camera gimbals to balance the center of gravity, and the converter is cooled by a downstream flow from the rotors. Figure 3a shows an appearance of the MUAV, Fig. 3b shows the setup of the voltage step-down converter on the MUAV, and Figure 3c shows the converter itself.

The weight of each module is 160 g, and the total weight of the two converters is lighter than the weight of the batteries normally used for its flight. The weight of the MUAV before installing any equipment is about 2.5 kg, and after installed, it is about 3.2 kg. The payload capacity of the MUAV is over 3 kg, thus the MUAV have more than 1 kg payload even consider about tension and weight of a tether.

Fig. 3
figure 3

Developed tether powered multirotor MUAV

Full size image

Next, we describe the power system in the helipad side. Typically, a commercial power source was used to handle a large power consumption in previous researches. However, in a disaster environment, it is impossible to use such a commercial power source. Therefore, a small-sized power source that has a sufficient power capacity is required to be mounted on the construction machine. In addition, the power source is required to be used in the vibration and inclination conditions.

There were two realistic candidates for the supply of 800 W of electricity: the battery type and gas-powered generator type power supply. In the case of the gas-powered generator type, it is difficult to estimate the remaining working time and to use it in the inclined condition. Therefore, we chose the battery type power supply in our implementation.

In the case of the battery type, there are two methods to obtain a high voltage to tether power feeding: (1) boosting the voltage from a battery with a voltage converter and (2) using a series connection of batteries. In the former case, it is possible to increase the choices of batteries. However, conversion loss occurs in the converter. In the latter case, there is no conversion loss, but the total voltage fluctuates greatly depending on the remaining battery power. As mentioned above, the chosen voltage step-down converter installed on the MUAV allows a wide-range input of between DC200 V and 420 V. Therefore, we chose the latter method.

According to the required conditions—low weight and possibility of use in the inclined condition–we did not choose lead batteries but lithium batteries. Specifically, lithium ion battery packs (23.1 V, 127 Wh, 62KSP545483-2, Hitachi Maxell, Ltd.) are chosen. This includes the circuit used for the estimation of the remaining battery level. In this research, we connected twelve packs in serial and use as a large battery unit (277.2 V, 1,524 Wh). It enabled over 3 h of operation, i.e., hovering, for our multirotor MUAV.

3.2 Winch with Controllable Tether Tension

To enable the appropriate control of the tether tension, we developed a winch with a controllable tether tension located on the helipad.

When both endpoints of a string-like object are fixed at any two points, the object forms a catenary curve. Our tether is sufficiently thin (approximately 5 mm outer diameter) and soft, such that the tether also shapes a catenary curve. When the lower side of the tether is fixed at the helipad, and the upper side of the tether is connected to the multirotor MUAV, the angle formed by the tether and helipad is expressed as a function of the tether tension. Therefore, the helipad can control the shape of the tether by controlling the tether tension, and it can avoid the contact of the tether with the surrounding objects.

The general tension control uses feedback control based on a tension measurement. The tension measurement is typically conducted by measuring the displacement of a movable pulley to which a spring is connected. However, when the acceleration is applied to the measurement device, it measures the total of the tether tension and acceleration. Furthermore, when the helipad is in an inclined condition, gravity acceleration is affected, and a measurement error occurs. Therefore, it is difficult to apply the typical feedback control method to our system.

Fig. 4
figure 4

Winch with controllable tether tension on helipad

Full size image

To solve the above problem, we chose to use a powder clutch that can specify the arbitrary torque with the open loop control, instead of a tension measurement of the tether. Figure 4a shows a CAD model of our winch that includes a powder clutch. The powder clutch uses magnetic powder, and it transmits torque from the motor to the spool according to the current. Once the torque control of the winch is realized, the tension of the tether is calculated based on the spool torque and spool radius. To estimate the tether tension accurately, an estimation of the spool radius, which changes according to the extended tether length, is very important. Therefore, in this research, we developed a mechanism to wind up the tether densely to estimate the winding position of the tether. With this mechanism, the helipad can accurately generate an arbitrary tension in the tether at any time, even under the conditions of vibration and inclination.

3.3 Communication System

In our system, control PCs are located at both the MUAV and helipad, and both the PCs should communicate with each other. On the other hand, it is necessary to establish a wireless communication between the operation room and the construction machine for teleoperation. To secure the wireless bandwidth, it was decided that the communication between the MUAV and the helipad should be wired.

The weight of the wires for the communication significantly affects the payload of the MUAV, and therefore, we chose a VDSL communication system that can be realized with only two metal lines. We mounted the VDSL modems on both the multirotor MUAV and the helipad and enabled a mutual conversion of the VDSL and Ethernet. The control signals for the flight and camera gimbals are sent from the helipad to the MUAV through the VDSL communication.

All the control signals are gathered in the control PC on the helipad, and the PC communicates with the operator’s PC on the wireless LAN. Based on the proposed communication system, various external communication devices can be used, and their integration with other systems on the construction machine becomes easy.

4 Field Test

In November 2016, we integrated our system on the teleoperated construction machine developed in the Impulsing Paradigm Change through Disruptive Technologies Program (ImPACT) and conducted an operation verification test of the proposed system. The objective of the test of the ImPACT Program was to confirm the efficient work made possible using a teleoperated construction machine in the case of a disaster situation, and the objective of the test of us was to provide images from various viewpoints to the operators by our proposed system. In this test, tether length is limited to 12 m for safety reasons. Also, our developed system fly over an hour continuously and battery capacity remains over 60% after the test.

Figures 5, 6 and 7 shows the photographs of the test. The photographs in the left column are taken from the outside, and the photographs in the right column show the images obtained from the multirotor MUAV. Figure 6 show the outside view and obtained image from the high flight altitude of the MUAV. Figure 7 show the outside view and obtained image from the low flight altitude and long horizontal length. In the last case, the tether is not slackened or tightened too much because of the appropriate tension control by the winch.

Fig. 5
figure 5

Test flight overview images in case of MUAV’s middle flight altitude

Full size image
Fig. 6
figure 6

Test flight overview images in case of MUAV’s high flight altitude

Full size image
Fig. 7
figure 7

Test flight overview images in case of MUAV’s low flight altitude and long horizontal distance from helipad

Full size image

The above results showed that the developed system worked as expected without the occurrence of any problems.

5 Conclusion

In this paper, we introduced the design and implementation of a tether-powered multirotor MUAV system to obtain images from various viewpoints for the teleoperation of a construction machine. The features of the system are as follows:

  1. 1.

    high voltage battery is used to transmit electric power through thin cables,

  2. 2.

    a powder clutch is used for the winch to enable tension control in the vibration and inclined conditions, and

  3. 3.

    a VDSL communication system is used for communication between the multirotor MUAV and the helipad.

Finally, we conducted an operation verification test of the proposed system to provide images from various viewpoints to the operator.

In future works, we intend to evaluate this system in detail and develop a relative positioning method for the MUAV based on the tether information to realize autonomous MUAV flights.