Keywords

1 Introduction

Among various transportation methods, pipelines are the most common fluid transportation method. Crude oil, natural gas and steam are highly utilized as energy sources utilized in both domestic and industrial applications, and for transportation and distribution of fluidic energy sources. Also, water and sewer transportation is done using pipes because of the ability to customize and extendibility [1]. Generally, pipeline networks are located in populated areas, whereas long-distance mass fluid transportation pipelines are located in deserted areas. Critical applications such as nuclear power plants utilize different types of pipes with suitable parameters according to the application, and these applications require a higher level of safety [2]. Due to these facts, pipe inspection and defect detection has become an interest of researchers at present. Pipeline accidents lead to monitory losses, environmental pollution and health hazards affecting human activities and in the worst case bring life risks to living beings. In the year 2010, two petroleum pipelines exploded in Dalian city in Liaoning province, China releasing about 11,000 barrels of oil to the sea causing a massive ecological disaster. In the year 2013, an oil pipeline of Sinopec Corp Oil Company exploded in Huangdao city in Shandong province, China resulting in 55 deaths. Likewise, massive oil pipeline disasters have taken place worldwide, negatively affecting the ecological system [3].

Commonly, indoor and outdoor pipe inspection task is carried out using human resource, and disadvantages such as lack of human resource, time consumption for training, quality of inspection results, safety-related problems and high cost for inspection have become a challenge in maintaining the growing pipe network resulting studies toward automation, robotics and mechatronics-based systems [4]. D. Misiunas studied on failure monitoring in water supply systems and presented different types of pipe defects, which are fundamentally categorized as geometrical defects, defects resulting in metal loss, planner discontinuities and changes in the pipe material [5]. Several pipe defects are shown in Fig. 1.

Fig. 1
figure 1

Pipe defects

Full size image

The literature is available on PDD systems with suitable sensing systems required for various testing methods and remote workstations. With time, nondestructive testing (NDT) became popular due to the nature of minimum alteration to the system, ability to iterate the test at the same location, reduce material wastage, comparative low cost for testing and reduce time consumption due to on-site testing capability [6, 7]. NDT methods are widely used to detect, localize and observe defects in both metallic and non-metallic pipes. Visual inspection, imaging techniques, penetrant testing, magnetic particle testing, ultrasonic testing, eddy-current testing, acoustic testing, vibration testing, microwave testing and radiographic testing are few common methods discussed in the literature under PDD [8,9,10,11,12]. At present, eco-friendly inspection techniques and testing methods are at interest of researchers rather than using testing methods that utilize chemicals, radioactive material to minimize imbalance of the present ecological system of the world.

Imaging techniques utilized in vision systems are rapidly advancing due to applicability, simple arrangement of sensor and higher feature extraction ability such as color, area and shape. D. Kragic et al. categorized vision systems in robotics into monocular and binocular, depending on the camera configuration [13]. W. Ting et al. proposed an in-pipe system using an active stereo omnidirectional vision sensor to detect and classify internal cracks and corrosions [14]. N.A.B.H. Yahya et al. developed a vision-based in-pipe inspection robot and successfully demonstrated identification of pipe geometry [15]. Therefore, studies on in-pipe vision-based defect detection systems have advanced, yet external surface defect detection based on vision systems lacks progress due to complexity of navigation and defects identification. The presented research is a study on external pipe inspection and external surface geometrical defects identification using a robotic system based on vision sensing.

2 Proposed System

The proposed system consists of a robotic system, communication system and a remote workstation.

2.1 Robotic System

Main components of the robotic system are the moving mechanism, sensor system, control hardware, control algorithm and the power source. The main objective in the development of the robot system is to navigate on the external surface of the pipeline and collect data to identify defected segments of the pipeline using a suitable sensor sub-system.

2.2 Communication System

Three communication requirements are identified in designing the proposed PDD system. The first is to control the navigation, the second is to activate and deactivate image acquisition algorithm (IAA), and the third is to transmit data acquired from the camera module to the remote workstation. These requirements are preferably satisfied by wireless communication methods, because wired methods dramatically reduce mobility and the range of the PDD system. Compared to wired communication methods, wireless methods have disadvantages in network safety and data loss.

2.3 Remote Workstation

The remote workstation of the proposed PDD system provides monitoring and analyzing features. Data acquisition system stores data in the remote workstation, and because of this arrangement, data manipulation is done according to the requirements remotely without interfering with robot control or image acquisition system control.

3 Design and Development of the System

3.1 Navigation on Pipelines

Pipeline robots are classified into two categories which are Internal Pipeline Robots (IPRs) [16] and External Pipeline Robots (EPRs) [17], depending on moving surface of the pipe. IPRs are only able to maneuver when the pipeline is at maintenance status, and because EPRs do not contact with the fluid conveyed inside the pipeline, EPRs have the capability to maneuver at operational status of the pipeline. Remotely controlled EPR, ExPIRo is the moving mechanism developed for the proposed robotic system. ExPIRo is developed with an ability to move on pipes with varying diameter and capable of vertical climbing. Figure 2 shows the design and the fabricated robot platform.

Fig. 2
figure 2

ExPIRo robot platform

Full size image

3.2 Vision System Hardware

Figure 3 shows the main hardware components of the vision system. A CMOS sensor module disassembled from a web camera is used as the camera module for the vision sensing system. Selected web camera is a with 480 × 640 pixel resolution. LATTEPANDA single board computer is selected as the onboard main controller for image acquisition setup of the PDD system and data transmission from the robotic system to the remote server. LATTEPANDA supports Windows 10 and performs with an Intel Quad Core 1.8 GHz processor. 4 GB RAM and 64 GB flash memory assist high performance for vision sensing. HP Envy 6-1012TX Ultra-Book laptop is used as the remote workstation to store acquired images and to perform image analysis to identify defected pipe segments. The selected remote server performs with an Intel Core i5 processor, 8 GB RAM and 2 GB VGA and runs on Windows 8.1 operating system.

Fig. 3
figure 3

Main components of the vision system

Full size image

Mounting the Camera Module and Onboard Controller Board

Camera module is mounted at the front end of the moving mechanism to restrain the robot system from running into defected areas of the pipe, and the LATTEPANDA is mounted on top of the battery for symmetric weight distribution and is shown in Fig. 4.

Fig. 4
figure 4

Camera mount and LATTEPANDA mount

Full size image

3.3 Image Acquisition Algorithm (IAA), Defect Identification Algorithm (DIA) and Graphical User Interface (GUI)

MATLAB software is used to develop the IAA and DIA. Acquire Images and Video from UVC Compliant Webcams (AIVUCW), hardware support package for MATLAB is utilized in IAA to initiate the connection between LATTEPANDA and camera module. IAA is deployed on LATTEPANDA, and DIA is deployed on the remote server. IAA captures images and stores the image in ‘.jpg’ format on the remote server. DIA reads the stored images and performs image analysis. Then, it displays the analysis result of the previous image until the next image is processed. Synchronically, DIA stores the result of each image analysis on the remote server for monitoring. GUI is developed using MATLAB GUI GUIDE, and Fig. 5 shows the GUI with additionally developed in-built options to test a pipe image without a defect and an image with a defect.

Fig. 5
figure 5

GUI for pipe inspection

Full size image

Moore-Neighbor tracing algorithm modified by Jacob’s stopping criterion is the image preprocessing algorithm used in DIA. Moore-Neighbor tracing algorithm identifies the contour of the pixel pattern, and the Jacob’s stopping criterion is used in order to avoid inability of the fundamental Moore-Neighbor stopping criterion to trace a large family of patterns in an image.

3.4 Communication and Data Transmission

Three communication requirements are present in the developed PDD system. The first is to control the ExPIRo robot navigation. The second is to activate and deactivate IAA. The third is to transmit data acquired from the camera module to the remote workstation. These requirements are preferably satisfied by wireless communication methods, because wired methods dramatically reduce mobility and the range of the PDD system. Radio frequency-based wireless communication module is used to send control commands from the remote controller to ExPIRo because the robot is navigated in the field. Bluetooth is used to send commands separately to the LATTEPANDA to initiate and terminate IAA. Wi-Fi is used to initiate communication link between the LATTEPANDA and the remote workstation. In-built Wi-Fi receiving module of LATTEPANDA and the in-built Wi-Fi receiving module in HP Envy 6-1012TX laptop are configured to a transmitting–receiving setup for image storing.

3.5 Power Source

Power requirement of the LATTEPANDA is 5 V and 2 A. The lithium-polymer battery used to drive motors for navigating ExPIRo robot platform satisfies the power requirement of LATTEPANDA, and the camera module is powered through the USB 2.0 port in LATTEPANDA. With this setup, the battery has the ability to supply power to the robotic system.

3.6 Control System Architecture

Figure 6 shows the control flow chart of the PDD system including robotic system, communication system and remote workstation.

Fig. 6
figure 6

Control system architecture

Full size image

3.7 Testing Platform

For the testing platform, a white PVC pipe is used having 110 mm outer diameter and 150 cm length. To represent defects, pieces of black tape with different shapes are pasted on the pipe for clear color difference between the pipe and defects. Thereby, the defect identification is simplified. In real applications, the color difference is not black and white, yet a noticeable color difference is present. Figure 7 shows the robot on the testing platform and Fig. 8 shows the images used to test the PDD system.

Fig. 7
figure 7

Testing platform of the PDD system (robot on the pipe)

Full size image
Fig. 8
figure 8

Test images (Image 1 top left corner, Image 9 bottom left corner)

Full size image

4 Results

Figure 9 a–c shows image analysis results of a horizontal pipe without a defect, with a circular shape defect on the outer surface and with a triangular shape defect on the outer surface, respectively.

Fig. 9
figure 9

Visualization of pipe inspection

Full size image

Image analysis result of each image is saved in one single file to identify defected images. Figure 10 shows the report generated with the analysis results of 16 images.

Fig. 10
figure 10

Image analysis results

Full size image

As shown in Fig. 10 among the 16 images captured, Image 6, Image 7, Image 8, Image 14, Image 15 and Image 16 are successfully identified as images with defected pipe segments.

5 Conclusion

Developed PDD system demonstrates successful integration of hardware including an EPR, camera module, single board computer and remote workstation using wireless communication and is capable of successfully detecting external surface defects having significant appearance abnormalities on the testing pipe. Using ExPIRo as the moving mechanism, the robot system demonstrates additional features such as moving on vertically fixed pipes and moving on pipes with varying diameter at a range from 110 mm to 120 mm. Furthermore, the developed PDD system operates outside of the pipeline it is suitable for pipe networks having long pipelines more than other alternative testing methods because this system operates while the pipe network is operational for fluid transportation where most of the other techniques require maintenance status for inspection. The vision system is adaptable to alternative navigation mechanisms with minor changes in the mounting mechanism and is developed as a stand-alone control system to increase adaptability and to reduce interferences from other systems. The vision system requires the defected area and the pipe to have a significant color difference for accurate defect identification; therefore, the presented prototype of the system is not suitable to operate under low light conditions.

Ongoing work under this research is implementing an expandable communication network, developing an artificial neural network (ANN) and implementing machine learning techniques to increase adaptability of the PDD system to real applications. Additionally, multiple camera configuration and infrared thermal imaging technique are identified as suitable advancements for pipe imaging in future developments to increase suitability of the vision system for pipelines having large diameters, higher thicknesses and longer in length.