Keywords

1 Introduction

Piping and instrumentation diagrams, P&IDs, are a type of technical drawing that provide a graphical representation of a process system that include the piping, vessels, control valves, instrumentation, and other process components and equipments in a given system. Such diagrams are the primary schematic drawing used for laying out a process control system’s installation and is most commonly used in the engineering field, such as during the designing of a manufacturing process in a processing plant. These facilities usually require complex chemical or mechanical configurations that are represented through the use of symbols and connections on a P&ID. They can be used to streamline a process, keep track of different pieces of equipment or guide the design of a new facility.

There are standard symbols used to represent the components in P&IDs, this representation is not made to scale, which add to the complexity of the task. Most of these symbols are also associated to words, letters or numbers to further identify and specify the components being represented. Even though standard symbols are used there can exist some variations on how they are depicted across P&IDs from different origins and industries.

In recent years, several attempts have been made at performing this task of automated reading. Given the schematic aspect of P&IDs, the goal of this work is to identify different symbols, their associated text and the different connections between components.

The rest of the paper is structured as follows: Sect. 2 describes some P&ID approaches in the state-of-the-art. Section 3 details the proposed pipeline and some experiments done so far. Section 4 concludes the paper and discusses possible future directions.

2 State of the Art

Digital engineering drawings have seen great attention across industries. Analyzing and processing these drawings are becoming important for decision making. In this context, several efforts have been made in different domain applications such as planning, construction, oil, gas, etc. In this section, we detail some works related to P&ID data extraction and automatic digitizing.

Dong-Yeol Yun et al. [8] propose a new object detection approach in order to recognize the different symbols in P&ID. To do that, three steps have been defined: i) region proposal using a sequential image-processing, ii) dataset annotation iii) regions classification where a convolutional network has been trained to classify the proposed regions and extract the information related to the symbols. In [7], authors introduced a new texts and symbols recognition in a P&ID based on local binary pattern (LBP) and spatial pyramid matching (SPM) for image feature extraction. Arroyo et al. [1] aim to convert the P&ID documents into an object-oriented plant description. This is by applying optical recognition and semantic analysis.

Premanand Ghadekar introduces an end-to-end GPU algorithm [3]. It starts by extracting the meta data of each instance related to the instruments, valves and equipment’s present in the P&ID. Then it detects the lines and texts. The authors combine computer vision methods and deep learning models (Retina net with a Resnet-101 backbone) to classify symbols, semantically analyze texts and detect connections between equipment’s. In [5], authors use template matching for symbols recognition, these have been extracted and registered automatically in the database. For line and texts recognition, the sliding window method and aspect ratio calculation has been used. A recent paper extends the previous works by applying graph search to detect the connections between symbols in P&IDs [6]. This paper is based on state-of-the-art methods to detect symbols where a Convolutional Neural Network has been used, recognize and interpret text and detect connections between symbols across lines. The proposed method can be applied in different applications such as diagram search, equipment-to-sensor mapping, and asset hierarchy creation.

Fig. 1.
figure 1

Preliminary line and text detection results

Full size image

3 Methodology

In this paper, we propose a pipeline for automatically extracting key information from P&ID’s. This pipeline is based on computer vision methods to detect lines, text and symbols.

3.1 Line Detection Using Kernels

As the lines encountered in a P&ID are either vertical or horizontal and always straight in shape the use of convolutional kernels for their detection becomes well adapted. We tested a combination of a vertical and a horizontal kernels to make the line detection. Since thicker and thinner lines are connecting the different elements, we take this information within our kernels in order to capture as many line width variation as possible. We can see an example of detected lines in Fig. 1a.

As it can be seen in the case of the line detection, improvements are still needed. For future work we aim at removing spurious line detections, Some possible implementation could be to consider the areas with text as non line regions, as not to mix the detection of text and lines. This could also be extended to the regions with detected symbols. Besides, an improvement in the detection of dotted lines would also provide an easier to read result.

3.2 Text Detection Using Character Region Awareness for Text Detection

Several algorithms exists in the literature to perform this task, we chose to implement the CRAFT algorithm [2] and apply it to P&ID scanned images. Without any pre-processing of the image, the results show a good detection performance. We can see an example of detected text bounding boxes in Fig. 1b.

3.3 Symbol Detection

For this section we consider two possible approaches when a small amount of data is available and when we have access to a bigger dataset. Having very few examples in our dataset and the lack of annotated images, we opted for a morphological approach using the circle Hough Transform. As P&ID come in different sizes and resolutions, different symbols may come in different sizes. To try this approach, we opted to focus on the detection of primary device and control symbols. In order to have a robust detection, we varied the radius of the circles in order to detect all of them. We provide an example of detected primary device and control symbols in Fig. 2a.

When datasets contain a large number of examples, other approaches can be considered, such as the use of deep neural networks. To illustrate that, annotated and trained a YOLOv5 neural network [4] to detect gate valves. We only had access to 8 P&ID’s with varying amounts of gate valves. In order to have enough examples for the training we realized random crops around the object of interest. We can see the detected valves in Fig. 2b.

The use of object detection neural network also provide for an adaptable tool to detect different types of symbol, given access to a sufficiently big dataset.

Fig. 2.
figure 2

Preliminary symbol detection results

Full size image

4 Conclusion

In this paper we proposed a pipeline for automatically extracting key information, such as text, symbols and connections from P&ID’s. This pipeline is based on various computer vision and deep learning methods inspired from state of the art approaches. The proposed symbols detection allows for a high flexibility and can easily be trained on new classes, given large volume of annotated data or the right morphological approach is used.

Despite the recent advances in the automatic feature extraction of P&ID’s using computer vision and machine learning approach, it remains a challenging task. The lack of a benchmark dataset and open access P&ID’s further increase this challenge. As text and symbol detection methods improve the focus should be given to finding the links between different symbols.

As future works, our proposed pipeline can be improved by combining recent models for better performance. It is also possible to develop a new deep learning method for symbols detection and text associations. Experiments on different real-world datasets will also be tested.