1 Introduction

The ocean occupies 70% of the earth’s surface and is rich in mineral resources, oil and natural gas energy. Therefore, the ocean is the second largest space for human survival and development. In the exploration and practice of underwater environment, underwater object measurement and underwater topographic mapping are the basis of underwater environment application. Sonar has better effect in complex water environment, so sonar imaging is often used for underwater object detection and underwater terrain mapping, such as seabed target detection [8], underwater environment mapping [18], hull detection [10], fish type detection [1], underwater Operations [7], submarine rescue [11], etc.

At present, two-dimensional sonar imaging is often used in underwater target detection and target classification. Yu et al. studied the application of sonar to detect image targets based on depth learning technology. A sonar image denoising method based on multi-resolution tool is proposed. The natural image is divided into blocks at an appropriate rate, and the underwater natural image is detected [24]. Wei et al. proposed a sonar image target detection and recognition method that can run in real time on low-power equipment. It uses the support vector machine model to classify the manually extracted target features, and uses the category voting mechanism to determine the target category [17]. In order to overcome the shortcomings of traditional algorithms in sonar image feature extraction, Tang et al. Applied mask RCNN instance segmentation network to sonar image feature extraction. The experimental results show that the average accuracy of the proposed network is improved [22]. Alan studied the potential of three self supervised learning methods to learn high-quality sonar image representation without manual labels. The experimental results show that the classification performance of self supervised pre training is equivalent to that of supervised pre training [19].

Object measurement based on image is a solution process of reverse engineering [9], this way records three-dimensional objects in the form of two-dimensional images, and calculates size information and angle information of the object according to the relevant features in two-dimensional images [6]. Most of the reverse engineering are stereo measurement based on binocular or multi vision [20], monocular measurement [16], scanning measurement based on structured light [12] or surface measurement based on moving optical flow [25]. These measurement methods can be summarized into two categories, one is using the known length and angle to calculate the relevant information of the object, the other method obtains the position information of object in the unified scene and restores the size information according to the geometric relationship between different angles [26]. In our paper, we use sonar images and landmarks to measure underwater objects, so our method belongs to the first method. In previous studies, sonar imaging is not used for underwater object measurement with a third-party reference object [4]. This non reference measurement method is suitable for objects with known size and work that does not need accurate data. However, when size of the object is unknown or accurate data are needed in the experiment, the traditional non reference underwater object measurement can not meet the requirements. Therefore, a reference underwater measurement method based on sonar image is needed.

By analyzing the existing advanced research methods, we found that scholars did not measure the underwater target according to the sonar image, but more based on the image for target detection and recognition. We guess that the sonar image is noisy and has low resolution, which is easy to cause large measurement error. Therefore, it is necessary to explore a more accurate underwater object measurement method based on sonar images and third-party reference objects. The known size of the third-party reference object can make up for the defect of sonar image measurement method. Based on the above problems, this paper proposes an underwater measurement method based on sonar images and artificial landmarks.

The main contributions of this paper are as follows:

  1. I.

    In our method, the angle and size information of the target object can be solved through the known size of the landmarks, in the process of solving the target angle, we propose a method to measure the target angle based on the cosine theorem.

  2. II.

    Based on the sonar images of artificial landmarks and objects, we can directly measure the size and angle information of the objects. Before solving, we use image processing technology to extract the key points of the artificial landmarks image, so as to provide accurate coordinates for the subsequent solving process.

  3. III.

    In the process of solving the size, we propose a three-dimensional conversion matrix, which can realize the direct conversion from sonar image to real size, and the accuracy can reach the millimeter level.

2 Angle measurement method

Sonar data acquisition needs to be completed with the help of underwater robots or unmanned boats. But the underwater robot is unstable during the movement [21], so the position, size and direction of underwater objects in sonar images are variable [5]. In our research the sonar equipment we use is carried on an underwater robot named Blueprint Subsea-M750d. It can provide two operating frequencies, namely 750khz and 1.2 MHz. We use the 1.2 MHz frequency because the detection range of sonar is relatively small, and the contour of the measured object is clearer, which is suitable for subsequent sonar image processing. The collected sonar files have continuous images, so we obtain sonar images by intercepting frames, and the intercepting time interval is set to 1 second.

Different from optical images, sonar can record relevant parameters in the inspection process, such as angle and distance. As shown in Fig. 1, when the sonar image is opened with Oculus software, the angle and distance of the target point can be displayed in real time. As shown in Fig. 1a, under the working frequency of 1.2 MHz, the sonar detection range is 70°. We stipulate that the angle on the right side of the center line of the detection range is positive and the left side is negative, so the angle range is [−35°, 35°]. And the image The distance of the upper target point represents the actual distance from the object to the sonar. In order to better express the points on the image, we divide the one-meter detection range into four areas, and the adjacent areas are separated by 0.25m. Therefore, the point on the image is uniquely determined by the angle and distance, and expressed in polar coordinates, as depicted in Fig. 1b.

Fig. 1
figure 1

Show of sonar image in Oculus software

Full size image

In the underwater experiment, the positional relationship between the sonar equipment and the artificial landmark is shown in Fig. 2. This photo was taken perpendicular to the water surface. Sonar equipment and artificial landmarks have a certain degree of inclination in the horizontal direction, and we use about 15°. Figure 2 is simplified to Fig. 3 [15]. Point S is the position of the sonar, and the quadrilateral ABCD represents the landmark.

Fig. 2
figure 2

Relative position of sonar equipment and landmark

Full size image
Fig. 3
figure 3

Simplified diagram of the relative position of the sonar and the landmark

Full size image

In sonar image the angle and distance information of points B and E can be measured. We use polar coordinates to express B and E as B(ρB, θB), E(ρE, θE), the position of the two points is based on point S. Then α can be expressed as

$$ \alpha =\left|{\theta}_B-{\theta}_E\right| $$
(1)

By cosine theorem, we calculate the length of BE, as follows

$$ {\displaystyle \begin{array}{c}{l}_{EB}^2={l}_{SB}^2+{l}_{SE}^2-2{l}_{SB}{l}_{SE}\cos \alpha \\ {}=>{l}_{EB}=\sqrt{l_{SB}^2+{l}_{SE}^2-2{l}_{SB}{l}_{SE}\cos \alpha}\\ {}=>{l}_{EB}=\sqrt{\rho_B^2+{\rho}_E^2-2{\rho}_B{\rho}_E\cos \left({\theta}_B-{\theta}_E\right)}\end{array}} $$
(2)

where lEB is the length of EB, lSB is the length of SB and lSE is the length of SE.

We take Fig. 4 as an example to calculate the angle β in the figure. First, we get the polar coordinates of O, P and Q separately called O(ρO, θO), P(ρP, θP) and Q(ρQ, θQ). According to the above method, the length of OP and OQ can be expressed as

$$ {l}_{OP}=\sqrt{\rho_O^2+{\rho}_P^2-2{\rho}_O{\rho}_P\cos \left({\theta}_O-{\theta}_P\right)} $$
(3)
$$ {l}_{OQ}=\sqrt{\rho_O^2+{\rho}_Q^2-2{\rho}_O{\rho}_Q\cos \left({\theta}_O-{\theta}_Q\right)} $$
(4)

where lOP is the length of OP and lOQ is the length of OQ.

Fig. 4
figure 4

Display of angle to be measured in sonar image

Full size image

The angle β can be calculated by the cosine theorem of the triangle OPQ,

$$ {\displaystyle \begin{array}{c}2{l}_{OP}{l}_{OQ}\cos \alpha ={l}_{OP}^2+{l}_{OQ}^2-{l}_{PQ}^2\\ {}=>\cos \alpha =\frac{l_{OP}^2+{l}_{OQ}^2-{l}_{PQ}^2}{2{l}_{OP}{l}_{OQ}}\\ {}=>\upalpha ={\cos}^{-1}\frac{l_{OP}^2+{l}_{OQ}^2-{l}_{PQ}^2}{2{l}_{OP}{l}_{OQ}}\end{array}} $$
(5)

where lPQ is the length of PQ, and it can also be expressed as

$$ {l}_{PQ}=\sqrt{\rho_P^2+{\rho}_Q^2-2{\rho}_P{\rho}_Q\cos \left({\theta}_P-{\theta}_Q\right)} $$
(6)

Combine Formula (3), (4), (5) and (6) we obtain the expression of angle as

$$ \upalpha ={\cos}^{-1}\frac{\rho_O^2+{\rho}_P{\rho}_Q\cos \left({\theta}_P-{\theta}_Q\right)-{\rho}_O{\rho}_P\cos \left({\theta}_O-{\theta}_P\right)-{\rho}_O{\rho}_Q\cos \left({\theta}_O-{\theta}_Q\right)}{\sqrt{\rho_O^2+{\rho}_P^2-2{\rho}_O{\rho}_P\cos \left({\theta}_O-{\theta}_P\right)}\sqrt{\rho_O^2+{\rho}_Q^2-2{\rho}_O{\rho}_Q\cos \left({\theta}_O-{\theta}_Q\right)}} $$
(7)

In this section, we use the sonar image and software to calculate the length of the two points of the landmark, and further we use the trigonometric function to calculate the actual size of the target angle in the three-dimensional space. The fourth part of the experiment shows that the error of our method is very small, within 5%.

3 3D mapping algorithm

In our proposed algorithm, the landmark is used as a third-party reference and its location is fixed. We use sonar-equipped underwater robots to obtain underwater environment data from different angles. As shown in Fig. 5, we display sonar data on the software.

Fig. 5
figure 5

Sonar images at different measurement angles

Full size image

In our method, P is the artificial landmark plane and Q is the projection plane. In underwater measurement, the size and angle information of the target is unknown. If M can be calculated, the target information can be reversed from the sonar image. For artificial landmarks, P is known data, and Q can also be obtained by the above method, so the task of solving M is handed over to the artificial landmarks. The process of solving M is described below.

In the underwater sonar measurement, the angle between detection plane and the horizontal plane is α, the angle between the landmark horizontal plane and the detection plane is β, the angle between the landmark and the projection plane is γ and the projection plane is perpendicular to the detection direction, as shown in Fig. 6. Therefore, the relationship between the three is as follows

$$ \left\{\begin{array}{c}\upbeta =\upalpha \\ {}\upgamma +\upbeta =\upgamma +\upalpha =90{}^{\circ}\end{array}\right. $$
(9)
Fig. 6
figure 6

The position relation between object, projection and detection plane

Full size image

We equate the object plane to the above-mentioned P plane and the projection plane to the above-mentioned Q plane. Then, we establish a rectangular coordinate system on P, which has the same scale as the real object and a polar coordinate system Q based on the sonar image. In order to facilitate the calculation, the polar coordinates are changed to rectangular coordinates. The point on P is transformed into the point on Q through the mapping of M, as shown in Fig. 7.

Fig. 7
figure 7

The mapping from P to Q by M

Full size image

We record points on P as A(x1, y1), B(x2, y2), C(x3, y3), D(x4, y4) and points on Q as \( {\mathrm{A}}^{\prime}\left({x}_1^{\prime },{y}_1^{\prime}\right),{\mathrm{B}}^{\prime}\left({x}_2^{\prime },{y}_2^{\prime}\right),{\mathrm{C}}^{\prime}\left({x}_3^{\prime },{y}_3^{\prime}\right),{\mathrm{D}}^{\prime}\left({x}_4^{\prime },{y}_4^{\prime}\right) \).The relationship between them can be expressed as

$$ \left\{\begin{array}{c}{A}^{\prime }=\boldsymbol{M}A\\ {}\begin{array}{c}{B}^{\prime }=\boldsymbol{M}B\\ {}{C}^{\prime }=\boldsymbol{M}C\\ {}{D}^{\prime }=\boldsymbol{M}D\end{array}\end{array}\right. $$
(10)

M is a 3 × 3 matrix, which can do rotation and translation transformation. We express M as

$$ \boldsymbol{M}=\left[\begin{array}{ccc}{m}_{11}& {m}_{12}& {m}_{13}\\ {}{m}_{21}& {m}_{22}& {m}_{23}\\ {}{m}_{31}& {m}_{32}& {m}_{33}\end{array}\right] $$

For ease of calculation, we add a dimension to the coordinates to form (x, y, 1)T, and formula (10) can be expressed as

$$ \left[\begin{array}{cc}{\mathrm{x}}_1^{\prime }& {\mathrm{x}}_2^{\prime}\\ {}{y}_1^{\prime }& {y}_2^{\prime}\\ {}1& 1\end{array}\kern0.5em \begin{array}{cc}{\mathrm{x}}_3^{\prime }& {\mathrm{x}}_4^{\prime}\\ {}{y}_3^{\prime }& {y}_4^{\prime}\\ {}1& 1\end{array}\right]=\left[\begin{array}{ccc}{m}_{11}& {m}_{12}& {m}_{13}\\ {}{m}_{21}& {m}_{22}& {m}_{23}\\ {}{m}_{31}& {m}_{32}& {m}_{33}\end{array}\right]\left[\begin{array}{cc}{x}_1& {x}_2\\ {}{y}_1& {y}_2\\ {}1& 1\end{array}\kern0.5em \begin{array}{cc}{x}_3& {x}_4\\ {}{y}_3& {y}_4\\ {}1& 1\end{array}\right] $$
(11)

M is invertible matrix, it can also be expressed as \( M=\left[\begin{array}{ccc}{m}_{11}^{\prime }& {m}_{12}^{\prime }& {m}_{13}^{\prime}\\ {}{m}_{21}^{\prime }& {m}_{22}^{\prime }& {m}_{23}^{\prime}\\ {}{m}_{31}^{\prime }& {m}_{32}^{\prime }& 1\end{array}\right] \), according to the properties of the matrix. So formula (11) can be simplified as

$$ \left[\begin{array}{cc}{\mathrm{x}}_1^{\prime }& {\mathrm{x}}_2^{\prime}\\ {}{y}_1^{\prime }& {y}_2^{\prime}\\ {}1& 1\end{array}\kern0.5em \begin{array}{cc}{\mathrm{x}}_3^{\prime }& {\mathrm{x}}_4^{\prime}\\ {}{y}_3^{\prime }& {y}_4^{\prime}\\ {}1& 1\end{array}\right]=\left[\begin{array}{ccc}{m}_{11}^{\prime }& {m}_{12}^{\prime }& {m}_{13}^{\prime}\\ {}{m}_{21}^{\prime }& {m}_{22}^{\prime }& {m}_{23}^{\prime}\\ {}{m}_{31}^{\prime }& {m}_{32}^{\prime }& 1\end{array}\right]\left[\begin{array}{cc}\begin{array}{cc}{x}_1& {x}_2\\ {}{y}_1& {y}_2\\ {}1& 1\end{array}& \begin{array}{cc}{x}_3& {x}_4\\ {}{y}_3& {y}_4\\ {}1& 1\end{array}\end{array}\right] $$
(12)

Construct a matrix A, which satisfies the following relationship

$$ \left[\begin{array}{cc}{x}_1& {x}_2\\ {}{y}_1& {y}_2\\ {}1& 1\end{array}\kern0.5em \begin{array}{cc}{x}_3& {x}_4\\ {}{y}_3& {y}_4\\ {}1& 1\end{array}\right]\boldsymbol{A}=\mathrm{E} $$
(13)

According to formulas (12) and (13), we continue to deduce

$$ {\displaystyle \begin{array}{c}\left[\begin{array}{cc}{\mathrm{x}}_1^{\prime }& {\mathrm{x}}_2^{\prime}\\ {}{y}_1^{\prime }& {y}_2^{\prime}\\ {}1& 1\end{array}\kern0.5em \begin{array}{cc}{\mathrm{x}}_3^{\prime }& {\mathrm{x}}_4^{\prime}\\ {}{y}_3^{\prime }& {y}_4^{\prime}\\ {}1& 1\end{array}\right]\left[\begin{array}{c}\begin{array}{ccc}{a}_{11}& {a}_{12}& {a}_{13}\\ {}{a}_{21}& {a}_{22}& {a}_{23}\end{array}\\ {}\begin{array}{ccc}{a}_{31}& {a}_{32}& {a}_{33}\\ {}{a}_{41}& {a}_{42}& {a}_{43}\end{array}\end{array}\right]=\left[\begin{array}{ccc}{m}_{11}^{\prime }& {m}_{12}^{\prime }& {m}_{13}^{\prime}\\ {}{m}_{21}^{\prime }& {m}_{22}^{\prime }& {m}_{23}^{\prime}\\ {}{m}_{31}^{\prime }& {m}_{32}^{\prime }& 1\end{array}\right]\left[\begin{array}{cc}\begin{array}{cc}{x}_1& {x}_2\\ {}{y}_1& {y}_2\\ {}1& 1\end{array}& \begin{array}{cc}{x}_3& {x}_4\\ {}{y}_3& {y}_4\\ {}1& 1\end{array}\end{array}\right]\left[\begin{array}{c}\begin{array}{ccc}{a}_{11}& {a}_{12}& {a}_{13}\\ {}{a}_{21}& {a}_{22}& {a}_{23}\end{array}\\ {}\begin{array}{ccc}{a}_{31}& {a}_{32}& {a}_{33}\\ {}{a}_{41}& {a}_{42}& {a}_{43}\end{array}\end{array}\right]\\ {}=>\left[\begin{array}{cc}\begin{array}{cc}{\mathrm{x}}_1^{\prime }& {\mathrm{x}}_2^{\prime}\\ {}{y}_1^{\prime }& {y}_2^{\prime}\\ {}1& 1\end{array}& \begin{array}{cc}{\mathrm{x}}_3^{\prime }& {\mathrm{x}}_4^{\prime}\\ {}{y}_3^{\prime }& {y}_4^{\prime}\\ {}1& 1\end{array}\end{array}\right]\left[\begin{array}{c}\begin{array}{ccc}{a}_{11}& {a}_{12}& {a}_{13}\\ {}{a}_{21}& {a}_{22}& {a}_{23}\end{array}\\ {}\begin{array}{ccc}{a}_{31}& {a}_{32}& {a}_{33}\\ {}{a}_{41}& {a}_{42}& {a}_{43}\end{array}\end{array}\right]=\left[\begin{array}{ccc}{m}_{11}^{\prime }& {m}_{12}^{\prime }& {m}_{13}^{\prime}\\ {}{m}_{21}^{\prime }& {m}_{22}^{\prime }& {m}_{23}^{\prime}\\ {}{m}_{31}^{\prime }& {m}_{32}^{\prime }& 1\end{array}\right]\left[\begin{array}{ccc}1& 0& 0\\ {}0& 1& 0\\ {}0& 0& 1\end{array}\right]\\ {}\begin{array}{c}=>\left[\begin{array}{cc}\begin{array}{cc}{\mathrm{x}}_1^{\prime }& {\mathrm{x}}_2^{\prime}\\ {}{y}_1^{\prime }& {y}_2^{\prime}\\ {}1& 1\end{array}& \begin{array}{cc}{\mathrm{x}}_3^{\prime }& {\mathrm{x}}_4^{\prime}\\ {}{y}_3^{\prime }& {y}_4^{\prime}\\ {}1& 1\end{array}\end{array}\right]\left[\begin{array}{c}\begin{array}{ccc}{a}_{11}& {a}_{12}& {a}_{13}\\ {}{a}_{21}& {a}_{22}& {a}_{23}\end{array}\\ {}\begin{array}{ccc}{a}_{31}& {a}_{32}& {a}_{33}\\ {}{a}_{41}& {a}_{42}& {a}_{43}\end{array}\end{array}\right]=\left[\begin{array}{ccc}{m}_{11}^{\prime }& {m}_{12}^{\prime }& {m}_{13}^{\prime}\\ {}{m}_{21}^{\prime }& {m}_{22}^{\prime }& {m}_{23}^{\prime}\\ {}{m}_{31}^{\prime }& {m}_{32}^{\prime }& 1\end{array}\right]\\ {}=>\left[\begin{array}{ccc}{m}_{11}^{\prime }& {m}_{12}^{\prime }& {m}_{13}^{\prime}\\ {}{m}_{21}^{\prime }& {m}_{22}^{\prime }& {m}_{23}^{\prime}\\ {}{m}_{31}^{\prime }& {m}_{32}^{\prime }& 1\end{array}\right]=\left[\begin{array}{cc}\begin{array}{cc}{\mathrm{x}}_1^{\prime }& {\mathrm{x}}_2^{\prime}\\ {}{y}_1^{\prime }& {y}_2^{\prime}\\ {}1& 1\end{array}& \begin{array}{cc}{\mathrm{x}}_3^{\prime }& {\mathrm{x}}_4^{\prime}\\ {}{y}_3^{\prime }& {y}_4^{\prime}\\ {}1& 1\end{array}\end{array}\right]A\end{array}\end{array}} $$
(14)

The final result of M is expressed as

$$ \boldsymbol{M}=\left[\begin{array}{cc}{\mathrm{x}}_1^{\prime }& {\mathrm{x}}_2^{\prime}\\ {}{y}_1^{\prime }& {y}_2^{\prime}\\ {}1& 1\end{array}\kern0.5em \begin{array}{cc}{\mathrm{x}}_3^{\prime }& {\mathrm{x}}_4^{\prime}\\ {}{y}_3^{\prime }& {y}_4^{\prime}\\ {}1& 1\end{array}\right]\boldsymbol{A} $$
(15)

We record the inverse matrix of M as M−1 and suppose an unknown coordinate on P is E(x5, y5), the corresponding coordinate on Q is \( {\mathrm{E}}^{\prime}\left({\mathrm{x}}_5^{\prime },{\mathrm{y}}_5^{\prime}\right) \). Therefore, E(x5, y5) is solved as

$$ \left[\begin{array}{c}{\mathrm{x}}_5\\ {}{\mathrm{y}}_5\\ {}1\end{array}\right]={\mathrm{M}}^{-1}\left[\begin{array}{c}{\mathrm{x}}_5^{\prime}\\ {}{\mathrm{y}}_5^{\prime}\\ {}1\end{array}\right] $$
(16)

In this section, we establish two coordinate systems in the real space and the sonar image respectively. Our method can map the points in the two coordinate systems, so that the real position point information can be solved from the known points on the sonar image, and the target length can be solved by the position information of the two points in the real space coordinate system. The key of this algorithm is to solve the invertible matrix M.

4 Experiment

This paper consists of three parts. In the first part. We use image processing technology to obtain sonar image key points. The second part and the third part respectively carry out experimental verification under horizontal and inclined conditions. Our experimental equipment includes Oculus M750d side scan sonar and CHARPIE STRAIT-100 ROV underwater robot. All experiments are based on Windows10 operating system, image processing technology is based on OpenCV. In addition, we use ViewPoint software to view sonar visualization images, and use Python 3.6 resource package to process experimental data.

4.1 Processing sonar images and acquiring key points

As mentioned above, if we want to obtain the transformation matrix M, we must know the information of any four key points of the landmark. Common sonar image feature point detection methods include Oriented Fast Rotated and BRIEF (ORB) and Scale Invariant Feature Transform (SIFT) [13, 14]. We used these two methods to be applied to our sonar image respectively, as shown in the following figure. Because the sonar image is very noisy, the uncertainty of feature point detection is very high, and the key point information we really need can not be extracted. Therefore, we have chosen a key point acquisition method combined with image processing technology, which will be described below.

figure a

The shape of our landmark is quadrilateral, so it is very appropriate to select the four vertices of the landmark as the key points. However, the pixel quality of sonar image is poor and there is a lot of noise. It becomes difficult to obtain the key point information manually, and manual acquisition will also lead to a large error in calculation results. Therefore, it is particularly important to process the image to enhance and reconstruct the landmark edge in the image for further obtaining key points. Low frequency and low gray level are the inherent characteristics of sonar image, which can not be effectively optimized. Therefore, we mainly deal with the noise in sonar image before extracting edge information. The noise interference in sonar image is large, and the most common noise is speckle noise [2]. At present, median filtering is the most preferred way to deal with sonar image noise [3], and it is also recognized as a more effective filtering method. Thus, we use median filtering to process sonar images, and the result are shown in Fig. 8.

Fig. 8
figure 8

Filtered sonar gray image

Full size image

Further, in order to enhance the edge information of the landmark, we binarize the sonar image [9]. Because different parts of the sonar image have different brightness, in this case, we use the adaptive threshold method to process the image, as shown in the Fig. 9. The threshold is calculated according to each small area on the image. In this way, we distinguish the brightness difference between the boundary and the surrounding environment to enhance the target boundary.

Fig. 9
figure 9

Binarized sonar image

Full size image

As shown in the figure, the boundary of the landmark has been significantly enhanced. Next, we need to extract the boundary. There are many edge detection algorithms in image processing, such as Robert algorithm, Sobel algorithm, Laplace algorithm and so on. And Canny edge detection algorithm is proposed by John F. Canny, which is a multi-level detection algorithm [9]. It can accurately find as many edges as possible in the image and reduce missed detection and false detection as much as possible. Based on the above considerations, we finally select Canny edge detection operator to process the binarized image as shown in Fig. 10a. After processing, we get a closed edge curve at the landmark position.

Fig. 10
figure 10

Image edge detection and line fitting

Full size image

In the end, we use the least square method to fit the four boundary curves into straight lines [23]. Taking the red curve in Fig. 10b as an example, we express the fitted linear relationship as y = ax + b.

Then we get N pairs of points on the red curve and record it as (xi, yi), i = 1, 2…, the error between the predicted value axi + b and the real value yi can be expressed as

$$ \sum \limits_{i=1}^N{\left[{y}_i-\left(a+b{x}_i\right)\right]}^2 $$
(17)

For the linear fitting of equal precision observations, when a is the smallest, the result is the most accurate. We calculate the partial derivatives of a and b in formula (17) respectively,

$$ \left\{\begin{array}{c}\frac{\partial }{\partial a}\sum \limits_{i=1}^N{\left[{y}_i-\left(a+b{x}_i\right)\right]}^2=-2\sum \limits_{i=1}^N\left({y}_i-a-{b}_i\right)=0,\\ {}\frac{\partial }{\partial b}\sum \limits_{i=1}^N{\left[{\mathrm{y}}_i-\left(a+b{x}_i\right)\right]}^2=-2\sum \limits_{i=1}^N\left[{y}_i-\left(a+b{x}_i\right)\right]{x}_i=0\end{array}\right. $$
(18)

After finishing, we get the equations,

$$ \left\{\begin{array}{c} aN+b\sum {x}_i=\sum {y}_i\\ {}a\sum {x}_i+b\sum {x}_i^2=\sum {x}_i{y}_i\end{array}\right. $$
(19)

By solving the above equations, the best estimates of line parameters a and b can be obtained,

$$ {\displaystyle \begin{array}{c}\overline{\mathrm{a}}=\frac{\left(\sum {x}_i^2\right)\left(\sum {y}_i\right)-\left(\sum {x}_i\right)\left(\sum {x}_i{y}_i\right)}{\begin{array}{c}N\left(\sum {x}_i^2\right)-{\left(\sum {x}_i\right)}^2\\ {}\end{array}}\\ {}\overline{\mathrm{b}}=\frac{N\left(\sum {x}_i{y}_i\right)-\left(\sum {x}_i\right)\left(\sum {y}_i\right)}{N\left(\sum {x}_i^2\right)-{\left(\sum {x}_i\right)}^2}\end{array}} $$
(20)

The other three curves use the same method, and the fitting results are shown in Fig. 10c. In eq. 21, the intersections between the four line segments is the four vertices of the landmark which is the key points we need.

$$ \left\{\begin{array}{c}\begin{array}{c}{y}_1={\overline{a}}_1x+{\overline{b}}_1\\ {}{y}_2={\overline{a}}_2x+{\overline{b}}_2\end{array}\\ {}\begin{array}{c}{y}_3={\overline{a}}_3x+{\overline{b}}_3\\ {}{y}_4={\overline{a}}_4x+{\overline{b}}_4\end{array}\end{array}\right.--->\left\{\begin{array}{c}\begin{array}{c}A\left({x}_1,{y}_1\right)\\ {}B\left({x}_2,{y}_2\right)\end{array}\\ {}\begin{array}{c}C\left({x}_3,{y}_3\right)\\ {}D\left({x}_4,{y}_4\right)\end{array}\end{array}\right. $$
(21)

In the following two experiments, the four key points A, B, C and D are obtained by the method in this section.

4.2 Horizontal measurement

In this section, we discuss a representative case of underwater experiments. In this case, the artificial landmark is located in the center of the sonar detection range, and the posture of the landmark in the sonar image is straight, as shown in Fig. 11. Under these conditions, artificial landmarks are clear in the sonar image and are suitable for preliminary object measurement experiments.

Fig. 11
figure 11

Horizontal measurement

Full size image

We first calculate the conversion matrix M according to the known information of the landmark, and then use M to calculate the position information of the point to be measured in reality, and finally solve the actual size and angle information of the target. Obtain and analyze errors by comparing with real data.

In order to facilitate the calculation, we use formula (22) to convert the polar coordinates in the sonar image into a Cartesian coordinate system, and then establish two coordinate systems, as shown in Fig. 12.

$$ \left\{\begin{array}{c}x=\rho \sin \theta \\ {}y=\rho \cos \theta \end{array}\right. $$
(22)
Fig. 12
figure 12

Coordinate system in sonar image and coordinate system in real space

Full size image

In the sonar image, as shown in Fig. 12a, the sonar is located at the origin of the coordinate system, the y-axis is a straight line at the center of the detection range, and the x-axis passes through the horizontal origin. In reality, we choose the lower left corner of the landmark as the origin, and the x-axis and y-axis coincide with the edge of the landmark respectively.

In the two coordinate systems, we mark the information of key points. The marked points of the real coordinate system is A—I, the marked points of the sonar image coordinate system is A′ − I′, and the same letter represents the same position. Points A (A’), B (B′), C (C′), D (D′) are used to derive the conversion matrix M, points E (E’), F (F′), G (G’) are used to calculate the error size, and points H (H′), I (I′) are used to calculate the size and angle. Among them, C and D are the midpoint of A and B as the error calculation point. The information of each point is recorded in Table 1.

Table 1 Coordinates of key points in real coordinate system and sonar image coordinate system
Full size table

The transformation matrix M is calculated by four pairs of point pairs A (A’), B (B′), C (C′), D (D′), it is expressed as

$$ \mathbf{M}=\left[\begin{array}{ccc}1.044474& 0& -9.45\\ {}-0.265263& 0.945& 40.949999\\ {}-0.005526& 0& 1\end{array}\right] $$

It should be noted that the two elements of 0 are not actually 0, but are limited by the expression of calculation accuracy. Therefore, the inverse matrix of M can be expressed as

$$ {\boldsymbol{M}}^{-\mathbf{1}}=\left[\begin{array}{ccc}1.005291& 0& 9.047619\\ {}0.052910& 1.058201& -40.793652\\ {}0.005291& 0& 1\end{array}\right] $$

When calculating the error, we first solve E’, F′ and G’ in the sonar image through M, and record results as E”, F″ and G”. Then, compare them with the actual three points of E, F and G, and calculate the error. Further, we calculate the length between two points and the angle between two lines, and calculate the error. Some error comparison items are listed in Table 2. Through error results, only the G (G”) ordinate error is large, more than 5%. This matter may be caused by the blur of G point in sonar image, and there is still room for improvement. In comparing the length and angle, the calculated values are close to the real values, and the error is less than 5%.

Table 2 Comparison of error between calculated value and real value
Full size table

We use the same method to calculate the target length GH and target angle GIH in Fig. 12. The calculation results and errors are recorded in Table 3

Table 3 Calculation size and error prediction of underwater target
Full size table

In this part, we discuss the case of horizontal measurement. We use the angle measurement method and 3D mapping algorithm proposed above to solve the known information, compare the results with the real values, and further apply them to the actual measurement. Experiments show that our method can control the error within 5%.

4.3 Inclination measurement

In this section, we discuss the other case of the position of the artificial landmark in the underwater experiment. The landmark is in the center of the sonar detection range, and the attitude of the landmark in the sonar image is inclined, as shown in Fig. 13. Compared with the previous section, this situation is a more common state of artificial landmarks in underwater sonar detection.

Fig. 13
figure 13

Inclination measurement

Full size image

Select the same way as Fig. 12 to establish two coordinate systems and mark key point positions. The establishment of real coordinate system is the same as Fig. 12b, and the establishment of sonar image coordinate system is shown in Fig. 14. The information of key points is recorded in Table 4.

Fig. 14
figure 14

Sonar image coordinate system in case of Inclination measurement

Full size image
Table 4 Coordinates of key points in real coordinate system and sonar image coordinate system
Full size table

M and M−1 are calculated as

$$ \boldsymbol{M}=\left[\begin{array}{ccc}1.113492& 0.496010& -16.095022\\ {}-0.339109& 1.204596& 39.174676\\ {}0.002531& 0.005061& 1\end{array}\right] $$
$$ {\boldsymbol{M}}^{-\mathbf{1}}=\left[\begin{array}{ccc}0.71683127& -0.40321758& 25.716322\\ {}0.30444965& 0.81101722& -25.28154\\ {}-0.0031565013& -0.0029019448& 1\end{array}\right] $$

Same as the previous section, we calculate length of EF, GF and EG and angle of EFG from point E, F, and G. Analyze the error between the calculated result and the real value and record it in Table 5.

Table 5 Comparison of error between calculated value and real value
Full size table

And then, calculate the target length GH and target angle GIH in Fig. 14. The calculation results and errors are recorded in Table 6.

Table 6 Calculation size and error prediction of underwater target
Full size table

In this part, we discuss the case of Inclination measurement measurement. The accuracy of the calculation results reaches the millimeter-level, and the calculation results of the target length are more accurate, some with an error of less than 3%. Combined with the calculation results in the previous section, the calculated values in two different scenarios are relatively close, and the range of calculated actual values has intersection, which mutually verifies the measurement results of underwater target size and angle.

5 Conclusion

This paper presents an underwater target measurement technique based on sonar images and artificial landmarks. In the process of solving the target angle, we use the cosine theorem to propose an angle measurement method. This method can calculate the target angle in sonar image. A three-dimensional mapping algorithm is proposed in the process of solving the size. The transformation of target points from sonar image to real space is realized by solving the mapping matrix. Finally, the target length is calculated by using the two-point position information in the actual spatial coordinate system. In order to find the key points more accurately, we use image processing technology to restore the contour of artificial landmarks and obtain the key point information. The experimental results at two different relative positions show that the accuracy of the calculation results of our proposed measurement technology reaches the millimeter level, and the error is controlled within 5%. The calculation results of the target length are more accurate, and can reach less than 3%. For the first time, the body proposes to use the third-party reference object in sonar imaging to measure the target. Our technology is more accurate than the previous non reference sonar measurement, and it is an effective attempt to improve the sonar measurement accuracy.