1 Introduction

Since the beginning of this century, CubeSats have received very much attention both from space research centres and universities, due to their short development time, extremely low cost (compared to larger satellites) and specific purpose applications, features that pose them as an ideal tool for research, technology demonstration and academic purposes. With novel and more resource-demanding missions for CubeSats, particularly for remote sensing like the Flock CubeSat Constellation which provides Earth images with a 3-5 m spatial resolution [1, 2], a high stress is placed in the development and performance of attitude determination and control system (ADCS). In addition, the robustness of ADCS and the determination of the current satellite orientation and orbit behaviour depend on the number of installed sensors, and these are always mission oriented [3, 4].

Beside Earth observation or remote sensing satellites, there are missions that demand high ADCS capabilities, such as narrow beam communication systems. Nowadays, new communication techniques have been implemented to increase the data rates and bandwidth, which involve modern modulation schemes or the design of highly directive antennas to operate at higher frequencies [5]. In this sense, conventional RF-based communication systems employed in CubeSat satellites involve a deployable monopole as well as patch antennas. While deployable antennas are arranged in a way to allow omnidirectional transception of signals, patch antennas for this type of satellites require a more directional treatment of signals since their radiation patterns are narrower. Nonetheless, the later ones allow a better space configuration to fit the size and power of CubeSats [6, 7].

As current demand of satellite services is continuously increasing, implying higher requirements for bandwidth and power consumption, it is necessary to develop modern designs and technologies to overcome such problems, not only in nano-satellites, but also in conventional satellites. The later fact is of particular importance since the technological limit of conventional satellite communication devices has been reached [8].

Optical communications or space-based laser communications (lasercom) have the potential to transform scientific, defense, and commercial spacecraft communications platforms. Most CubeSats have not been able to achieve more than a few MB of total downloaded information over the course of an orbit since their data rates are \(\sim 9.6\) kbps, particularly due to their low earth orbits (LEO). According to [9, 10], it has been possible to reach a data rate up to 3 Mbps by implementing amplitude shift keying (“ASK”) systems. However, such data rates are insufficient for the increasing demands of the market. Lasercom has the potential to increase data rate substantially, as it has proved to be able to provide values higher than 5 Gbps [11,12,13,14].

Compared with RF communications, lasercom offers higher bandwidth, reduced size and mass of transceivers, lower power consumption, interference immunity and a higher gain, avoiding the significant regulatory hurdles of radio frequency allocation [15, 16]. Particularly, nano-satellites would be largely benefited from lasercom as they are heavily constrained in size, weight and power (SWaP).

However, lasercom faces several challenges for in-orbit CubeSats. Lasercom main drawbacks are atmospheric loss since involved frequencies are very susceptible to atmospheric effects, as well as the accuracy of pointing, acquisition, and tracking (PAT) subsystem. While narrow transmission beam widths increase lasercom link efficiency, the trade-off is the necessity for a very precise PAT.

Lasercom systems ought to align the optical line-of-sight (LoS) with an accuracy of the order of submicroradians, depending on the satellite altitude. Typical pointing accuracy for laser satellite communication systems could vary between 1 and 200 \(\mu rad\) [17, 18]. The precision of the pointing system must be at least of \(0.25^\circ \) to achieve a 50 Mbps consistent transmission data rate for a LEO-ground link (at 350 km) [19, 20]. Optical technology has been used for communications since the last century. However, its implementation into satellite communications has only been tested since 2001 in different inter-satellite experiments, and since 2005, in satellite to ground station linking experiments, from several space agencies and research centres worldwide.

Up to now, the best reported pointing accuracy for nanosatellites was achieved by the BRITE mission in 2014 and the MinXSS mission in 2016 [21]. The BRITE constellation achieved a pointing accuracy of \(0.0115^{\circ }\) using a miniature star tracker, a sun sensor, a magnetometer, as well as 30 mNms miniature reaction wheels [22]. The MinXSS mission achieved a pointing accuracy of \(0.002^{\circ }\) on a 3U CubeSat using a solar position sensor and a miniature X-ray solar spectrometer, while the ADCS hardware consisted miniature reaction wheels, torque rods, a star camera, a coarse sun sensor, inertial measurement units, and magnetometers [23]. Clearly, the larger the number of sensors and actuators, the better the satellite’s attitude, providing more accurate angles for pointing. Nevertheless, the number of sensors and actuators is always restricted by the SWaP constrictions imposed by the CubeSat standard [24].

Fig. 1
figure 1

PAT sequence for an optical communications link [50]

Full size image
Fig. 2
figure 2

Flowchart of the proposed detection algorithm

Full size image

In recent years, a large number of lasercom links has been demonstrated, in different linking scenarios including LEO-to-ground [25,26,27,28], LEO-to-LEO [29, 30], GEO-to-ground [31,32,33,34], GEO-to-LEO [35,36,37], deep space-to-ground [38, 39], ground-to-deep space [40], GEO-to-aircraft [41], aircraft-to-ground [42,43,44], aircraft-to-aircraft [45, 46], and stratospheric balloon-to-balloon [47].

The vast majority of lasercom systems use beacon tracking to locate the ground station or satellite. In the approach undertaken in this work, the ground station sends up a wide beam at a predetermined wavelength towards the spacecraft and vice versa. Initial conditions for the spacecraft consist of orbital pointing acquaintance. The sequence begins with initial pointing towards the expected location of the satellite. Orbital acquaintance usually comes from global positioning system (GPS), radar tracking, or two-line element sets (TLEs), and any error in the spacecraft’s position is translated into mispointing. The mispointing induced by position error increases when the spacecraft is closer to the ground station. Pointing acquaintance also induces error as the spacecraft relies on a combination of gyroscopes, magnetometers, accelerometers, sun sensors, Earth horizon sensors, or star trackers to determine its orientation with a limited accuracy [48].

In this sense, this paper proposes a low-cost computer vision system prototype as an on-board PAT device, capable of assisting the ADCS subsystem in early stages of the beacon signal reconnaissance. The proposed computer vision system is a visual feedback method for the correction of orientation and aligning of the satellite, providing the coordinates of the detected signal to reduce the response time of the ADCS, so to be able to establish a lasercom link.

This system is intended to begin its operation before the first pointing stage, when the satellite begins to transmit its orientation to the ground station’s approximated location, and it remains under operation during the link maintenance stage when the satellite is in LoS. In this way, the visual feedback provided by the system yields the desired coordinates to locate the ground station. The purpose is to decrease PAT errors and response time. Furthermore, parameters from the system can be transmitted through the RF telemetry system to aid in finding the satellite from the optical ground station. In this sense, CubeSat lasercom does not completely substitutes RF communications, which are intended for operations of the satellite, but it is intended to transmit payload data.

The low-cost computer vision system developed in this work, implemented in a CubeSat prototype as a fine pointing mechanism to locate a theoretical ground station from the CubeSat point of view, relies in the tracking algorithm optimised to reduce the visual perturbations for high-lighting inferences. This system exploits the on-board computer capabilities and the remote sensing camera to determine the location of the ground station beacon and it is fully developed using commercial COTS components. The structure of this work is as follows: in Sect. 2, we present the computer vision algorithm and the PAT stages, while in Sect. 3, we show the image detection and processing phase. In Sect. 4, we show the results while in Sect. 5 we pose interesting final remarks.

2 Computer Vision and PAT System Stages

The PAT proposal herein presented consists of a computer vision software able to identify and follow a ground station beacon from the satellite’s point of view, considering the constraints of current CubeSat technology in terms of SWaP requirements. In this sense, it is necessary to extract spots from the image to further determine which are to be followed and which not. To achieve the blob extraction of a single image, some criteria must be considered, such as the colour of the object, shape or a specific pattern.

Computer vision implementation consists of the four stages [49]: image acquisition, segmentation, parametrisation, and interpretation. Image acquisition requires sensors and the capability to digitalise the produced signals. The objective of the segmentation stage is to divide the image into its constitutive parts. In general terms, the autonomous segmentation is one of the hardest issues of digital image processing. The outcome of the segmentation stage process is an image containing different pixels’ regions that were distinctly classified. The parametrisation consists in extracting data from the image to generate quantitative information, i.e., characteristic features to distinguish one object class from another. The interpretation stage refers to assigning a meaning to the recognised sets of objects.

Figure 1 shows the typical stages for a PAT system to establish satellite-to-ground lasercom. By adapting this scheme for the use of computer vision as a coarse scan mechanism, stage 1 begins with the pointing of the transmitter towards a predicted rough location of the ground station. Stage 2 executes the computer vision algorithm to locate the beacon emitted by the ground station. Finally, stages 3–5 perform the tracking and acquisition of ground station via a fine pointing mechanism. In case of losing the link, there is a loop that reboots the process from the second stage. This fact settles the importance of the beacon detection so the system can proceed to perform a correct tracking [48].

3 Image Detection and Processing

The proposed system was developed using low-cost COTS components. Since most CubeSats are endowed with a remote sensing camera as one of the payloads, this computer vision system takes advantage of it. The system is implemented in a Raspberry® Pi 3, model B, which assists the on-board computer. This module features a Quad-core Cortex A7 at 900 MHz GPU, a VideoCore IV CPU, and 1 GB DDR2 RAM. Its dimensions are 85 mm length by 57 mm width, and it has a mass of 45 g. Programming is through Matlab® environment. The voltage and current requirements of this embedded system are of 5 V DC at 2 A. This power resources fit well within the operational margins of an on-board computer for a CubeSat. For the prototype, a Picam camera module V2.1 and a fisheye lens are used together with the Raspberry module. The camera module has a 5MP resolution as well as video modes of 1080p30, 720p60, \(640\times 480\) p60/90. The camera resolution used in these tests is \(640\,\times \,480\), since it is the only video mode supporting a refresh rate of 90 FPS, providing shorter image acquisition times. Furthermore, the fisheye lens offers a 180\(^{\circ }\) field of view. The Raspbian© lite operative system was employed because it operates fast and it was able to correctly execute the algorithm.

Figure 2 depicts the flux diagram of the computer vision algorithm, where the input of a digital image is the image acquisition stage, thresholding and binarisation are part of the segmentation stage, the beam detection corresponds to the parametrisation stage and finally, the interpretation stage is comprised by the plotting of centroid and tracking. In Fig. 2, initial parameters [51, 52] correspond to the initialisation of the system as well as to the verification of the correct connection of the camera and lens. Input a digital image [51, 52] stage refers to the camera recording and the extraction of digital images frame by frame from the video recording. The basic nature of an image, represented by f(x, y), is mainly described by two components: the illumination i(x, y) and reflectance r(x, y). The forming process of an image involves the following elements: the object, the radiant source, and the image forming system. Such system consists in an optical system, a sensor and a digitalisation device. Digital image can be depicted as a matrix f of dimensions of \(N\times M\), whose elements are called pixels,

$$\begin{aligned} f=\begin{bmatrix} f(1,1)&{} \cdots &{}f(1,M) \\ \vdots &{} \ddots &{} \vdots \\ f(N,1) &{} \cdots &{}f(N,M) \\ \end{bmatrix} \ . \end{aligned}$$
(1)

For this particular experimental setup, we consider that the acquired images contain the beam, which is directly pointing towards the CubeSat, since the ground station along with the beam are constantly following the satellite’s orbit. However, minor corrections in the CubeSat attitude are required, which provoke only small distortions of the beam. This detection and tracking mechanism helps to align the beam with the camera correctly, and it implies the manipulation of the CubeSat through ADCS, aiding to the CubeSat’s lasercom link as well as to the first stage of the orientation of the satellite.

The image processing is performed on every frame obtained from the 90 FPS video. To achieve the beacon detection, the beam colour emitted by the optical ground station (OGS) ought to be considered. After selecting the beam colour, thresholding is performed to convert the image from RBG to a two-level image (black and white). Thus, upper and lower threshold limits must be assigned. Figure 3 shows thresholding of the beacon as the camera receives the image.

Fig. 3
figure 3

Selection (tuning) of upper and lower threshold limits of captured images

Full size image

Thresholding in Fig. 2 is responsible of selecting the corresponding pixels to be tracked. For instance, Fig. 3 depicts a beam directly incident to the camera. By thresholding such image, the system recognises that the area to be tracked is the white one, while the green background is not useful and can be discarded. The binarisation stage (see Fig. 2) is performed by comparing each input image pixel with the upper and lower limit range of colours obtained by the thresholding tuning [51, 52]. The lower and upper thresholding RGB components are (rl, gl, bl) and (ru, gu, bu), respectively, and they provide a range of colours with the value 1, thus discriminating the background pixels which are assigned to 0.

Once the binarised image is obtained, morphological operations are used to correct free pixels, to improve the image segmentation. The morphological processing herein used is based on the closing method that consists in the dilatation and erosion of the binarised image. Erosion removes small-scale details from a binary image, and it simultaneously reduces the size of the regions of interest. The dilatation of the binarised image increases the recognised area by assigning 1 to the neighbour pixels of the detected object. In the dilatation process, a disc morphological structure is created to improve the binarised image quality; the purpose of this process is to detect a circular area of about 10 pixels of radius,Footnote 1 i.e. beam detection. The dilatation process is performed over a region of 10 pixels around the centroid, improving the quality of the binarised image.

For the binarised picture, there is a filtering process called “black and white area open” (bwareaopen) to eliminate white pixel regions under a certain pixel density, thus correcting the noise that the binarisation generates for areas of under 30 illuminated pixels. The final image is a black and white representation of the original picture, being in white the centroid and discriminating everything else. The calculation of the centroid and the tracking (see Fig. 2) are the final stages corresponding to a graphic that indicates the location of the beam [51, 52]. Finally, the process repeats itself for every recorded frame. Figure 4 shows a block diagram of the system for each step of the process.

Fig. 4
figure 4

Computer vision algorithm hybrid programming

Full size image
Fig. 5
figure 5

Prototype stages

Full size image

4 Algorithm Results

Once the morphological operations have been applied, the command regionprops is used to obtain the centroid’s coordinates. At the beginning of the algorithm, two empty vectors are declared \(C_{x}=[]\) and \(C_{y}=[]\), which store the detected object centroid coordinates, to print the object tracking.

Considering lighting changes in controlled and non-controlled conditions, a threshold tuning mechanism is proposed for the detection stage to avoid errors in the beam recognition and detection due to environmental lighting variations. This denotes the importance of the herein proposed thresholding process since by selecting upper and lower limits for each RGB component, the discrimination of the background image pixels is realised. Figure 5a shows the computer vision system embedded in a 1U CubeSat prototype. To increase the field of view of the system, a fisheye lens is used as the image-capturing device to allow a better centring of the image and to accomplish an improved space detection. This system proved to be effective for distances of less than 75 m. However, a second iteration of the prototype is presented in Fig. 5b, which includes a coupled gimbal-oriented camera automated by servo motors, thus achieving better results for distances up to 2 km in the performed tests.

Fig. 6
figure 6

Results of the detection and recognition of the beam

Full size image
Fig. 7
figure 7

Tracking of the ground station beam trajectory, considering sudden motion of the light source

Full size image

The controlled lighting environment test were performed in a closed laboratory, where the distance between the light source to be detected and the CubeSat is of 75 m, and the beacon light was a non-coherent source to test the capabilities of the system. This test revealed that the fisheye-based system only detects the beacon up to a distance of 50 m. For larger distances, this tracking technique clearly requires the implementation of a high-power beacon emitted from the ground station. The non-controlled tests were performed in outdoors conditions, for both the fisheye and the automated versions of the prototype. The automated version features a servo motor-controlled camera, and it increased the accuracy of the system, allowing to establish a land-to-land link from 75 m and up to 2 km. Non-controlled lighting tests were performed with a coherent beacon, which consists of a 3 W laser working at a wavelength of 532 nm. We did not perform tests to establish land-to-land links for further distances due to the difficulty to find clear LoS in an urban environment, as well as to the lack of more powerful laser beacons. Nevertheless, [53] considers that the minimum required power to establish a LEO-to-ground station link is of 6 W. If such theoretical laser beacon has a divergence angle of \(1.4^{\circ }\), it could guarantee a reconnaissance diameter of 8.55 km considering a 350 km of altitude LEO. With this configuration, the beacon could be detected from the perspective of the mentioned LEO satellite, so it should be directed towards its the predicted orbital position.

Figure 6 shows the results of the recognition, detection and tracking of the beam trajectory. The behaviour of the computer vision algorithm in the different lighting condition proves to provide a good performance for the detection and recognition of the object. Furthermore, Fig.6 shows the original video, beam detection and tracking in real-time sections. The beam detection stage proves that the closing method works well for the elimination of the noise, improving the segment to be tracked. The tracking stage in Fig. 6 shows the detection of the object’s centroid, where the cross mark represents the coordinates of the centroid in a two-dimensional plane.

To verify the monitoring of the followed trajectory, a gimbal-controlled prototype is used to locate a moving light source. This fact changes the luminosity received by the computer vision system. Figure 7 shows the tracking after the beacon source is moved. The centroid’s coordinates are captured in real time, and the system reacts well in controlled and non-controlled lighting environments. Further tests were performed in more complex lighting conditions for both the controlled and the non-controlled environments. To perform them, additional light sources and reflective elements were added to the outdoors setup, changing the distances between the beacon and the system between 10 m and 2 km. These tests results confirm that the system is still able to determine the beacon light in these complex lighting scenarios. Furthermore, since the used camera is a low-resolution COTS option, more demanding tests are unable to be performed well under higher interference conditions. However, the camera used in this work allowed to prove the effectiveness of the proposed algorithm to distinguish the ground station beacon from other light sources, which is the main purpose of this work. Tests with more powerful camera options are part of future work.

Figure 8 shows the tracking results in a more hostile lighting environment for the detection process in the controlled lighting conditions, which includes reflections of the image. Figure 9 adds more intense light sources to the environment. Figures 8 and 9 show the quality in the tracking performed by this system. The original video section shows the interference around the beacon. Moreover, the beam detection screen always shows the white centroid of the theoretical ground station beacon. Figures 8 and 9 also reveal that, by moving the CubeSat in a severe way, the tracking screen always finds the beacon, not showing important delays in the tracking and not losing the centroid. Even in highly reflective environments, the system identifies the correct beacon and tracks it correctly.

Fig. 8
figure 8

Tracking of the ground station beacon in a reflective environment, with different colour light sources

Full size image
Fig. 9
figure 9

Tracking of the ground station beacon in a highly contaminated luminous environment

Full size image

With respect to response delays of the algorithm, the processing time of the proposed computer vision system depends upon the image’s size, the camera shutter speed and the processor frequency. The used camera offers two different progressive resolutions: 720p at 60 fps and 1080p at 30 fps, providing image reception speed per frame of about 16.66 ms and 33.33 ms, respectively. Table 1 shows the execution times for key stages of the algorithm, for the video modes of this camera.

Table 1 Execution times for different stages of the computer vision system
Full size table

As resolution increases, execution times diverge from a real-time tracking (see Table 1). However, the algorithm is executed in a low-cost COTS processor frequency of 1.2 GHz and 1 GB of RAM, achieving an execution time of 0.361 s. In this sense, other processing platforms can provide almost real-time tracking for higher resolutions, as independent on-board graphics cards [54].

5 Conclusion

In this work, a computer vision-based PAT system for a 1U CubeSat is presented. The image processing by binarisation, thresholding and segmentation stages works well in environments with varying light conditions. The closing method proved that it is able to improve the received image quality to ease the object detection. The system is capable of achieving an accurate tracking, even in adverse lighting conditions with simultaneous light sources, as well as considering sudden motion of the receptor with respect to the beacon source.

The system, executed in a low-cost COTS platform, proves to be fast, stable and solid when operating from a Raspberry PI 3 platform endowed with one of its most basic operative systems. This prototype follows miniaturisation trends so it can be embedded in a 1U CubeSat structure, because its SWaP stay within the range imposed by the CubeSat standard for a 1U satellite [24], not stressing power and mass budgets for design and development purposes. This proposal can be considered as a support system that works efficiently for the fine pointing mechanism required for lasercom, particularly for LEO-to-ground links.

The usage of low-cost COTS components to implement this computer vision system allowed to determine its precision and effectiveness, opening opportunities for students and universities to stimulate learning and research in aerospace science and optically based space communications. Moreover, these results help the reader wishing to implement the system in a real CubeSat mission as a guide in the selection of a processing platform, a camera and the required beam source, fostering the introduction and popularisation of lasercom for LEO-to-ground optical links in nano-satellites, as those built under the CubeSat standard [24].