A survey of quantum image representations

Abstract

Quantum image processing (QIMP) is devoted to utilizing the quantum computing technologies to capture, manipulate, and recover quantum images in different formats and for different purposes. Logically, percolating this requires that representations to encode images based on the quantum mechanical composition of any potential quantum computing hardware be conjured. This paper gathers the current mainstream quantum image representations (QIRs) and discusses the advances made in the area. Some similarities, differences, and likely applications for some of the available QIRs are reviewed. We believe this compendium will provide the readership an overview of progress witnessed in the area of QIMP while also simulating further interest to pursue more advanced research in it.

1 Introduction

Image processing is popular in our daily life because of the need to extract essential information from our 3D world. The nature of visual information, algorithm complexity, and the representation of 3D scenes in 2D spaces are all popular research topics [96]. Due to the rapid development of quantum computation and quantum information, notably Feynman’s quantum computation model [20], Deutsch’s quantum parallelism assertion [16], Shor’s integer factoring algorithm [80], and Grover’s database searching algorithm [25] in the past several decades, it is considered that the analysis of previously mentioned problems under the light of quantum computation and quantum information may result in new ways of understanding the nature of visual information [96].

Quantum image processing (QIMP), an area focused on extending conventional image processing tasks and operations to the quantum computing framework, is the new sub-area that has emerged in that regard. The time line for this sub-discipline’s development shows its modest beginning started in 2003, when Venegas-Andraca and Bose proposed the Qubit Lattice representation to encode quantum images [96]. This was closely followed by Latorre’s Real Ket description for quantum images [49]. Years later, the third and final representation completing the trio, collectively regarded as the pioneers of the sub-discipline of QIMP, Le et al.’s FRQI (flexible representation for quantum images) was proposed in 2010 [51] and later reviewed in 2011 [54].

In fact, these pioneering representations and developments that followed them provide the essential proof that image processing on the quantum computing domain could speed up processing tasks, reduce the computational requirements, and facilitate the secure transmission by utilizing the characteristics of quantum computation and information, e.g., entanglement and parallelism.

Available literature on QIMP can be broadly classified into two groups [31]. The first, which is referred to as quantum-inspired image processing, aims to exploit some of the properties responsible for the potency of quantum computing algorithms in order to improve some well-known classical or digital (i.e., conventional or non-quantum) image processing tasks and applications. Some of the available literature that fall under this group include: the work by Beach et al. [4], which concentrated on showing that existing quantum computing algorithms (such as Grover’s algorithm [26]) are applicable to image processing tasks. In addition, the use of quantum computing and reverse emergence in classical image processing was discussed in [3]. The key idea involved in this work is the use of cellular automata as a complex system and quantum inspired algorithms as a search strategy. The second group of the available literature in QIMP derives its inspiration from the expectation that quantum computing hardware will soon be physically realized and, hence, such research focuses on extending classical image processing tasks and applications to the quantum computing framework. Accordingly, such work is being referred to as classically inspired QIMP [105, 107].

As mentioned earlier, the work by the trio of Venegas-Andraca, Latorre, and Le is credited with pioneering what is today regarded as QIMP. They provided the framework to formalize representations to capture and encode images on the quantum computing paradigm, the main requirements for which are simplicity and efficiency of the storage and effectiveness of the image retrieval [31, 38].

In particular, Venegas-Andraca posited that if we assume an apparatus that could detect electromagnetic frequencies and produce a quantum state as output, we could store color in a qubit by translating given frequencies to quantum states. And by updating the indices to specify the pixels in the image, a full image could be stored in a Qubit Lattice [96]. Following that, entangled images, in which geometric shapes are encoded in quantum states [95]; and Real Ket, where the images are quantum states having gray levels as coefficients of the states [49], were proposed by Venegas-Andraca and Latorre, respectively. More recently, Le et al. [51] proposed a flexible representation of quantum image (FRQI), which is a normalized state that captures the essential information (i.e., its color and position) about every point in an image. It utilizes the idea of using the angle $\theta$ to represent the color information of an image as in [96], and by using the two-dimensional position information (Y-axis and X-axis), the representation is more similar to the pixel representation for images on conventional computers. The FRQI representation maintains a normalized state and significantly decreases the number of qubits required to prepare images compared to the Qubit Lattice description. For example, the FRQI representation requires 2n + 1 qubits for representing a $2^n\times 2^n$ image compared with $n^2$ qubits in Qubit Lattice model. In addition, it is a very flexible representation because it facilitates both the geometric and the color transformations on an image. Following its sturdy formulation, many of the available QIMP literature are either focused on its use, modification, extension, or applications as in quantum watermarking [34, 122], quantum movie [32, 33], image database search [106, 109], or in many other applications such as [94, 110].

Inspired by the pioneering representations above, numerous other quantum image representations (QIRs) have been suggested. In terms of the QIMP time line, we call this group the recent QIRs. They include multi-channel representation for quantum images (MCQI) [86], the so-called Caraiman’s QIR approach (presented as CQIR for brevity) [10]; novel enhanced quantum representation of digital image model (NEQR) [125]; quantum image representation for log-polar images (QUALPI) [126]; simple quantum representation of infrared images (SQR) [120]; color image representation utilizing two sets of quantum states for M colors and N coordinates, respectively (QSMC & QSNC) [58]; and multi-dimensional image representation using a normal arbitrary quantum superposition state (NAQSS) [60]. These representations and their operations will be further introduced in the subsequent sections.

In terms of efficiency and resources for storage, these QIRs, basically, utilize similar strategies. The first step is the color storage, where an apparatus to convert the energy to an initialized qubit is needed (the apparatus acts like an injective function from the set of energy to the set of quantum states) [96]. This energy could be electromagnetic waves which can be detected and recorded, and then a magnetic field proportional to the stored frequency could be applied to a spin-half particle, which is originally prepared in either the spin-up or the spin-down state, wherein different quantum states for representing the color could be stored [96]. The energy could also be an infrared radiation where the energy values are in proportion to radiation intensity that can be measured with the number of received photons. In this way, radiation energy of objects is transformed into the required quantum state for storing color information [120]. It should be noted that as discussed in [96], this storage protocol allows us to store energy from all the spectrum (visible and invisible spectrum). The second step is storing an image in the QIR model. The apparatus we mentioned earlier can be an energy detector array with the size $N_1\times N_2$, and we then update the indices according to the way visual information is made available to facilitate the storage of an entire quantum image.

The remainder of the review will be as follows: In Sect. 2, we briefly review the current QIRs as well as their relations by dividing them into three categories. In Sect. 3, we specialize the FRQI, MCQI, NEQR, CQIR, QUALPI, SQR, QSMC & QSNC, and NAQSS representations and their features. The basic geometric, color, and frequency domain transformation of these QIRs is discussed in Sect. 4. The applications of aforementioned QIRs are reviewed in Sect. 5. In Sect. 6, we discuss the future developments of QIMP and offer a few concluding remarks.

2 Quantum image models

The objective of classically inspired QIMP or simply QIMP is to utilize the quantum computing technologies to capture, manipulate, and recover quantum images in different formats and for different purposes [31]. Based on the different requirements to capture the content of an image, we could divide the available QIRs into three models as discussed in the remainder of this section.

2.1 Image color model

This QIR model comprises of all three images that form the pioneer QIRs and the more recent ones. All of the QIRs in this group utilize the color model or visual difference to encode the content of an image. In their Qubit Lattice QIR, Venegas-Andraca and Bose [96] averred that frequency of the physical nature of color could represent a color instead of the traditional color models, e.g., RGB model or the HIS model, so a color could be represented by one qubit in the quantum computing system [96]. Similarly, all the other QIRs in this model are built on the chromatic content of the image. Typically, the QIRs in this model could be further grouped as follows:

• Binary-based QIR

• Grayscale-based QIR

• RGB-based QIR

• Infrared image-based QIR

QIR research has witnessed a similar evolution with the classical image representation: In [95], the entangled image state was proposed for storing and retrieving binary geometrical shapes in quantum mechanical systems. It harnesses the entangled state to express the realization of some certain pixels so that the object shapes could be reconstructed without any use of auxiliary information. The binary-based QIR in [95] is a simple representation and easy to discuss the storage, retrieval, and related operations, e.g., it is always used in digital image processing as masks or as the result of certain operations such as segmentation, thresholding, and dithering. However, the limitation of the simple color representation [actually, black (0) for the foreground color and white (1) for the background color of the image] provides enough evidence that a more complex color model is needed to focus on the geometric and chromatic details in the image.

Instead of merely describing the outline of some objects in an image by using binary encodes (0 and 1), more levels to express color, i.e., grayscale information, are needed to indicate more color details. In [51], a flexible representation for quantum images, FRQI, was proposed to integrate the grayscale color and position information about an image into a normalized state that facilitates the geometric and color transformations on the image. A single qubit is dedicated for encoding color information, thereby ensuring that transformations on the image content can target the color only or both the color and position simultaneously. Specified quantum gates applied on predetermined areas of the image could transform the color information as desired.

In order to effectively mimic the human perception of visual effects, a true color image with three primary colors R, G, and B, is a natural extension in QIMP area. Color image representations either utilize two sets of quantum states, to respectively represent M different colors and N pixels in an image [58], or use different level of angles for RGB information and the tensor product with location information (Y-axis and X-axis) to represent an image [86, 114]. They are different from the only simple color operations that can be performed on the quantum image encoded in FRQI representation (because of the fact that only a single qubit is used to represent the color information for each pixel), and the RGB-based QIRs can separate the image into not only its R, G, and B components but also an additional $\alpha$ channel, which helps facilitate additional operations on the image’s content as in the multi-channel quantum images MCQI in [86], an extension of FRQI representation, that could operate on the color of interest (CoI), color swapping (CS), and $\alpha$ blending ($\alpha \mathbf{B}$) transformations which will be reviewed in Sect. 3.2. While in QSMC & QSNC [58], it uses two sets of quantum states for M colors and N coordinates, respectively, which could both represent grayscale and color information of an image, and in addition, it could provide a lossless compression with acceptable compression ratios and a quantum search-based image segmentation method that is universal for grayscale images and color images.

The last type of QIR that completes the color model is the representation for infrared quantum images, which differs from the other types described so far in that it is a result of thermal-based image in infrared imaging system with its visual capacity in darkness and fog; it can almost work in any environment and all weather conditions. Therefore, infrared image processing has a wide range of applications in satellite sensing, weapon navigation, meteorological monitoring, and environmental pollution supervision, etc. Inspired by the idea of producing quantum states from certain frequencies and using the angle parameter of a qubit to store a color as in [96] and [51], probability of projection measurement is used to store the radiation energy value of each pixel in a simple quantum representation for infrared images, i.e., SQR [120]. SQR provides more processing operations than Qubit Lattice model and exhibits a quadratic decrease in quantum gates used in the image preparation procedure compared with FRQI representation. In addition, the processing operations, such as global operations (GO) and local operations (LO), could be performed on the SQR representation [120].

2.2 Image coordinate model

In this subsection, we focus on QIRs that utilize different coordinate systems to capture the information about an image. Some of the available QIRs that fall under this model include:

• Cartesian coordinate system-based QIR

• Log-polar coordinate system-based QIR

• Multi-dimension-based QIR

The typical example of a coordinate system is the Cartesian coordinate system. In the two-dimensional plane, two perpendicular lines are chosen and the coordinates of a point are taken to be the signed distances to the lines. We always describe the physical and mathematical model on the Cartesian coordinate system. The quantum image representation is also originated from the basic Cartesian coordinate in order to satisfy human’s senses to the nature by presenting the content of an image on the plane. The pioneer QIRs, e.g., Qubit Lattice, which maps every pixel to a single qubit and the images are stored in two-dimensional arrays of qubits without using any preprocessing step [96], and the Real Ket, which utilizes the coefficients of a basis state of a qubit sequence to represent the grayscale value of every pixel and stores an image into a quantum superposition [49], could both be described as QIRs processed on the basis of two-dimensional Cartesian coordinate systems that are intuitionistic, and most importantly, they facilitate the basic geometric transformations on an image due to their special coordinate structure. For example, the third pioneering QIR, the FRQI representation [51], falls also under the Cartesian coordinate system; the two-point swapping, flip, coordinate swapping, orthogonal rotation, and their variants for quantum images have been fully studied [52, 55].

Image representations on different coordinate systems oftentimes yield different operations and applications. In the earlier discussion on QIRs based on the image coordinate model, many complex affine transformations cannot be easily performed because a lot of irreversible interpolations are needed. In the field of classical image processing, log-polar coordinates are widely employed as sampling methods where the sampling resolutions of the log-radius and the angular orientations are assumed to be $2^m$ and $2^n$ in a $2^m\times 2^n$ sized log-polar image, respectively. A quantum log-polar image, QUALPI, was proposed in [126] for storing and processing images sampled in log-polar coordinates. Complex geometric transformations can be performed based on QUALPI, e.g., symmetry transformation including quantum centrosymmetry and quantum axisymmetry could be efficiently performed, and the rotation transformation by shifting the angular directions is lossless and reversible because there is no interpolation operation. In addition, a fast rotation-invariant quantum image registration algorithm was designed for log-polar images, using which the exact rotation difference between two quantum images can be found out. The relationship between these two kinds of two-dimensional image sampling methods (the coordinate transformation between Cartesian coordinate system-based and Log-polar coordinate system-based images) is also presented in [126].

In our 3D world, 3-dimensional description of objects (3D images) could always provide us with an intuitive impression. In the mathematical and physical contexts, we even discuss some problems or build some models in multi-dimensional Cartesian systems using n-dimensional Euclidean spaces. A new representation for multi-dimensional color image, NAQSS, was presented in [59, 60], where the position information of an image expanded to k binary arrays in k coordinates in the k-dimensional space was broached. The NAQSS requires n + 1 qubits, where n qubits represent colors and coordinates of $2^n$ pixels and 1 qubit represents an image segmentation information to improve the accuracy of image segmentation, while only the image (not including additional segmentation information) is represented by n + 1 qubits in FRQI representation. In addition, due to the bijective function between color and angle, it has no constraints in terms of the color representation (it can both represent the grayscale and RGB images by adjusting the bijective function).

2.3 Image color information encoding model

Finally, similar to the different classical image file formats, e.g., BMP, TIFF, and JPEG, the two fundamental information, i.e., color and position, of an image could be encoded in different approaches. Considering this, we divide the QIRs under the color information encoding model into two categories:

• Encoded using one qubit through the angle parameter of it

• Encoded in the basis states of a sequence of qubits

In the first category, we see some of the pioneer QIRs, and the representation models encode the color information in the state of one qubit by establishing a bijective relationship between the frequency of the monochromatic electromagnetic wave that determines the color and the angle parameter of a qubit such as Qubit Lattice [96], FRQI [51], and QSMC & QSNC [58]. Even though the pixel position encoding of them varies each other among these three representations, e.g., no explicit quantum encoding of the position information in Qubit Lattice model, the position information of every pixel can be stored in a basis state of a 2n qubit sequence in FRQI representation, and the position information can be encoded in the angle parameter of a qubit by creating a bijection between the set of pixel coordinates and a set of angles in QSMC & QSNC method. It should be noted that in this classification, the original classical image cannot be accurately retrieved notwithstanding a finite number of measurements. This is because the quantum measurements only provide probabilistic results of the color angle [67]. Therefore, multiple copies of the same quantum image are required to be prepared, following which a statistical procedure is applied according to the sampling theory in order to estimate the probability amplitudes of the quantum states encoding the color of each pixel within a given accuracy [13].

With the development, the CQIR in [10] and NEQR in [125] were simultaneously and independently proposed. In both QIRs, the color information is encoded in the basis states of a sequence of qubits so that the whole image is stored in the superposition of the two qubit sequences. This is the second category that we discussed earlier. Assume m qubits are used to encode all possible gray levels presented in the image which means the color model C is an m-qubit register. In this category, both the color and position of a pixel in the image are encoded in the basic quantum states (instead of the superposition state with the complex numbers as the coefficients); therefore, both the color and position information in this representation can be possibly retrieved deterministically through a finite number of projective measurements. In addition, the number of representable colors and positions in an image does not depend on the actual physical implementation of the quantum system, and a larger class of more complex image processing operations can be applied using this model because the color is represented using a computational basis state that can act as control for applying value-dependent color transforms and for computing statistics in the image according to Caraiman addressed in [13]. However, due to the complex representation, additional qubits are needed to encode the color information of this kind of images.

3 Quantum image representations and features

As mentioned earlier, the FRQI representation has a widespread appeal in the QIMP literature over the last five years. In this section, we introduce this representation and highlight some of its properties [50, 51, 54]. Later, we review some recent QIRs which are broached each with one operation or the other in mind by modifying and/or extending aspects of the pioneering QIRs. To make our review self-contained, we adopt the formalization and description in the literatures [10, 51, 58, 60, 86, 120, 125, 126] to elaborate these QIRs.

3.1 FRQI representation and its features

FRQI, flexible representation for quantum images, is similar to the pixel representation for images on traditional computers. It captures the essential information about the colors as well as the corresponding positions of every point in an image and integrates them into a quantum state having its formula in Eq. (1) as:

$$\begin{aligned} {{\vert I\rangle } = {{1\over 2^n}{\sum _{i=0}^{2^{2n} - 1}{\vert c_i\rangle }}\otimes {\vert i\rangle }}}, \end{aligned}$$

(1)

where

$$\begin{aligned} {\vert c_i\rangle }= & {} {\cos {\theta _i}\vert 0\rangle } + {\sin {\theta _i} \vert 1\rangle }, \end{aligned}$$

(2)

where $\vert 0\rangle$ and $\vert 1\rangle$ are 2D computational basis quantum states, $\vert i\rangle$, i = 1, 2,..., $2^{2n}-1$, are 2n-D computational basis quantum states and $\theta = (\theta _0,\theta _1,\ldots ,\theta _{2^{2n}-1}), \theta _i\in [0,\pi /2]$, is the vector of angles encoding colors. There are two parts in the FRQI representation of an image: $\vert c_i\rangle$ and $\vert i\rangle$ which encode information about the colors and corresponding positions in the image, respectively.

An example of a $2\times 2$ FRQI quantum image with its quantum state is shown in Fig. 1.

Fig. 1

A $2\times 2$ FRQI quantum image and its quantum state (figure adapted from [51])

In quantum computation, computers are usually initialized in well-prepared states [51]. As a result, the preparation process that transforms quantum computers from the initialized state to the desired quantum image state is necessary. The polynomial preparation theorem (PPT) as developed by using Theorem 1 shows a constructively efficient implementation of the preparation process.

Theorem 1

Given a vector $\theta = (\theta _0, \theta _1, \ldots , \theta _{2^{2n}-1})$, $(n \in N)$ of angles, there is a unitary transform $\mathcal {P}$ that turns quantum computers from the initialized state, $\vert 0\rangle ^{\otimes 2n+1}$, to the FRQI state in Eq. (1), composed of polynomial number of simple gates [51].

Proof

There are two steps to achieve the unitary transform $\mathcal {P}$ where Hadamard transforms are used in Step 1 to change $\vert 0\rangle ^{\otimes 2n+1}$ to $\vert H\rangle$, and then, controlled-rotation transforms are used in Step 2 to transform $\vert H\rangle$ to $\vert I\rangle$.

Considering the 2D identity matrix I and the 2D Hadamard matrix, applying the transform $\mathcal {H} = I \otimes H^{\otimes 2n}$ on $\vert 0\rangle ^{\otimes 2n+1}$ produces the state $\vert H\rangle$,

$$\begin{aligned} \mathcal {H}(\vert 0\rangle ^{\otimes 2n+1}) = \frac{1}{2^n}\otimes \sum _{i=0}^{2^{2n}-1}{\vert i\rangle } = \vert H\rangle . \end{aligned}$$

(3)

Let us also consider the rotation matrices (the rotations about y-axis by the angle $2\theta _i$), $R_y(2\theta _i)$, and controlled-rotation matrices, $R_i$, with $i = 0, 1, \ldots , {2^{2n}-1}$,

$$\begin{aligned} R_y(2\theta _i)= & {} \left( \begin{array}{cc} \cos \theta _i &{} -\sin \theta _i \\ \sin \theta _i &{} \cos \theta _i \\ \end{array} \right) , \end{aligned}$$

(4)

$$\begin{aligned} R_i= & {} \left( I \otimes \sum _{j=0,j\ne 1}^{2^{2n}-1}\vert j\rangle \langle j\vert \right) + R_y(2\theta _i)\otimes \vert i\rangle \langle i\vert . \end{aligned}$$

(5)

Applying $R_lR_k$ on $\vert H\rangle$ provides us that:

$$\begin{aligned} \begin{aligned} R_lR_k\vert H\rangle&= \frac{1}{2^n} \left[ \vert 0\rangle \otimes \left( \sum _{i=0,i\ne k,l}^{2^{2n}-1}{\vert i\rangle \langle i\vert }\right) + (\cos \theta _k\vert 0\rangle + \sin \theta _k\vert 1\rangle )\otimes \vert k\rangle \right. \\&\left. \quad + \left( \cos \theta _l\vert 0\rangle + \sin \theta _l\vert 1\rangle \right) \otimes \vert l\rangle \right] . \end{aligned} \end{aligned}$$

(6)

It is clear from Eq. (6) that:

$$\begin{aligned} \mathcal {R}\vert H\rangle = \left( \prod _{i=0}^{2^{2n}-1}{R_i}\right) \vert H\rangle = \vert I\rangle . \end{aligned}$$

(7)

Therefore, the unitary transform $\mathcal {P} = \mathcal {R}\mathcal {H}$ is the transform turning quantum computers from the initialized state, $\vert 0\rangle ^{\otimes 2n+1}$, to the FRQI state, $\vert I\rangle$. Therein, the computational complexity of the whole preparation for FRQI could be calculated as $O(2^{4n})$ [51]. $\square$

3.2 MCQI representation and its features

By extending the grayscale information in FRQI to color representation, a multi-channel representation for quantum images, MCQI, was proposed in [86, 88], which uses R, G, and B channel to represent the different color information of image and also keeps the normalized state. It is accomplished by assigning three qubits to encode color information about images. Based on the MCQI representation, all RGB information about an image is stored simultaneously. This representation is in the form presented in Eq. (1) as:

$$\begin{aligned} |I\rangle =\frac{1}{2^{n+1}}\sum \limits _{i=0}^{2^{2n}-1}|C_{\mathrm{RGB}\alpha }^{i}\rangle \otimes |i\rangle , \end{aligned}$$

(8)

where the color information $|C_{\mathrm{RGB}\alpha }^{i}\rangle$ encoding the RGB channels information is defined as:

$$\begin{aligned} |C_{\mathrm{RGB}\alpha }^{i}\rangle= & {} \cos \theta _\mathrm{R}^i|000\rangle +\cos \theta _\mathrm{G}^i|001\rangle +\cos \theta _\mathrm{B}^i|010\rangle +\cos \theta _\alpha |011\rangle \nonumber \\&+ \sin \theta _\mathrm{R}^i|100\rangle +\sin \theta _\mathrm{G}^i|101\rangle +\sin \theta _\mathrm{B}^i|110\rangle +\sin \theta _\alpha |111\rangle , \end{aligned}$$

(9)

where $\{\theta _\mathrm{R}^i,\theta _\mathrm{G}^i,\theta _\mathrm{B}^i\} \in [0,\pi /2]$ are three angles encoding the colors of the R, G, and B channels of the $i^{th}$ pixel, respectively, and $\theta _\alpha$ is set as 0 to make the two coefficients constant ($\cos \theta _\alpha = 1$ and $\sin \theta _\alpha = 0$) to carry no information.

The computational complexity of the whole preparation for MCQI quantum image is the same with FRQI’s, i.e., $O(2^{4n})$, as the authors stated in [86]. An example of a $2\times 2$ MCQI quantum image with its quantum state is presented in Fig. 2.

Fig. 2

A $2\times 2$ MCQI quantum image and its quantum state (figure adapted from [86])

As discussed earlier, the MCQI representation leads consequently to the following major advantages:

• MCQI representation provides a solution to encoding R, G, and B channel information in normalized quantum states using many fewer qubits. Such a merit originates because in the MCQI representation, the property of quantum parallelism is utilized to encode the color information and their corresponding positions. This results in MCQI quantum images requiring far fewer qubits than the Qubit Lattice [96] and also significantly less computing resources than the same-sized classical RGB image.

• Using MCQI makes it easier to design color image operators with much lower complexity. Color information operators applied to the whole MCQI quantum image are designed by using unitary quantum gates within these 3 color qubits irrespective of position qubits. This means that the complexity of MCQI-based color information operators is image size-independent, whereas color information must be shifted pixel by pixel when executing similar operations on classical computers.

• MCQI representation provides the potential to design quantum cryptography-based color image watermarking algorithms. Combined with the quantum measurement-based key generation procedure, MCQI may be used to design the color image watermarking algorithms, which will be reviewed in Sect. 5.2.

An MCQI quantum image is stored in the preparation process using the MC-PPT theorem that extends the vector in Theorem 1 to three vectors of angles [86]. MCQI representation shares the same method with FRQI to encode position information, i.e., the same arrangement for position qubits. The notable difference between the two representations is color qubits because FRQI uses 1 qubit to encode an image color, whereas MCQI applies 3 qubits to carry multi-channel color information. An improved version of MCQI, color quantum image based on phase transform (CQIPT) [85], was proposed where the color information is prepared by controlled phase gates which are especially flexible for many image processing and security algorithms based on phase encoding.

3.3 NEQR representation and its features

Zhang et al.’s NEQR representation (novel enhanced quantum representation) for digital images uses the basis state of a qubit sequence to store the grayscale value of every pixel instead of an angle encoded in a qubit in FRQI. Therefore, to store the digital image in NEQR using quantum mechanics, two entangled qubit sequences are used to store the whole image, which represent the grayscale and positional information of all the pixels [125].

Suppose the gray range of an image is $2^q$ with a binary sequence $C_{yx}^0C_{yx}^1\ldots C_{yx}^{q-2}C_{yx}^{q-1}$ encoding the grayscale value f(y,x) as:

$$\begin{aligned}&f(y,x) = C_{yx}^0\dots C_{yx}^{q-2}C_{yx}^{q-1}, \nonumber \\&\mathrm{where}\, C_{yx}^k\in [0,1] \text { and } f(y,x)\in [1,2^q-1]. \end{aligned}$$

(10)

The representative expression of a $2^n\times 2^n$ version of such a quantum image can be written as:

$$\begin{aligned} \vert I\rangle= & {} \frac{1}{2^n}\sum _{y=0}^{2^n-1}\sum _{x=0}^{2^n-1}\vert f(y,x)\rangle \vert yx\rangle \nonumber \\= & {} \frac{1}{2^n}\sum _{y=0}^{2^n-1}\sum _{x=0}^{2^n-1}\bigotimes _{i=0}^{q-1}\vert C_{yx}^i\rangle \vert yx\rangle . \end{aligned}$$

(11)

An example of a $2\times 2$ NEQR quantum image and its quantum state is shown in Fig. 3.

Fig. 3

A $2\times 2$ NEQR quantum image and its quantum state (figure adapted from [125])

To prepare a NEQR quantum image, the first step is similar to the preparation for FRQI that was reviewed earlier in Sect. 3.1, and so, for brevity, we skip repeating it here. In the second step, the grayscale value for every pixel is set. This step is divided into $2^{2\textit{n}}$ sub-operations to store the grayscale information for every pixel [125].

The NEQR representation utilizes the basis state of qubit sequence to represent the grayscale of pixels instead of probability amplitude of a single qubit used in the FRQI representation. Therefore, the time complexity of preparing the NEQR quantum image exhibits an approximately quadratic decrease, i.e., $O(qn\cdot 2^{2n})$, compared to FRQI; for image compression, based on minimization of Boolean expressions, the compression ratio of NEQR increases 1.5 times more than the FRQI [125]. In addition, through quantum measurements, the original classical image can be retrieved accurately from the NEQR quantum image, not probabilistically as FRQI. Interestingly, the authors, in [125], assert that more image operations can be performed conveniently based on NEQR than on FRQI, especially some complex color operations such as partial color operations and statistical color operations.

However, it should be stressed that NEQR representation uses more qubits to encode a quantum image. From the representation of NEQR, q + 2n qubits are needed to construct the quantum image model for a $2^n\times 2^n$ image with gray range $2^q$, wherein 2n qubits for the position information that is the same amount requirement with FRQI, while for the color information, q qubits are needed instead of only 1 qubit required in FRQI representation.

3.4 CQIR representation and its features

Similar to NEQR that uses a qubit sequence to encode the color information in quantum images [125], Caraiman and Manta proposed an approach, i.e., CQIR, that facilitates histogram equalization of a quantum image. It is particularly useful for better processing operations such as computing negative, binarization, and histograms, where $m = \log _2L$ qubits are used to encode the L gray levels presented in the image [10]. This approach could have potential applications for medical, radar, and astronomical image processing by increasing the image contrast to enhance the details in the image. The CQIR representation is presented below:

$$\begin{aligned} \vert I\rangle = \vert C\rangle _m\times \vert P\rangle _{2n} = {{1\over 2^n}{\sum _{i=0}^{2^{2n} - 1}{\sum _{j=0}^{2^m - 1}\alpha _{ij}\vert j\rangle \vert i\rangle }}}. \end{aligned}$$

(12)

Pixel positions are encoded in a register $\vert P\rangle$ using 2n qubits, while color information for each pixel is represented using $m=\log _2L$ qubits encoding the L gray level colors. Similarly, the whole quantum image preparation will cost no more than $O(\log _2L\cdot n\cdot 2^{2n})$ [10]. Coefficients $\alpha _{ij}$ with $0\le i\le {2^{2n}}$ are used to express the color of a pixel with position i by means of a superposition of all possible colors. For a given pixel i, coefficients $\alpha _{ij}$ take value 1 if the color of the pixel is j, and 0 otherwise [11]. This is illustrated in Fig. 4 with a simple example of a $2\times 2$ image with four colors.

Fig. 4

A $2\times 2$ CQIR quantum image and its quantum state (figure adapted from [11])

3.5 QUALPI representation and its features

Most of the existing models can only store and process image sampled in Cartesian coordinates. Many complex affine transformations such as rotation and scaling cannot be easily performed with these models because a lot of irreversible interpolations are needed. Restricted quantum image transformations have limited the development of complex QIMP algorithms. This motivated the need to find a better way of quantum image representation to overcome the above limits.

This prompted Zhang et al. to propose a quantum representation to store and process the log-polar images, QUALPI, in [126]. The sampling resolutions of the log-radius and the angular orientations of a log-polar image are assumed to be $2^m$ and $2^n$, respectively. The whole QUALPI quantum image is stored in a normalized and equiprobable quantum superposition, in which each basis state represents one pixel. The representation of QUALPI quantum image is presented as:

$$\begin{aligned} {\vert I\rangle } = {1\over \sqrt{2^{m+n}}}{\sum _{\rho =0}^{2^m - 1}\sum _{\theta =0}^{2^n - 1}{{\vert g(\rho ,\theta )\rangle }\otimes {\vert \rho \rangle }\otimes {\vert \theta \rangle }}}, \end{aligned}$$

(13)

where $g(\rho ,\theta )$ represents the grayscale of the corresponding pixel. The gray range of the image is assumed to be $2^q$. Thus the grayscale can be encoded by binary sequence $C_0C_1\dots C_{q-2}C_{q-1}$ as seen in the following equation:

$$\begin{aligned} g(\rho ,\theta ) = C_0C_1\dots C_{q-2}C_{q-1}, g(\rho ,\theta )\in [0, 2^q - 1]. \end{aligned}$$

(14)

Equation 13 shows that the basis state of QUALPI consists of the tensor product of three qubit sequences, where all the information of a pixel, i.e., the grayscale, the log-radius position, and the angular position, is stored in. The whole quantum image preparation will cost no more than $O(q(m+n)\cdot 2^{m+n})$ for a $2^m\times 2^n$ log-polar quantum image with gray range $2^q$ according to Theorem 1 in [126].

An example of a $2\times 8$ log-polar quantum image with its quantum state is shown in Fig. 5.

Fig. 5

A $2\times 8$ QUALPI quantum image and its quantum state (figure adapted from [126])

3.6 SQR representation and its features

Similar to Qubit Lattice representation, radiation energy of objects is transformed into a quantum state $\vert I\rangle$, which is named SQR model [120], through a converter C, i.e., C is capable of detecting an recording infrared radiation energy and then producing a quantum state $\vert I\rangle$ as output. As a result, the infrared image information for all the pixels is stored in quantum state $\vert I\rangle$ that is composed of a set of qubit states $\vert \varphi _{ij}\rangle$ as presented in Eq. (15). A 2D detector array is responsible to distribute the quantum states to the assigned position. The SQR quantum image is mathematically described as below:

$$\begin{aligned} {\vert I\rangle } = {\vert \varphi _{ij}\rangle }, i = 1,2,\ldots ,N_1, j = 1,2,\ldots ,N_2, \end{aligned}$$

(15)

where

$$\begin{aligned} \vert \varphi _{ij}\rangle = \cos \theta _{ij}\vert 0\rangle + \sin \theta _{ij}\vert 1\rangle , \end{aligned}$$

(16)

stores the normalized infrared radiation energy collected by detector unit.

An example of a $2\times 2$ SQR quantum image with its quantum state is shown in Fig. 6.

Fig. 6

A $2\times 2$ SQR quantum image and its quantum state

3.7 QSMC & QSNC representation and its features

As discussed in Sect. 2.1, QSMC & QSNC uses two sets of quantum states to store an image, where QSMC represents M colors and QSNC represents the coordinates of N pixels in an image [58]. In order to store the color and coordinate information of an image using quantum states, two bijective functions that from M different colors to M angles and likely from N coordinates to N different angles are created. The mathematical expression of QSMC & QSNC quantum image is presented in Eq. (17),

$$\begin{aligned} {{\vert I\rangle } = {{1\over 2^n}{\sum _{i=0}^{2^{2n} - 1}{\vert \mathrm{QSMC}_i\rangle }}\otimes {\vert \mathrm{QSNC}_i\rangle }}}, \end{aligned}$$

(17)

where

$$\begin{aligned} {\vert \mathrm{QSMC}_i\rangle }= & {} {\cos {\phi _i}\vert 0\rangle } + {\sin {\phi _i} \vert 1\rangle }, \end{aligned}$$

(18)

$$\begin{aligned} {\vert \mathrm{QSNC}_i\rangle }= & {} {\cos {\theta _i}\vert 0\rangle } + {\sin {\theta _i} \vert 1\rangle }, \end{aligned}$$

(19)

$$\begin{aligned} \phi _i, \theta _i \in \left[ 0,{\pi \over 2}\right] , i= & {} 1, 2,\dots , 2^{2n} -1. \end{aligned}$$

(20)

An example of a $2\times 2$ QSMC & QSNC quantum image with its quantum state is presented in Fig. 7.

Fig. 7

A $2\times 2$ QSMC & QSNC quantum image and its quantum state

3.8 NAQSS representation and its features

A multi-dimensional color image representation, named a (n+1)-qubit normal arbitrary quantum superposition state, NAQSS, was proposed by Li et al. [60], where n qubits represent colors and coordinates of $2^n$ pixels and the remaining 1 qubit represents an image segmentation information. A bijective function which sets up a one-to-one relationship between color and angle is created so that all the pixels can find a corresponding value in the interval $[0, \pi /2]$. A k-dimensional digital image is represented as the function $f: V\rightarrow R$ where V notates the position information of an image in a k-dimensional Euclidean space and f(V) is the above-mentioned color set of pixels corresponding to the position V. The NAQSS quantum image is represented as:

$$\begin{aligned} {\vert I\rangle } = \sum _{i=0}^{2^{n} - 1}{\theta _i}{\vert v_1\rangle \vert v_2\rangle \cdots \vert v_k\rangle }\otimes {\vert \chi _i\rangle }, \end{aligned}$$

(21)

where

$$\begin{aligned} \vert \chi _i\rangle = \cos \gamma _i\vert 0\rangle + \sin \gamma _i\vert 1\rangle , \end{aligned}$$

(22)

represents the segmentation information, where $\gamma _i$ corresponds to m to set up a bijective function if we divide an image into m sub-images that contains the pixel corresponding to the coordinate $\vert v_1\rangle \vert v_2\rangle \cdots \vert v_k\rangle$.

An example of a $2\times 2\times 2$ NAQSS quantum image with its quantum state is shown in Fig. 8.

Fig. 8

A $2\times 2\times 2$ NAQSS quantum image and its quantum state

3.9 Summary of available properties of QIRs

As noted in the aforementioned discussions, many of the development in the QIMP sub-area is widely attributed to the three pioneering QIRs and most of the recent QIRs are also focused on modifying them. Most often, this is geared toward enhancing the available results, improving the applications they are used for, and/or facilitating new applications for QIMP. These QIRs are summarized in the sequel.

The NEQR representation uses two entangled qubit sequences to store the whole image, and in this manner, it reduces the time complexity and provides more accurate information retrieval and more complex operations compared to FRQI. Similar to NEQR, the basis states of a sequence of qubits are used to represent the color information, but different number of qubits with different gray levels to encode the color information of an image in CQIR so that different processing operations could be performed. The multi-channel representation for quantum images, MCQI, proposed by Sun et al. is an extension of the FRQI representation that facilitates more advanced color image processing by applying different operations on the R, G, and B channels of the image. While also for representing the color information, QSMC & QSNC chose to employ two bijective functions from color and coordinate to angles for representing the color and position information so that compression and segmentation likely operations could be well performed. Finally, three special quantum image representations are concluded here. QUALPI on its part is a representation that exhibits the image in log-polar coordinate. In this way, the affine transformations such as rotation and scaling could be easily performed. In addition, SQR is a representation for infrared quantum images where the color information (a quantum state) is produced from infrared radiation energy that could improve the visual capacity in almost any environment. Last, NAQSS, where the usual position notation/information i is divided into k binary expansions for representing a coordinate $(v_1, v_2,\ldots , v_k)$ in the k-dimensional space V, was proposed to satisfy the image processing research in multi-dimensional space.

At the end, we present the features including their equations, the color information encoding strategies, the computational complexities for preparing these quantum images, and the measurement-based retrievals on them among aforementioned QIRs as shown in Table 1.

Table 1 The features concerning the aforementioned QIRs

4 Basic quantum image transformations

Basic quantum image transformations are the foundation of the QIMP. In this section, we will introduce some well-studied transformations which will be used in some senior QIMP algorithms.

4.1 Geometric transformations

Geometric transformations are the operations that are performed based on the geometric information of images, i.e., information about the position of every point in the image [52].

A fast geometric transformation based on FRQI quantum images (GTQI) was firstly proposed in [52]. Using GTQI operations, classical-like transformations such as two-point swapping, flip, coordinate swapping, and orthogonal rotations can be performed on the images encoded in the FRQI representation using the basic quantum gates, e.g., NOT, CNOT, and Toffoli gates. Often, we would like to constrain the desired geometric transformations to some specific parts of an image, which leads the proposal of restricted geometric transformations on FRQI images (RGTQI) in [35, 55].

Quantum image translation (QIT), which is a kind of geometric transformation that is designed to map the position of each picture element in an input (or original) image into a new position in an output (or translated) image, was proposed in [100]. Two types of QIT based on NEQR representation, i.e., entire translation and cyclic translation, were studied in [100] by giving the quantum translation circuits.

In addition, an image scaling method (that resizes an image) was proposed based on the NEQR representation [42]. It uses interpolation to create new or delete redundant pixels. The quantum image scaling-up on one direction is realized by repeating the value to every pixel $2^{m-n}$ times along the direction scaled. While the scaling-down strategy is fulfilled by scaling down an image using nearest neighbor interpolation.

Because of the peculiar properties of the log-polar images, the symmetry transformation and rotation can be performed flexibly and quickly. Symmetry transformation is a kind of important geometric transformation including centrosymmetry and axisymmetry. Rotation is a common geometric transformation of images sampled in the log-polar coordinates. Different from the images sampled in Cartesian coordinates, rotation transformation of a log-polar image is lossless and reversible because there is no interpolation operation. Meanwhile, the quantum rotation will be more flexible and faster.

4.2 Color transformations

Color transformations are targeted at converting the color information of the whole image or a portion of it for image enhancement or information hiding [53].

A general form of color-related transformations on FRQI quantum images (CTQI) was proposed based on the single-qubit gates such as X, Z, and H gates. Efficient color transformations based on FRQI quantum images were initially proposed in [53]. Similarly to RGTQI, with more controls to the original operations, the color transformation to smaller areas in a quantum image could be specified.

The MCQI representation [86] encodes the R, B, and B color information of quantum images. Therefore, its color transformation includes: the Col operator on MCQI quantum image which entails shifting the grayscale value of the preselected color channel (R, G, B, or $\alpha$ channel); the CS operator which is used to swap the grayscale values between two channels (R and G, R and B, or G and B channels); and the ($\alpha$B) operator that is employed to blend an image (A) with a background image (B) to create the appearance of partial or full transparency.

In addition, the common image processing methods based on CQIR ($\log _2L$ qubits are used to encode the L gray levels present in the image), e.g., the negative of a quantum image, quantum image binarization, the histogram of a quantum image, and histogram equalization, could be easily computed as discussed in [10].

In [125], with the proposal of NEQR representation, Zhang et al. discussed the color transformations on all the pixels in the whole image, a certain range of an image by utilizing the control conditions, and the operation of statistical computation according to the grayscale information in the whole image. The time complexities for each operation were also presented which indicates the efficiency for these operations.

4.3 Frequency domain transformations

The frequency of an image reflects the gray intensity variation in the image. The low-frequency information composes the basic grayscale of the image, while the min-frequency information provides the main edge structure of the image. The high-frequency information enhances the image content in the min frequency and emphasizes the details of the image. Commonly used frequency domain transformations in QIMP (FTQI) include: quantum Fourier transform (QFT) [27, 104], quantum wavelet transform (QWT) [21], and quantum discrete cosine transform (QDCT) [47, 73, 92]. They are often adopted for quantum image compression and information hiding. These applications will be introduced in Sect. 5.

5 Quantum image-based applications

In the QIMP, some basic quantum logic gates, i.e., AND, OR, and NOT operations [119], and some operators could be combined to realize some arithmetic operations, e.g., quantum adder [40, 98], and quantum counter [65], and quantum comparator [99, 101], which are always utilized for realizing the senior QIMP algorithms or applications.

In addition, using the classical image processing as reference point, some methods and modules, e.g., quantum neutral network (QNN) [29, 57], quantum pattern recognition (QPR) [71], quantum genetic algorithm [5, 91], and quantum chaotic map [2, 24, 76], were studied and applied.

In this section, we will based on the above-mentioned transformations and modules introduce some senior QIMP algorithms and the applications based on them which were also mentioned in Sects. 3 and 4. Mainly, they are quantum image-related security technologies and the applications based on the quantum image strip.

5.1 QIMP algorithms

(1) Extraction

Feature extraction targets the detection and isolation of various desired portions or shapes (features) of a digital image. Zhang et al. made a lot of efforts on this area. In 2014, they proposed a novel quantum image edge extraction algorithm (QSobel) based on FRQI representation and the classical edge extraction algorithm Sobel [124]. In their discussion, they demonstrated that QSobel can extract edges with the computational complexity of $O(n^2)$ for a FRQI quantum image with a size of $2^n\times 2^n$. It shows a significant and exponential speedup compared with existing edge extraction algorithms, which possibly resolve the real-time problem of image edge extraction. Later, this group proposed a quantum feature extraction framework based on NEQR representation [127]. By utilizing the quantum addition and subtraction operations and some quantum image transformations, the feature points (corner points) could be extracted by comparing and thresholding the gradients of the pixels. The feature points extracted from a quantum image are stored in a quantum superposition via a quantum procedure with a approximate complexity $O(n^2 + q^2)$ for image with size $2^n \times 2^n$ and gray range $[0, 2^q -1]$; in addition, the extracted feature points can be used to construct quantum graph.

(2) Segmentation

Image segmentation is the process of separating the foreground of one or more objects from the background in a digital image. A quantum algorithm for threshold estimation and a segmentation algorithm based on iterative thresholding were proposed in [12]. The algorithms exhibit significant speedups compared to the analogous classical procedures because they exploit the quantum mechanism of amplitude amplification and QFT. Meanwhile, Caraiman et al. proposed another threshold-based segmentation method that applies a quantum oracle in a single computational step [13]. It allows for an accurate retrieval of the segmented image using a finite number of quantum measurement operations.

(3) Registration

Image registration is the process of transforming different sets of images (the same scene or object that may be from different sensors, times, depths, or viewpoints) into one coordinate system. Even though some quantum-inspired registration methods [90, 91] have been proposed, it is still rarely touched in QIMP. Due to the special properties of log-polar coordinate, a fast rotation-invariant quantum image registration algorithm was proposed based on the QUALPI quantum image model [126]. This quantum algorithm comprises four steps: quantum image preparation, quantum image expanding, quantum image searching, and quantum measurement. It achieves a quartic speedup compared with most of the classical image registration methods.

(4) Morphology

Morphology is an approach for analyzing and processing the geometrical structures of digital images. Dilation (the object of an image “grows” or “thickens”) and erosion (the object of an image “shrinks” or “thins”) are fundamental to morphological operations. Yuan et al. proposed a quantum morphology operations based on NEQR representation in [119]. Quantum binary dilation and erosion operations can be realized based on AND, OR, and NOT logic operations supplemented by quantum measurement operations. In addition, Yuan et al. also provide the design of flat grayscale dilation and erosion operations, where quantum bit string comparator [68] is used as an oracle in Grover’s search algorithm. Later, Zhou et al. uses the quantum loading scheme (QLS) [28, 70] and Boyer algorithm [9] to build the quantum dilation and erosion algorithm [128]. They claimed that the time complexity of both the dilation and erosion algorithm is $O(\sqrt{M}N)$, which is much smaller than the classical method for the quantum computing parallelism.

(5) Pseudocolor

Pseudocolor processing dyes grayscale images to color images to make the images more beautiful or to highlight some parts on the images. Jiang et al. proposed a quantum image pseudocolor coding scheme based on the density-stratified method [45]. A quantum colormap representation (QCR) is given in [45], which uses $q+t$ normalized qubits to store a colormap with $2^t$ colors. The color depth of each color is $2^q$. The preparation and retrieving of QCR with their quantum circuits were also discussed there. The main advantages of the quantum version for pseudocolor processing over the classical approach are that it needs less memory and can speed up the computation.

(6) Filtering

Image filtering aims to suppress the noise without losing the details and features in an image, which is an indispensable operation in image preprocessing. Caraiman et al. described a method to achieve the filtering of a quantum image encoded in CQIR by exploiting QFT and the principle of the quantum oracle in [11]. It is a common approach to convolve the image with a filter function, which in the frequency domain translates into a multiplication operation. It should be noted that there are fundamental differences between classical and quantum operations, the latter being necessarily invertible due to the reversible nature of quantum computation. Therefore, some classical processing operations, such as convolution and correlation, cannot be directly applied to quantum images [64].

(7) Denoising

Denoising is a process of removing noise from an image for aesthetic or practical purposes. Mastriani proposed a quantum Boolean image processing methodology in [66], where two new interfaces, i.e., classical to quantum and quantum to classical, are built to facilitate the boolean image denoising in quantum computing domain. By utilizing the quantum Boolean mean filter, the algorithm replaces central pixel value of a portion of the quantum boolean image affected by the selection mask. Meanwhile, color decomposition/recomposition and bit slicing/reassembling are considered for the preparation and completion of this procedure.

(8) Scrambling

Image scrambling, a technique widely used to transform an image into a disordered image, has been extensively studied. The popular traditional image scrambling algorithms, i.e., Hilbert transform, Arnold transform, and Fibonacci transform, are extended to the QIMP. The Arnold and Fibonacci scrambling quantum circuits in [40, 44] take advantage of the plain adder and adder modulo N by modifying operations on the input and output in order to scramble the images, while the Hilbert scrambling quantum circuit in [43] is executed by using Hilbert Scanning Matrix that is generated by a recursive generation algorithm. Most of the existing quantum image scrambling schemes are basically position space scrambling strategies. However, Zhou et al. [130] proposed a quantum image gray-code and bit-plane scrambling scheme based on NEQR representations. They reported that the scheme’s cost is rather low, and the scrambling speed is very high compared with other quantum image scrambling like the quantum Hilbert scrambling [43].

At the end of this subsection, we provide some quantum-inspired image processing methods (processing the classical image by utilizing the quantum mechanics and properties). We refer interested readers to more detailed discussions on image denoising in [6], image information hiding in [29, 76], image segmentation in [7, 61, 62, 118], edge detection in [18, 103], image compression in [57, 72], and image registration in [90, 91].

5.2 Quantum image security technologies

Traditional image security techniques, such as digital watermarking and image encryption, are widely used for effective protection of digital images. Digital watermarking is utilized to embed some watermark information into the carrier image. Such a strategy, however, should not change the visual acuity of the carrier image [31, 111]. This application is very important for the authentication of the copyright in entertainment industry. Additionally, image encryption offers a medium to translate the original image into a meaningless image (where visual acuity is lost) and become the information similar to the channel random noise. This is significant for some special communication applications such as secure transmission of military satellite images and patients’ records when they are sent across the Internet [56, 123]. In the remainder of this subsection, we will review some pioneering efforts for quantum image security applications, notably quantum image watermarking, steganography, and encryption strategies.

In 2010, Iliyasu et al. [35] pioneered what is today widely regarded as quantum image watermarking with the proposal of the WaQI scheme (watermarking and authentication strategy for quantum images based on restricted geometric transformations). The enhanced WaQI strategy in [34] is secure, keyless, blind, and computationally efficient, and can be perfectly used to authenticate the identity of an image owner. Subsequently, its two-tier grayscale version, or simply GWaQI scheme, was proposed in [37] that includes embedding a conspicuous watermark logo in a predetermined sub-area of the cover image; meanwhile, the same watermark signal is embedded to cover the rest of the image in an obscure manner. Although the WaQI scheme proves effective in authenticating whether the carrier image belongs to a party laying claim to the image’s ownership, its main drawback is that the content of the watermark is required in order to recover the watermark authentication circuit needed to validate the ownership of the carrier images. To solve this, a watermarking scheme for quantum images based on QFT was proposed in [122]. This watermark technique authenticates ownership of the carrier image actively (i.e., by extracting the watermark image without knowing what it is). However, as its drawback, the whole computation process realized using this technique is incapable of maintaining the normalized quantum image state, because their embedding strength controlling parameter is a fixed parameter [115, 121, 122]. In addition, two dynamic watermarking schemes, i.e., QWT-based watermarking [84, 117] and Hadamard transform-based watermarking [82], which utilize dynamic vectors to control the embedding strength instead of a fixed parameter for embedding process in [122], were proposed. These schemes suggest it is possible to get better visual quality under a higher embedding capacity, which outperform other existing schemes.

The watermarking strategies above are all based on the FRQI representation. However, we should recall that the FRQI is a grayscale-based quantum representation, so all of the watermarking methods introduced earlier are only available for dealing with the grayscale images. Based on the MCQI representation, a new multi-channel watermarking strategy for quantum images, MC-WaQI, was proposed in [111]. Using it, watermarking of colored quantum images appears realizable. In MC-WaQI, two keys are generated in the preprocessing stage of the scheme, one of them (color information key or briefly CIK) is dynamically produced by the quantum measurement to protect the color information of the watermark image for embedding information in the spatial domain of the cover image, while the other key (position information key or briefly PIK) is fixed and preassigned by the watermark owner to scramble the position information of the watermark image for the embedding in frequency domain [122]. The adoption of MCQI representation for the carrier and watermark images facilitates the protection of colored quantum images and provides a duple protection on the carrier image, which also improves the capability of watermarked images to withstand malicious attacks [111].

In addition, Jiang et al. proposed a Moiré pattern-based NEQR quantum image steganography strategy (a technique that hides a secret message into quantum images) in [41]. It is designed primarily as a steganographic algorithm and corresponding quantum circuits that hides a binary image into a grayscale image. The embedding algorithm begins with the choice of an initial Moiré grating, i.e., a stochastic image, as the cover image, and then deforms the initial Moiré grating according to the secret image, and the deformed Moiré grating is the Moiré pattern. Finally, the Moiré pattern is altered to obtain the stego image. Then in [46], this work was further enhanced with the proposal of two blind LSB (least significant bit) steganography algorithms in the form of quantum circuits based on NEQR representations. One algorithm is plain LSB, and it uses the message bits to substitute for the pixels’ LSB directly. The other is block LSB, and it embeds a message bit into a number of pixels that belong to one image block. The experimental results in [46] demonstrate that the invisibility is good, and the balance between the capacity and the robustness can be adjusted according to the needs of applications. In addition, Wang et al. also proposed a concrete least significant qubit (LSQb) information hiding algorithm for NEQR quantum images [101]. LSQb information hiding embeds a secret message qubit stream into the last qubit of color of quantum cover image. Moreover, the information hiding is researched on the frequency domain which can increase the security of quantum cover image.

Another aspect for quantum image security is the image encryption as introduced earlier. An encryption and decryption algorithm using geometric transformations on FRQI quantum images was investigated in [131]. In doing this, the correlations among adjacent pixels were reserved which means that the encrypted images were not noise-like and the sketches can be identified visually. As an improvement, in [83], the encryption scheme used the quantum image geometric transformations to shuffle the codes of pixel positions, and further the color transformations were performed to recode the color codings in the FRQI images. In addition, the double random-phase encoding technique is generalized to quantum scenarios, which was used together with QFT to realize a robust quantum image encryption/decryption method [116]. Similarly, a novel encryption algorithm for FRQI quantum image based on QWT and double diffusions was discussed in [102]. A seemingly more secure FRQI image encryption method based on image correlation decomposition was discussed in [30]. Therein, the whole quantum image is divided into a series of sub-images and stored into a complete binary tree array, after which they were randomly performed by one of the operations, i.e., quantum random-phase gate, quantum revolving gate, and Hadamard transform. Finally, the encrypted image can be obtained by superimposing the resulting sub-images using the superposition principle of quantum states.

5.3 Quantum image strip-based applications

A dextrous property of the strip representation used to encode $2^m$-ending quantum images (STQI) [33, 109] lies in its ability to utilize the parallelism inherent to quantum computation in order to transform multiple images using very few quantum resources. The $2^m$-ending FRQI/MCQI quantum images encoded in a strip (which we shall call F-strip and M-strip, respectively) can be horizontally oriented or vertically oriented [109]. Control-condition operations on strip wires could control which image is being processed. Combined with the control-condition operations from the position $\vert y\rangle \vert x\rangle$ to the color wire, every pixel in this strip can be accessed and/or manipulated.

Based on the FRQI representation for quantum images and its basic geometric transformations on them, a method to compare multiple pairs of quantum images in parallel was proposed in [109, 113]. The F-strip is used to compare multiple quantum images of the same size. It does this because an operation on the strip wire can transform the information in all the images in the strip simultaneously. The strip does not only make the comparison of quantum images possible, but it also provides an efficient way to execute multiple operations on multiple pairs of quantum images in parallel. There are three steps to complete the parallel comparison, namely the quantum image preparation, image comparison, and measurement results’ observation. The layout of the scheme to compare quantum images in parallel is presented in Fig. 9. The proposed quantum image comparison method is proven to require less computational resources and hence offers a significant speedup in comparison with performing the same task on traditional computing devices. Similarly, a complete scheme of solving the similarity of two multi-dimensional quantum color images, i.e., NAQSS, was proposed in [129], where two cases, i.e., the color image with segmentation information and without segmentation information, were considered, respectively.

Fig. 9

Scheme to compare quantum images in parallel on quantum computers (figure and descriptions adapted from [109])

Based on the method of parallel comparison of quantum images, a quantum image searching method was further explored in [106]. The proposal was built on a modified definition for the strip to allow a horizontal combination of two F-strips, which are located on the left and right side, called the Z-strip. The operation to search for images on quantum mechanical systems is realized by using only a single Hadamard gate and several measurements. On a classical computer, however, such an image searching scheme can only be achieved by comparing one pair of images at a time. Hence, the proposed method offers a significant speedup compared to how it is performed using classical computing resources.

Although the research targeting quantum images has already made a lot of progress as discussed before, the advances in quantum video are just beginning to gain traction. Perhaps, this could be attributed to the large size (in terms of a number of frames) and the enormity of resources required to process the data encoded in these frames.

As reviewed earlier, Iliyasu et al. [33] were credited with proposing the quantum movie framework, which in its generalized form can be seen as the utilization of F-strip. Quantum movies consist of multiple frames (quantum images) that comprise of slight changes from one frame to the next which combine to produce a single shot. The flexibility inherent to the FRQI quantum image as well as the feasibility of representing and controlling multiple quantum images is some of the properties that allow the use of the F-strip in quantum movies. The quantum movie framework itself is built on three conceptual devices: the quantum CD, quantum player, and movie reader [33] are used to achieve the preparation, manipulation, and measurement of the quantum movie. The quantum CD is defined as a device required to prepare, initialize, and store $2^m$ multiple frames, while the main objective of quantum player is to manipulate the content of these frames to convey a scene from the movie. As the movie reader, it measures the contents of the sequence of frames in order to retrieve and recover the contents of the movie [33].

There are three types of frames in a quantum movie, i.e., a key frame, viewing frame, and makeup frame [33], which are all FRQI quantum states. Figure 10 outlines the proposed schematics for these three types of frames as explained. Extending the formalism described in the quantum movie framework to the MCQI representation and utilizing the M-strip, the notion of color quantum video is realized as the basis on which the quantum video encryption and decryption on quantum computers is built [112].

Fig. 10

m-shots from a video showing the key, makeup, and viewing frames, $\vert F_m\rangle$, $\vert \mathcal {F}_m^c\rangle$, and $\vert f_{m,q}\rangle$, respectively (figure and descriptions adapted from [33])

As presented in [33] and utilized in [112], two or more key frames that exhibit very little resemblance to each other could delineate one shot. Using Fig. 10 as an example, we focus on the key frames, $\vert F_1\rangle , \vert F_2\rangle , \ldots , \vert F_m\rangle$, by preparing many copies of them on which quantum measurements to generate the CIKs are performed. The eight different elements in the CIK as well as the corresponding operations based on a specified Col or CS operation [87] are adopted so that the color information in the frames could be transformed. According to the CIKs obtained from the key frames, we then execute the same color transformations on the viewing frames and the makeup frames to complete the quantum video encryption procedure. The decryption procedure is used to recover the content of the encrypted video. Since all the transformations used in quantum computation are unitary transformations [67], it follows that the encryption procedure is completely reversible.

Generally, video encryption should make the quantum video indiscernible in order to avoid the illegal copyright and tampering by the unauthorized users, while the video decryption recovers the encrypted video back to its pristine format for access by the authorized users (with the correct keys). The method is considered secure, fast, and flexible [112]. The combination of these two procedures facilitate the secure transmission and sharing of video in quantum communication.

In addition, in [108], Yan et al. proposed a method for moving object detection in multi-channel quantum video (MTD) which could detect the location of the moving target in each frame of the quantum video, thereby providing a motion trail that could be used to describe the path or trajectory of the object. This work presents a modest attempt to utilize quantum computing properties for moving target in quantum video research. Further, such a work will inspire the electronic image stabilization in quantum video (QIS), which we will investigate in the future.

To summarize, in these three subsections, we have tried to highlight some of the astounding properties that have been made in QIMP as evinced by the many applications it has been used for. Figure 11 presents a brief graph simplifying the progress made in QIMP. All the acronyms therein have been mentioned earlier in this review.

Fig. 11

QIRs (in blue) and their transformations (in red), algorithms (in green), and applications (in yellow) (Color figure online)

6 Future developments of QIMP and concluding remarks

QIMP is a field full of exciting open problems for physicists, mathematicians, computer scientists, and engineers. So far, efforts in this area have been focused on developing the basic tools required to process images in quantum mechanical computers, and consequently, many areas of cross-fertilization remain to be explored yet. In order to transit from an emergent branch of quantum computing to an established field of research and technology development, we must identify, design, and implement disruptive algorithms and ‘killer apps’ (i.e., high-impact applications) that clearly show the power and advantages of QIMP over its classical counterpart. In the following lines, we share with the readership some thoughts about future research trends in QIMP.

6.1 Algorithmic advantages of existing QIMP protocols

(1) Comparison with best classical image processing protocols

In this review article, we have summarized several protocols devoted to storing, processing, and retrieving visual information using quantum mechanical systems. Indeed, some computational advantages and disadvantages of those protocols arise when making comparisons among them (e.g., the number of qubits used for storing images), but in order to clearly exhibit the true value and usefulness of those quantum protocols in both theoretical and applied computer science as well as in engineering applications, a comprehensive comparison analysis is yet to be produced.

For each existing QIMP protocol, we must compute accurate formulae for the computational complexity of algorithms used for storing and retrieving images as well as the amount of (quantum and classical) physical resources employed for those tasks. This endeavor must be undertaken so that its results can be meaningfully compared with the computational complexity and resource consumption of corresponding classical algorithms. This is crucial for two reasons:

• The complexity of QIMP algorithms is usually computed in terms of quantum gates, while classical complexity is usually computed in terms of algorithm running time.

• Quantum hardware designers must clearly identify the amount of classical and quantum resources needed to implement a given QIMP protocol.

(2) Advanced toolkits

In order to use existing and upcoming QIMP protocols to develop quantum algorithms in fields like computer vision, automatic surveillance and manufacturing, a set of basic and advanced image operations must be developed for each protocol. These toolkits should include quantum routines for performing basic tasks as rotations and region swapping, as well as advanced procedures for performing image filtering, scrambling, filtering, encryption/decryption and segmentation, among several others.

6.2 Applications for solving open problems in science and engineering

(1) Classical and quantum information

Quantum algorithms can be used to solve problems defined within the realms of classical or quantum information. For example, quantum algorithms can be developed to solve hard instances of the 3SAT problem [74] or to simulate quantum systems [23].

So far, published results in our field have mainly focused on processing classical images in quantum mechanical systems, i.e., processing classical information. Of course, this work has been of paramount importance to build the foundations of our field as well as to attract the attention of applied computer scientists and engineers who work on areas related to image processing. Meanwhile, research on storing, processing, and visualizing quantum information is a territory largely untouched. Thus, in addition to further progress on processing classical images on quantum systems, new research efforts should be devoted to designing algorithms to process signals/images produced by sources of quantum information.

Algorithms developed to process and visualize quantum information could become the building blocks of new methods to present complex data to experts as well as to the general public (data visualization is an active area of contemporary research [8]). For instance, processing and visualizing data emanated from areas like quantum light sources [78] using QIMP techniques may lead to novel methods to analyze and present data.

(2) Science and engineering

Scrutinizing models and understanding results via data visualization are a fruitful approach for efficient problem solving. Due to the immense amount of visual information available from a plethora of fields and the everlasting quest for efficient methods, computer image processing is a pervasive discipline in many areas of science and engineering like computer vision [89], astrophysics [39], pattern recognition [79], and medicine [17, 22], among many others. We foresee two scenarios for further development of our field:

a) Basic scientific research. Visualization of experimental results is of great value in many scientific disciplines (e.g., astrophysics, bioinformatics, and nanotechnology). In this regard, the development of QIMP algorithms could contribute toward solving open challenges in image processing (for example, enhancing superresolution algorithms [63]) as well as on realizing novel methods to process classical and quantum information (e.g., QIMP of phytoplankton data produced by multicolor flow cytometry techniques using quantum dots [97]).

b) Applications in engineering. There is plenty of room for novel applications of QIMP, particularly in highly visible fields like medical images, automatic surveillance, pattern recognition, (nano) manufacturing, and technology for law enforcement. Applications of QIMP algorithms in all branches of engineering are a key area of future work in our field for the following reasons:

• Developing algorithms to solve complex and real-life problems is an effective way to attract the attention of funding organizations. Hence, in order to increase the amount of financial resources in our field, we should create ‘killer apps’ of QIMP algorithms in high-impact fields, i.e., in areas that are both complex and visible to society at large.

• Engineering solutions focus on performance and accuracy, either as a result of market consumer demands (e.g., digital cameras: In less than twenty years, products on this market ranged from image size of $756\times 504$ pixels and 500 g of weight (Kodak DC40, Table I of [1]) to image size of 18 megapixels and 245 g of weight [15] ) or as key technical requirements within a given domain of application (for example, digital processing of medical images [19]). In both contexts, researchers working on QIMP could make use of the physical and mathematical properties of quantum computing (e.g., entanglement and quantum parallelism) as well as to propose novel uses of existing quantum algorithms and protocols (e.g., quantum teleportation) to design innovative solutions.

• Algorithms for specific-purpose hardware. One of the main goals in quantum computing is to design quantum algorithms that are more efficient (i.e., faster) than their classical counterparts. This goal usually has an implicit assumption: Both quantum and classical algorithms are to be executed on general-purpose and universal computers. Unfortunately, the quest for quantum algorithms running on universal computers has left somewhat behind the development of advanced quantum algorithms for specific-purpose architectures. Meanwhile, specific-purpose architectures are used in many areas of computer and communications technology (e.g., content-addressable memories for network routers [69]). In our opinion, specific-purpose hardware is a blue-sky arena for developing QIMP algorithms as well as to attract funding from industrial partners (for example, QIMP algorithms to run on specific-purpose devices like advanced digital cameras or next generations of TV technology [66]).

(3) Military applications

In addition to its key role in National Security programs, research on military scientific and technological topics is a most interesting activity, due to the very challenging nature of problems faced in those fields and the interdisciplinary teams that usually aim at solving those problems.

Image processing is a major component in military technology (e.g., [77, 81, 93]) and quantum information and quantum computation are areas of interest for military application [14, 75]. Hence, developing QIMP algorithms that can be used in scientific and engineering applications in this domain (e.g., quantum radar [48]) is a potentially fruitful area.

6.3 Concluding remarks

Beyond the ongoing efforts toward physical realization of quantum computing hardware, research is focused on what can be done with these technologies when realization has been intensified [31]. Encouraged by the de rigueur ubiquity and benefits of image and video processing applications and devices in today’s world, the last half decade has seen a surge in efforts to extend conventional image and video processing to the quantum computing paradigm. This effort has given birth to a new sub-discipline known as QIMP. At the onset, QIMP is devoted to conjuring representations to capture images based on the quantum mechanical composition of any potential quantum hardware.

The main purpose of this review is to gather the current mainstream QIRs and discuss the advances made in the area. In doing so, some similarities, differences, shortcomings, and likely applications for some of the available QIRs were reviewed. Having established itself as the paragon among QIRs, the FRQI representation has the highest number of applications and algorithmic frameworks built on it, and so, most of our review was based on it. The mathematical formalism, computational requirements and outlook, and likely applications emanating from most of the available QIRs were also reviewed.

Buoyed by the growing interest in QIMP in general, as well as the enthusiasm generated by FRQI in particular, Iliyasu et al. espoused the realization of FRQI application-specific hardware to the advances made in photonic (or optical) quantum computing technologies to provide an insightful compendium of the level of maturity in the area and the likely impediments to the realization of QIMP quantum computing hardware. We recommend interested readers to find more information about it from [36] as a basis to simulate interest geared toward the realization of QIMP-specific quantum computing hardware.

With additional efforts to improve on this work or offer similar espousitions along different quantum computing technologies, attention should be devoted on how to realize some low-level image and video processing applications and hardware for QIMP. All these efforts are essential for the realization of smooth, effective, and secure QIMP technologies, thereby allowing mankind to take full advantage of the immense potential of quantum computing.