Keywords

1 Introduction

Numerous resources are available from various digital and nondigital sources. Over time, scholars from different areas have started exploring various fields of study to gather information from different fields. For traffic surveillance and security surveillance, moving object detection and segmentation are critical. Detecting moving objects in dynamic environments is more difficult than it is in static environments. The inclination of research scholars and industries to transform the quality and quality of unstructured data has increased over time. Many data are available in the field of object identification and classification for researchers dig into to investigate various techniques for the identification of objects.

Furthermore, thanks to the immense growth and availability of online resources, users have a lot of exposure to various ideas, approaches, opinions, and recommendations on various methods. Such ample numbers of data open up new opportunities for scholars to analyze the existing techniques in their area of interest. Image processing has various applications in different fields of study. Among all these applications, the contributions of image-processing techniques and computer vision in the field of security and surveillance are remarkable. Moving object surveillance is an active area of research that detects, identifies, and tracks objects from a moving sequence of images. Objects in the video frame sequence are identified for video sequencing [1]. In order to track objects from moving vehicles or video sequencing of moving vehicles, it is important to detect an object that appears first in the video sequence or detect objects from every image frame of a video sequence. Using a good-quality, high-intensity video camera is required to capture and acquire inputs for high effectiveness and clear object detection. The identification of objects in videos follows three important steps:

  1. 1.

    Detection of object of interest

  2. 2.

    Tracking objects from each image frame

  3. 3.

    Identification of object behavior in an image

Morphology provides the operations for analyzing objects of different forms and shapes. This aids in object analysis and recognition. The required information is extracted from the image for analysis to help yield an improved image. Morphological procedures are contingent on the comparative assembling of pixel values [2]. The structuring component supports determining the method into which the structuring element fits to identify an image.

1.1 Morphological Operations on Image

  1. 1.

    Erosion: In this process, boundaries are eroded away. This shrinks the object. The mathematical erosion of image I can be defined as follows:

$$ I+T=\left\{G\mid \right[(T)A\in I $$
(1)

Erosion reduces the size of objects by etching the object borders. Structural elements pass through all the pixels of the image.

  1. 2.

    Dilation: This operation allows objects to expand, filling in small holes and connecting disjointed objects.

$$ I+T=\left\{G\;|\left[(T)\;A\cap I\right]\in I\right\} $$
(2)

For descriptor extraction, grayscale image representations are employed. Such representations simplify and minimize the computing requirements [21,22,23] (Fig. 1).

Fig. 1
Two illustrations depict spiral-like structures and are labeled R G B and grayscale respectively.

Image scale

Full size image

1.2 Impact of the Environment on Objects

Humans infer object-to-object interactions, part-to-whole object hierarchies, object properties, and 3D scene structures in addition to identifying and finding items in a scene. Having a better understanding of situations would help with applications such as robotic interactions, which often need information beyond item identification and position. This requires not only scene perception but also knowledge of the physical surroundings. An image has pixels, and every pixel of an image has a numerical intensity value. In order to classify images, it’s very important to identify objects that are static or moving, and they must be clearly visible. Many factors influence the visibility of the image. One such key factor includes environmental conditions such as fog, rain, snow, dust, and so on (Fig. 2).

Fig. 2
Four photographs of automobiles traveling through sandstorms, rain, thunderstorms, and fog.

Moving vehicle in (a1) dust wind, (b1) rain, (c1) hail storm, and (d1) fog

Full size image

The environment also significant affects the identification of images. Images taken in hazy or foggy weather are difficult to identify because of their unclarity. Fog and dust, in particular, drastically diminish visibility distance. Because of light dispersion and attenuation, the color of nearby objects seems to be quite similar, with poor saturation. Under such circumstances, it is difficult to distinguish between objects because the edges between the background and the item in the foreground become blurred. Many cameras are installed in modern automobiles, and they are used for a wide range of purposes. The detection of fog from images captured by a photographic camera mounted on a vehicle is a difficult task that could be useful in a variety of situations [3]. So far, methods have focused on the attributes of nearby items in the image, such as lane markings, traffic signs, vehicle backlights, and approaching vehicle headlights. In contrast to all these previous studies, some researchers suggest adopting methods that utilize image descriptors and take a cataloging approach to distinguish between images with fog and those without fog.

2 Methodology and Research Description

Image-processing techniques are used to visibly improve image appearance. These techniques help to improve the interpretation of an image by a human or an automated system. Image-processing techniques may also be used to identify images and vehicles under adverse weather circumstances, such as fog, rain, hail, etc. Table 2 presents research conducted in similar fields.

Some researchers have proposed conducting a literature review on studies that aimed to identify objects. For the analysis and identification of objects, the Scopus database has been selected because this database indexes a wide range of engineering literature from conferences and journals. The scholars also can explore a wide range of research articles from the Scopus database. For this study, papers from 2011 to 2022 are included for analysis. For the selection of research papers, we focused on keywords such as “object detection,” “object recognition,” “moving vehicles in the fog,” “vehicle in the fog,” “morphology,” “neural network,” and “deep learning.” We also used Boolean connectors “AND,” “OR,” and the symbol “+” to influence the search so that more specific and meaningful data could be gathered. In order to retrieve the desired research papers, a search using keywords such as “object identification,” “fog,” and “moving vehicle” was conducted; the search yielded 7300 research papers in the first attempt. To refine the results, the second phase of the search for studies published in journals was limited to 1100 publications. Further, in the third phase, we focused primarily on object identification in fog, because of which the number of research papers dropped to 423 (Figs. 3 and 4).

Fig. 3
A word cloud presents the keywords for the selected research papers. Some of the words include structuring element, edge detection, object detection, moving vehicle, and morphological processing.

Keyword cloud of the selected research papers

Full size image
Fig. 4
A word cloud depicts the keyword used for paper searching. Some of the words include image processing, image recognition, r c n n, morphology, deep learning.

Keywords used for paper searching

Full size image

3 Findings and Results

This section is divided into three sections: year-by-year statistics, journal-by-journal statistics, and theme-by-theme reporting. In the first subsection, the study displays the year-by-year distributions of publications and a list of periodicals. In the second subsection, the study shows the high-frequency words from the title and author keywords of the publications studied. In the third subsection, the study shows diverse themes across several image-processing approaches. Various functional factors, such as contrast-sensitive images, brightness, transparency, texture gradient, and light, affect the visibility of an image display. Further, a few more key performance indicators, also known as image-quality factors, influence image quality and visibility, such as image sharpness, noise, dynamic range, color accuracy, alterations, homogeneity, blaze, artifacts, compression, and links. In addition to these factors, a few environmental factors also influence image visibility. Such factors include rain, hail, dusty wind, and fog. Various researchers have carried out significant research on all these factors and their effects, and they have proposed suggestive and corrective techniques to overcome all these problems, except for fog.

Even though a few researchers have conducted research on object identification in foggy weather, they have not put any significant weight on the identification of objects in moving vehicles in fog or video sequencing of the vehicles moving in fog. This leaves a huge gap to fill. We have searched through research papers in this field and have identified and focused on their techniques for identifying objects in fog.

This section is divided into three subsections: year-by-year statistics, journal-by-journal statistics, and theme-by-theme reporting. The first subsection displays the year-by-year distributions of research articles and a list of periodicals. The study shows high-frequency words from the title and author keywords of the articles studied in the second subsection. The study shows the diverse themes across several image-processing approaches in the third subsection. Various researchers have proposed different methods for identifying objects in moving vehicles so that the vehicles themselves can also be identified (Figs. 5 and 6).

Fig. 5
A horizontal bar graph plots the year of publication versus the number of research papers. The values are, 2011, 2. 2012, 8. 2013, 6. 2014, 12. 2015, 8. 2016, 6. 2017, 5. 2018, 9. 2019, 7. 2020, 23. 2021, 4.

Year-by-year distributions of research articles

Full size image
Fig. 6
A horizontal bar graph plots journal versus count of research papers. Zoological record 2. S C I 38. I JEAT 3. I E E E 12. Emerging source citation index 2.

Research articles per journal

Full size image

All the referenced papers were analyzed, read, and categorized according to vehicle identification and fog estimation. Various researchers have proposed different techniques for vehicle identification under adverse environmental conditions, such as rain, haze, and fog. Researchers classify fog density as a visibility feature of a vehicle preceding another vehicle and the distance to that other vehicle. Researchers proposed a method to recognize daytime images in dense fog and applied Fourier transformation with global features. The proposed method was applied to many images. Here, 96% of fog-free photographs were categorized as fog-free, whereas 93% of fog images were classified as fog images. The researchers used annotated photos with fog values, as shown in Table 1. The data did not include profiles of roads [4]. To categorize fog, various calculations have been performed on the basis of visibility distance.

Table 1 Visibility distance range
Full size table

While following the proposed methodology, researchers assessed fog images and fog-free images. Afterwards, they proposed using a semisupervised convolutional neural network. A sparse Laplacian filter was applied to gather information about the vehicles. The SoftMax classifier layer was used as the output layer on the labeled vehicles. We worked with the BIT-Vehicle data set. This data set includes 9850 images to test the proposed method. Only 10% of the images were nighttime images [3, 4] (Fig. 7).

Fig. 7
An illustration of a network. Input photograph of a car passes through convolutional neural network, pre trained with a large number of unlabeled vehicles and a small number of labeled vehicles, points to probability of type, and further to identification of vehicle type as sedan.

Representation of the analyzed works that use local features [1,2,3,4]

Full size image

First, foreground objects are extracted from the video. After this, the hierarchical multi-SVMs (support vector machines) method is applied for vehicle classification. For final precision, a voting-based correction scheme is used. Vehicle classification is achieved in complicated traffic scenes by taking the proposed approach. Singh et al., Zhuo et al., Murugan et al., Chowdhury et al. presented a vehicle classification technique that is based on neural networks. In the pretraining stage, GoogLeNet is applied to the ILSVRC-2012 data set, and after fine-tuning, a vehicle data set of 13,700 images is used for the final classification [5,6,7,8]. Liu et al. have proposed techniques to detect the number of vehicles on the road in real time. They applied image subtraction on foreground and background images and used image binarization, a method of counting objects that results in a faster computing technique. Wang et al. have proposed classifying images on the basis of deep learning and generative adversarial networks (GANs) [9, 10]. Jyothi et al. have proposed using a high-accuracy convolutional neural network (CNN) technique for vehicle classification. The vehicles in focus were cars and trucks [11]. Morphological operations were applied in vehicle detection and identification. Video sequencing of vehicles was reviewed by Chandrika et al., and frames were extracted from it [12] (Fig. 8).

Fig. 8
Four photographs. Two at the bottom have vehicles moving on highways, each marked and labeled. The top images depict the extracted images of the vehicles on a roadway, and only the vehicles.

Vehicle extraction using a deep neural net [9,10,11,12]

Full size image

Kalyan et al., Shyamala et al., Şentaş et al., Hedeya et al., Zahra et al., Jagannathan et al., Miclea et al. have applied image binarization and Sobel edge detection. Morphological operations were applied to the image to identify objects. Researchers used their own video data set. They tested the SVMs classifier and tiny-YOLO on a data set, and their paper proposed an adaptive histogram equalization and the Gaussian mixture model. Researchers have implemented a technique to enhance vehicle image quality and detected vehicles from denoised images [13,14,15,16,17,18,19]. Kim et al. have proposed an efficient technique for visibility measurement under foggy weather conditions [20]. International Commission on Illumination (CIE) defined meteorological visibility distance as the distance beyond which a black object of adequate size with a contrast of less than 5% can be seen. Jiang et al. have applied the Canny–Deriche filter [21]. The purpose of this research was to propose a framework for recovering the contrast of photographs captured from a moving vehicle. Nam and Nam first computed the weather conditions, then inferred the scene structure, which was refined during the restoration process [22]. The proposed visibility enhancement algorithm was not designed for road photos [23]. Hautière et al. proposed techniques in which the fog region is segmented using the calculation from the direction charts. They also computed fog density and used a method that restored the contrast and assumed that the road was flat, to detect vertical objects [24]. Abbaspour et al. used a technique on a single in-vehicle camera. This method has better performance in terms of accuracy and speed to detect fog density from images [25]. Hautière et al. proposed image improvement techniques appropriate under daytime fog conditions in differential geometries, where the partition of unity was the base of this proposed method. This model was the most suitable for contrast restoration under foggy conditions [26]. These researchers suggested an algorithm for reducing the turbidity of an image. In their method, they assumed that an image has a reference intensity level and a characteristic intensity level. A low-pass filter was used to obtain the reference intensity level: intensity level = original intensity level – reference intensity level [27] (Fig. 9).

Fig. 9
Two sets of photographs of a moving vehicle and an aerial view of a city block processed through a low-pass filter to obtain an enhanced image.

Image identification by applying image enhancement [24,25,26,27]

Full size image

Negru et al. conducted an experiment to contrast an image by providing quantifiable proof that road safety is increased by Advanced Driver Assistance Systems (ADAS). Next, using a modified Piéron’s rule as a foundation, a quantitative model was developed for target visibility (Vt), which is calculated from onboard camera images [28]. Other researchers presented a pixel-based technique to eradicate haze artifacts from images by using a single image-based dehazing framework. Halmaoui et al. recommended conducting a haze density analysis to determine the level of atmospheric light. A transmission map can then be estimated and refined by using a bilateral filter [29] (Fig. 10).

Fig. 10
A set of four photographs of a natural scene with trees on either side of a path processed through different filters to obtain an enhanced image.

Image identification by removing haze from an image [28,29,30]

Full size image

For hazy photos and videos, Kim et al., Su et al. have proposed a fast and optimal dehazing method. The contract term and the data loss term were combined into a cost function by these researchers. The suggested approach improves the contrast and maintains information by minimizing the cost function. This method is expanded to real-time video dehazing from the static-image-dehazing algorithm [31, 32]. Sharma and Arya provided a technique for retrieving image data from a single blurry image. The dark-channel-prior (DCP) algorithm has a tendency to underestimate bright region transmission. The predicted value of the shady network of a hazy image was utilized as an estimation of this offset to adjust the transmission because the intensity in a dark channel that was influenced by haze created the same offset [33].

For the real-time processing of haze removal from high-definition videos, a GPU-accelerated parallel computing solution was proposed [34]. For real-time processing, Rai et al. proposed a single image haze reduction approach that implements hardware. The suggested method uses computationally efficient image-processing techniques [35]. This research investigated aureole pieces generated by scattering and nonuniformly dispersing lighting in low-light foggy circumstances. To reduce the influence of multiple scatterings, an image was subdivided into a halo and scene layer. Next, following the Retinex theory, they calculated the spatially variable ambient light [36]. In the transmission map, they employed the mean shift segmentation technique to separate the sky areas from the foreground, which were obtained by using the dark-channel-prior approach. Afterward, they used guided image filtering to smooth the map by separately increasing the brightness of each sky region in the transmission map [37]. Yuan et al. proposed dark channels prior to masking the sky regions focused on road edge recognition, and dehazing was similarly focused—together resulting in improved visibility under foggy conditions [38].

In their study, Mou et al. proposed an algorithm consisting of sky segmentation and area-wise medium-transmission-based image dehazing [39]. Another approach was proposed that first detects the sky and divides the image into sky and nonsky regions and then independently estimates the transmissions of the two sections, followed by a refining step [40]. This is an efficient strategy for improving fog-degraded traffic images. The fog-degraded image is divided into blocks. The block with the least amount of sparsity is chosen to compute the local transfer function [41]. To simulate the mathematical model for the fog, Hu et al. used a deep neural network [42].

The edge-sharpening cycle-consistent adversarial network was proposed as a generative adversarial network, namely ESCCGAN [43]. To solve the performance limitations of using an atmospheric disintegrating archetypal-based method, a residual-based dehazing net was developed [44]. The generative adversarial network (GAN) dehazing method has been used to dehaze image areas. This technique considers the many degrees of haze concentration that need to be adjusted while preserving the original image’s details [45]. The suggested algorithm employs a supervised machine-learning technique to approximate the transmission medium’s pixel-wise extermination factors and uses a unique compensation scheme to correct the erroneous enlargement of white objects after dehazing.

Feng et al. used an edge-preserving maximum reflectance prior (MRP) method to reduce the color effect of hazy photos taken at night. The transmission map was then obtained by feeding the hazy image with no color effect into the self-encoder net [46]. Feng et al. continued their research by considering the following parameters: contrast, intensity, image noise, image resolution, illumination, heavy occlusions, visibility distance, classification accuracy, speed, changes in illumination, aspect ratio, compactness, the accuracy of a crossing vehicle, hue, precision, accuracy, light angle, size variety, relative width, length and area, Histogram of Oriented Gradients (HOG), precision, vehicle class, classification accuracy, computation complexity, processing speed, reliability, standard deviation, entropy, visibility distance, saturation value, chroma, visible threshold, attenuation, meteorological visibility distance, object distance, camera response, pitch angle sensitivity, inflection point, processing time, reaction time, and radiance. Our proposed review is carried out on the basis of the effect that the parameters have on the input image under consideration.

Table 2 highlights the implemented techniques and the key points extracted from all the referenced research papers (Fig. 11).

Table 2 Techniques and dimensions used by different researchers
Full size table
Fig. 11
A bar graph of the percentages of different types such as view-based, intensity-based, feature, natural, graph, and others with a decreasing trend, and an increasing line on top.

Representations based on various feature types

Full size image

4 Conclusion

According to the key findings of all the research papers referenced in the survey, a flexible fog-estimating method is required. A hybrid algorithm is not used/proposed by any of the researchers to detect the edges of moving/dynamic objects. In addition to this, none of the researchers focused on identifying moving vehicles in foggy weather, which leaves a huge gap to fill. According to the research conducted to date, researchers have proposed many techniques and methods to identify objects but have not focused much on the identification of moving objects. The few researchers who considered moving object identification did not include the impact of the environment on moving vehicles, especially the impact of foggy weather on the visibility and identification of moving vehicles. Therefore, an algorithm that can efficiently identify moving objects in fog needs to be developed.