Abstract
Keeping crowdsourced maps up-to-date is important for a wide range of location-based applications (route planning, urban planning, navigation, tourism, etc.). We propose a novel map updating mechanism that combines the latest freely available remote sensing data with the current state of online vector map data to train a Deep Learning (DL) neural network. It uses a Generative Adversarial Network (GAN) to perform image-to-image translation, followed by segmentation and raster-vector comparison processes to identify changes to map features (e.g. buildings, roads, etc.) when compared to existing map data. This paper evaluates various GAN models trained with sixteen different datasets designed for use by our change detection/map updating procedure. Each GAN model is evaluated quantitatively and qualitatively to select the most accurate DL model for use in future spatial change detection applications.
Access provided by Autonomous University of Puebla. Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
Conventional (manual) crowdsourced map updating procedures utilises remote sensing imagery as a background layer to guide mappers as they manually digitise objects (e.g. buildings, roads, etc.). For example, OpenStreetMap (OSM) allows for the use of multiple satellite image sources when updating their maps [1]. However, to detect changes in satellite imagery automatically when comparing to the latest versions of online vector maps is an important next step for many GIScience related problems, including mapping.
Previously, we introduced our methodology for detecting changes (both constructions and destructions) between vector maps and raster images [2]. Consequently, a series of experiments was conducted to evaluate the accuracy of this OSM-GAN procedure. This paper reports on these experiments and other related outcomes of OSM-GAN predictions with various datasets.
Specifically, this study evaluates the prediction accuracy of various OSM-GAN models on several spatial datasets to select the best change detection model for use in further map updating operations. Two different raster and vector data sources were tested: 8-bit (panchromatic) and 24-bit (RGB) raster image data with spatial resolution 15 cm/pixel and 30 cm/pixel [5] and; OpenStreetMap (OSM) vector map data plus Ordnance Survey IrelandFootnote 1 (OSi) building footprint data of Dublin city centre were also used in conjunction with the raster data mentioned above.
1.1 Data Sources
Raster Data
Raster satellite image data was used to train the OSM-GAN models for detecting any changes to the map in a given Area of Interest (AoI). First, a satellite image dataset was created using freely available Google Earth satellite images discovered online using customised data crawlers that considers both spatial resolution (15 cm and 30 cm) and AoI. Second, a 25 cm resolution aerial orthophoto dataset of Dublin area registered to the Irish Transverse Mercator (ITM) coordinate system was acquired from OSi with an academic research license.
These TIFF (Tagged Image File Format) orthophotos needed to be pre-processed before inputting to the deep neural network – e.g., resampled to 30 cm pixels, co-registered, tiled, and served from QGISFootnote 2. Figure 1 shows the qualitative differences between both data sources (Google Earth and OSi). Note how some buildings currently visible in the Google Earth imagery are not present in the OSi orthophotos as they have since been demolished in preparation for constructing the new TU Dublin campus.
Vector Data
OSM vector data was the primary map data source checked for changes in this study. As OSi building footprint (vector) data is produced by Ireland’s National Mapping Agency, it was used as ground truth for model training and prediction purposes. The OSM vector data was downloaded using their Overpass API [3] by first parsing the minimum bounding rectangle (MBR) of a user generated AoI. The OSi building footprints were provided in DWG (AutoCAD) format. A series of operations converted the DWG formatted data into GeoJSON format to be compatible with further processing steps. Table 1 gives a summary of both the OSM and OSi building footprint datasets.
1.2 The Kay Supercomputer
The Irish Centre for High-End Computing (ICHEC) allows institutional users (e.g. academic researchers) access to its super computing infrastructure, named Kay [4]. Kay is comprised of five sub-components: Cluster, GPU, Phi, High Memory, and Service and Storage. Specifically, for experiments in this study, the GPU service was utilised.
The GPU service is a partition of 16 nodes where each node has 2 × 20-core 2.4 GHz Intel Xeon Gold 6148 (Skylake) processors, 192 GiB of RAM, a 400 GiB local SSD for scratch space and a 100 Gbit Omni-Path network adaptor. Two NVIDIA Tesla V100 16 GB PCIe (Volta architecture) GPUs are integrated on each node. Each GPU has 5,120 CUDA cores and 640 Tensor Cores. In order to reduce training times, this study tested the Kay Supercomputer with various parameter settings [4]. As such, overall training times per model reduced from a few days spent training on a high-end “gamer spec” laptop, to just a few hours on Kay.
2 Related Work
Spatial change detection is a well-researched area in both the GIScience and computer vision domains. Historically, many different image processing techniques, including Markov Random Field [6] and Principal Component Analysis [7], were used to perform spatial change detection operations. More recently, artificial neural network-based techniques have been introduced to address various limitations of more traditional approaches (e.g. to overcome low performance, low segmentation accuracy, higher time complexity, etc.).
Now, common solutions to this problem rely on applying Machine Learning (ML) techniques such as U-Net [8], SegNet [9, 10], Mask R-CNN [11], and Pix2Pix among others [12, 13]. These approaches follow the encoder-decoder architecture to perform image segmentation, a critical step in any change detection process. In particular to GIS, image segmentation attempts to isolate various entities in the environment visible in aerial imagery. For example, entities such as buildings [14,15,16], road networks [17,18,19,20,21,22,23,24], and land-use classifications [25,26,27,28].
With the emergence of Generative Adversarial Networks (GANs), image segmentation procedures have been redefined as image-to-image translation. GANs are a ML technique of training a generative neural network model by representing the task as a supervised learning problem with two sub-models: a generator that learns to generate new examples; and a discriminator that tries to classify these generated examples as either real or fake (generated) [29]. Several studies have proposed GAN-based solutions for various spatial change detection problems, such as generating heat-maps of possible changes [30], seasonal change detection [31], and image classifications [32].
2.1 OSM-GAN for Spatial Change Detection
The OSM-GAN approach presented in this paper suggests a change detection methodology that employs spatial imagery (satellite images) and OSM vector map data [2] to train its models. The deep learning model should be accurate enough to detect image objects (e.g. buildings) to predict any change detection outcomes to these map features. Technically, the OSM-GAN model needs to perform a satellite image to feature-map translation with a high confidence level [2]. This paper proposes a methodology to evaluate various GAN models (trained with different datasets to perform satellite image to feature-map translation) to detect spatial changes accurately.
Producing our OSM-GAN model begins with the data crawling process. Freely available raster and vector data sources are crawled (mined) and saved in local directory structures ready for further processing. Geo-referenced satellite images are merged to construct the left half of the training sample, and OSM vectors are merged into a binary (black & white) single image to create the right half of the training sample. This process results in a single 600 × 300 pixel sized training sample as shown in Fig. 2.
An object-density based data filtering mechanism is used to remove false-negative data samples from the training dataset [2]. The filtered dataset is then split into a 3:2 ratio of training:validation sets of data. Finally, these datasets were fed into the training algorithm on the Kay supercomputer to generate the resulting OSM-GAN model.
To initiate the change detection process, a feature-map (binary image that represents particular map features (e.g. buildings) as white blobs) needs to be predicted for a particular satellite image using the OSM-GAN model generated previously. Then the predicted feature-map is segmented into separate objects and compared to current OSM vector data using an Overlap Score Matrix (percent overlap of a feature-map object and its OSM vector footprint). Finally, any detected changes are post-processed to compose OSM-acceptable changesets. Figure 3 illustrates the overall workflow for spatial change detection based on OSM-GAN.
3 Experiments and Results
For this study, a series of experiments were conducted to evaluate the accuracy of OSM-GAN models qualitatively and quantitatively. Sixteen OSM-GAN models were trained with different datasets. A combination of two different spatial resolutions (15 cm and 30 cm) with two different types of images (panchromatic and RGB) were used to create the raster image segment (left half of the training sample). OSM and OSi vector data were used to create the right half of the training sample (Fig. 2). For instance, the Google_OSi_8bit_z19 dataset was created from panchromatic (8-bit) Google Earth satellite images with 30 cm/pixel and OSi vectors. After the training process, the final model is named the same as the name of the dataset used to create it.
3.1 Modelling OSM-GAN with OSi Data
OSi raster (orthophotos resampled to 30 cm and 15 cm pixels to match the satellite data) and vector building footprint data was used in this experiment. Four datasets with different spatial resolutions and bit-depth were created from the above-mentioned sources. These datasets were smaller than the OSM datasets since the data provided by OSi was of a limited area of Dublin city centre only. Table 2 summarises the two datasets produced for this experiment.
Upon completion of the training process, each model was evaluated on a new dataset within the same AoI. These results were qualitatively and quantitatively evaluated, and Accuracy, Recall, Precision and F1 score measurements were calculated for each model (Table 3). The model trained with 30 cm/pixel resolution RGB images can be considered more accurate than the other three models.
Figure 4 qualitatively compares the outcomes of the above-listed OSM-GAN model predictions. It can be seen in Fig. 4a that the OP_OSi_24bit_z19 model gives comparatively more accurate results, demonstrating the importance of a qualitative analysis of testing. For example, the OP_OSi_8bit_z20 model predicted a large building that could be identified as an “extension” to existing OSi vector data by the subsequent change detection process.
3.2 Modelling OSM-GAN with OSi-OSM Data
The second experiment was designed to investigate the consistency/coherence between OSi Orthophotos and OSM vectors. Apart from a difference in the spatial reference system used, it was observed that current OSM vectors of the test area are outdated. Therefore, many data samples were filtered out in the data filtering phase. Table 4 summarises the generated datasets using OSi Orthophoto images and OSM vectors.
A quantitative analysis of the testing results is listed in Table 5. In this case, the model trained with a 30 cm (z19) 8-bit dataset showed the highest accuracy, while the first experiment produced a model trained on RGB images as the most accurate; in both cases with the same resolution.
Figure 5 illustrates the qualitative comparisons of the models in Table 5. Since visual comparisons show only two random instances taken from the test dataset, the visual results perhaps do not support well the quantitative measurements obtained above. However, to improve the change detection workflow, the results should be accurate both quantitatively and qualitatively.
3.3 Modelling OSM-GAN with Google-OSi Data
Models with Google Earth satellite images and OSi building footprints were trained in a third experiment. Four datasets were created using different spatial resolutions and bit-depths. A relatively small AoI fitting OSi boundary constraints was applied to the Google Earth image crawler to collect the relevant satellite images or the area. Table 6 summarises the details about these datasets.
Table 7 lists the quantitative measurements calculated for the trained models. A model trained with 15 cm RGB images scored better quantitative results than the other three models. After comparing to previous experiments (OSi-OSi and OSi-OSM), the model trained with higher resolution images is quantitatively more accurate.
Figure 6 shows a qualitative comparison of some prediction samples. The predictive results of the Google_OSi_24bit_z20 model (Fig. 6b) agree with the above quantitative results. The predicted polygons can be used in the subsequent change detection process since they are allied to ground truth polygons.
3.4 Modelling OSM-GAN with Google-OSM Data
Finally, yet importantly, Google Earth satellite images and OSM vectors were combined to perform another training phase. Since both data resources are free and unlimited, a wider AoI was chosen and crawled to create the following datasets.
The above-listed datasets were used to train four OSM-GAN models. These models were then evaluated using the same accuracy measurements such as Accuracy, Recall, Precision, and F1 Score (Table 9). The model trained with 30 cm RGB Google Earth satellite images and OSM vector footprints performed better. Significantly, this is the most accurate OSM-GAN model obtained when compared to all the models evaluated in the four experiments.
Figure 7 qualitatively compares the prediction results of models trained with Google Earth satellite images and OSM building footprints. In this case, the prediction results of Google_OSM_24bit_z19 shows the best qualitative test result, agreeing with the quantitative results above. Comparing all 16 models tested, the results of model Google_OSM_24bit_z19 suggests to train 30 cm Google Earth satellite images with OSM vector data to obtain the most accurate OSM-GAN models for change detection.
4 Conclusions
This paper presented sixteen different OSM-GAN based experiments with a quantitative analysis of each model produced, as well some qualitative observations. It evaluated different OSM-GAN models against different raster and vector data sources. Each dataset offers its own benefits and limitations and the qualitative results motivated continued training with larger area datasets.
The study concludes that the vector footprint generated by OSM-GAN image-to-image translation could be extended to spatial change detection procedures. Experiments show that model training with larger datasets (i.e. datasets built from Google Earth satellite images and OSM building footprints) yielded more accurate feature-map predictions.
Since the proposed spatial change detection methodology is highly based on OSM-GAN model accuracy, the model should be as accurate as possible to translate a satellite image to its corresponding feature-map. The final experiment reveals that training on larger sets of geographically similar areas could be a solution to generating more accurate OSM-GAN models. In other words, models trained with Dublin data should not be used for mapping Paris, for example. The model trained with Google Earth satellite images (24-bit, 30 cm/pixel) and OSM building footprints scored the highest accuracy (88.4%) among all the experiments. Moreover, the predictions of the Google_OSM_24bit_z19 model can also be observed qualitatively as more accurate than the other model predictions.
The ultimate objective of this research is to build an end-to-end workflow to update crowdsourced maps automatically with the use of freely available data (satellite images, vector footprints) and Artificial Intelligence (AI) techniques. Automated map update success ultimately depends on the accuracy of the ML change detection process since correctly identifying spatial changes in the map is an initial key step in this process. Compared to contemporary approaches for automatically detecting spatial changes, the proposed OSM-GAN approach offers an appropriate mechanism to follow.
As a next step, a new OSM-GAN model will be trained on the original 25 cm Dublin orthophotos with a larger AoI to potentially produce even more accurate feature-map results. The training process will utilise a Transfer Learning approach and so begins by initiating the process using the Google_OSM_24bit_z19 model parameters as the base input model. In order to evaluate the performance and accuracy of OSM-GAN against other spatial change detection models in the literature, a new test phase will also be carried out on the crowdAI Mapping Challenge datasetFootnote 3 in future work.
References
OpenStreetMap. https://www.openstreetmap.org. Accessed 22 Sept 2021
Niroshan, L., Carswell, J.D.: OSM-GAN: using generative adversarial networks for detecting change in high-resolution spatial images. In: 5th International Conference on Geoinformatics and Data Analysis (ICGDA 2022), Paris, France, January 2022, Springer Lecture Notes on Data Engineering and Communications Technologies (2022)
Overpass API. https://wiki.openstreetmap.org/wiki/Overpass_API. Accessed 29 Oct 2021
Kay. https://www.ichec.ie/about/infrastructure/kay. Accessed 15 Sept 2021
What is bit depth? https://etc.usf.edu/techease/win/images/what-is-bit-depth/. Accessed 15 Oct 2021
Gong, M., Su, L., Jia, M., Chen, W.: Fuzzy clustering with a modified MRF energy function for change detection in synthetic aperture radar images. IEEE Trans. Fuzzy Syst. 22(1), 98–109 (2013)
Yousif, O., Ban, Y.: Improving urban change detection from multitemporal SAR images using PCA-NLM. IEEE Trans. Geosci. Remote Sens. 51(4), 2032–2041 (2013)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 5–9 October 2015, pp. 234–241 (2015)
Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39, 2481–2495 (2017)
SegNet. https://mi.eng.cam.ac.uk/projects/segnet/. Accessed 20 Sept 2021
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017, pp. 2961–2969 (2017)
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017, pp. 5967–5976 (2017)
Image-to-Image Translation with Conditional Adversarial Networks. https://phillipi.github.io/pix2pix/. Accessed 20 Sept 2021
Tiecke, T.G., et al.: Mapping the World Population One Building at a Time. arXiv 2017, arXiv:cs/1712.05839
Iglovikov, V., Seferbekov, S.S., Buslaev, A., Shvets, A.: TernausNetV2: fully convolutional network for instance segmentation. In: Proceedings of the Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA, 18–22 June 2018, vol. 233, p. 237 (2018)
Microsoft/USBuildingFootprints. https://github.com/microsoft/USBuildingFootprints. Accessed 20 Sept 2021
Zhou, L., Zhang, C., Wu, M.: D-LinkNet: LinkNet with pretrained encoder and dilated convolution for high resolution satellite imagery road extraction. In: Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA, 18–22 June 2018, pp. 192–1924 (2018)
Oehmcke, S., Thrysøe, C., Borgstad, A., Salles, M.A.V., Brandt, M., Gieseke, F.: Detecting hardly visible roads in low-resolution satellite time series data. arXiv 2019. arXiv:1912.05026
Buslaev, A., Seferbekov, S.S., Iglovikov, V., Shvets, A.: Fully convolutional network for automatic road extraction from satellite imagery. In: Proceedings of the Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake Cit, UT, USA, 18–22 June 2018, pp. 207–210 (2018)
Xia, W., Zhang, Y.Z., Liu, J., Luo, L., Yang, K.: Road extraction from high resolution image with deep convolution network—a case study of GF-2 image. In: Multidisciplinary Digital Publishing Institute Proceedings, MDPI: Basel, Switzerland, vol. 2, p. 325 (2018)
Wu, S., Du, C., Chen, H., Xu, Y., Guo, N., Jing, N.: Road extraction from very high resolution images using weakly labeled OpenStreetMap centerline. ISPRS Int. J. Geo-Inf. 8, 478 (2019)
Xia, W., Zhong, N., Geng, D., Luo, L.: A weakly supervised road extraction approach via deep convolutional nets based image segmentation. In: Proceedings of the 2017 International Workshop on Remote Sensing with Intelligent Processing (RSIP), Shanghai, China, 19–21 May 2017, pp. 1–5 (2017)
Sun, T., Di, Z., Che, P., Liu, C., Wang, Y.: Leveraging crowdsourced GPS data for road extraction from aerial imagery. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 16–20 June 2019, pp. 7509–7518 (2019)
Ruan, S., et al.: Learning to generate maps from trajectories. In: Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA, 7–8 February 2020
Albert, A., Kaur, J., Gonzalez, M.C.: Using convolutional networks and satellite imagery to identify patterns in urban environments at a large scale. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada, 13–17 August 2017, pp. 1357–1366 (2017)
Rakhlin, A., Davydow, A., Nikolenko, S.I.: Land cover classification from satellite imagery with U-Net and Lovasz-Softmax loss. In: Proceedings of the Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA, 18–22 June 2018, pp. 262–266 (2018)
Cao, R., et al.: Integrating aerial and street view images for urban land use classification. Remote Sens. 10, 1553 (2018)
Kuo, T.S., Tseng, K.S., Yan, J.W., Liu, Y.C., Wang, Y.C.F.: Deep aggregation net for land cover classification. In: Proceedings of the Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake Cit, UT, USA, 18–22 June 2018, pp. 252–256 (2018)
Schmidhuber, J.: Unsupervised minimax: adversarial curiosity, generative adversarial networks, and predictability minimization. arXiv 2019, arXiv:cs/1906.04493
Albrecht, C.M., et al.: Change detection from remote sensing to guide OpenStreetMap labeling. ISPRS Int. J. Geo Inf. 9(7), 427 (2020)
Lebedev, M.A., Vizilter, Y.V., Vygolov, O.V., Knyaz, V.A., Rubis, A.Y.: Change detection in remote sensing images using conditional adversarial networks. International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences, 42(2) (2018)
Papadomanolaki, M., Verma, S., Vakalopoulou, M., Gupta, S., Karantzalos, K.: Detecting urban changes with recurrent neural networks from multitemporal Sentinel-2 data. In: IGARSS 2019–2019 IEEE International Geoscience and Remote Sensing Symposium, pp. 214–217. IEEE, July 2019
Acknowledgements
The authors wish to thank all contributors involved with the OpenStreetMap project. This research is funded by Technological University Dublin College of Arts and Tourism, SEED FUNDING INITIATIVE 2019–2020. The authors wish to acknowledge the Irish Centre for High-End Computing (ICHEC) for the provision of computational facilities and support. We also gratefully acknowledge Ordinance Servey Ireland for providing both raster and vector data for the experiments.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Niroshan, L., Carswell, J.D. (2022). Post-analysis of OSM-GAN Spatial Change Detection. In: Karimipour, F., Storandt, S. (eds) Web and Wireless Geographical Information Systems. W2GIS 2022. Lecture Notes in Computer Science, vol 13238. Springer, Cham. https://doi.org/10.1007/978-3-031-06245-2_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-06245-2_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-06244-5
Online ISBN: 978-3-031-06245-2
eBook Packages: Computer ScienceComputer Science (R0)