Abstract
The Indian subcontinent is a south geographic part of Asia continent which consists of India, Bangladesh, Pakistan, Sri Lanka, Bhutan, Nepal, and Maldives. Different rulers or the empire of different periods have built various buildings and structures in these territories like Taj Mahal (Mughal Period), Sixty Dome Mosque (Sultanate Period), etc. From archaeological perspectives, a computational approach is very essential for identifying the construction period of the old or ancient buildings. This paper represents the construction era or period identification approach for Indian subcontinent old heritage buildings by using deep learning. In this study, it has been focused on the constructional features of British (1858–1947), Sultanate (1206–1526), and Mughal (1526–1540, 1555–1857) periods’ old buildings. Four different feature detection methods (Canny Edge Detector, Hough Line Transform, Find Contours, and Harris Corner Detector) have been used for classifying three types of architectural features of old buildings, such as Minaret, Dome and Front. The different periods’ old buildings contain different characteristics of the above-mentioned three architectural features. Finally, a custom Deep Neural Network (DNN) has been developed to apply in Convolutional Neural Network (CNN) for identifying the construction era of above-mentioned old periods.
Access provided by Autonomous University of Puebla. Download conference paper PDF
Similar content being viewed by others
Keywords
- Computational archaeology
- Digital cultural heritage
- Computer vision
- Feature detection
- Deep neural network (DNN)
- Convolutional neural network (CNN)
- Deep learning
- Machine learning
1 Introduction
Building detection and feature detection are vital research areas in the study of computer vision. There are numerous old and ancient building sites in the Indian subcontinent region, such as Taj Mahal (Mughal era), Sixty Dome Mosque (Sultanate era), etc. Generally, the archaeologists can identify the construction period of old building by using its architectural characteristics or features. In this point of view, this research has established a computational technique for recognizing the construction period of old architectures by differentiating the building’s architecture.
In previous years, some researches have been published, where computer vision is used in archaeology sections [1, 3]. An artificial neural network based feature recognition technology is used to identify the features of the ancient structure [4]. A CNN method focuses on visualization for primitive Maya hieroglyph [5]. The deep learning method is being utilized for recognizing the ancient Roman coin [6]. China’s ancient warrior terracotta has been visualized by computer vision [7] and it is effective for 3D modeling [8]. Photogrammetric method has been enabled the image analysis of the Turkish ancient heritage site [9]. Moreover, some researches have been revealed where machine learning is also used in period identification [10, 11].
Furthermore, any technique for recognizing the building period of old architectural structures like the old building, mosque, and temple is not available. That’s why this research has committed a technique that helps the archaeologists for recognizing the construction period by detecting the old spectacular architecture.
For establishing the CNN, a deep learning model has been developed where four features detection methods are applied. These are Canny Edge Detector [12], Hough Line Transform [13], Find Contours [14], and Harris Corner Detector [15]. After utilizing these methods, three diverse architectural features have been classified which are Dome, Minaret, and Front because different old structures contain different forms of these three features. A deep feed-forward neural network [16] model has been developed where three features have been used for identifying the era. Moreover, this research has identified three ruler periods, such as the Mughal period (1526–1857), Sultanate period (1206–1526), and British period (1858–1947).
Recently a deep learning model has been expressed [17] for identifying the old era for ancient buildings. Here, only Canny Edge Detector method and two eras’ (Mughal and Sultanate) datasets have been used. The updated research has developed a more custom neural network where the remaining methods (Hough Line Transform, Find Contours, and Harris Corner Detector) has been utilized. Moreover, the British dataset is used here in addition to Mughal and Sultanate datasets.
The contributions of this manuscript are in three areas: (1) Identifying construction era based on Dome, Minaret and Front features of Mughal (1526–1857), Sultanate (1206–1526) and British (1858–1947) eras’ buildings; (2) Edge, Line, Contour and Corner elements are raised for identifying the different features (Dome, Minaret and Front) of different heritage buildings; (3) A Deep Neural Network (DNN) has been developed and applied in CNN where three features (Dome, Minaret and Front) of four different methods (Canny Edge Detector, Hough Line Transform, Find Contours, Harris Corner Detector) have been used for classifying old periods.
2 Era Identification Process
This research has illustrated a computational archaeological model that has described how a program can identify the construction era of an old building. At first, a photo was sent to Canny Edge Detector, Hough Line Transform, Find Contours, and Harris algorithm functions. These techniques have been used for collecting the features of Dome, Minaret, and Front from the old building image. The architecture and process of the era identification model have been illustrated in Fig. 1.
3 Experimental Methods
3.1 Canny Edge Detection
Edge recognition covers a diversity of mathematical processes that goals at identifying the points in an image. In this experiment, Canny edge detection method has been utilized for acknowledging the edges from a photo. At first, vertical direction (Gy) and horizontal direction (Gx) were filtered by finding the gradient intensity of an image. After applying the Canny algorithm, gradient was constantly perpendicular to edges and it was rounded to the angles for illustrating vertical, horizontal, and diagonal directions. The direction and edge gradient [18] for each pixel were found as follows:
3.2 Hough Line Transform
Hough line transform is a feature extraction technique. It was related to the line identification on the picture. In this technique, the parameters m, b mentioned [19] for Cartesian coordination and parameters r, θ for Polar coordinate system [20]. These coordination approaches were used for identifying the line of ancient buildings. In this research, a line had been represented as y where, y = mx + b or in parametric form, as r = x cos θ + y sin θ. Hence, the line equation for an image is as follows:
3.3 Find Contours
Contours can be narrated entirely as a curve or turn joining all the continuous points’ boundary and it is an adjuvant tool for shape or object detection. Image Moment technique has been used for finding the counters of an image. The spatial structure moment of an image was declared as mij where i and j are nested for loop order. This image moment had detected different features from the ancient buildings by matching different shapes. The image moment [21] was computed as:
3.4 Harris Corner Detection
Harris corner detection method extracts the corners and concludes the features of an image. It generally searches the corners in image intensity for a prolapsed of (u, v). In this method, there is a Window function that is Gaussian Window and gives weights to the image pixels down. In Eq. (5) [22], E is the distinction between the original and the moved window. Here, I parameter is the image intensity. The window’s translocation in the direction x is u and the direction y is v. Window w(x, y) is at (x, y) position. The I(x + u, y + v) portion is moved window’s intensity. Last portion I(x, y) is the original intensity. The window function w(x, y) is a Gaussian function.
3.5 Training Dataset and Classification
For recognizing the construction period or era three classifications had been created for Sultanate, Mughal, and British periods. After using the feature detection techniques, a Decision Tree [23] had been created based on output of above techniques. Decision tree creates classification in the form of a tree formation. It improves an “if-then” ruleset which is reciprocally exclusive. These rules are learned orderly using the training data one at a time. Table 1 showed the types of data classification of the training dataset. Here, the data were classified by three periods (Mughal era, Sultanate era, and British era). Every era contains three different features (Dome, Minaret, and Front) of four different methods (Canny Edge Detector, Hough Line Transform, Find Contours, and Harris Corner Detector).
4 Deep Neural Network (DNN) Model
In this experiment, a DNN approach has been developed. In the input layer of DNN, there are five nodes (x1, x2, x3, x4, and bias unit). The inputs of the input layer have been displayed in Table 2, Figs. 1 and 2. The mathematical structure [24, 25] of the node at neural network in this experiment has been illustrated in Fig. 2. Here, a is Activation, b is Bias and W is the ‘Weight’ of input layer. A bias unit allows changing the activation to the left or right, which is used for successful learning.
From Fig. 2, the equation for each activation node (a) is as follows:
Here, Index = i; Activation = a; Current Layer = L; Previous Layer = L − 1; Input node = x; Bias Unit = b. The computational algorithm of the developed DNN is represented as follows:
Layer, L = 2 (Hidden Layer 1):
Layer, L = 3 (Hidden Layer 2):
Layer, L = 4 (Hidden Layer 3):
Layer, L = 5 (Output Layer):
In Fig. 2, we have applied node to also denote the inputs to the network. The nodes labeled “+1” are called bias units corresponding to the intercept. We denoted ni, the number nodes (without bias unit) in neural network. Weight W (L−1)i denoted the parameter which connected with the link between i unit in layer L and this weight comes from previous layer L − 1. The bias units don’t have inputs and links going into them. The bias units always output the value +1. Here, we have denoted the activation a (L)i of unit i in layer L. For L = 1, we declared a (L)i = xi to denote the ith input. The parameters W, b defines the hypothesis h (x)w,b that outputs a real number.
5 Convolution Neural Network (CNN) Model
A CNN has been created which is based on the developed DNN model. Generally, the CNN consists of multiple hidden layers having convolution and pooling layers. Here, CNN has been developed on three convolution layers, two max-pooling layers, two fully connected layers, and a dropout (Fig. 3). After using the feature detection methods we had got two datasets: training set and test set. Then the neural network model provides the prediction result of the old period.
6 Results and Analysis
The outputs of the developed model consist of identifying the construction era, where a program provides a probable output by learning the ancient buildings’ features. This work has indicated how a computer program learns several old buildings’ features such as Dome, Minaret, and Front. For evaluating the performance of such systems, the data in the matrix has been used. The CNN model has been trained with the modified dataset and calculated the accuracy. Figure 4 has shown the composition of the CNN model, where the process successfully predicted the period from the picture of the ancient or old heritage building. Accuracy is also used as a statistical grade of the test calculations. The law for calculating accuracy is as follows:
where, TP = True Positive; FP = False Positive; TN = True Negative; FN = False Negative.
In this research, total tested 500 images data have been used. The Sultanate era contains 270 data, Mughal era 130, and British era 100 data. We get TP = 254, TN = 227, FP = 3, FN = 16. Following the above Eq. (17) for the raw data, 96.20% accuracy achieved from this research.
7 Conclusion
This study has represented a model that demonstrates how an intelligent program can identify the construction era from an ancient or old heritage building. This research is mainly focused on the construction period and features of the heritage building by using artificial neural network and feature detection techniques. This research achieved much better accuracy over the previous method by using three periods (Mughal, Sultanate, and British eras) and four feature detection methods (Canny Edge Detector, Hough Line Transform, Find Contours, and Harris Corner Detector).
Still there are some limitations to this study. There are further issues to be resolved. Furthermore, if the model is tested on the low pixel picture, it cannot determine the target result. This drawback would also lead this research to the future work to make the raised model more robust and more significant to recognize precise objects from the image. These issues will be looked forward to solve in the future experiment.
References
Tang, T., Chen, B., Hu, R.: Combined with DCT, SPIHT and ResNet to identify ancient building cracks from aerial remote sensing images. In: Liang, Q., et al. (eds.) Artificial Intelligence in China. Lecture Notes in Electrical Engineering, vol. 572, pp. 313–318. Springer, Singapore (2020)
Kabir, S.R., et al.: Performance analysis of different feature detection techniques for modern and old buildings. CEUR Workshop Proc. 2280, 120–127 (2018)
Barceló, J.A.: Computational Intelligence in Archaeology. Universidad Autonoma de Barcelona, Spain (2008)
Zou, Z., et al.: Feature recognition and detection for ancient architecture based on machine vision. In: Proceedings of SPIE 10602, Smart Structures and NDE for Industry 4.0, p. 1060209 (2018)
Can, G., et al.: How to tell ancient signs apart? Recognizing and visualizing Maya Glyphs with CNNs. ACM J. Comput. Cult. Herit. 11(4), Article no. 20 (2018)
Schlag, I., Arandjelovic, O.: Ancient Roman coin recognition in the wild using deep learning based recognition of artistically depicted face profiles. In: 2017 IEEE International Conference on Computer Vision Workshops, Venice, Italy, pp. 2898–2906 (2017)
Bevan, A., et al.: Computer vision, archaeological classification and China’s terracotta warriors. J. Archaeol. Sci. 49, 249–254 (2014)
Brutto, M.L., Meli, P.: Computer vision tools for 3D modeling in archaeology. Int. J. Herit. Digit. Era 1(1), 1–6 (2012)
Toz, G., Duran, Z.: Documentation and analysis of cultural heritage by photogrametric methods and GIS: a case study. In: XXth ISPRS Congress, Istanbul, Turkey, pp. 438–441 (2004)
Min, Y., et al.: Real time detection system for rail surface defects based on machine vision. EURASIP J. Image Vide. 2018, 3 (2018)
Zdravevski, E., et al.: Automatic machine-learning based identification of jogging periods from accelerometer measurements of adolescents under field conditions. PLoS ONE 12(9), e0184216, 1–28 (2017)
Ramnarayan, Saklani, N., Verma, V.: A review on edge detection technique “Canny edge detection”. Int. J. Comput. Appl. 178(10), 28–30 (2019)
Tatsubori, M., et al.: A probabilistic Hough transform for opportunistic crowd-sensing of moving traffic obstacles. In: 2018 SIAM International Conference on Data Mining, California, USA, pp. 217–215 (2018)
Soomro, S., Munir, A., Choi, K.N.: Hybrid two-stage active contour method with region and edge information for intensity inhomogeneous image segmentation. PLoS ONE 13(1), Article: e0191827 (2018)
Sun, Y., Ientilucci, E., Voisin, S.: Improvement of the Harris corner detector using an entropy-block-based strategy. In: SPIE 10644, Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery XXIV, 1064414, Florida, United States (2018)
Zheng, W., Wang, H.B., et al.: Multi-layer feed-forward neural network deep learning control with hybrid position and virtual-force algorithm for mobile robot obstacle avoidance. Int. J. Control Autom. Syst. 17(4), 1007–1018 (2019)
Hasan, M.S., et al.: Heritage building era detection using CNN. IOP Conf. Ser. Mater. Sci. Eng. 617(1), Article: 012016 (2019)
Mordvintsev, A., Revision, A.K.: Canny Edge Detection. OpenCV-Python Tutorials (2013)
Hough Line Transform: OpenCV (2017)
Xu, G., Zheng, A., Li, X., Su, J.: A method to calibrate a camera using perpendicularity of 2D lines in the target observations. Sci. Rep. 6, Article number: 34951 (2016)
Structural Analysis and Shape Descriptors: OpenCV (2014)
Nelli, F.: OpenCV & Python—Harris Corner Detection—A Method to Detect Corners in an Image. Meccanismo Complesso (2017)
Mesarić, J., Šebalj, D.: Decision trees for predicting the academic success of students. Croat. Oper. Res. Rev. 7(2), 367–388 (2016)
Higham, C.F., Higham, D.J.: Deep learning: an introduction for applied mathematicians. SIAM Rev. 61(4), 860–891 (2019)
Durstewitz, D., Koppe, G., Meyer-Lindenberg, A.: Deep neural networks in psychiatry. Mol. Psychiatry 24, 1583–1598 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Hasan, M.S. et al. (2021). Identification of Construction Era for Indian Subcontinent Ancient and Heritage Buildings by Using Deep Learning. In: Yang, XS., Sherratt, R.S., Dey, N., Joshi, A. (eds) Proceedings of Fifth International Congress on Information and Communication Technology. ICICT 2020. Advances in Intelligent Systems and Computing, vol 1183. Springer, Singapore. https://doi.org/10.1007/978-981-15-5856-6_64
Download citation
DOI: https://doi.org/10.1007/978-981-15-5856-6_64
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-5855-9
Online ISBN: 978-981-15-5856-6
eBook Packages: EngineeringEngineering (R0)