Keywords

1 Introduction

Heritage interpretation is an educational activity that uses authentic materials, firsthand experience, and illustrative media to reveal meanings and relationships rather than merely communicating factual information. Visual or firsthand interactions help people connect to information better than theory [8]. AR technology has become a well-accepted technology among the scientific community and public, which combines real and virtual objects and mixes them into the real environment. In virtual heritage, this technology is used for improving the visitor experience of a cultural heritage site. Heritage interpretations can have many forms, material workshops, heritage walks, conjectural models, Virtual Reality (VR) experiences, and AR applications. In the current scenario in India, we see workshops and heritage walks on a broader scale. VR and AR experiences are scarce and often seen in museums. High-quality experiences are achieved with high-quality and precise documentation. Today, superior methods are used to perform measurements and digital documentation. Photogrammetry is a popular tool in the Architecture, Engineering & Construction (AEC) sector [3].

In the field of cultural heritage, virtual modelling and 3D reconstruction are standard tools for recreating, analysing, and visualising large objects (for example, archaeological sites and architectural buildings) as well as small objects (for example, sculptures, ceramic tiles, silver, marble, and wooden artefacts) [6]. At this time, various technologies can be used to create accurate photo-realistic 3D models, and Photogrammetry is extensively used as it is effortless to use. Whereas, precise modelling of existing 3D data is typically complex and costly since “reality” is complicated in and of itself; the more complex the thing, the more complex the model. Photogrammetry models are frequently created to visualise the historical state of monuments and authentic real-life models.

1.1 Photogrammetry

Photogrammetry and Augmented Reality are tools that have become a keen interest of professionals in all sectors. Photogrammetry procures measurements of size, shape, position, and texture from high-resolution photographs. Photographs are captured from all angles of the structure with a high-resolution camera. These photos help create a 3D model with real-time texture. In its most basic form, a pair of overlapping images are utilised to construct a three-dimensional model, which may then be quantified using proper instruments [9]. These proportions were traditionally depicted on maps and plans as elevations, facades, and/or contours. Photogrammetry is the science of using photos to derive measurements of an object's size, shape, location, and texture. In its most basic form, a pair of overlapping photos is utilised to generate a three-dimensional model, which may then be quantified using suitable equipment [2]. Photogrammetry has a long history of use as a tool to aid in the documentation of cultural assets and is well-established as a measuring science. The data sources begin with photogrammetric data, which includes terrestrial, aerial, and satellite pictures; second, spatial (GIS) data, which includes maps, vector files, and point event locations, as well as lines ( counter lines and roads) [2]. Scientific advancements have made the processes far more adaptable in their use, opening up new possibilities for portraying structures as diverse as aboriginal rock painted shelters, historically significant buildings, and Ruins.

1.2 Augmented Reality

AR is widely being used in many applications such as education, entertainment, virtual heritage, simulation and games. In virtual heritage, AR is used to enhance the overall experience of the visitor of a cultural heritage site. Furthermore, the interactive, realistic and complex AR system can enhance, motivate and stimulate students’ understanding of certain events, especially for the traditional notion of instructional learning that has proven inappropriate or difficult [7]. Museums have been at the forefront of experimenting with how these new technologies may be utilised as educational aids, touting these breakthroughs as part of their democratisation goal. As a result, we may observe a shift from traditional audio guides to PDAs (Personal Digital Assistants) and eventually to mobile applications. The increasing use of mobile devices for various purposes has increased in the application-development sector. People can use this type of software to acquire critical information and communicate with others creatively [5]. Multiple virtualisations have influenced AR to be utilised quickly and pleasantly because of improved device capabilities, 3D sensor equipment, and graphics technologies, allowing it to reach a broader market [1]. When it comes to using this tool in educational settings, recent research has shown that AR has helped students better understand reality, examine elements from various perspectives, and construct scenarios that promote simulation or information contextualisation, to mention a few advantages [4].

In the field of heritage, certain qualities of AR can be beneficial, particularly in terms of acquisition, management, and distribution. There are three aspects of AR experience. First, due to the current scenario's lack of modelling, it aids in the reduction of time and ultimate expenses associated with acquisition, modelling, and management. Second, it can create hybrid settings (real and virtual), combining past (non-existing portion, virtually modelled) and present (actual part, not modelled) scenarios to increase comprehension of heritage. Third, AR improves user immersion over VR systems since users can move around, see objects in their actual size, and explore them more naturally, allowing applications to be developed in real-time, on-site.

1.3 Historical Background

Kashi Vishweshwar temple, Kalabgoor, Telangana, is a masterpiece constructed solely out of black granite. The whole site is spread on an approximate 10,000 sq. ft. area. It is believed that temple was built by the Kakatiya dynasty of Telangana and was built in the eleventh century CE. Kashi Vishweshwar temple's architecture is similar to Warangal's Thousand Pillars Temple in Telangana state. The sculptures on these pillars are excellent examples of sculptural mastery, and Architectural Elements are also Exquisite in aesthetic features.

2 Methodology

In the AR application for heritage interpretation, augmentation was applied to a part of Kashi Vishweshwar Temple. The structure in focus was the Nandi Mandapa of Kashi Vishweshwar Temple. The Mandapa was chosen for its excellent condition and details, which covered up to 88. A high-resolution precision camera, Nikon D5600, was used to click photographs covering the maximum surface area of the Mandapa. Shadows in photos hamper the capturing of proper texture in pictures. A cloudy day was preferred to click the photographs to avoid significant shadows. The desired weather also provided precise detail of textures. The whole process was completed within one day. Figure 1 describes a sample of photos captured for this research. Camera properties for these photos are as follows:

Fig. 1
A photograph of Nandi Mandapa of Kashi Vishweshwar Temple.

Nandi Mandapa of Kashi Vishweshwar Temple, Kalabgoor

Full size image

Camera Model: Nikon D5600, Exposure Time: 1/400 s, ISO speed: ISO-200, Focal Length: 28 mm, Max aperture: 4.1

Reality Capture software was used to rebuild the 3D model using the raw data acquired realistically as shown in Figs. 2 and 3. This software was used as it provides additional tools to enhance the texture. The software aligns the photos that are obtained. In the event of any discrepancies, the images were aligned manually using a reference point.

Fig. 2
An image depicts the 3 dimensional sample photo prepared from the raw data obtained from an actual photograph with the reality capture software.

Raw data (Textured) obtained after uploading to reality capture

Full size image
Fig. 3
An image depicts a grayscale three dimensional digital image of Nandi Mandapa prepared from the raw data obtained from the original picture.

Raw data (Mesh Model) obtained after uploading to reality capture

Full size image

The final product is available in two formats: basic and dense model. The dense model mesh comprises 72.5 million triangles, whereas the simple model mesh is made up of 3 million triangles, as shown in Fig. 4. The simple model was chosen because of its smaller file size and ease of use. The high-resolution texturing on the preview model gave it a more realistic appearance. After unwrapping the model, 167 textures with a resolution of 4096 × 4096 pixels were obtained, as shown in Fig. 5.

Fig. 4
A set of three images depict the screenshots of the adjustments made for the mesh, unwrap tool, and the color and texture of the real picture captured.

Settings used in Reality Capture for mesh modelling, unwrapping and texturing the model

Full size image
Fig. 5
An image depicts the texture model of the photograph with a resolution of 4096 multiplied by 4096 pixels.

167 Texture obtained from the data of photographs

Full size image

MeshLab software was used to access the file, which was exported in wavefront object format. Unwanted surfaces were modified in the mesh lab to improve the model.

The final polished 3D model was then imported into the Unity Augmented Foundation Android Platform. While importing the 3D model into Unity, the Pixels of the material image are enhanced and changed to 16,384 × 16,384 pixels, as shown in Fig. 6a. Also, the unlit texture is applied as material, as shown in Fig. 6b, to retain the same exposure value. An application on the android platform was created for accessing the final product. In Unity, both the possibilities of Floor based tracker and Image-based tracker were explored. In-Floor based tracking, this app detects the Floor and places the model. This model is provided with interactive features such as Pinch to scale, Drag to translate and Drag to Rotate on Axis. And in the Image-based tracker Plan of Nandi mandapa was drawn and was set as an Image tracker to show the AR model in Unity.

Fig. 6
A set of two screenshots depict a texture upgrade of the photograph to 16384 multiplied by 16384 pixels. b, unlit texture application to retain the exposure value.

Texture enhancement in unity

Full size image

3 Results and Analysis

Tests were conducted for augmented reality on the floor-based tracker and Image-based tracker. The AR model was provided with interactive accessibilities like Pinch to scale, drag translate and drag-rotate on-axis. The application developed was for 32-bit and 64-bit supported Android phones.

Due to the high-resolution texture wrapped around the model, the object appeared to be more realistic. In Fig. 7a, the 3D virtual model reconstructed from Photogrammetry is placed inside the natural environment, and in Fig. 7b Realistic appearance of the 3D model can be observed along with the original photograph in the background. Also, in Fig. 7c 3D virtual model is placed alongside the existing Nandi Mandapa.

Fig. 7
A set of three photographs depict three dimensional views for a, a virtual model of mandapa in a natural environment and b, a real model. c, a virtual model in view of the existing Nandi Mandapa.

AR virtual model placed in real-world

Full size image

This research used Image in Fig. 8 as Image Tracker for the Augmented reality app. In this research, the possibility of distortion of the Image was checked. The hand-drawn plan of Nandi mandapa, as shown in Fig. 9, was tested on the app. Proportions were maintained while drawing this plan, and as a result, it was found that the Augmented reality app is able to detect Fig. 9 as an image tracker. However, input was given in Fig. 8.

Fig. 8
A computer generated schematic of the top view of the Nandi mandapa. The platform and roof are represented as two large rectangles and the pillars are represented by squares.

Image-based tracking with the plan of the Nandi Mandapa

Full size image
Fig. 9
A hand drawn schematic of the top view of Nandi mandapa. The platform and roof are represented as two large rectangles and the eight small squares depict the pillars.

Image-based tracking with the hand-drawn plan of the Nandi Mandapa was found successful with the generated application

Full size image

4 Conclusion

In this study, it is observed that augmented reality can be used as a powerful tool in historical interpretation, and the end-user experience can be enhanced and can be more immersive. In addition, the possibility of making AR more interactive gives users the freedom to have better interaction with the model and improve the heritage interpretation. Photogrammetry based AR models of heritage structures can be explored by the user from micro-scale to macro scale. Photogrammetry gives the hyper-realistic appearance of a 3D model, which may also be used in the depiction of a conjecture of dilapidated sites or ruins. Also, another aspect of this study in which Image-based tracking was explored it is found that images with slight distortion but with the same proportions can be detected as image markers. This can help people to draw their own image tracker markers anywhere and get access to the model. These markers can be set as any iconography or pattern associated with the heritage structure so people can have better knowledge about it and can relate to it.