Abstract
Throughout the evolution of humanity, technologies have served as support for new evolutionary horizons. It is an unambiguous fact that technologies have positively influenced the masses, but they have also brought a remoteness with local cultures, often making them oblivious. Among the new technologies and forms of interaction, we have augmented reality and screen readers that allow the device to read the content. This paper presents AIMuseum. It aims to facilitate accessing and interacting with cultural environments for people with different abilities, combining the use of technologies with local museums, artworks, and exhibitions. The work was evaluated with 38 users, ranging from 16 to 41 years old, and five declared having one type of disability. They used the application and answered a questionnaire. The results showed a positive experience and improved the users’ interest in the artworks and their additional information.
Access provided by Autonomous University of Puebla. Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
Throughout the evolution of humanity, technologies have been used for new evolutionary horizons. From the typewriter to the most modern devices, humans search for something that meets their desires in terms of ergonomics, usability, automation, and utility.
The so-called augmented reality (AR) can be defined as an interface based on the overlap of digitally generated virtual information (dynamic images, sounds, etc.) with the user’s real environment, using modern devices such as smartphones. When this virtual information is brought into real space, using the user’s natural interactions, we have the presence of augmented reality.
It is important to consider people with different abilities. Inclusive applications help people with special needs to interact and use them. Accessibility is essential for allowing people to access and interact with multiple devices, enhancing independence and freedom.
This paper presents AIMuseum. It aims to facilitate accessing and interacting with cultural environments for people with different abilities, combining the use of technologies with local museums, artworks, and exhibitions.
2 Related Works
AR is nowadays a widely accepted technology, however not yet so widespread socially. It keeps a system with three characteristics: it combines the real with the virtual, brings interactivity in real-time, and adjusts virtual objects in the three-dimensional environment.
We can use QR codes to track printed markers, which will be captured by a camera, and subsequently mapped by software [8]. This type of application allows visitors in an exhibition to have eye contact with details not perceived in real life.
Neto et al. [7] propose a tourism guide for smartphones using AR and QR codes’ technologies. Tillon et al. [10] describe the results of experiments conducted during museum visits with a mobile guide using markerless augmented reality. Heikkinen [3] used a JSARToolKit library with APIgetUserMedia WebRTC to run augmented reality applications on the Web. Some other works applied augmented reality to museums’ exhibitions, such as [1, 9], and [4].
The use of immersive technology, like VR, has also been used for works in Museum Exhibitions, such as [2, 6], and [12]. Likewise, recent works present natural interaction [5] and experience model for mixed reality in the museum [11].
3 Enhancing Interaction and Accessibility
We developed AIMuseum, an application to facilitate accessing and interacting with cultural environments for people with different abilities, combining the use of technologies with local museums, artworks, and exhibitions.
We built the application on the Unity game engine using the C# programming language. We used Vuforia as API (Application Programming Interface) to create augmented reality. It was done by associating information received through the camera (from QR codes) with its predefined database, then projecting a two or three-dimensional image from this association.
QR Codes were selected as markers for the present work, because, as they use black and white, it becomes easier to identify the image even in low light conditions. We adapted their size and quality according to our preliminary tests.
To develop the Screen Reader accessibility feature, we used a UI Accessibility Plugin for Unity that makes it possible to read QR codes associated texts in different languages.
While developing the application, it was necessary to choose, catalog, and model some artworks. For the 3D modeling, we used Blender, which consists of an open-source program to make models, animations, textures. One 3D example, while being modeled, can be seen in Fig. 2.
We select the current artworks from local museums and free collections available. The works include paintings, objects, and photographs.
Objects were photographed at different angles to allow modeling. It is important to note that this is not a digital reconstruction of the works but manual modeling, seeking to reproduce faithfully. We do not intend to substitute the users’ experience with the real artworks, but to make them more accessible and easy to understand.
The application interface is intuitive and straightforward. It uses a responsive User Interface (UI) design, adaptable to the user’s device (smartphone or tablet). It also allows users to configure settings such as volume, font size, and language.
The application UI can be found in Fig. 3. In Fig. 3a the main screen. Pressing Next, the user will find the instructions page, Fig. 3b. Pressing on options, Fig. 3c, the settings will appear to personalize the app.
When the users configure the application and open it, they can scan a QR code automatically. It is just necessary to point the device to the QR code. We can check an example of scanning in Fig. 1.
The user can access extra information about the artwork, such as title, year of creation, description, the context of production, and copyright.
If the screen reader is on, the user can listen to all extra information. This inclusive feature is essential for people with disabilities, such as visually impaired or intellectually disabled people.
One example of a 3D model scanned by AIMuseum is available in Fig. 4. The QR code and the white background are real-life objects, while the sword (Rapier) and its label are virtual objects generated by the application.
4 User Evaluation
To validate the development, we tested the application at a local exhibition. We aim to promote the interaction of the community with artworks and making them easily accessible.
We evaluated the work for two afternoons. We invited participants using banners and social network invitations.
We had 38 participants in total, ranging from 16 to 41 years old, with a mean age of 28. The participants were 54% female and 46% male. Three participants declared to have low vision, and two participants reported to have reading difficulties.
They used the application freely for about 5 min and then answered a questionnaire. There were ten artworks available, and we can check some of the AIMuseum generated artworks used in the user evaluation in Fig. 5.
On a Likert scale of 1 to 5, being one very unsatisfied and five very satisfied, we obtained:
-
87% very satisfying and fast QR code scanning experience
-
92% very satisfying screen reader (text-to-speech) experience
-
89% very satisfying and intuitive interface experience
-
89% very satisfying usability experience
-
87% very satisfying overall experience
The results showed a positive experience and improved the users’ interest in the artworks and their additional information. The majority of participants found the interaction easy, and the public’s approach to the works was relaxed and straightforward, demonstrating that the use of new technologies became part of our current lives.
Some interesting comments: two participants that declared having low vision said they could understand the artwork in a novel way, and wanted to have this extended for other artworks. One participant that stated having reading difficulties commented that the screen reader helped to focus.
5 Conclusions and Future Work
Developing this research was a very productive experience. During the development, it was possible to be in contact with several artworks and regional artifacts. Also, the presentation of the application to the target audience exceeded expectations, making the community’s interaction with works of art and their feedback very significant.
Comparing AIMuseum with other similar works interacting with available artworks, the screen reader enhanced the experience for AR. Our work made artworks accessible for the population with low vision and reading difficulties. Also, it can be fun for children and older people.
Unfortunately, we did not have the number of participants with special needs that we wanted. So we could analyze all-new challenges that they could face. We are sure some adjusts have to be made in future work to include, for example, blind users and people with difficulty moving.
Therefore, we suggest a few more activities for future work:
-
Add new forms of reading and accessing markers
-
Add pictures and audios to the gallery
-
Share content in social networks
-
Test future versions with people having other types of disabilities.
References
Braga, I.F., Landau, L., Cunha, G.G.: Realidade aumentada em museus: as batalhas do museu nacional de belas artes. Virtual Reality Sci. Visual. J. 4(1), 35–55 (2011)
Haguenauer, C., Cunha, G.G., Cordeiro Filho, F., Araujo, M.C.M., Pastorino, L., de Freitas Lohmann, A.: Projeto museu virtual: Criação de ambientes virtuais com recursos e técnicas de realidade virtual. Virtual Reality Sci. Visual. J. 1(2), 1–13 (2008)
Heikkinen, I.: Writing augmented reality applications using JSARToolKit, February 2012. https://www.html5rocks.com/en/tutorials/webgl/jsartoolkit_webrtc/
Keil, J., et al.: A digital look at physical museum exhibits: designing personalized stories with handheld augmented reality in museums. In: Digital Heritage International Congress (DigitalHeritage), 2013. vol. 2, pp. 685–688. IEEE (2013)
Kyriakou, P., Hermon, S.: Can I touch this? Using natural interaction in a museum augmented reality system. Digit. Appl. Archaeol. Cultural Heritage 12, e00088 (2019)
Marques, M.A., Costa, T.K., Machado, L.S., Nettos, C.: Representação do sítio arqueológico da pedra de ingá com realidade virtual. In: Anais do Workshop de Realidade Virtual e Aumentada, Santos, Brasil (2009)
da Silva Neto, P.C., Nunes, C., dos Santos Nunes, E.P.: Integrando recursos de realidade aumentada e código de barras bidimensionais no desenvolvimento de um guia de turismo (2011)
da Silva, U.L., Braga, R.F., Scherer, D.: Uso de qr code e realidade aumentada como suporte a visitação de museu. In: Anais dos Workshops do Congresso Brasileiro de Informática na Educação, vol. 1 (2012)
da Silveira, A.L.M., pela UFRGS, I., Biazus, M.C.V.: A realidade aumentada aplicada a museologia (2010)
Tillon, A.B., Marchal, I., Houlier, P.: Mobile augmented reality in the museum: can a lace-like technology take you closer to works of art? (2011)
Trunfio, M., Campana, S.: A visitors’ experience model for mixed reality in the museum. Curr. Issues Tourism 23(9), 1053–1058 (2020)
Wojciechowski, R., Walczak, K., White, M., Cellary, W.: Building virtual and augmented reality museum exhibitions. In: Proceedings of the Ninth International Conference on 3D Web Technology, pp. 135–144. ACM (2004)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Guedes, L.S., Marques, L.A., Vitório, G. (2020). Enhancing Interaction and Accessibility in Museums and Exhibitions with Augmented Reality and Screen Readers. In: Miesenberger, K., Manduchi, R., Covarrubias Rodriguez, M., Peňáz, P. (eds) Computers Helping People with Special Needs. ICCHP 2020. Lecture Notes in Computer Science(), vol 12376. Springer, Cham. https://doi.org/10.1007/978-3-030-58796-3_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-58796-3_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58795-6
Online ISBN: 978-3-030-58796-3
eBook Packages: Computer ScienceComputer Science (R0)