Keywords

1 Introduction

The transition from an information society to a communication one that has taken place in recent years has led to large-scale changes in many areas of human activity, including culture. The functions of various cultural and educational institutions have changed, new formats of their interaction with the audience have appeared, brand-new platforms on which the activities of a museum, gallery, or festival can take place have emerged [1]. Communication processes related to the transfer of knowledge and information are being carried out today not only in educational institutions but also in libraries, theatres, archives, museums, and art spaces, as well as on their virtual “equivalents” such as web sites, social media pages, and specialized mobile applications. The explosive growth of information in the field of culture forces viewers to control and filter their news feeds more thoroughly and find relevant data. Professionals - curators, art critics, lecturers, artists, etc. – are now to develop effective communication strategies and new tools to retrieve and structure relevant information and offer the best means to report it to consumers of the cultural product.

In this study, we set out to analyze the changes taking place in art museums, spurred on the one hand by the expectations of the audience, and on the other hand, new opportunities for the development of digital technologies open up. The focus of our attention is precisely on the museum (and in some cases – galleries) of fine arts, since this is the most visited type of museum and it is here that digital transformation is proceeding at an accelerated pace, since the cultural and aesthetic demands of the audience are changing significantly due to general trends in the field of communication. Our research is based on both systematic research in the field of museology and the described experience of art museum employees directly involved in the process of digital transformation, we will argue that today there is a large-scale transformation of the idea of the museum itself. Previously, the emphasis was placed on storing and displaying exhibits, and today, the communicative aspect becomes more important [2]. One of the significant cases for our analysis will be the renewal of the Pushkin State Museum of fine arts, which is one of the critical art museums in Russia, combining ancient and modern art in its collection, as well as focused on the use of the latest technologies and following the trends adopted in the international art and Museum community. The museum is becoming a more open, flexible and interactive space, aimed not only at giving viewers access to certain items and information about them but also to captivate the viewer and form an absolute “gamut of emotions,” which includes both the building itself and its virtual continuation.

In this article, we will consider examples of the use of advanced information technologies for the formation of a modern Museum exhibition space within the framework of the multisensory environment concept [3,4,5,6,7]. The use of virtual, augmented reality, artificial intelligence, Internet of things technologies in the formation of the museum space gives rise to a lot of interdisciplinary scientific issues that are at the intersection of design, art, museum businesses, and information technology. Thus, our research is empirical, focusing on the relationship between technological solutions and the humanitarian tasks of a modern art museum, focused on the storage of antiquities and the non-exhibition of modern art. Our analysis necessarily involves a review of the most effective digital technologies involved in transforming the traditional museum space into a multisensory, inclusive environment that is open to intensive interaction with the viewer and has a “media expansion” in the global space network. Besides, this research presupposes a systematic analysis of how digital transformation is related to the fundamental research and educational tasks of the museum, which in the XXI century require rethinking due to changes in the communication society and the place of culture in it, as well as the updated continuously requests of viewers focused on multimedia consumption of information [8].

Working on present research we are considering several key concepts that go as follows:

  1. 1.

    Digitalisation of museum oeuvre and archive – by the means of the latest technologies, museum staff apply new methods of collecting, storing and organizing information about works of art. This work is primarily related to the aims of preservation and study of cultural heritage.

  2. 2.

    Digital transformation of a museum – systematic reformation in the structure and functions of the museum under the influence of digital technologies. This process significantly affects the communication with the audience, new ways of presenting objects of the museum collection are being developed.

  3. 3.

    Big data management – a museum is not only accumulating information about the preserved objects but also is constantly monitoring both the behavior of visitors and the requests of an ever expanding audience. With the help of digital technologies, this data gets processed and later it forms the basis for the development of more effective curatorial strategies.

communicative interactions and educational/scientific activities. That is, interaction with exhibits and interaction with information are two different things

  1. 4.

    Communication society as a term reveals the features of contemporary globalized community where digital interaction and communication is structuring anew all types of social activities. Unlike to the previous period named “information society” where the usage, creation and distribution of information became a socially and economically significant activity, current communication society requires not passive recipients but participating users, digitally active citizens of a globalized world.

2 Contemporary Museums Facing Technological Challenges

A new cultural product has a hybrid character and combines traditional artistic mediums and the latest information and communication technologies [9]. Modern museums and interactive exhibitions suggest this trend most clearly and visibly. Museums’ digital transformation has experienced steady progress over the last decade. These processes are now greatly intensified by improving the software and, secondly, enhancing audience requests to the technical equipment of the museum the virtual extension. Thirdly, due to the “line shock” that emerged amid the spread of a new coronavirus infection and the associated stringent security measures, suddenly forcing the museum to “virtual serve” to maintain and strengthen the connection with the audience.

In this regard, the problem of adaptation of museum mechanisms for storing and presenting art to the audience of the communication society is of particular relevance today. Therefore, the museum is increasingly involved in digitization using new technologies such as mobile applications, virtual and augmented reality, artificial intelligence, the Internet of things, and data design, which we consider the critical technologies used in a new “expanded museum.”

Museum’s functions are now way beyond the traditional storage of works of art, their study, and display. A museum today, as an art institution, offers a viewer new formats of communication and knowledge acquisition. Museum’s scientific networks are developing, their activities are highlighted in social media networks (short sci-fi posts about significant artifacts, exhibitions, and events appear daily on the museum’s pages, lectures and workshops are broadcast online reaching a growing audience). Moreover, large-scale digital platforms are created to present digitized materials on the Internet, for example, the “Artefact” app, allowing the viewer to gather information in a virtual environment and build their collection. The concept of an “imaginary museum” described by Andre Malraux has now become a reality for every user. In addition, we offer specially designed applications created per the needs of each independent museum visitor. It primarily includes guides, which are available in the form of individual players with headphones, as well as mobile applications such as “Izitravel”. Specially printed on 3D-copies of the works of art, virtual exhibitions, and tours of this digital space enable physically challenged visitors to feel and explore them.

In recent years, there has been increasing attention to digital museum transformation and the use of technology in exhibitions [9, 10]. In this article, we continue to study the problem of using innovative technologies, including information and communication technologies, as a useful tool for artistic and socio-cultural practices in the context of the development of cultural industries that require understanding the possibilities of their practical use in a communication society.

A modern art museum is supposed to be flexible and democratic enough to create a fascinating, exciting atmosphere for the visitor to interact with [11]. The immersive component comes to the foreground, since the modern viewer is no longer limited to the role of a passive recipient of a narrative, but strives for interactive communication, for understanding art through some interaction with it. This trend is manifested at a basic level in the way a museum visitor “curates” their pages in “Pinterest” and posts in “Instagram” mini-reports on the visit of a particular exhibition. However, these processes can be actively involved in the initiative of the exhibition site itself. For example, in the course of Jean-Hubert Martin’s project “Carambolage” which was a large-scale visual study where works of art and material culture related to entirely different chronotypes collided. Besides, they happened to have unexpected similarities and therefore opened up new research prospects and consequently invited the viewers to participate in this process by comparing virtual “doubles” of the works presented at the exhibition and thus continuing the so-called “game” in discovering unexpected connections between different cultural layers. Taking a closer look at this case, it is a kind of billiard stroke in which the cue ball (the hit ball) makes a consecutive collision with two other shots the player is aiming at, accordingly. Jean-Hubert Martin acted as a billiard player, choosing a particular work of art and then “hitting” it with two other similar ones, but not historically related art objects. The exhibition space was designed in such a way that the viewer could see these unexpected coincidences due to the placement of art objects in the same showcase, which is a traditional, chronologically arranged exhibition that could not possibly be together. The show is supplemented with the directory, including the “threesome.” However, even more, notable was the website of “Carambolage” and the eponymous mobile app, where not only curatorial idea is revealed to the audience in every detail, but also a continued dialogue is established: a visitor can find and compare art objects applying Martin’s principle and may adopt this methodology to their studies.

Taking into consideration a steady interest in the format of visual research in today’s humanitarian scientific field, we can assume that the web version of “carom” may end up even more resonant than the original exhibition. It is important to note that this project was not the first experiment of its kind since many art historians who specialized in iconology, structural analysis, and semiotics developed art research projects based on formal criteria, certain visual parallels, and “rhymes.” The most famous of these ideas are described in the works of Andre Malraux [12], Abi Warburg [13], Erwin Panofsky [14], Ernst Gombrich [15] and other contemporary researchers [16, 17]. Jean-Hubert Martin’s curatorial projects are an example of the practical implementation of these concepts in the museum halls. However, structural analysis and the creation of collections based on formal features have reached a new level thanks to artificial intelligence; thus, the Google Art Project platform is currently opening more and more sections devoted to visual research. For example, an experiment called “X Degrees of Separation” comes down to the following: the user selects two works of art as two poles, and the machine based on visual analysis selects a “palette” of similar art objects located between these two specified points. The pool of such experiments is continuously expanding. The viewer working with the Google Art Project is no longer just a consumer of a cultural product, but a user-researcher who actively studies the phenomena and contexts of art history, accordingly. The principles of curatorship are increasingly integrated into the independent work of each visitor’s perception of the cultural site. In 2020, Jean-Hubert Martin is planning to hold a project at the Pushkin State Museum of Fine Arts. Structurally reminding of “carom,” under the working title, “The Ancients stole all our ideas,” it is very likely that the interactive component will be core here as well.

Now The Pushkin Museum pays excellent attention to working with the audience. Head of IT department, Vladimir Opredelenov, in his speech at the eNano site noted the statistics of mentioning trends in the professional literature and highlighted such technologies as virtual reality, artificial intelligence, big data processing, virtual assistants, blockchain and machine learning. Further, he revealed the results of a survey of Muscovites about what technologies they would like to see integrated into the museum space. Virtual and augmented reality, virtual assistants, mobile applications, 3D graphics, and modeling topped the list. According to the results of this survey, a specific agenda was formed including the task of integrating information about the museum and its exhibits into the systems of virtual assistants (such as “Alice” created by Yandex), which will serve as mediators for users in the future [18].

Thus, we argue that a museum armed with new technologies should not only present works of art and provide specialists with access to the maximum possible amount of information about stored objects, but also include these parts in a more abundant flow of information, make them visible to the audience, to create conditions for a lasting dialogue [19, 20]. We also state that this process involves not only the technical development of the museum but also the development of a new approach without telling the history of art.

It should be noted that the Pushkin Museum adapted to the crisis that arose in connection with the spread of new coronavirus infection in 2020. As soon as the quarantine measures were introduced in Russia, meaning that the museum was closed entirely to visitors, not only did it keep research and educational activities, but also increased its intensity. Among the most effective measures taken by the museum are the following:

  1. 1.

    virtual tours on the Zoom platform, which involved the recruitment of a limited number of spectators, who were invited by the guide to evaluate a virtual copy of the Museum halls, accompanied by an interactive video with stories about the exhibits and a discussion;

  2. 2.

    open and free lectures on art: both Museum employees and invited specialists were involved in this project. The broadcasts were conducted in real-time and actively promoted on social networks;

  3. 3.

    «Ask the curator» project, which involves a series of meetings with the museum’s curators, narrow-profile specialists, to whom everyone could ask questions about the most valuable exhibits;

  4. 4.

    Olga Viktorovna Shishko curated the project “100 ways to live a minute”, in which artists, curators, and researchers of modern art recorded short analytical videos about the category of time in modern culture [21]. In parallel, a large-scale media program unfolded, during which several dozen broadcasts of works of video art and experimental cinema took place.

Due to that they have been keeping constant track of their audience’s demand and expectations Pushkin Museum specialists quickly identified the growing call for educative and aesthetically entertaining content: before the “Lockdown” youtube channel of Pushkin Museum introduced 5–7 videos per month and during the spring-summer of 2020 there has been appearing 40–50 videos per month. Furthermore, on the Facebook page of the museum reveals lectures, webinars and short educative videos several times a day.

Analyzing this case, we could outline a certain trend: digitalization of museum archives and creation of online content which have been gradually performed throughout the last 15 years built up the base for a digital transformation of Pushkin Museum. Having faced the “Lockdown” art museum immediately switched all its activities to online formats – and introduces some new ones. The above-mentioned project “100 ways to live a minute” united video art and media art pieces from various collections around the globe, virtually exhibiting oeuvre that would be almost impossible to see together in any international Biennale of art fair. The team of Olga Shishko thus presented new media art – by the means of new media (and we could point out that Russian was initially not too familiar with this type of pieces). In this case the “Lockdown” turned out to be a resourceful moment. So, using creatively the newer technologies Pushkin Museum has been “holding the line” on both scientific and educative interactions with the committed viewers and reaching out for a wider audience.

Similar measures were taken in other museums and centers of contemporary art in Russia, which corresponds to international trends [22]. Museums such as the Metropolitan Museum of art, the Louvre, the Prado, the Tate, and other vital sites have strengthened their media presence during the quarantine, proving the feasibility and effectiveness of digital transformation of cultural sites.

In the course of the development of the information society, cultural and educational institutions are responsible for a vast amount of tasks. Modern people are constantly absorbing information. It is no coincidence that today the generation of «sciensters» is viral, young people are passionate about scientific knowledge, book markets marked an unprecedented growth in sales of popular science books. More and more channels devoted to science appear on such platforms as “Youtube” and other social media networks, but with new and useful information, there is still much unnecessary random material. The current educational institutions, libraries, theaters, archives, and galleries have unique data and must present it so that the viewer-user can get valuable knowledge. The concept of “curator” in this area goes to an entirely new level in the processing of big data and their correct presentation. Among such institutions is a museum. It is necessary to continually analyze changes in the cultural field to remain relevant to the consumer, note what the museum was like, how it transforms, what trends are most suitable for its development, and how its visitor has changed.

In our world, non-traditional museums are of increasing interest to visitors. In a rapidly changing society, museums have to adapt more quickly to a new system of understanding the world. The preservation of all the material available in a traditional museum is not exactly favorable for attracting new visitors and making them want to return. In such a museum, almost nothing changes; all the items on display are in their places. However, the traditional museum has its positive aspects. In such museums, the structure is clear and understandable to the viewer, and some visitors with conservative views have a desire to return to the museum with a permanent exhibition. It is convenient for a long and detailed study of the material, natural movement through the halls among famous expositions, and employees. One must never forget that digitizing works of art and placing information about them in databases is an essential tool for inter-museum communication and a means of establishing systemic international research projects [23].

3 AR, VR, and IoT in Exhibition Space

Modern society does not exclude mixed museums with their traditions and customs, but at the same time, it is trying to “upgrade” itself. These can be various temporary exhibitions or partial changes, as well as stationary expositions, additional seminars, training programs, and further work with the visitor.

The multimedia approach to the dissemination, communication, and exploitation of cultural heritage is a recognized trend worldwide. Several studies [24,25,26,27,28] demonstrate that the use of new and combined media improves the perception of culture. The advantage lies in the number of people with access to information and the quality of data dissemination. In this regard, it uses augmented, virtual, and mixed reality technologies for various purposes, including education, improving exhibitions, research, reconstruction, and virtual museums. These technologies provide a user-oriented view and make cultural heritage available digitally, especially when physical access is limited. Besides, museums offer technological and digital opportunities to improve the experience of visiting.

The convergence of digital technologies such as the Internet of things, virtual and augmented reality, artificial intelligence, big data, and machine learning allows us to entirely modify the approach to the organization of the exhibition space, based on data from social networks of visitors to the exhibition space, in which users describe, evaluate and distribute information about works of art or the exhibition itself. Information from social networks can be extracted to make recommendations for museum tours, and such judgments can be improved by combining and applying data mining from the Internet of things sensors installed in museums.

For example, the article [29] offers a generalized approach to developing recommendations for visiting museums, including the architecture of wireless sensors for local positioning of visitors to the exhibition space, including some technologies based on semantic analysis, data mining, and machine learning. This approach combines different data sources to create and recommend indoor and outdoor visualized routes for museums using augmented reality. The user route is based on opinions and ratings from social networks, semantic classification of exhibition spaces, and cultural events, as well as data received from sensors located in the exhibition space. As a result, the user receives a unique individual tour using augmented reality technologies, which contains a set of recommendations on how to visit a specific set of exhibition spaces and improve the immersiveness of the visit.

Free audio guides are currently used in museums to improve the experience of visitors. Over the past few years, some augmented reality (AR) solutions have been researched in this context, allowing for smartphones acting as additional visual and audio incentives. The unique conditions of the Casa Batlló Museum in Barcelona, where artificial markers cannot be used, and the exhibition space is a small crowded room, forced the developers to use a hybrid approach of applying augmented reality to the existing architectural framework [30] using indirect augmented reality. The issues of creating immersive museum guides utilizing the Internet of things and augmented reality technologies are discussed separately. In the article [31], the authors propose a new type of audio guide for museums, consisting of a headset equipped with a camera that takes pictures of exhibits and a computer vision device. The developed electronics can recognize works of art using functions from crucial points of accelerated segment testing and the random forest classifier and can be used throughout the day without the need to recharge batteries. The results show that the system performs better than its counterparts due to the ease of use and user preferences of the proposed method compared to traditional audio guides.

In the field of cultural heritage, wearable augmented reality devices (for example, Microsoft Hololens) are also widely used. The HoloMuse app [32] allows users to actively interact with archaeological artifacts from the museum’s collection in ways that would otherwise be impossible. Such an application can be used not only in the exhibition space but also in classrooms since the spatial augmented reality technology does not impose severe restrictions on the physical area around the user. The user is provided with a wide range of interactive elements for interacting with the virtual environment. It can select, rotate, zoom, and change the hologram of the original exhibit using gestures. Similar technology for interactive holographic images was used at the Royal Ontario Museum in Toronto using the TombSeer app at the Tomb of Kitines exhibition [33].

New exhibition space undergoes digital transformation not only on the side of the visitor but also on the side of the exhibits themselves—these changes h influenced by additive technologies, programmable microelectronics, and multimedia. The need to increase immersiveness, in this case, is bi-directional, and the existing set of technologies allows us to talk about creating a new type of exhibition exhibits - cyber-physical ones. The problem of creating such artifacts is their exceptional level of immersiveness, interactivity, and tangibility. The article [34] describes the methodological basis and tools for creating such cyber-physical exhibits on the example of the Bonobo material skull in the Darwin State Museum (Moscow) and the titanophone material skull in the Perm Museum of Antiquities (Perm). Another example of applying this methodology is the exhibition of an extinct synapsid species Dimetrodon grandis [35].

An essential factor in this context is how to organize the exhibition space and the location of various exhibits using modern digital technologies. The article [36] analyzes technological innovations to create a “smart space” in which items can communicate with visitors and with each other to generate a rich, personalized, and stimulating experience. The authors propose an architecture called “history” designed to provide the appropriate infrastructure to be used in museums and exhibition centers to support intelligent exhibits.

A reasonably important area of application of the discussed technologies is in the safety and security of exhibits and cultural heritage. The preservation of cultural heritage artifacts remains of paramount importance. Among the technological possibilities, we can highlight the regulation of microclimate parameters, such as humidity, temperature, and brightness. In connection with the digital transformation of the museum space, its systems face the problem of minimizing human interaction. The article [37] proposes a CHPC system for automatically regulating microclimate parameters and assisting employees in choosing the right exhibition halls for exhibits depending on the materials of items for reasons of different store conditions, based on the Internet of things (IoT) and artificial intelligence (AI), in particular on the Semantic Web technologies. However, exhibition space and exhibits are vulnerable not only to poor environmental conditions but also to human vandalism. It is the responsibility of humanity to preserve the content of museums. The article [38] describes a specialized system that signals suspected or unintentional attempts at vandalism and allows remote control of the environment through authorized devices with Internet access. A key distinctive aspect of the proposed system is the use of always switched on and energy-intensive sensors for comprehensive and accurate monitoring, while power provided by collecting radio is frequency (RF) energy freely available in the museum. It contrasts with the technologies proposed in the literature that use RF energy harvesting to power simple IoT devices. The authors use antenna arrays that collect radio frequency energy and convert it into electrical energy to extend the life of sensor nodes. Another essential feature of the proposed system is the use of deep learning to identify daily trends in the collected environmental data. In this way, the museum environment is further optimized, and the system becomes more resistant to failures in the detected data. Data analysis and machine learning methods are also used in the external burial grounds of the Yang Mausoleum of the Han Dynasty in China [39]. The data collection system installed in the pavilion collected more than 7,000,000 heterogeneous data. Traditional algorithms did not allow accurate prediction of various situations inside the building, so methods of training the neural network with feedback were implied. The use of machine learning for data analysis and modeling is an effective forecasting algorithm for the preventive protection of exposition space and exhibits.

In addition, one can pay attention to the introduction of current AR and VR technologies in installations, that is, works that occupy the exhibition space almost entirely.

As for the study of virtual reality as an aesthetic phenomenon, the book by Oliver Grau, “Virtual art: from illusion to immersion” [40] plays an important role. Where the author shows that, since the Renaissance, artists have sought to create an immersive image that is, drawing the viewer, using the latest visual methods: linear perspective in painting, then panoramic photographs, cinema, television, and finally a virtual reality - the effect of immersion is becoming stronger. Grau concludes that, since immersion is the opposite of a distant view, it takes the viewer away from analytical work and offers a rather vivid bodily and emotional experience. However, this process is not just an attraction; it allows a person to have a new experience for them in its integrity, to use all the senses, and thus transform their lives.

Virtual space is a computer model of an environment whose physical properties are approximated in reality allowing the user to interact with an artificially created world and objects in it. The effect of full immersion in the program is achieved by influencing different channels of perception: now, there are high-quality versions of VR glasses and helmets that provide image and sound. Besides, VR gloves that provide tactile signals are being developed. Moreover, there is a project of VR costumes (VPL Dataset, 3rd Space Vest), with which the program could affect the entire body of the user, but now, it is clear that the critical channel of perception in virtual reality is vision. In any case, this model is currently available on the market.

What is virtual reality? On the one hand, we see the apotheosis of an opto-oriented culture, and on the other hand, being in virtual reality involves bodily experience and motor response. It means that objects in VR are seen by us as three-dimensional, having borders and weight, each of them separately is like a sculpture (it is no coincidence that one of the first commercially successful programs for VR was “Tilt Brush,” a graphic editor where you need to draw three-dimensional strips). If VR is a continuation of cinema and computer games in its visual component, then the plastic principle inherent in it is related, somewhat, to minimalist sculpture and installation. The key to aesthetic impact was the viewer’s sense of the materiality of the art object and in connection with it their physical presence. To explore the potential of VR technologies available to the curator of the art institute, director Alejandro Gonzalez Iñárritu creates his now-famous short film “Flesh and sand” (“Carne y Arena,” 2017). This virtual installation is dedicated to the lives of migrants crossing the border between Mexico and the United States. The director has been working on this project for more than five years. As part of the project, he has talked to many people who have experienced this transition. Unfortunately, many do not manage to survive it. Alejandro gathered their experience into a series of images that strongly affect the viewer, in which elements of physical reality supplement virtual reality. When this project was brought out during the 2017 Cannes film festival, it was shown in an airport hangar. The viewer is taken to a cold “transit” room, where they are asked to take off shoes and put them next to rows of shoes that belonged to migrants, then a few minutes of uncomfortable waiting, and the viewer is provided with a heavy backpack and a VR helmet, together with which he must make a journey in a specially designed space. Film journalist Tatyana Shorokhova described the experience inside the installation as one of the most difficult ones in her life:

«I found myself in the desert at dawn. A light breeze is blowing, and you can hear the tumbleweed rustling. I spin around, peering at the landscape. Suddenly, a group of people appears on the horizon. They can hardly walk straight in my direction. Men, women, and children are the same illegal immigrants that some Americans are so afraid of. Tired and exhausted. [ … ] I am gazing at people. They are very close. I think you can reach out and touch them. However, the project has the subtitle “Present virtually, physically invisible.” It can be attributed to refugees, but now I understand that it is primarily us, the guests of the installation. The feeling of presence is complete, though, I cannot see my hands and feet—this is not typical for VR projects. There are also no boundaries that are usually drawn for those immersed in virtual reality. As I retreat, a helicopter appears in the sky. It hovers above us, and the searchlight beams across the ground, illuminating the figures. Another couple of seconds and I have the border patrol behind me. American border guards with a dog jump out of two cars. I jump to the side, hitting one of the terrified Mexicans and involuntarily “look” inside him. There is a heart beating. “I look” at the border guard—the same heart. It is clear what I wanted to say to Inarritu: Yes, we are all the same, despite the walls that divide us. By the way, it is not easy to “look” at a person. People, though generated on a computer, have weight and are incredibly realistic. The word “discomfort” describes my feelings very poorly». [41]

In terms of genre, this project is a “walker”. However, the narrative component is clearly shown in everything, from the details of the appearance of migrants to the sequence of “pictures.” A physically and emotionally exhausting journey across the border ends tragically for the viewer-hero: the border guard kills them. The Iñárritu project was awarded a special prize for experimenting with form and promising a new experience of cinematic perception, meaning the lesson of history in a space free from the borders of the frame, stimulating sensory cognition. Inarritu’s installation is a complex involving space, objects, and people, which, undoubtedly, only enhance the effect of immersion.

Another exciting project based on virtual travel is “Psychosis” (2019), developed by the AES+F group together with Alexander Zeldovich. One part of this work was a theatrical production by Zeldovich based on the text of Sarah Kane, which described her experience of hallucinations and subsequent stay in a hospital. The second part was a VR video in which the viewer found himself in the role of a mentally ill person. The installation space in the MARS gallery was designed as a hospital corridor. The viewer had to fill out a short questionnaire, wait for an invitation, then put on a virtual reality helmet, and sit in a wheelchair. When the session started, hero identification’s video took place almost instantly thanks to this wheelchair: in VR-glasses, the viewer found himself in the same hospital corridor, with his head down, saw his own body, motionless legs and hands resting on the wheels of the chair, and at the same time could feel the cold of these metal wheels. The movement inside the VR space was performed by pressing a unique pedal, and turns were achieved by a relatively active movement of the hands turning the stroller. Thus, the body “believed” that it was in the same hospital, and hallucinogenic images floated before the eyes, and the further away, the more terrible. Huge insects, severed limbs, flying multicolored nails, giant mushrooms, and powerful explosions - all these images, elegant and at the same time repulsively textured, in the traditional spirit of AES+F, rose before viewer’s eyes, and it was impossible to hide from them. This video evoked a strong sense of being in a nightmare and complete powerlessness, the inability to escape from the shroud of hallucinations, which, even though we know about their artificial nature, remain frightening.

3.1 What Else Can Virtual Reality Do?

Today, many people call VR technology a “machine for empathy” - because it enables the user to get a dose of a new subjectivity, being-with-the others or - to a certain extent-being the others. This idea forms the name of the interdisciplinary group of students BeAnotherLab from the University of Barcelona named after Pumpeu Fabra. Within the laboratory, the project “The Machine to Be Another” is being implemented. The essence of which comes down to the following: two participants (a man and a woman) put on VR glasses, and each begins to see what the other sees. For example, a man sees women’s hands in front of him. The experiment participants are asked to perform simultaneous movements with their hands and feet to enhance the effect of the illusion. As a result, participants have a vivid sense of “body exchange”; they get a new experience of gender identity [42].

BeAnotherLab is not the first team of researchers to engage in such researches. It is also committing to study Thomas Metzinger and Olaf Blanke, who, together with cognitive scientists Bigna Langgenhager and Ted TEDI, created a VR system designed to evoke the experience of being outside the body, destroying the illusion of stability of its model in consciousness.

In their work, Metzinger and Blanke proceeded from the following premises: we live “inside” not only our ideas about the world, but also inside the models of our bodies, minds, and selfhood that we represent. These models, like all others, are subject to manipulation and transformation, they are not constant. When we receive new bodily changes, we gain a unique experience of subjectivity or “expand” an existing one.

From 2010 to 2015, two other researchers, Marie Sanchez-Vives and Mel Slater, worked with Metzinger and Blanke on the project “Virtual incarnation and robotic reincarnation”. The project was aimed at giving the viewer the experience of “incarnation” in another body. At the moment, Marie Sanchez-Vives, along with other researchers, is developing a VR project in which perpetrators of violent crimes in the family get the opportunity to switch places with their victims.

“To be in someone else’s skin”, “to walk in someone else’s shoes” - various modifications of this idiom must be present in most world languages. It is not accidental – because it is often difficult for us to relate ourselves to different states that we are not too familiar with, events that have never happened to us, and alternative value systems. Thanks to VR, we can be “in someone else’s shoes” literally.

3.2 Multisensory Environment Problems

A brief analysis of how the Internet of things technology can be used in the cultural industry we have recently reviewed [43] gives us a track for outlining the strategy of Iot development in the context of a modernised museum. As the Internet of things is a multi-level system that includes sensors and controllers associated with specific exhibits, works of art or elements of the theater scene, means of transmitting the collected data and their visualization, powerful analytical tools for interpreting the received information, with the possibility of remote control and management in an automated mode.

In this regard, the problem of adapting museum mechanisms for storing and presenting works of art to the audience of a communication society is becoming particularly relevant. Therefore, the museum is becoming increasingly involved in the digitization process using new technologies, such as mobile applications, virtual and augmented reality, artificial intelligence, the Internet of things and data design, which we consider to be the key technologies used in the new “expanded museum”. Unfortunately, now there is no integral methodology for designing the exhibition space and topologies of immersive and multisensory environment using advanced information technologies and aimed at increasing the visitor’s immersion.

Usage of information technologies in multisensory environments leads to new research challenges. The goal of such research is to investigate and develop new methods for designing an exhibition space and for creating new topologies of immersive and multisensory environment using advanced information technologies to enhance the immersiveness and quality of user and audience experience.

To attain this goal, the following objectives were identified:

  1. 1.

    To analyze studies on designing an exhibition space and creating new topologies of immersive and multisensory environment using advanced information technologies to define the connections between the exhibition space design and advanced information technologies.

  2. 2.

    To create the map the technologies that enable to design spaces for immersive simulations, with strong attention for innovations and future trends.

  3. 3.

    To develop and investigate new methods for designing an exhibition space and for creating new topologies of immersive and multisensory environment using advanced information technologies.

  4. 4.

    To evaluate the effectiveness of the proposed methods using simulation.

4 Conclusion

Summing up, we can say that the process of digitalization of art museums and similar cultural institutions is now gaining momentum and becoming an integral part of the development strategies of these sites. The engine of this process is not only the needs of specialists to work with artifacts, but also the needs of viewers who expect the modern museum to actively implement innovative technologies that improve the quality of interaction with the historical heritage stored by the institution. Today, the museum is increasingly using immersion mechanisms and feedback tools to allow each viewer to get a vivid and” customized” experience of meeting art.

However, there is a need to ask to what extent modern curatorial strategies respond to the challenge of such innovations.

In the 1920-s, the first museums of contemporary art had only appeared in the USA (MOMA), Russia (State Museum of Modern Western Art), and Europe. These institutions aimed to build up not just a collection of art pieces but a narrative about the history of contemporary art, they tried to reveal a kind of genealogy of this phenomenon, to name all the art movements, to make their goals visible and to outline what the very term “art” meant at the time [44]. And their resources were classical ones: exposed artifacts, exhibition catalogs, lectures, and specialized issues. And today, a museum faces new challenges: not just the exhibition, but the whole field of information relied on it has to be curated [45].

Thus, we may conclude that the fundamental change in communication processes one sees in the work of an “expanded museum” has resulted in a substantial transformation that needs to be further reviewed in an interdisciplinary analysis. We argue that neither engineering nor humanitarian approach has to be dominant in this development of a future museum but both of them – it is crucial for technical specialists and art historians to engage in a continuing dialogue in order to build up the right track for innovations. A “smart” museum means not only attractiveness and “liveness” but also an updating of the basic narratives about art, a way of providing visitors with a set of correctly outlined concepts and facts.

If in the museum of the 20th century the guide was a human being: authors of texts posted on the walls and in catalogs, authors of scripts for audio guides and art critics who covered the exhibitions. Then today, when the viewer-user works with mobile applications, virtual assistants and other products of artificial intelligence that process big data downloaded from the museum’s archive, the question of who is the “narrator” of the history of art becomes acute, who is responsible for preserving and transmitting significant narratives about the nature of art to new generations of viewers [46]. In other words, today art historians, museum employees, engineers and programmers must face the task of not only loading specific data into artificial intelligence systems and designing interfaces that make it convenient for the viewer-user to access this data, but also creating algorithms that can present information in a high-quality way. In short, an artificial intelligence working with the materials of an art museum must not only be able to “pick up pictures,” but also learn to “understand” the history of art. Keeping in mind that a museum not only presents but also introduces the artefacts and builds up a storyline of aesthetic development and variation, we argue that the role of a historian/curator remains crucial in a digitally transforming institution.

Therefore, we would underline that the mission of an art museum and a gallery remains: preservation and demonstration of cultural heritage, spreading knowledge and aesthetic formation. And in order to achieve these goals specialists have been analyzing the currently changing needs and desires of their audience, introducing the newer information technologies and following the communication trends. Digital transformation of a museum is an incessant process that requires adequate tools (AI, AR, VR, social networks) to structure the great amount of content which is being curated and to make it “user friendly” for contemporary audience. Today it becomes obvious that the museum can reach its main goals only in the format of a continuous dialogue with a viewer, which is necessarily carried out using the most up-to-date technological tools that empower both the “voice” of a curator and the “feedback” of the audience.