Enabling the Metaverse with the Development of PopupView: An Augmented Reality Platform

Abstract

PopupView is an augmented reality (AR) platform that provides audio-visual digital content through mobile phones via visual “triggers” such as photographs and artwork. It builds on Visioning Lab’s vision for the metaverse as integrated virtual and physical environments, conceptualised as “storyworlds”, through which people move freely. AR becomes a tool for viewing and accessing digital content in the metaverse whilst on the move.

1 Company Description

If “Web 3.0 is the world”, we want to make sure citizens are engaged in its development.

At Visioning Lab, we take broad ideas and visions for the future and identify emerging concepts and technologies that have game-changing effects on societal structures and environmental challenges. We specialise in the Web 3.0 infrastructure technologies such as augmented and virtual reality, 3D digital worlds, digital placemaking, blockchain, and NFTs. We develop work using the “storyworld” approach, incorporating these technologies and sharing our insights with others.

Our emphasis on placemaking insists that digital data, tools, and technologies should be connected to specific localities. We promote digital placemaking strategies to ensure direction over how civic decision-makers control digital activity in their region’s geographical contexts.

Visioning Lab is led by Dr. Jessica Symons who specialises in emerging technologies, sustainability, creativity, and public engagement. The company has delivered innovative products and services across the public, private, and third sector including the Universities and Councils of Salford and Manchester, Quays Culture, and Manchester Museum. Our team have also delivered projects funded by National Lottery, Innovate UK, the European Commission and DCMS.

2 Project Summary

PopupView is available on the Google and Apple app stores as a simple, free augmented reality app ready for download onto mobile phones and tablets. Using this app, cultural venues and producers add interest to their content through a “digital” layer, bringing creative and social dimension to their audience experience.

Our vision for the metaverse is an integrated physical and virtual world of multiple domains through which people move freely, whether as digital avatars or as their physical selves in real life.

If we take the emergence of the metaverse seriously, then people will need to access digital content or assets as easily as they can pick up a physical chair or pair of trainers. Augmented reality (AR) becomes the enabling technology that facilitates engaging with digital content on the move. To demonstrate this conceptualisation of AR as providing access to “the digital all around you”, we created PopupView in 2019.

PopupView.com, www.popupview.com is an app and an aggregating platform for AR experiences, i.e. the “YouTube for AR” where people access augmented reality experiences through their phone camera much like they use YouTube for seeing short films. The PopupView platform works by using “trigger images” to launch additional digital content. These images can be photographs, drawings, real-world assets, or any other visual content.

PopupView is provided as limited open-source AR software in contrast to proprietary platforms such as Snapchat/Instagram filters and Pokemon Go/Lightship and bespoke AR apps built by digital design companies for clients such as the National Gallery in London.

The code for the PopupView app and platform is available with permission for download on GitHub and alongside the demonstrator app which contains third-party AR “campaigns” using multiple “trigger” images to show digital content produced in collaboration with artists, writers, and musicians. It is non-proprietary and open-source with permission, available for customisation and collaboration.

3 Project Details

Imagine the following scenario

You are walking down the road when you see the PopupView logo. You recognise that digital assets are available nearby. You tap the side of your glasses and the digital asset materialises. If you want it, you capture it and save to your digital wallet for future use.

PopupView.com is a platform for showing third-party AR experiences just as YouTube shows third-party films. It is a site for providing a “taster” experience of augmented reality using 2D images to “trigger” the display of 2D/3D/animated/audio digital assets. The platform contains over 30 different AR experiences designed by artists, musicians, writers, and producers.

PopupView content is accessed via an app on a phone/tablet. The app uses “trigger” images to show additional digital content in the camera view of the device using “augmented reality” or AR.

Each AR “campaign” on the PopupView platform was developed to share an idea or add interest to an existing offer. For example, at Manchester Museum, we created digital butterflies to emerge from the insect artefacts contained in gallery display cases. For artist Peter Kalu, we used photographs of his artwork to trigger his song about the slave trade.

To develop AR campaigns for the app, Visioning Lab works in collaboration with the campaign “client” to identify images or scenes that could be augmented with additional digital content. These images could be at a physical location such as an art gallery or tourist attraction, or they could be in a book or on a website.

We identify existing digital content or produce additional work as required. This content is incorporated as a campaign into the PopupView platform and so available for viewing through the app. We provide information to the campaign client to incorporate the PopupView app into their marketing material and monitor the downloads and progress of the app use over the campaign period.

We also develop 3D digital content by running workshops for people to make and view their own creative work in public settings. This approach worked well in the Lightwaves PopupView app where digital sketches drawn in virtual reality by visitors to the Immersive Lab in MediaCityUK were exhibited at an art exhibition in Salford Quays. People liked the hand-drawn digital animations by their peers, available as part of an art festival and sharable via social media.

4 Case Studies

One of our most popular AR campaigns is the Birdsong digital experience that helps people recognise birdsong better.

#Birdsong is a digital nature experience for walking trails, visitor centres, museums and galleries and for games and educational packs. The experience is customisable and made available via phones and tablet devices.

#Birdsong works by playing sounds of birds on a device when a user scans a “trigger” image as well as providing information about the bird. The trigger images can be images of the birds, habitats, plants, the natural environment or real-world scenes in the landscape/venue itself.

Groups of People at Lightwaves Festival

Visioning Lab developed an AR app for the Lightwaves Festival using digital drawings sketched in virtual reality by visitors to the Immersive Lab. The work was inspired by LS Lowry and also used photo montages created by fine artist Amber McCormack.

Find the butterflies at the Royal Horticulture Show in Tatton (RHS 2019)

Visioning Lab collaborated with Tatton and Cheshire East Council to animate the Tatton stand at the 2019 Royal Horticulture Show. Artist Jacki Clark made 3D digital sketches using virtual reality. The AR app worked on tablets at the Tatton stand and the butterfly case. Visitors released butterflies by pointing the device camera at the display.

5 Feedback from End Users

User comprehension is the biggest barrier to the adoption of augmented reality. People do not understand what it is, how it works, and its value. In an already saturated media content world, there has been little incentive for people to overcome the learning “hump” necessary to make the cognitive leap required.

Typically people need to work out how to use the app on their own after downloading from the app store. Many users will download the app, open it, open a campaign, and select an image. When their device camera opens for them to find that image and “trigger” additional content, they do not know what to do and will often abandon the app at that point. Users rarely recognise the need to point their device camera at a “trigger image”, not on their phone but in the world around them. This lack of awareness that the camera can show additional digital content 'triggered' by an image is finally slowly changing through the use of QR codes during the pandemic and filters on Snapchat/Instagram.

Over the past 4 years of working with augmented reality development, audience awareness of “trigger images” was our biggest challenge. To ensure user engagement, we needed to be physically present with a phone or tablet device and an image ready to “trigger” additional digital content. To maximise audience engagement, we would attend events and stand next to the images and invite people to view the AR experience together with us.

For example, referring to the case studies above, at the RHS show in Tatton, we spent 5 days at the Tatton stand with a tablet. The trigger images were photographs of butterflies pinned to a tree sculpture. As people approached the stand, we would say “would you like to see digital butterflies come out of the tree?” or “would you like to wake up the butterflies” which worked particularly well with children. Then we opened the PopupView app on the tablet, hovered the device camera over the photograph of a butterfly, and showed them digital butterflies designed for the art experience as they came into view on the tablet.

Once experienced, augmented reality has a powerful effect on the viewer. Responses are usually very positive with words such as “wow!”, “that’s amazing”, “I love that”.

QR codes during COVID helped increase awareness of images as “triggers” accessed through phone cameras. When demonstrating AR campaigns, we can now refer to QR codes and this now produces a positive response and an instant awareness that the phone camera needs to be used and that something happens as a result of hovering over an image.

Once users become aware of the functionality of augmented reality, they adopt it readily. It usually only needs to be explained once and then they will view multiple AR experiences. It was due to this issue around user awareness that we focused on producing PopupView as a platform with very simple AR campaigns where the key feature was using trigger images to “pop up” additional content and we collaborated with artists as the most likely to intrigue audiences and stimulate enough curiosity to work out how to use the app.

6 Future Outlook

The world’s leading tech companies are due to launch augmented reality glasses in the next 3 years with Apple Glasses and Snapchat Spectacles in 2023, Meta AR glasses in 2024 and Microsoft, Amazon, and Chinese companies Oppo (Air Glass) and TCL also preparing. The glasses work through a transparent overlay on the lenses and mini speakers built into the sides of the glasses so people can see and hear 2D and 3D animated content in front of their eyes. This industry is projected at $33 billion worldwide by 2027 with AR glasses eventually replacing mobile phones as the main way people access digital content whilst on the move (Statista 2022).

Visioning Lab’s augmented reality platform PopupView (www.popupview.com) was designed to take advantage of the emergence of the AR glasses industry. We released PopupView as an app initially, showing 2D/3D animated and audio digital content through mobile phone cameras. We recognised that whilst currently augmented reality experiences are shown through a mobile phone camera, soon the digital view would be through a pair of AR glasses.

To further build the PopupView platform, we need a rich source of “trigger images” with associated digital content (YouTube used sports and music videos to rapidly expand their platform). We have identified the hugely popular graphic novels as a significant source of material for “trigger images” which combined with AR spectacles would create a whole new way of consuming digital content experiences. Our goal is to rapidly produce volume AR experiences using online content such as webtoons as trigger images and audio as digital content. For the millions of webtoon fans worldwide, their platform can offer additional audio and 2D/3D animated content to accompany the existing visuals. We are also always looking for artists, musicians, writers, and producers who want to collaborate in the development of AR experiences.

We are ready to integrate NFTs with the PopupView platform, having recently carried out an R&D project investigating the process. This included a system architecture, review of popular NFT platform APIs, and industry standards for digital assets.

By making PopupView limited open-source, we hope to increase opportunities for co-producing work globally. Interested parties are encouraged to get in touch.

7 Conclusion

Augmented reality is a technology in transition. It needs to be considered in context with the wider digital environment and how this will manifest over the next 10–20 years. It is a critical enabling technology between digital placemaking (where digital assets are placed into context with physical locations) and the metaverse (where digital assets are located in virtual environments).

As a technology in transition however, the process of using augmented reality, particularly with a phone, has not yet become an automatic reflex for the general public. This can be likened to awareness of hyperlinks and the mouse technology when the Internet first came into being over 20 years ago. Back then people were not aware that an underlined blue word on a “page” on a “website” indicated it could be “clicked on” by the round plastic object next to their keyboard (called a “mouse”). Now this process is second nature to people. Indeed, whilst on their phones, people will tap any word on the screen to see if it will take them to additional content.

Similarly, people are beginning to recognise that an image may trigger additional digital information when viewed through a device camera. This awareness increased rapidly during COVID through the use of QR codes for tracking purposes and now widespread use of QR codes for multiple purposes.

The release of AR spectacles by leading international technology companies will further stimulate the takeup and use of augmented reality experiences. Whether or not it will continue to be called “AR” or “augmented reality” however is unlikely. New terminology will emerge that is better suited to the devices used. We predict that the use of “HUD” or “Head-Up Display” is likely to transition across when the use of AR glasses becomes more mainstream.

“A head-up display, or heads-up display also known as a HUD is any transparent display that presents data without requiring users to look away from their usual viewpoints.” (Wikipedia)