Keywords

1 Introduction

The advancement of technology in the mobile device industry has completely changed the way people record and share their memories. Previous studies of have shown that retrieval cues, which are environmental or internal stimuli presented in multimedia forms such as the image of a familiar face, the sound of a particular song, or the smell a specific scent, are essential for the recall of a specific memory [1]. Hence, digital storytelling often uses tools such as podcasts, videos, and blogs for creating a more interactive and engaging experience [2]. Learning that multimedia elements can be used to enhance the storytelling experience and engage the user [3], many mobile applications have been developed to support the creation of multimedia stories, providing users with a more interactive and dynamic way to record their lives [4].

However, social media apps currently available in the market are primarily focused on image editing, video editing, and text creation. They are still adopting traditional and singular storytelling methods. For instance, image editing functions commonly use text, image stickers, and multi-picture collages to create a narrative. Video editing functions use a timeline concept, where material on the same timeline is edited based on duration, and then combined into a complete video file. Text creations are usually provided with a text workspace, where various text editing tools can be used to create text and images can be inserted to enhance its meaning [4].

Unfortunately, the above methods fail to create the most immersive storytelling experiences, as some apps cannot integrate all sensory elements, while others may be time-consuming and lack logical structure, making it difficult to tell a compelling story [5]. Previous studies of Wang and Tan has presented a solution to this using a series of snapshots of a piece of specific memory with the associated context, namely time, location, people, activity, imagery and emotion in digital albums [6]. But the storytelling experience is still lack of usability and logic attachments since it’s still adopting a traditional way of a movie-like slide show as a visual form of memory.

ProtoLife aims to meet the growing need for simple and effective tools to capture, store, and share multimedia content. It provides a user-friendly platform for creating multimedia stories by connecting multimedia elements using the principles of prototype design, which lowers the learning curve and enhances the user experience.

Prototype design is a mature method in interface design that helps designers build a quick and viable working model to evaluate and improve the feasibility of each interaction before final implementation [7]. In ProtoLife, this technology is applied to the process of creating multimedia stories, enabling users to quickly organize the story's logic and add more narrative elements to it. An intuitive and easy-to-use interface allows users to effortlessly create dynamic and captivating multimedia stories [7, 8].

2 Design and Implementation

2.1 System Overview

The ProtoLife platform was designed and implemented to provide a fast, user-friendly, and effortless solution for recording and creating multimedia stories. To achieve this goal, we simplified the process of storytelling by studying the narrative concept and combed the process into a system structure where multimedia elements are able to link and dynamically refer to each other (see Fig. 1). We developed the program using SwiftUI and Objective-C and created a MVP (Minimum Viable Product) demo of our application based on iOS platform.

Fig. 1.
figure 1

An example of the functional structure of ProtoLife. Left: ProtoLife simplified the process of storytelling into a systematic structure where multimedia elements are able to link and dynamically refer to each other. Right: ProtoLife is able to link friends and circles to different stories that helps to create a story network.

Full size image
Fig. 2.
figure 2

Modular card-form canvas used in ProtoLife. There are currently six multimedia canvas options available, linked by dynamic arrows represent the logical relationship between them.

Full size image

2.2 Fluid Interface

The fluid storyboard interface is a key feature of ProtoLife. The interface uses the common "canvas" concept in modern prototyping methodologies [9], representing each narrative element in a modular card form (see Fig. 2).

There are currently six multimedia canvas options available, including photo, video (share the same portal with Photo Canvas), music, audio, location, and personnel canvas. Dynamic arrows are used to connect these cards, expressing the logical relationship between each element. Each modular canvas card has a specific function in building the story, which is outlined below:

  • Photo Canvas: the function designed for users to upload pictures from system album, allowing users to capture and record the surrounding visual information when the story occurs.

  • Video Canvas (share the same portal with Photo Canvas): the function designed for users to upload videos from files, allowing users to capture and record the dynamic environment around the story.

  • Music Canvas: clicking on the music canvas allows the user to select a music work provided by the system with complete intellectual property rights, enabling users to record the music environment around the story.

  • Location Canvas: obtains the user's geographic location information and presents it on a small interactive map on the card, allowing users to record the geographic information around the story.

  • Audio Canvas: a recording function similar to a WeChat voice message or an Instagram voice message. When the user presses and holds the button, the built-in microphone of the phone can be used to record audio, allowing users to record the sound information around the story.

  • Personnel Canvas: the function to link to nearby friends and strangers or friends already in the user's friend list. The user can choose to add the person to the personnel canvas, allowing users to record the character information around the story.

2.3 Navigation

We use a large “map” to hold all modular cards on the page, but only one card is displayed in the screen display area at a time. This is typically designed to integrate all the multimedia elements to form a consistent storyline [10]. When the user clicks on the overview navigation button at the bottom, the map will be displayed, allowing the user to know the relative position of the card they are editing.

2.4 Interactions

When using the application, users can use simple and natural interactive gestures, such as single tapping, long-pressing, dragging, and sliding, to select and connect elements. This provides users with the ability to seamlessly weave various multimedia elements together to create a unified and strongly logically related story [11].

Users can create or connect two canvas cards in two ways:

  1. I.

    Start by long-pressing a card on the screen and dragging out a dynamic and changing arrow while holding it, the location where the arrow is released will be the object that the user wants to add or connect to (see Fig. 3-a).

  2. II.

    By clicking the floating plus button on the right border of the card, users can select a connecting object or create a new card (see Fig. 3-b).

Fig. 3.
figure 3

Interactive options to start with. a: Start by long-pressing a card on the screen and dragging out a dynamic and changing arrow while holding it. b: click the floating plus button on the right border of the card to select a connecting object or create a new card.

Full size image

2.5 Social Sharing

Along with the storyboarding and commenting feature, users can easily share their stories with friends or the public through social media platforms or by generating a shareable link to their story. Moreover, users can invite friends to collaborate on a story and grant them editing permissions, making it an ideal platform for creative collaboration.

3 Evaluation

The evaluation of ProtoLife was conducted using a mixed-method approach that included both qualitative and quantitative data. This allowed for a comprehensive assessment of the effectiveness of the application in supporting the recording and creation of multimedia stories.

3.1 Usability Test and Data Collection

Data was collected through in-depth interviews with 37 participants who had used the application for at least two weeks. We developed a minimal viable product, which retained the primary storyboard features and allowed users to share their story creations on social media. We sent this minimal viable product to the users for usability testing and conducted follow-up interviews two weeks later (see Fig. 4). The interviews were conducted in a semi-structured format, allowing participants to provide insights into their experience using the application. The interviews were recorded and analyzed using thematic analysis to identify common themes and patterns in the participants' responses.

Fig. 4.
figure 4

User conducting a series of tasks during usability test. Left: User drag to create a new Photo Canvas. Middle: User speaks to the microphone on the Audio Canvas. Right: User taking a picture on the Photo Canvas.

Full size image

While we want the product to mainly highlight its features on emotional connections, we specifically asked users about their subjective perceptions of the emotional value of the memories our products provide for them. This method is widely recommended in designs with emotional connections taken into consideration [12].

Descriptive statistics were used to analyze the data to determine the mean, median, and mode of each question, and inferential statistics were used to test for statistical significance.

3.2 Data Analysis

The results of the qualitative data analysis showed that the majority (20/23) of participants considered ProtoLife to be a valuable tool for recording and creating multimedia stories. Participants reported that the application was easy to use and its user-friendly interface allowed them to capture and organize their experiences in multimedia format easily. Participants also appreciated the application's ability to enhance their life narratives and memories, with some reporting that they were able to retrieve past memories more vividly, giving them a unique experience.

The results of the quantitative data analysis showed that participants had a high level of satisfaction with the application. On a 5-point Likert scale administered to the 38 respondents, the overall satisfaction score averaged 4.5, indicating that participants were generally very satisfied with their experience using the application. The average score for the application's usability was also high, averaging 4.3, indicating that participants considered it a valuable tool for recording and creating multimedia stories. Participants also reported that the application was effective in enhancing their life narratives, with an average score of 4.1 on the same scale.

Inferential statistical analysis showed that there was no significant difference in participants’ satisfaction with the application based on demographic factors such as age, gender, or previous experience with multimedia storytelling. This suggests that the application is effective in supporting diverse user groups in recording and creating multimedia stories. However, this can occur due to the shortfall in the number of testers.

4 Conclusion and Future Work

The evaluation results of ProtoLife demonstrate that this application is a valuable tool for users who want to record and create multimedia stories. The implementation of prototyping methods in storytelling using multimedia is effective and accessible. The participants in the evaluation were highly satisfied with ProtoLife, and they found its interface to be user friendly, which made the process of recording and organizing their experiences in a multimedia format much simpler. In addition, the participants look forward to using ProtoLife to enhanced their narratives about their lives, with a number of them reporting feeling more connected to their experiences because of using the application.

Since the result might occurs due to the shortfall in the number of testers, in the future we are seeking a massive evaluation towards the usability of ProtoLife to further optimize our solution. In addition, future research should continue to explore the potential of ProtoLife to support the recording and creation of multimedia stories in more application scenarios.