1 Introduction

With the rapid development of the Internet industry, users now receive a vast amount of news information daily via electronic devices, such as smartphones. As a result, data journalism has become a key driving force in the news industry, effectively enhancing the accuracy and attractiveness of information dissemination by combining data with narrative visualization. However, some narrative visualizations may compromise data accuracy and completeness, affecting the credibility and effectiveness of data, and hindering effective communication and exchange among users (Lin et al. 2020). Providing high-quality data journalism works is therefore crucial for increasing public attention and understanding of significant social issues.

Several researchers in previous studies have proposed models for visual design. For example, Sacha et al. (2014) introduced a visual analytics model that extensively analyzes the process of generating knowledge from data, along with the theories and methods involved at each stage. Munzner (2009) introduced a nested model for visual design and validation, comprising four layers: user and task analysis, data abstraction, visual encoding, interaction techniques, and algorithm implementation. These models offer researchers distinct perspectives and approaches for handling various aspects of visualization. However, these models primarily concentrate on the theories and methods within the realm of data visualization, overlooking the overall user experience when engaging with visual works. Therefore, the focus of this study is to pay attention to the overall user experience and address the shortcomings of the aforementioned models in this aspect.

To address these challenges, this study seeks to integrate user experience with narrative visualization to enhance the adoption and satisfaction of information products and services. Although researchers have made strides in improving visualization user experience, these achievements often focus on specific fields or partial experiences, lacking a comprehensive consideration of the overall user experience. In this work, we propose a user-centered model for creating narrative visual works that encompasses the user’s perceptual, interactive, cognitive, and emotional experiences when browsing visual works. This model explicitly defines the key design elements of narrative visualization, providing a comprehensive reference framework for designers and researchers to effectively optimize the user experience of narrative visualization. Additionally, we evaluate the impact of this model on narrative visualization design through in-depth interviews with six experts. The results show that this model can effectively help designers enhance the user experience on various levels.

2 Related work

In this section, we first conducted a literature review of the relevant research based on user experience. Then, we separately introduced the definitions and current research status of user experience, data journalism, and narrative visualization.

2.1 Narrative visualization

With the advancement of new media technology, visualization design as a form of communication that transcends language barriers has become increasingly prominent. Data visualization assumes a pivotal role in presenting and exploring complex information in this domain. Narrative visualization consists of a series of data visualizations with a narrative sequence and serves as a medium for storytelling (Segel and Heer 2010). Kosara and Mackinlay (2013) argue that storytelling is a natural process in visualization, from exploration to analysis to presentation. The narrative structure becomes even more important when making decisions because it can express important facts concisely and logically. Research has shown that effective story composition and visual storytelling can guide readers through data, improving their understanding and memory abilities. In summary, narrative visualization is designed to better convey information from data (Borkin et al. 2015; Hullman et al. 2013).

2.2 User experience

User experience (UX) covers multiple domains, with differences between disciplines (Forlizzi and Battarbee 2004). With the development of technology, UX involves various aspects of experience, such as usability, aesthetics, pleasure, and effectiveness (Hassenzahl and Tractinsky 2006). Donald Norman expanded the scope of UX to emotional factors, such as enjoyment, aesthetics, and entertainment (Norman 1988). According to ISO 9241–210, UX is defined as individuals’ cognitive and emotional responses to the use or anticipated use of products, systems, or services.

Bernd Schmitt’s categorization (Schmitt 1999), UX is classified into five types: sensory, affective, creative cognitive, physical, and behavioral experiences. Norman (1988) proposed three levels: visceral, behavioral, and reflective. The visceral level focuses on intuitive reactions, encompassing pleasure, safety, and comfort. The behavioral level underscores reactions during operation and use of things. The reflective level involves thinking, analyzing, and interpreting experiences and situations.

2.3 Narrative visualization based on user experience

Recently, researchers have been focusing on combining user experience with data visualization to improve data presentation and satisfaction. They have explored various technical means and methods to enhance user experience and proposed effective methods and models from different perspectives. For example, Lu et al. (2016) proposed a user experience-based visualization model by analyzing human cognition. Harrison et al. (2015) suggested creating more attractive information graphics by measuring human visual senses. Lan et al. (2021) optimized the visual design of information graphics from the perspective of emotional experience. Sallam et al. (2022) proposed creating data videos that can generate positive emotions for users from the perspective of user personality. Adding interaction in visualization works can greatly enhance user experience, according to researchers (Hasan et al. 2022; Arjun et al. 2018). These methods and models offer valuable support for user experience-based narrative visualization techniques. Research has shown that incorporating principles of user-centered design can greatly enhance user satisfaction, engagement, and comprehension (Tyagi et al. 2021). Therefore, by placing emphasis on optimizing user experience during the production of data visualization, it is possible to effectively promote user understanding (Saket et al. 2016).

2.4 Data journalism

Data journalism is a reporting approach that utilizes data to convey news stories in a clear and concise manner. This involves the process of collecting, refining, filtering, and presenting complex data through the use of visualization techniques (Na 2018).

Paul Bradshaw proposed the “double pyramid model” (Bradshaw 2011), summarizing data journalism production as an inverted pyramid structure (data collection, cleaning, understanding, integration) and data journalism communication as a positive pyramid structure (visualization, narrative, socialization, humanization, personal customization, usage). Skelton (2014) divided data journalism production into seven steps: needing data, acquiring data, cleaning data, analyzing data, displaying data, interviewing the public, and telling stories that include data. In general, the creation of data journalism can be divided into three stages: data collection, inquiry, and presentation (Anderson et al. 2015).

3 Construction of narrative visualization model based on user experience

In the present era of information, users are no longer just passive recipients of information. They have the ability to actively choose and experience information, which helps them to extract knowledge from it. The ultimate goal of design is to serve the audience, and it is essential to consider whether the audience can truly and effectively understand the information contained in the data (Wu 2016). From a user experience perspective, optimizing solutions in data visualization can be an effective way to enhance user understanding (Shihu and Fang 2011).

3.1 Construction of narrative visualization model

User experience is important in the product interaction process (Wu 2016). Researchers have created various models to understand and measure user experience. Garrett (Jesse James Garrett 2022) analyzed user cognition and experience, establishing a model with and grouped ten elements into five planes. Building on Garrett’s model (Jesse James Garrett 2022), Lu et al. (2016) analyzed user experience and identified visual elements and interactive behavior, proposing a data visualization user experience model. In this study, we further refined Lu et al.’s model (Lu et al. 2016) by integrating it with specific dimensions to create a narrative visualization design model based on user experience. In the realm of data visualization theory, researchers have suggested four key steps for visualization design: demand discovery, information processing, interface design, and sensory design. Each stage encompasses a range of specific tasks. According to Lu et al.’s data visualization user experience model (Lu et al. 2016), there are four critical stages: strategy, scope, structure, and surface, each corresponding to different planes of the experience.

3.1.1 Strategy plane—demand discovery

In the narrative visualization model, the strategy plane is primarily to clarify the project purpose, including clarifying user needs and determining the project objective. To clarify the user needs, a detailed analysis of user needs is necessary to understand what information users want to obtain through data visualization. This level directly affects the overall plane of design and is closely linked to subsequent design work (Chu and Tian 2016). After understanding user needs, we can grasp the overall goal of the project and make the data obtained later more targeted to the problem and goals (Lu et al. 2016). To achieve this goal, designers can employ user experience research methods such as user interviews and persona modeling to truly understand the needs of the target users (Wu 2016).

3.1.2 Scope plane—information processing

The scope plane in the narrative visualization model focuses on defining the boundary of the analyzed data. This stage involves processing the acquired data, which consists of three parts: data cleaning, analysis, and story structure construction. Data cleaning ensures that only necessary data is retained. Data analysis explores the logical structure of the stored and organized data (Lu et al. 2016). Story structure involves organizing the data content into a compelling narrative. Thus, at the scope level, designers are tasked with transforming the relevant data into a coherent and logical data story.

3.1.3 Structure plane—interface design

In the structural plane, it is necessary to encode the data, determine the style of the visual charts, and based on the charts, select the interaction methods to associate different data and form an interactive data story. At this stage, information mapping and interaction design need to be completed (Lu et al. 2016). Information mapping involves mapping data to visual channels such as dots, lines, surfaces, and colors for user perception (Wu 2016). Interaction design focuses on user-terminal interaction and response. User operation is a crucial role in achieving good UX and perception goals. Therefore, according to the different data and user groups, the design of visualization prototypes needs to focus on interactivity, rationality, and fun (Lu et al. 2016).

3.1.4 Surface plane—multisensory design

The surface plane is the final step of the design process in the narrative visualization model. Its purpose is to present data mapping from the structural plane to the user in a way that is easily understandable (Lu et al. 2016). Designers need to study visual elements within the visual channels of the structural level from the perspective of visual perception and then apply highly perceptible visual elements to the design, considering the target audience. Additionally, designers need to integrate visual, tactile, auditory, and other sensory aspects in an organized manner to meet the user’s needs and aesthetic expectations. Throughout the process, designers need to continuously refine, modify, and reorganize information to present the design structure and expression form in a diverse manner.

3.1.5 Summary

The planes of the model are organized in a hierarchical manner, with each plane building upon the previous one (Jesse James Garrett 2022). The strategic plane focuses on prioritizing user needs, while the scope plane determines the relevant data based on those needs, addressing the question of “what to do”. The structure plane involves data mapping and interaction design, depending on the data selected in the scope plane, and primarily deals with the choice of the chart and user interaction. The surface plane, positioned at the top, determines how the information is presented. These planes are interconnected and mutually supportive, ultimately resulting in a positive user experience for the entire visualization system.

3.2 User experience model construction

3.2.1 Process of user viewing visualizations

Luo et al. (2002) proposed a three steps process for information interaction: information perception, comprehension, and processing. This process also applies to the interaction between users and visualization. First, users search for specific visual information according to their task goals. Next, users perceive and recognize the information presented in the visualization. Finally, users perform interactive operations to accomplish specific functions or tasks, and provide cognitive feedback on the outcomes. Likewise, Zhu et al. (2010) also suggest that the user’s process of viewing visualizations consists of four stages: perception (identifying visual information through visual features), identification (determining the target information), cognition (understanding the conveyed information), and interpretation (processing the information and taking interactive actions and measures).

Drawing upon the analysis of the aforementioned research, this study identifies and examines four distinct processes commonly encountered by users during their interaction with narrative visualizations. These processes include interface browsing, user interaction, information reception, and the provision of emotional feedback.

3.2.2 Experience gained during the viewing process

Wu (2016) proposed three types of experiences in data visualization: visual, interaction, and emotional. Yin (2018) examined information visualization from the perspective of cognitive experience and concluded that a good cognitive experience is crucial for user experience, as it can enhance the emotional experience and provide users with an exceptional experience. Wang (2018) emphasized that in the rapidly evolving technology and culture of today, visualization design should prioritize users’ cognitive and interactive experiences. Building upon the experience types proposed by these researchers, data visualization primarily encompasses four types of experiences: sensory, interactive, cognitive, and emotional.

Sensory experience Multisensory design is a critical aspect of user experience, involving various dimensions such as visual, auditory, tactile, olfactory, and gustatory. In the field of visualization, sensory experience primarily focuses on users’ visual perception of interface elements (Cai and Zhu 2010). As the top plane of the user experience model, sensory experience also serves as the instinctive plane of emotional design, encompassing users’ pleasure planes in terms of visual, auditory, and other aspects, as well as their overall contentment with the material. Essentially, sensory experience determines whether visualization design allows information receivers to absorb information in an enjoyable state and provides them with a sense of beauty (Jiang et al. 2016).

Interaction experience In the field of data visualization, interaction experience primarily encompasses modifications in visualization views and browsing view information (Wu 2016). The interaction experience focuses on whether users can achieve their goals efficiently and avoid unnecessary work (Jiang et al. 2017). Users’ perception of their interaction experience is primarily influenced by the layout of information (visual elements), specific interaction operations, and the three elements of interaction space (Wu 2016). This aligns with the behavioral layer in emotional design, which strives to streamline interaction behavior and minimize cognitive processing time (Chen 2020).

Cognitive experience The aim of cognitive experience design is to assist users in efficiently identifying and acquiring the desired information. To achieve a good cognitive experience, it is important to ensure that the data are accurate and appropriately categorized and has a coherent logical structure (Yin 2018). The reflection layer in emotional design (Norman 1988) is closely linked to cognitive experience. By integrating cognitive experience with the reflection layer, designers can gain a better understanding of the cognitive difficulties users encounter when interpreting and applying visual products. This enables them to strike a balance between usability and profound thinking in the design process.

Emotional experience Emotional experience refers to the attitudes that people have toward objective things that meet their needs. Thudt et al. (2017) consider emotional experience as a crucial aspect in visualization, and Kennedy and Hill (2018) found emotional reactions to be a key factor in user participation in the data process. Lan et al. (2021) discovered that the usability and expressiveness of visualization can have emotional impacts on users. This emotional experience is closely related to the reflection layer in emotional design, which emphasizes users’ attitudes and emotional reactions during long-term product use.

Based on the above analysis, the user experience of a visualization work can be seen as a hierarchical relationship. Initially, users perceive the work through visual perception, which provides them with a sensory experience. Subsequently, as users interact with the work through data filtering and querying, they form an interactive experience. As users gain a deeper understanding of the specific content, they undergo a cognitive experience. This process can elicit positive or negative emotional reactions based on the information provided, resulting in a complete emotional experience. As the level of experience deepens, users’ satisfaction increases, leading to greater acceptance and recognition of the visualization products.

4 Data journalism case study

This section introduces the data journalism cases to be analyzed and establishes the specific dimensions for analyzing these cases. It then provides a detailed explanation of the specific methods employed for the analysis of the data journalism cases.

4.1 Standard for selecting examples

The Sigma Awards for Data Journalism have gained international recognition since its establishment in 2020. This study selected 10 narrative visualization data journalism works that won the Sigma Awards in 2021–2022, based on three criteria shown in Table 1, 10 case screenshots are shown in Fig. 1.

Fig. 1
figure 1

Screenshots of 10 data journalism cases. See How the World’s Most Polluted Air Compares With Your City’s (The New York Times 2020). Why your smartphone is causing you “text neck” syndrome (South China Morning Post 2020). Danish Scam (Pointer (KRO-NCRV) 2020). Kloop’s investigation of femicide in Kyrgyzstan (Kloop 2020). Who Gets to Breathe Clean Air in New Delhi? (The New York Times 2020). The Digital Army (Pointer (KRO-NCRV) 2021). This is Myanmar’s State of Fear (Al Jazeera Media Network 101 East, AJLabs 2021). Black Snow: Big Sugar’s Burning Problem (ProPublica The Palm Beach Post 2021). Lanes of Death in East Cairo (Arab Reporters for Investigative Journalism (ARIJ) 2021). What the 1921 Tulsa Race Massacre Destroyed (The New York Times 2021)

Full size image

This study primarily focuses on narrative visualization in data journalism, requiring the selected works to possess both information narrative and visualization features. Additionally, interactivity plays a significant role in visualization design, as it allows for exploration and enables the audience to view different perspectives throughout the storytelling experience. Hence, the chosen works must incorporate interactive elements. Lastly, the selected works should emphasize providing specific descriptions of events to aid users in comprehending the event details and forming a comprehensive understanding.

This study collected three data journalism cases that received the Best Data Visualization Award in 2020, along with narrative visualization data journalism cases from 2021 to 2022, based on the aforementioned criteria. A total of 25 works received awards during this period. After evaluating the definition and characteristics of narrative visualization, ten narrative visualization data journalism cases that met the requirements were selected.

Table 1 The data journalism narrative visualizations that won awards between 2020 and 2022
Full size table

“Case ” (The New York Times 2020) uses geolocation data overlaid on real-world maps to visualize air pollution, allowing readers to intuitively understand the harm air pollution poses to public health. “Case ” (The New York Times 2020) visualizes the environmental pollution situation in New Delhi, India, particularly the PM2.5 values for low-income and middle-income families in different scenes. “Case ” (ProPublica The Palm Beach Post 2021) visualizes the harm that burning sugarcane does to the air, prompting officials in Brazil and other regions to stop using burning methods to process sugarcane. “Case ” (Arab Reporters for Investigative Journalism (ARIJ) 2021) is an investigation that records how the road reconstruction project in East Cairo overlooked pedestrian safety planning, leading to an increase in casualties from traffic problems. The author presented the impact of Cairo’s transportation development and the lack of road infrastructure through interactive visualizations. In the end, specific solutions to address this issue were proposed. The government officials adopted the recommendations and made improvements to the roads.

“Case ” (South China Morning Post 2020) uses data visualization to show changes in user behavior when using smartphones. The aim is to make readers aware, in an interactive and innovative manner, of how their smartphone usage habits can impact themselves and those around them. “Case ” (Pointer (KRO-NCRV) 2020) is a report about the use of other people’s information for fraud. The author recounts the process of the editorial room gradually finding Martyna in a first-person perspective through the restoration of the case, making readers feel as if they are searching for the truth with the investigator. “Case ” (Pointer (KRO-NCRV) 2021) mainly tells the story of a group of people who are skeptical about the COVID-19 pandemic spreading false and misleading information on social media. This report uses an innovative storytelling method to bring the audience into the chat group of the digital army and experience becoming a member of these telegram groups. Overall, these three articles vividly demonstrate, through innovative narrative approaches and data visualizations, how digital technology impacts our lives, serving as a reminder for us to remain vigilant when using digital technology.

“Case ” (Kloop 2020) focuses on femicide cases in Kyrgyzstan. The importance of this investigation lies in the fact that no research on female victimization cases has been conducted in Kyrgyzstan before. The author, by collecting various data, uses this work to confirm that the direct cause of female victimization is the presence of domestic violence in Kyrgyzstan. “Case ” (Al Jazeera Media Network 101 East, AJLabs 2021) reveals the atrocities of the Myanmar army during the power seizure. The report reveals the existence of secret bases and the occurrence of torture, mysterious deaths, and missing cases through audio interviews and narratives of the reporting team stationed in Myanmar. “Case ” (The New York Times 2021) is a work produced by The New York Times on the occasion of the 100th anniversary of the Tulsa Race Massacre, restoring the event in a linear narrative way and showing the truth of a century ago to readers. The work first shows the prosperous scene of the Greenwood District, and then describes the cause, process, result, and experience of the survivors of the massacre in chronological order. These reports vividly reveal social issues and historical events through investigation, analysis, and the profound impact on communities and individuals, arousing attention and reflection.

4.2 Indicators for case analysis

We adopted the analysis framework proposed by Tong et al. (2018) and McKenna et al. (2017) to systematically examine narrative visualization cases. This approach enabled us to conduct a thorough analysis of narrative visualization works using established analysis standards. As a result, we identified eight key elements of narrative visualization: visual elements, narrative visualization types, navigation input, animated transition, data sources, narrative structure, narrative theme types, and emotional feedback.

Visual elements are fundamental components used in visual design to convey information and ideas effectively. They play a crucial role in organizing and enhancing visual works, making them aesthetically pleasing and readable. Visual elements typically include points, lines, shapes, colors, textures, charts, maps, illustrations, photographs, videos, and other visual representations.

Narrative visualization types refer to different categories of data visualization works based on various story structures and presentation methods. These types aid in classifying narrative visualization works, enabling a better understanding and analyze their characteristics and purposes. In the recent study by Chen et al. (2023) seven types of narrative visualizations were identified: annotated chart, infographic, timeline, data comics, scrollytelling, slideshow, and data video.

Navigation input refers to the various ways in which readers interact with the narrative visualization while engaging with the storytelling process. These methods primarily include scroll, button, and slider (McKenna et al. 2017). In simpler terms, navigation input enables users to navigate through the work by clicking buttons, scrolling through the document, or using sliders to find specific locations within the story. These methods empower readers to independently explore and comprehend information within the narrative visualization.

Animated transition includes three types of changes: text, visual, and widget. This concept pertains to the dynamic transformation of text or visual elements in response to user input. Widgets are miniature applications that involve changes in text and images and enable direct users to participate, such as triggering interface changes through selecting menus. These dynamic changes play a crucial role in enhancing the interactivity of narrative visualization works, ultimately fostering greater user engagement and understanding.

Data source refers to the origin or provider of the data. Data sources can be categorized into official and unofficial sources. Understanding the data source helps to judge the reliability, accuracy, and authority of the data. In data visualization, clarifying the data source helps to increase the trustworthiness of the work, ensure the authenticity and reliability of the data, and follow data copyright and citation standards.

Narrative structure refers to the organization and arrangement of a story or information during the conveying process. It involves the beginning, development, and ending of the story and aims to communicate information effectively through a reasonable order, relevance, and emphasis distribution. Broadly speaking, narrative structures can be categorized into two types: linear and nonlinear.

Narrative theme type refers to the classification of themes or content involved in narrative visualization works. Narrative theme types can include social issues, historical events, technological advances, human stories, and various other types. Identifying the narrative theme type in narrative visualization assists designers in selecting suitable forms of expression and interaction methods to effectively convey information and engage the target audience.

Emotional feedback in the context of visualization refers to the emotional response that users experience during their interaction with the visualization. These emotional responses can range from positive emotions like joy, surprise, and interest, to negative emotions like anger and sadness (Lan et al. 2021). In narrative visualization, the goal of emotional feedback is to evoke emotional resonance in users by using effective visual expression, narrative structure, and interaction design, further enhancing the attractiveness and dissemination of the work.

4.3 Case analysis method

In this study, we analyze 10 narrative visualization cases that utilize eight key elements, discussed in Sect. 3.2.2. These elements encompass four crucial aspects of the user experience: sensory, interaction, cognitive, and emotional.

Sensory experience is related to how users perceive the visual elements in the visualization interface (Cai and Zhu 2010). Interactive experience mainly involves changes in the visualization view and browsing of information (Wu 2016). Cognitive experience refers to the subjective experience of individuals during the process of perception, thinking, and understanding (Tynan and McKechnie 2009). Emotional experience includes all the emotions that users experience while recognizing visualization information. To measure these experiences, we will use the following methods: analyzing visual elements and narrative visualization types to assess the sensory experience; evaluating navigation input and animated transitions to gauge the interactive experience; assessing data sources, narrative theme types, and narrative structure to measure the cognitive experience; and analyzing positive and negative emotions to evaluate the emotional experience. To evaluate the emotional experience conveyed by each case, we conducted interviews with 10 users. During the interview, users interacted with these 10 narrative visualization websites themselves. Subsequently, we interviewed the users and recorded the emotions they experienced during their interaction with the narrative visualization works.

5 Result

This study presents a comprehensive and in-depth analysis of ten award-winning narrative visualization works. The aim is to validate the practical application and effectiveness of the eight key elements in the narrative visualization model. Based on the data types of the 10 award-winning works, it can be seen that there are four themes related to the environment and public health (The New York Times 2020a, 2020b; ProPublica The Palm Beach Post 2021; Arab Reporters for Investigative Journalism (ARIJ) 2021), three themes related to the impact of digital technology on people’s lives (South China Morning Post 2020; Pointer (KRO-NCRV) 2020, 2021), and three themes related to violence and disasters (Pointer (KRO-NCRV) 2020; Al Jazeera Media Network 101 East, AJLabs 2021; The New York Times 2021).

This research summarizes the analysis results from four aspects: sensory, interactive, cognitive, and emotional experience, as shown in Table 2, based on the case studies.

Table 2 Analysis results of 10 cases
Full size table

5.1 Sensory experience

To create an optimal sensory experience, designers should prioritize two key aspects: visual elements and narrative visualization types. When it comes to visual elements, it is crucial to ensure consistency with the thematic content and maintain a unified overall style. In terms of narrative visualization types, designers must carefully select an appropriate display method based on the specific theme. Therefore, designers should first identify the desired experience effect that the theme aims to convey to users. Subsequently, they can choose a narrative layout method that aligns with the theme to maximize the user’s sensory experience.

5.1.1 Visual elements

To ensure coherence in visual design, elements should align with the theme and exhibit a consistent style. In data journalism works, bar charts, line charts, and pie charts are typically used, while ring charts, bubble charts, rectangle tree charts, and network charts may also be utilized (Kloop 2020). Maps are the most commonly used type of visualization and are found in almost all content related to regions or cities. The form of maps may vary depending on the theme. For instance, “Case ” (The New York Times 2021) and “Case ” (ProPublica The Palm Beach Post 2021) both employ maps. The former uses 3D modeling to showcase the city scene before the massacre, allowing users to engage with the page and understand the devastating loss caused by the event. In contrast, the latter employs semi-transparent color blocks on a map to demonstrate regional environmental pollution under different weather conditions. “Case ” (Arab Reporters for Investigative Journalism (ARIJ) 2021) offers insights into the city’s traffic conditions by utilizing satellite maps. They can serve as background information graphics that enhance the content’s understanding by incorporating annotations based on the map’s display purpose. The proficient use of maps in data journalism can help communicate complex information to users in an engaging manner.

To enhance the visual appeal of an infographic, designers can utilize techniques such as increasing color saturation and selecting images with minimal text to convey the story content effectively. For example, “Case ” (Pointer (KRO-NCRV) 2020) and “Case ” (Pointer (KRO-NCRV) 2021) both discuss the subject of “spreading false information.” These works employ colored backgrounds and vibrant color tones on the page to create a striking visual contrast with the main image. This technique enhances the audience’s long-term recollection of the works compared to other web pages.

5.1.2 Narrative visualization types

In this study, we analyzed 10 cases of narrative visualizations. We found these cases employed different types of narrative visualization to present thematic information based on the nature of the data. The most frequently used type was scrollytelling, which is suitable for displaying data news in long news reports. This format conforms to the logic of user operations, whether through scrolling with a mouse wheel or swiping up and down on a smartphone. To aid users in understanding their progress and the remaining content, a scroll bar should be displayed on the page.

In certain scenarios where complex and abstract concepts need to be conveyed, an infographic format can be employed. For example, in the work “Case ” (South China Morning Post 2020), an infographic was utilized to visually demonstrate the detrimental effects of prolonged phone use on posture by depicting people looking down at their phones. According to some interviewees, the inclusion of infographics in the content made them more inclined to explore the story. Furthermore, in “Case” (South China Morning Post 2020) and “Case ” (The New York Times 2020), data videos were also utilized, which would play automatically for brief durations of no more than 2 s when the user reached the relevant section.

In narrative visualizations, data comics and timelines are not as commonly used, likely for two main reasons. Firstly, data comics are a relatively new type of narrative visualization that has only gained limited popularity in recent years (Bach et al. 2018). Secondly, data comics are presented in a comic format, which might result in a visually disjointed experience for users (Wang et al. 2019).

5.2 Interactive experience

Interaction in narrative visualization design aims to minimize users’ thinking time and facilitate the quick acquisition of information. This can be achieved through direct interaction with the page or by presenting information through animated changes on the page. The goal is to enhance users’ understanding of the information and improve their overall interactive experience.

5.2.1 Navigation input

When it comes to navigation input, scrolling interaction is the primary method, supplemented by button interaction to operate the page. The design logic primarily depends on whether the page content needs to be paginated. Pagination is used to avoid difficulties in loading a single page when there is a large amount of data on the page (HKYoyo 2020). Another situation is to use button clicks to flip through pages to show different types of content for users to better understand. For example, in the work “Case ” (Kloop 2020), users can view different cases of women being killed by flipping through pages. Additionally, many cases utilize a combination of scroll and button interactions. Based on award-winning data news cases, pages that employ mixed interaction methods possess two characteristics. First, a single interaction method is insufficient to fulfill all the requirements of data presentation; secondly, authors want users to explore information through more interactive behavior in order to reduce browsing speed and view detailed information on the page more carefully. This design enables users to acquire a deeper understanding of the content by employing multiple interaction methods.

None of the analyzed works utilized the slider interactive mode. This could be attributed to the demanding precision needed for users to manipulate sliders, as well as the comparatively intricate design and implementation process involved. In contrast, buttons and scrolling typically offer more explicit navigation controls, such as page numbers or clearly labeled forward and backward buttons. The control of sliders can be more ambiguous, making it challenging for users to accurately determine their position and navigation direction. As a result, the absence of slider interaction is evident in these works.

5.2.2 Animated transition

Regarding animated transition, the focus is on visual element changes centered around user operations. In other words, when the user interacts with the page, the visual elements respond accordingly. For example, on a page containing a map, users can zoom in or out of the map through mouse operations and scroll to view detailed information on specific areas. There are mainly two forms of text changes: displaying related content in the form of annotations when the mouse hovers over a certain area, and adjusting the text color or size according to changes in the image. For example, in the work “Case” (The New York Times 2020), the font type and background color change accordingly as the city changes.

In data news pages, widget interaction is rarely used because this type of interaction usually requires the user to click a button to trigger the corresponding action, which may interrupt the user’s reading experience. Hence, when designing motion effects, it is advisable to keep them simple and clear, avoiding excessive use of animation effects that could potentially distract users’ attention.

The animated transition should be consistent with the overall design style to ensure coordination and unity of the page’s visual effects. Additionally, it is important to pay attention to the response time and method of page motion effects. According to research, the optimal duration of motion effects is between 200 and 500 milliseconds, taking into account human brain cognitive patterns and information processing speed (Google 2014). During the design process, it is recommended to minimize unnecessary motion effects and overly complex visual elements to ensure fast page loading speed.

5.3 Cognitive experience

To maximize the cognitive experience and effectiveness of data journalism, it is crucial to carefully balance three key aspects: data sources, narrative theme type, and narrative structure.

5.3.1 Data source

Data journalism relies on official published data or data collected through field investigations. The accuracy and truthfulness of the data, regardless of its source, are of utmost importance.

Most data journalism works involve researchers collecting data themselves. This is mainly because data journalism focuses on topics related to social livelihood, environment, health, military, education, and other aspects, where much of the information is difficult to obtain through official channels. For instance, military-themed data journalism works such as “Case ” (Al Jazeera Media Network 101 East, AJLabs 2021) and “Case ” (The New York Times 2021) require gathering private information, which often involves conducting interviews and employing other methods. For information from many years ago, researchers need to carefully study existing old photographs, briefs, and other archival materials to verify data accuracy. In relation to urban environment topics, such as “Case ” (Arab Reporters for Investigative Journalism (ARIJ) 2021), designers often face a challenge of limited official data availability. Therefore, they resort to collecting data on-site using diverse methods such as photography, filming, questionnaire surveys, and interviews. On the other hand, for topics closely connected to the users themselves, like “Case ” (The New York Times 2020), “Case ” (Pointer (KRO-NCRV) 2020), and “Case ” (Pointer (KRO-NCRV) 2021), researchers opt for observation and online surveys to conduct thorough investigations on users.

Data journalism encompasses a wide range of topics and relies on diverse sources of data. Its primary goal is to shed light on social phenomena and address pressing issues from a fresh and comprehensive perspective. In order to achieve this, researchers must prioritize the accuracy and truthfulness of the data they gather, ensuring that their work maintains objectivity and credibility.

5.3.2 Narrative theme type

The essence of data journalism lies in delivering information that aligns with users’ cognitive patterns. Award-winning data journalism focuses on identifying social issues and capturing widespread attention through innovative perspectives and presentation methods. For instance, in the work “Case ” (The New York Times 2020), the report revealed disparities in environmental pollution problems faced by people in the same city due to economic status differences. This was achieved by documenting the real-time experiences of two children from different families for 24 h. The research team aimed to enhance users’ attention to environmental pollution issues and evoke emotions through intuitive and perceptible forms, starting with a human story. As the New York Times team stated, “if we cannot make data intuitive and perceptible, then this data is meaningless (Brooks 2013).”

Moreover, the analysis of these works reveals that award-winning data journalism often lacks timeliness. It frequently addresses enduring social issues or summarizes past events rather than focusing on recent occurrences. The production of high-quality narrative visual data journalism requires advanced technical skills, lengthy production cycles, and relatively high costs. However, exceptional data journalism pieces typically provide comprehensive information and critical analysis of events, while also resonating with readers.

5.3.3 Narrative structure

After analyzing data journalism works, it has been observed that there are more works that employ nonlinear narrative structures compared to those that use linear narrative structures. Lan et al. (2021) pointed out that nonlinear narrative structures are more effective in capturing user attention, and these data journalism works have provided evidence to support this claim. Analysis of these works reveals that nonlinear narrative structures are commonly utilized for topics that do not have a clear chronological aspect, such as investigations into specific groups or behaviors. Additionally, nonlinear narrative structures are frequently employed when telling stories for targeted audiences. In the works “Case ” (ProPublica The Palm Beach Post 2021) and “Case ” (Arab Reporters for Investigative Journalism (ARIJ) 2021), the authors intend to draw the attention of government officials toward the corresponding issues and devise solutions. Both of these works start with a real and startling incident, and then examine the underlying causes, using nonlinear narrative structures to captivate the officials’ attention.

Nonlinear narrative structures defy the constraints of time and space, traversing narrative dimensions of both time and space (Hua 2021). These structures not only heighten the suspense of the story but also satiate the audience’s curiosity. However, as a consequence, nonlinear narrative articles tend to be lengthier compared to linear narrative ones. This is likely due to the fact that nonlinear storytelling disrupts the logical flow of events, making it challenging to effectively convey the narrative thread within a shorter length, thus impeding the exploration of the story. Hence, in narrative data journalism, if the story possesses a distinct time attribute or necessitates a concise explanation, it is advisable to minimize the usage of nonlinear narrative structures.

5.4 Emotional experience

In data journalism, emotional experiences at the perceptual, interactive, and cognitive levels are crucial for attracting user attention. Negative emotions have the power to capture attention and promote profound reflection. However, it is equally important to strike a balance and prevent overwhelming users with excessive negative emotions. To address this, innovative and captivating presentation formats can be employed to deliver content.

5.4.1 Emotional feedback

This research primarily examines the award-winning works of narrative visual data journalism, which primarily address contemporary social issues to engage readers. For example, in the case of “Case ” (The New York Times 2020),” several interviewees mentioned that it was their first exposure to this type of news reporting, and the distinct visual experience left a deep impact on them. The smooth interaction also provided them with a positive experience. However, six respondents expressed feelings of helplessness and unease after reading the news content. For example, respondent 2 expressed a sense of helplessness regarding the wealth gap in society and showed concern for the impoverished population in New Delhi, India. Likewise, respondents 7 and 8 also experienced unease after reading the article, as it highlighted the severe environmental pollution in their cities. The feedback suggests that respondents thoroughly considered the provided information in light of their specific personal circumstances.

Some studies utilize negative emotions in their design to capture users’ attention toward important issues. For example, in the case of “Case ” (Arab Reporters for Investigative Journalism (ARIJ) 2021), real-life maps are used to illustrate the city’s traffic conditions, highlighting the detrimental impact of road traffic problems on people’s lives. seven participants acknowledged that such works offered them a diverse emotional experience. They also acknowledged the potential of exceptional data journalism pieces to make a positive difference in people’s lives and instill hope in those facing challenges.

However, it is important to present the overall content in an engaging format to mitigate the negative impact of works that evoke negative emotions on users. For example, the study by “Case ” (Al Jazeera Media Network 101 East, AJLabs 2021) showcases the disasters brought by the military dictatorship to the country and its people through on-site photographs. Respondents 3, 7, and 9 all noted that after reading this piece, they felt deeply suppressed and saddened as if experiencing a suffocating sensation. They also expressed reluctance toward viewing this form of visual representation. On the contrary, when discussing authoritarian rule, respondent 4 mentioned that after reading “Case ” (The New York Times 2021), the interesting interactive approach and presentation format kept them engaged in viewing the entire content. They further added, “If it were a plain textual description, the information on this topic would have oppressed me, and I wouldn’t have completed reading it.”

In the analyzed award-winning cases, works that evoked negative emotions were more prevalent. This is because these works focused on social issues, and negative emotions proved to be more effective in capturing attention. Research has demonstrated that negative emotions enhance immersive experiences and facilitate the formation of long-term memories (Lan et al. 2022). Hence, it is not always the case that works with positive emotions are superior, and not all works need to convey positive energy. Utilizing negative emotions can stimulate critical thinking and closer attention to the presented information.

6 Refining a user experience-based narrative visualization model

Based on the research findings, this study presents an optimized user experience-based narrative visualization model, as shown in Fig. 2. The model is divided into three parts vertically. On the left side, it illustrates the four stages that designers go through during the process of creating narrative visualization, from bottom to top. On the right side, it describes the four stages and four types of experiences that users encounter when engaging with visual works, from top to bottom. The intermediary elements represent the eight key elements that designers should prioritize when creating narrative visualizations. These elements are also the core factors that influence the user experience.

Fig. 2
figure 2

A user-experience-based narrative visualization model for data journalism is presented. The left side illustrates the four stages (from bottom to top) that designers go through in creating narrative visualizations. The right side describes the four stages that users experience when engaging with narrative visualizations, as well as the four types of experiences they gain during this process (from top to bottom). The intermediary elements represent the eight key elements that designers should prioritize when creating narrative visualizations. These elements play an important role in shaping the result of user experience

Full size image

The model is enriched and expanded on four planes: strategy, scope, structure, and surface. These planes are interconnected, as shown in Fig. 2. By optimizing this model, designers can better understand how to improve narrative visualization works and enhance users’ experience. Furthermore, the model highlights the relationship between design elements and user experience, offering valuable insights and guidance for future research and practice.

7 Evaluation

To further confirm the applicability and effectiveness of this model, we organized a series of expert interviews. We invited six experts with extensive professional knowledge in their respective fields to participate in our study. These experts cover all the key areas of the model, including two interaction design experts who have a deep understanding of the user interaction experience with visualization, two visualization design experts who excel in displaying data through visual elements; a data story creator who specializes in combining data and story elements to create compelling data stories; and a data analyst who has deep knowledge of data processing and analysis. By incorporating feedback from these experts, our aim is to gain a better understanding of the model’s strengths and limitations, as well as its practical applications.

Interaction design experts: With the use of this model, I can systematically consider the user’s interaction experience with visualization. Interaction design plays a crucial role in data visualization. This model provides a new way of thinking that allows us to better understand how users interact with data. Through the model’s prompts, we can also better consider user needs and behavior patterns. The model emphasizes the importance of data filtering and querying, which reminds us to pay more attention to user browsing and search behavior in design. We can incorporate features such as search boxes and filters to facilitate users in finding the data they are interested in and reduce their information acquisition costs. In addition, through the interactive way, users can more deeply and intuitively understand the data, which enhances the user’s sense of engagement and user experience.

Visualization design experts: As a practitioner in visualization design , I have previously focused primarily on selecting colors, shapes, and layouts during the design process. However, I now realize that I may have included too much information in a single chart, which could overwhelm the user’s cognitive load. This model provides a comprehensive and detailed description of the user’s experience. It encourages me to consider not only my design solutions but also the user’s interaction, cognition, and emotions. In future designs, I will pay more attention to simplifying and presenting data clearly. Furthermore, in my designs I will take a holistic approach to consider how users interact and match corresponding visual designs based on their interaction preferences, aiming to achieve better user experiences.

Data journalist: This model provides a comprehensive framework for me to better understand and consider the key elements in creating data stories. The model emphasizes the importance of narrative structure, making me more attentive to integrating data and storytelling to create captivating stories. For example, after learning from this model, I will prioritize data acquisition, utilizing data to support the theme and plot of the story, and using storytelling to explain and present the data. The model also highlights multiple levels of user experience, prompting me to focus on designing and optimizing data stories to provide better sensory, interactive, cognitive, and emotional experiences. This approach is more comprehensive than relying solely on data and storytelling elements because it allows me to systematically consider and enhance user experience.

Data analyst: Previously, my data analysis focus was primarily on presenting data clearly to uncover hidden insights. However, I neglected to systematically consider presenting data as a story or the cognitive load that excessive data might impose on users. This model has highlighted the issues I overlooked in the past, where I was too focused on my own field and neglected the user’s cognitive experience. After understanding this model, I will adopt a more comprehensive approach that takes into account the impact of data presentation on user cognition, aiming to maximize understanding of the data while meeting user experience requirements.

However, some experts have pointed out that there are areas for optimization in this model. For example, the model covers a wide range of considerations. While relying on this model can help us create error-free works that align with user experience, it may be challenging to create a captivating masterpiece solely using this model . Additionally, some experts have suggested that the model’s complexity could be reduced by dividing it into three separate models for better comprehension . These are important considerations for our future research, and we will continue to explore and enhance this work.

8 Conclusion

This study aims to improve the user experience of viewing narrative visualizations by proposing user experience-based narrative visualization strategies to assist designers in creating visualizations from a user perspective. To accomplish this, a visualization model based on previous studies was established, and eight key elements that impact the user experience in narrative visualizations in data journalism were identified.

This study holds both practical and theoretical significance. On a practical level, it has constructed a narrative visualization model based on user experience, which can guide designers in creating data journalism works for narrative visualization. The design elements of this model can serve as inspiration for future research. For example, it was found that moderate negative emotions can enhance user attention, but excessive negative emotions may affect users’ attitudes toward viewing the work. Therefore, further investigation is needed to balance the proportion of negative emotions. It is important to note that the sample size of award-winning narrative visualizations analyzed in this research from 2020 to 2022 was relatively small, which may limit the generalizability of the design guidelines obtained. Future research should aim to include a wider range of narrative visualizations, increase the sample size, and identify techniques that can be universally applied to enhance the user experience and broaden the scope of research.