1 Introduction

A story refers to a series of ordered events that occur spatially and temporally [1]. In a story, there are complex relationships, such as causal relationships between events, collaborative relationships of entities and hierarchy relationships in locations. The skill of understanding dynamic relationships in a story is very important in many domains, such as in analysing social relationships, managing team cooperation, analysing narrative methods and watching movies and fiction. How to describe a story with abstract expression and in an analysable way is a popular problem and was the initial motivation for our work.

To help people better understand and analyse a complex story, researchers have proposed Storyline visualization techniques [2]. The automatically generated layout of previous research has achieved good results [3, 4], especially the StoryFlow visualization method [2], which generates a visual view in real time and takes hierarchy relationships into account. These techniques describe dynamic social relationships along the horizontal timeline. Visualization forms of non-horizontal timelines are presented in order to express the special characteristics and structure of a story, for instance, a tree-ring metaphor [5], time ring [6] and time folding [7].

The narrative structure of a story, such as a film, play or novel, contains specific plots, themes and resolution. In addition to the sequential narrative, flashbacks and interlaced narratives are also two widely used narrative methods. Due to the nonlinear timeline in Storyline visualization, the lines of flashback located in the visualization view are inappropriate. As a result, the key is to find a new visualization method that both effectively reflects the dynamic relationship between the characters and the association of driving events and supports the different narrative structures and methods.

By analysing and summarizing the rules of novels, script writing and film-making [8], we found that scenes are very common in film-making and fiction; thus, the method proposed to split the story into multiple sections is reasonable and effective. Based on these characteristics and previous research, a hierarchical plot visualization method in polar coordinates, StoryCake, is proposed in this paper. This method uses a fan-shaped design and a semi-circle timeline. Compared with previous works, it creates a hierarchical storyline layout to illustrate the relationships of hierarchies to help people better analyse and understand the story evolution. It is effective for screenwriters, game designers and movie or anime fans to analyse their favourite works. It could be an auxiliary tool for a project leader to manage collaboration between team workers in a project procedure or for analysts to analyse changes in social relationships in time and space. In addition to the linear narrative structure, the result view can show nonlinear structures. For example, the events in a flashback or interlaced narrative will be displayed in real time rather than the development time of the plot. To help people better analyse story evolution, interactions of line integration and session folding are provided. We will explain the method in detail in the following sessions.

Our work makes the following contributions:

  1. (1)

    A hierarchical plot visualization method to demonstrate the relationships of hierarchies in the stories clearly through the partition of session blocks.

  2. (2)

    A visualization method to demonstrate nonlinear narrative structures based on dividing the scenes and rich tags in the visualization view.

  3. (3)

    An interactive design to provide good scalability for storylines with increasing time-steps and individual lines.

2 Related work

2.1 Time-oriented event visualization

Commonly, time-oriented events can be classified into point-based events and interval-based events. The former are considered instants in time, and the latter are events that last for a period of time.

In visualization, point-based events are encoded as a point or a node with height. For example, a triangular-shaped point was used to represent the events of a patients electronic medical records [9]. Characteristics of events are visualized by node size and node colour (e.g., [10,11,12]). Interval-based events are generally represented by a line segment on the time axis. A Gantt chart is commonly used for visualization of interval-based events (e.g., [13,14,15]). Similarly, size and colour of lines are given specific meanings (e.g., [16,17,18]).

The works mentioned above are based on linear timelines. Many researchers have proposed nonlinear timelines to describe combined characteristics of events. For instance, Bach et al. [7] folded the timeline to explore event similarity. Thern [5] presented a tree-ring metaphor to display the hierarchy of events. A circular timeline is used to visualize an individuals daily life [19] and a famous persons lifetime [6]. Choosing appropriate timelines and consideration of time characteristics and space utilization enable a story to be better understood and space well used.

2.2 Storyline visualization

Storytelling visualization can be understood as visualizing a data story [1], which helps people discuss problems [20], analyse data [21], display a story [2] and create a story [22]. On the issue of displaying a story visually, researchers have carried out extensive exploration and research, such as ThemeRiver [23], OutFlow [24] and EvoRiver [25]. The above methods are used to explore contact between events, but they do not effectively show the relationships among entities. Relatively speaking, Storyline visualization technology has great advantages in representing the dynamic relationships between entities [26]. Inspired by XKCDs hand-drawn illustration Movie Narrative Charts [27], many researchers have conducted deep studies and extensive applications. Ogawa and Ma [3] formulated the rules of Storyline automatic layout and put forward a weak component clustering algorithm and a greedy algorithm to create the Storyline automatic layout. Tanahashi and Ma [4] adopted the genetic algorithm and introduced the concept of an interactive session.

Fig. 1
figure 1

The process of our method to implement the layout

Full size image

Based on previous research, the StoryFlow visualization method proposed by Liu et al. [2] can optimize a layout in a shorter time. Although this technique has achieved better results in visual effects and time complexity, it cannot reflect the story structure and plot changes, and it only supports Storyline visualization of linear continuous stories. It will be a great challenge for linear visualization to demonstrate its structure if montage is used. A horizontal view cannot effectively demonstrate the changes of the plot. In addition, a large number of lines may lead users to lose their focus.

2.3 Hierarchy data visualization

The hierarchy visualization technique is very important due to the iniquitousness of hierarchical data structures. Many researchers have conducted deep studies and evaluations; most of the research was concentrated on the challenge of displaying large hierarchies in a comprehensible form [28].

In all of the results, there are improved node and link diagrams, cone trees and their variations. However, the radial, space-filling (RSF) hierarchy visualization has been proven to be a useful approach. It has the advantage of using the display space efficiently while conveying the hierarchical structure in the view over other space-filling techniques [29]. However, an inherent drawback of the RSF technique is the loss of important information of the hierarchy. To solve this problem, Andrew and Heidegger [30] proposed Information Slices, which use cascading semi-circular discs to compactly visualize large hierarchies. The InterRing proposed in [31] and the FanLens proposed in [29] achieve better visualization quality by providing powerful interactive distortion functions, including circular distortion and radial distortion.

RSF techniques can demonstrate hierarchical data using the fan-shaped word as a visual element. The Storyline visualization technique is the most popular method for demonstrating dynamic relationships in a small group. To better present nonlinear information and preserve users mental maps, we proposed the hierarchical plot visualization method in polar coordinates, StoryCake, combined with the ideas of RSF techniques. In addition, we added tags in visualization view with the layout method of RSF techniques.

3 Summary of the method

We use XML files that describe characters and their relationships in a movie as the initial data. The data format is the same as it is used in reference [3]. A hierarchical tree, a location table and the interactive session table are built to record the dynamic relationships among characters. In visualization, entities are encoded as individual lines, and a session is encoded as bundled segments. We focus on optimizing the layout of the visualization view as well as line geometry to explore the development of plots and the relationships among characters. Figure 1 shows the process of our method to implement the layout of the fan-shaped visualization. The process contains three major parts: partition and computing of sessions, line layouts, coordinate computing in polar coordinates and adding tags and interactions. For details, we describe our method as follows: Sect. 3.1 introduces visual encoding, Sect. 3.2 is the main part of the layout strategy, and Sect. 3.3 describes the user interaction.

There are many sessions in the initial file. Each session contains five properties, including the ID of this session, the IDs of members that are in touch with each other during two adjacent time frames in the session, the start time frame, the end time frame and the location ID in which it occurs. Each session can be regarded as an event in a story. The hierarchical tree represents the hierarchical relationship of the story, which determines the positions of the sessions. Each tree node represents a location and includes all the session IDs. These nodes form the relationship tree, which defines the dynamic hierarchical relationships between entities at individual time frames. Each keyword as a label in the location table assists users in better understanding the visualization view.

3.1 Visual encoding

According to the design principles of the traditional Storyline visualization [2], we can obtain an initial layout that cannot reflect those stories with nonlinear narrative structures. To solve this problem, we add a new dimension of hierarchical division on the basis of the traditional Storyline visualization using the narrative structure in the literature and film-script writing as a reference. To help users understand the view of the visualization, we add tags to different stages of the story. Considering that the horizontal view cannot demonstrate multilayered stories and integrate information effectively, a new fan-shaped and hierarchical visualization method is proposed. Here are some principles:

  1. (1)

    Lines drawn along the direction of the arc represent entities, and the central angle of the arc represents the life cycle of an entity.

  2. (2)

    Multiple arc segments bundled together in a period of time represent those entities which the arcs represents interacting with each other. The convergence and separation of the arcs represent the start and the end of the interactive session, respectively.

  3. (3)

    The hierarchical relationship of session blocks determines the radius of the fan-shaped view.

  4. (4)

    Session blocks in different layers are filled with different colours so that diverse levels can be clearly distinguished.

  5. (5)

    The fan-shaped block corresponds to the tag in that area.

3.2 Layout strategy

Based on the design principles presented in Sect. 3.1, we describe the main layout strategy in this section. As Fig. 1 illustrates, this part contains five detailed steps: partition of session blocks, determining the layer of session blocks, layout adjustment of entities in session blocks, calculation of the line coordinates and adding tags. Each step is discussed in the subsection below.

3.2.1 Partition of session blocks

In this step, the rule of dividing session blocks is instituted by referring to the development of the plots; the narrative technique, the time span, the spatial span, the narrative structure and the emotional changes also influence the rule. The tree structure that is generated by the characteristics, for example, the inclusion of time or space, of the story itself is applied to store the session blocks and the hierarchical relationships among them. We assume that a film is a root node, and multiple session blocks are child nodes according to the film’s plot. For instance, the movie StarWars would be divided into seven session blocks corresponding to seven planets in the movie, and all of the sessions occurring on one planet would be represented by the corresponding session blocks.

3.2.2 Determining the layer of session blocks

The hierarchical division helps express the structure and hierarchy of the story more clearly, and the optimization of the layout makes the view more readable for users to understand. In general, people prefer to get information from the centre to the circumference of the circle. Thus, the session blocks that are closer to the centre of the circle are more relevant to the tags. Additionally, we set an attribute named level for every session block, and the value of each level is dependent on the length of each session block. For example, the session block with the longest length has the minimum value 0, and that with the second has a value 1.

The position of a session block is determined by the value of level. Different levels result in different positions, and the session blocks may be distributed in different layers; this is a waste of space. Therefore, compressing the layers is very necessary. The session blocks are put on the same level unless they overlap adjacent sessions. The process of compressing is as follows:

step1: :

For the level of each block, perform the operation: \(level=level-1\);

step2: :

Judge whether the level is greater than \(-1\). If true, go to step 3; otherwise, go to step 4;

step3: :

Check whether there are session blocks overlapping with each other. If true, go to step 4; otherwise, go back to step1;

step4: :

\(level=level+1\), end.

This procedure is schematically shown in Fig. 2 as an example. After compressing, wasted space is reduced, and the initial relative position of session blocks is also maintained. In most cases, our algorithm is effective. However, there is an extreme case that if the session blocks overlap with each other in the beginning, they will retain their initial layer even after compressing. Considering the narrative of the story, this case is rare, and the view in this condition does not affect the display.

Sometimes, dividing sessions purely on the basis of the length is incomplete. Other logical relationships between two session blocks, such as causality, contain, progressive and parallel, also need to be taken into account. For instance, in the movie Inception, the containment relationship of dreams is clear to see. Hierarchical relationships will be determined by the block hierarchy tree. The level can be assigned by referring to the data in the table of the hierarchy tree.

Fig. 2
figure 2

Determining the layer of session blocks . a Initial layer layout depended on the length of each session blocks, in this example, the session block with the longest length has the minimum value 0, b compressing steps, for each session block, compress it down to lower layer if there are no session blocks overlapping with it, c the final result after compressing

Full size image

3.2.3 Layout adjustment of entities in a session block

The major goal of our layout optimization is to minimize the following four metrics: line crossing numbers, wiggle numbers, wiggle distances and white space. For each session, the optimal layout of entities will be generated after the steps of sorting, aligning and compressing with the StoryFlow method. The StoryFlow method is applied to each session block to obtain their positions in horizontal coordinates.

Sorting To reduce line crossings, the sessions and entities at successive time series inside the layer will be sorted with an extended DAG sweeping algorithm [2]. This algorithm contains two main parts: The first part produces an initial layout considering the hierarchical relationships and the layer of session blocks. Then, the second part iteratively reduces the number of crossings by carefully adjusting the order of visual elements. In each iteration, we sweep the session segments in each time-step back and forth. The detailed steps of sweeping are as follows:

step1: Ordering the entities at the initial step \(t_{start}\, (t_{start}>0)\) of session block \(V_{L}\), and \(L=\{l_{1},l_{2},\dots ,l_{n}\}\) is the entity list to be sorted. We define a comparator to determine the rule of sorting. With this rule, the entities will be sorted in ascending order. The comparison function of the comparator is as follows:

figure d

where \(l_{i}\) represents character number, and \(t_{n}\) represents time. We define Y(\(l_{i}\), \(t_{n}\)) to represent the level value of the session block which character \(l_{i}\) belongs to at \(t_{n}\).

step2: The order of the current step that is determinate may be used as a reference to sort the lines of the next step with the barycentre method. If the current step is the initial step of the session \(\{V_1,V_2,\dots ,V_m\}\), jump to step3. This process will be repeated until the last step or the end of the successive time series.

step3: Order the session inside the layer by the weight of a session node. The weight of session \(V_i\) can be calculated as follows:

$$\begin{aligned} weight(V_i) = \sum _{l\in (M(V_i))} weight(l)/count(M(V_i)) \end{aligned}$$
(1)

The l in the formula above indicates the entities in the session \(V_{i}\). We can use the barycentre method to order the sessions according to their weights.

step4: The positive sweep is completed.

The steps of reverse sweeping are similar to those of positive sweeping. The initial step is the last step of session \(V_{L}\). We iteratively improve the ordering result by sweeping back and forth until the number of line crossings reaches a stable value or the number of iteration reaches a maximum value. To shorten time-steps, we can set the maximum value of iterations smaller. In our implementation, the maximum iteration number is set at 10.

Aligning and compressing To reduce the wiggles of lines between sessions, we align the sessions in the same layer. We can get a session cluster {\(cluster_i|i\in [1,n]\)} for each layer using the weighted Longest-Common-Subsequence. \(cluster_i\) contains the sequence of sessions to be aligned \(\{V_{1},V_{2},\dots ,V_{n}\}\).

To minimize the unnecessary empty space caused by line wiggles, we calculate the wiggle distance between the adjacent sessions \(V_L, V_R\). \(S_L=\{l_{l_1},\dots ,l_{l_m}\}\) and \(S_R=\{l_{l_1},\dots ,l_{l_n}\}\) represent the entities sets of \(V_L,V_R\). Suppose that \(l_{li},l_{rj}\) are the first pairs of lines to be aligned; then, the distance of the wiggle is calculated as follows:

$$\begin{aligned} d\_value= & {} (l_{l_i}\cdot t+(l_{l_i} + 1)\cdot t_{space})\nonumber \\&-(l_{rj}\cdot t+( l_{rj} + 1)\cdot t_{space}) \end{aligned}$$
(2)

Then \(cluster_i\) will compact with the non-aligned session that we call seExtra, and the top coordinate top(i) of \( cluster_i\) will be obtained. We will calculate every coordinate of sessions in the cluster \(cluster_{i}\) according to top(i) and \(d\_value\).

3.2.4 Calculation of the line coordinates

In this paper, we choose a fan-shaped mapping with a central angle of \(120^{\circ }\) because we have determined that the running time of a film is approximately 120 min. Furthermore, reading the fan-shaped mapping will be more difficult if the degree is too large or too small. Although the time-steps of diverse films are uncertain, we can adjust the unit angle of the time-step. Here, the unit angle of each time-step equals \(120^{\circ }\) divided by the total steps. In addition, an appropriate radius can be obtained according to the size of the screen. We specify the centre of a circle named O as the polar circle centre dot and the start edge named OX as a polar axis whose central angle degree is named \(\theta _0\). We regulate the clockwise direction as the positive direction for the polar coordinate system. Each session and entity in the view should be armed with three information elements: the start angle \(\theta _{1}\), the end angle \(\theta _{2}\) and the radius r of the arc. The formulas for \(\theta _{1}\), \(\theta _{2}\), and r are as follows:

$$\begin{aligned} \theta _{1}= & {} \frac{start}{allsteps}\times \frac{2\pi }{3}+\theta _0 \end{aligned}$$
(2a)
$$\begin{aligned} \theta _{2}= & {} \frac{end}{allsteps}\times \frac{2\pi }{3}+\theta _0 \end{aligned}$$
(2b)
$$\begin{aligned} r= & {} {\left\{ \begin{array}{ll} y+R_0 &{} level=0\\ y+max(r^2) &{}level>0 \end{array}\right. } \end{aligned}$$
(2c)

Above, the variables start and end represent the start and the end of a session, respectively. \(R_{0}\) is the initial radius of the 0th layer. The \(max(r^2)\) indicates the maximal radius of the other layers, and the y coordinate in the view was calculated in the Sect. 3.2.3.

3.2.5 Adding tags

A keyword tag represents a continuous sessions information and reflects the changes of theme, time, location or other elements in the story. In this paper, the keyword tags are arranged in an arc near the centre to show structural and location information. To get a compact layout, we calculate the normal vector of the ring and the coordinates of each letter referring to paper [32]. The essential part of this work is to select tags that conform to the completeness of time and that can fill a 120\(^\circ \) sector when grouped together. The group with more tags is closer to the centre of the fan-shaped visualization view.

However, the remaining tags are sorted according to the span of their time and will be drawn in order. The time for adding tags is mainly spent seeking the group with completeness of time, which is then placed innermost. It is necessary to try each tag which will possibly be a member of the group. Its time complexity is \(O(n_{t}^{2})\), and \(n_{t}\) represents the number of tags.

3.3 Line integration and user interaction

To help users better understand the changes of story structure and complex character relationships, we also provide a group of user interactive methods.

Line integration The visualization view may still be confusing when there are many characters. To solve this problem, we use the method of Level-of-Detail (LOD) to integrate the entities in successive sessions in this paper. We use a thick line with a certain width to represent the entities in sessions. The colour is determined by the superposition of colours of each entity, and the width is determined by the number of entities. The ligature between sessions is transited smoothly with nonlinearity interpolation. As shown in Fig. 3, the bundled lines are flowing when there are many entity lines. Therefore, users can switch to the visualization view with line integration when they are interested in changes in global relationships rather than detailed information.

Fig. 3
figure 3

The visualization view without line bundle (a) and with line bundle (b)

Full size image

Folded session block The session blocks are put in the view in order of the session happened in the story. However, the order of the events in the story may not be the real development of the story itself. The authors and directors may use skills, such as flashback or other types of narratives, to raise the interest of readers and viewers. These types of narratives may increase the difficulty for storyline users to focus on the plot in continuous time in the real story. In this paper, referring to fan metaphors, the session block folding mechanism is presented to solve this problem, as shown in Fig. 4. The session blocks that the users choose will be folded by the mechanism such that users can more easily concentrate on the story.

Fig. 4
figure 4

The result of folding different session groups. Show the session block in dreams, reflect the inclusion relation between dreams, strengthen the users attention to the dynamic relationship of characters inside dreams

Full size image

4 Results and discussions

As shown in Figs. 5 and 6, we tested our method for visual analysis of the movie Inception and the novel KingLear. In the process of StoryCake, layout for session blocks, sorting inside sessions and arranging keyword tags are the three main time-consuming portions. The time in layout for session blocks is negligible because the number of blocks is much smaller than the number of interactive sessions. The time complexity of this method is expressed as \(O({n_{e}}^2T+T^{3}+{n_t}^{2})\), where \(n_{e}\), T and \(n_{t}\) represent the number of sessions, time-steps and labels, respectively. Compared with StoryFlow [2], this method only spends more time on label arrangement.

Fig. 5
figure 5

The visualization view with our method using the experimental data of the novel King Lear

Full size image
Fig. 6
figure 6

The visualization view with our method using the experimental data of the movie Inception

Full size image
Fig. 7
figure 7

The visualization view with StoryFlow method using the experimental data of the movie Inception

Full size image

Case study 1: King Lear novel The data for KingLear involve 489 time-space, 119 interactive sessions, 14 entities, 4 levels and 10 location blocks. The result is shown in Fig. 5. From the figure, we learn that most characters are gathered together in the first scenarios. Then, the story develops along two main lines in GLOUCESTER’S CASTLE and ALBANY’S PALACE. Second conflicts among all entities occur in the middle of the story at GLOUCESTER’S CASTLE. In the last part of the story, there is a war between the French camp and the British camp. We can see that entities cross between location blocks, and there are many dynamic relationships. In summary, development of the story and relationships between entities are well described with our method.

Case study 2: Inception movie The data for Inception comprise 491 time-space, 116 interactive sessions, 10 entities, 5 levels and a number of keyword tags. The film is divided into 12 blocks according to location. Referring to Fig. 6, in the beginning of the movie, the director leads the film viewers to two layers of dreams in order to make them interested in the movie. Then, he explains the concept of inception in the story through four short dreams, as the blue blocks shown in the second layer. Finally, the story reaches a climax in a multi-level dream just as the stacked blocks in last part of the view. In conclusion, the narrative structure of the story has been demonstrated, and the simultaneous hierarchical relationship of dreams performs better.

Sessions in different blocks make it easier for users to locate the plot and observe the dynamic relationships of characters. Although the block mechanism increases line crossings, wiggles or blank spaces, these occur when the character relationships and plot developments change. Therefore, if we attach no significance to the wiggle distance, we can reflect the plot development of stories at the expense of increasing the wiggle distance. Reviewing Fig. 7, although the horizontal thrusts of the lines are regular, they cannot show the contacts of storyline and dynamic entity relationships. As shown in Fig. 7, it is difficult for people to associate a line with the corresponding plot. Readers must search from the beginning if they want to explore changes in relationships. For example, in Fig. 6, there remains only one entity line both in FISCHERs DREAM and FISCHERs DREAM LAYER 2. The first shows that Yusuf (the green line) is responsible for carrying COBB, who escapes the unconscious Defender into lower dreams. The second corresponds to the plot that Arthur (the deep brown lines) was left to wake teammates up, but we cannot get these through Fig. 7.

If the user wants to display a layer individually or does not care about the flashback sessions, he can choose to fold the unconcerned sessions. Figure 4 shows the result of folding session blocks in the 0 level and reflects the inclusion relationship between dreams to enhance users’ concerns of the dynamic relationships of entities within dreams. Folding interaction techniques can help users to analyse the story, especially parallel narrative and flashbacks.

5 Experiment evaluation

5.1 Comparison to visualization designs

A good visual design for storytelling should be able to help users track how the story evolves and analyse the dynamic relationships between characters in a story. Given that the current visualization methods, such as Storyline [3, 4] and StoryFlow, focus on presenting an efficient approach to generate an aesthetically pleasing and legible storyline visualization, it is challenging to illustrate the evolution of stories with a flashback or a narration interspersed with flashbacks. In contrast, using our method, in which the location dimension is considered in the visualization view, users are allowed to follow the hierarchy to better understand the stories. There may be graphic designs that can be used to show story evolution. For example, the graph could have: (1) a horizontal or fan-shaped timeline, (2) a discrete or mixed location layout or (3) a poor or abundant text explanation. The theoretical analysis below shows the advantages.

Horizontal versus Fan-shaped The timeline was linear in previous works. However, rendering the visualization view along a horizontal timeline results in a serious visual cluster when the number of time-steps increases to a great extent. The fan-shaped timeline is a nonlinear timeline; it can show nonlinear information as well as preserve users mental maps. The radius of the fan-shaped view is adjusted adaptively according to the number of time-steps and therefore has good scalability for increasing time-steps.

Discrete versus Mixed The location layout in recent research on Storyline visualization was compact, resulting in mixed visual perception when users analyse the evolution of hierarchies. In contrast, the discrete location layout in a fan-shaped visualization view can clearly show the relationships of hierarchies.

Poor versus Abundant In the visualization field, captions and text-interpretation are very important to help users better understand the visualization view. In our results, in addition to the labels that describe the individual entities and scenes, keyword tags are arranged in the arc area near the centre of the view to reflect structural and location information.

Therefore, what kind of visualization design will be more popular? To this end, a comparative experiment was performed as follows.

5.2 Experimental design

5.2.1 Subjects and procedure

We invited 27 participants as our respondents, including 4 doctors, 16 masters students and 7 undergraduates. They came from different subjects, and their ages ranged from 21 to 30 years old. Approximately 85 per cent of them were involved in our research field for the first time. Therefore, they were trained to use and become familiar with the two visualization designs shown in Figs. 6 and 7 before the study was conducted.

Table 1 The average time (standard deviation) and correct rate to complete the task
Full size table
Table 2 The supporters of the graph designs of two visualization methods
Full size table

To better complete the survey, the participants were divided into two groups in random order, and they watched the movie clips together. Then, the two groups completed the five tasks and filled out the questionnaire according to the view of StoryFlow (Fig. 7) and StoryCake (Fig. 6). All the user studies were performed in a quiet environment, and the questionnaire survey took almost 1 h, i.e., 10 min for work introduction, 30 min for a film show, 15 min for task completion and 5 min for questionnaires on preference and satisfaction.

5.2.2 Five tasks

First, the participants were asked to complete five tasks after watching the designated film clips. The tasks were designed according to the suggestions of film aficionados familiar with Inception. The types of tasks in the questionnaire were simple statistical issues: tracking the change of locations, tracking the evolution of the relationships between characters, finding out the specified session according to the plot and retelling the plot by the visualization view. The tasks were finished based on the visualization view generated by StoryFlow and StoryCake.

The details of the questionnaire were as follows:

  1. (1)

    Quickly point out the level of the deepest dream of the film Inception.

  2. (2)

    In the film Inception, find the sessions in which Mal and Cobb interacted with each other. Then, point out the first and the last session.

  3. (3)

    List the characters that were left in the hotel in the second level of dreams and were responsible for waking up team members.

  4. (4)

    List the characters that entered the dream Limbo.

  5. (5)

    Answer the questions when Fischer died in the dream and who entered the next level of dreams to rescue him.

5.2.3 Questionnaires on preference and satisfaction

After completing the tasks, participants were asked to fill out preference and satisfaction surveys. First, participants were asked to express their attitude towards each pair of visual indexes (support or oppose). We kept a record of the participants attitudes and their suggestions. Then, the two visualization views were evaluated with a 10-point Likert scale. The four indicators included the degree of beauty, legibility, interactivity and practicability. The scores range from 1 to 10, where 1 represents totally dissatisfied, and 10 indicates totally satisfied.

5.3 Results

5.3.1 Tasks completion result

In completing the five tasks, we collected the answering time, the correct rate and the satisfaction scores of the participants. Table 1 shows the average time and correct rate of completing the questionnaire. As seen from the table, using our visualization system, users took less time to answer questions and achieved higher correct rates. This phenomenon was more apparent in the third and fifth questions because the questions were related to the plot. When the questions were simple, such as the first and fourth questions, the advantage of our system was slightly weakened. For the second question, the correct rate of our evolution visualization system was slightly lower than that of Storyline because some participants got confused when looking for the relationship between characters across levels; however, our evolution visualization system had a huge advantage in terms of task completing time.

The hypothesis in the user study was that our hierarchical plot visualization view would outperform StoryFlow in terms of answering time. We applied Student’s t test to analyse the efficiency factor and verify this hypothesis after the study. As shown in Table 1, a comparison of StoryCake with Storyline for the aspect of efficiency of task achievement was significant at the \(p<0.05\) level. We can conclude that our work has a great advantage in tasks with regard to the hierarchy of the visualization view. Block and layered structure weakens the constraints of time and space, leading to new means of interaction, which greatly adapted to the parallel and flashback narrations.

5.3.2 Questionnaire results

In the first part of the questionnaire, each participant was asked to express their attitude towards each pair of visual indexes (support or oppose). Then, we made a comparative analysis and an investigation of the graph designs of the two visualization views according to the statistical result of the participants attitudes and their suggestions. Table 2 shows the survey data; the statistical result is demonstrated in Fig. 8. It can be seen that the visual design of our work in multiple areas, including the manifestation of entities, time frames, labels, locations, the structure of the visualization view and the information dimensionality, received broad support, especially for the label expression and the structure of the visualization view.

Every participant gave four scores ranging from 1 to 10 for four aspects, including the degree of beauty, legibility, interactivity and practicability. We collected and counted the ratings of every participant for the two visualization views from the indicators. Table 3 shows the average scores and standard deviations in the four satisfaction indexes for StoryCake and StoryFlow. Figure 9 shows the comparison of the average satisfaction scores of the two views. As seen, StoryCake was much better than StoryFlow in terms of beauty and legibility but had no significant advantage in interactivity or practicability.

Fig. 8
figure 8

The support rate of the two graph designs

Full size image
Table 3 Four satisfaction index of the average score and standard deviation (in parentheses)
Full size table
Fig. 9
figure 9

The average satisfaction score of the two visualization results

Full size image

In addition, we also asked for suggestions from the participants. The participants spoke highly of the beauty and the feasibility of the new visualization design, even though some of them were not familiar with the designs because they rarely participate in our research field. They were happy to see the visualization view after our introduction as they believed that it can provide a global view for the development of the film. They preferred our visualization view to the traditional Storyline visualization view because the distinct hierarchical structure of locations in our work provided a more intuitive way to help users better understand the view compared with StoryFlow. In previous works, the locations were represented by closed contours with different background colours, and the hierarchy of locations was described by the inclusion of contours. Generally, after completing the tasks, most participants confirmed that our visualization design is useful. Some users also looked forward to richer interactions to meet the diversity of their requirements.

In summary, the graphic designs of StoryCake had wide support from many users, and the method can achieve a high accuracy rate within a short time in terms of understanding movie plots and analysing relationships between characters. This visualization method reflects the changes of plots by introducing the concept of story level and adds effective information of the story using labels to make it easier for users to better understand relationships and locate targets quickly. From the satisfaction survey, we conclude that the beauty and legibility of our visualization view are obvious improvements compared with previous works, especially in analysing the narrative mode of recall and flashbacks; however, interactivity still needs to be improved.

6 Conclusion and future work

In this paper, we presented a hierarchical plot visualization method in polar coordinates that puts session blocks in different layers to reflect plot developments. This visualization combined with keyword tags enables the user to understand story structure and better analyse dynamic character relationships. Compared with the existent method, StoryCake also applies to discontinuous events and a nonlinear storyline. This method highlights the hierarchical relationships of a story, reflects the structural arrangement and uses the fan-shaped design to show more information in a limited screen to help users understand and analyse the storyline effectively and quickly. The evaluation of our graph designs and the conducted user study demonstrate the feasibility of our method.

There are some limitations in our work. First, in some cases, a person might be involved in multiple tasks at the same time; thus, an entity occurs in different sessions at one time-step. Using a simple line, which can only reach along one direction, to represent an entity caused confusion. A considerable solution to this problem is to use a line with width to represent an entity and branches of the line to represent simultaneous task participation. This may be an interesting inspiration for our future work. Second, up to now, only the session blocks can be folded to show the parallel or interlaced narrative. Thus, we are planning to support a different blocking mechanism to show more story structure. We will try to increase interactions, such as moving session blocks, to cope with flashbacks and multiple linear narratives.