Keywords

1 Introduction

User Engagement is a relation of emotion, cognitive, and behavior between users and resources at a specific time or range of time [1]. In the past years, user engagement analytics on websites has been widely used by web developers to gather feedback from web users. In a survey of datanyze.com, 97.6 % of Alexa top one million websites use web analytics tools to analyze their user behavior data to obtain the insight of web usage [2]. The example of the analytics tools with the highest market share are Google Analytics, Google Universal Analytics, Yandex Metrica, comScore, and Quantcast, where Google Analytics occupies 45.5 % among these tools.

The analytics tools can provide web developers with the insight and feedback of web usage including what the users do on the websites, when they visit, how they interact with the websites, and many more. Once equipped with such information, the web developers can find ways to improve their web sites. For example, using an analytics tool, an e-commerce company [3] could identify the lost revenue due to high shopping cart abandonment rate and improved its website features to finally obtain an increase in checkout to payment page. Another case of a company [4], using an analytics tool to analyze which versions of its landing page should be used, could obtain an increase in homepage engagement and a boost to e-commerce conversion rate.

Fig. 1
figure 1

Web usage analytics based on a webpage, b web contents on a page

Full size image

To analyze user engagement, a number of metrics have been developed to measure web usage and calculate user engagement. Click-Through Rate, Time Spent on a Site, Page Views, Return Rates, and Number of Unique Users are examples of popular metrics [5]. However, available analytics tools and metrics focus on analytics of user engagement with the whole webpages, rather than the web contents. Figure 1 depicts web contents, namely c1, c2, and c3, on a webpage, where Fig. 1a shows page-based engagement data that can be collected from users, i.e. mean visit to a page, mean hover over a page, and mean click to a page. However, the usage information regarding a particular content on a page is not known. If we can enhance page-based engagement metrics with content-based ones, i.e. mean visit to a content, mean hover over a content, and mean click to a content as in Fig. 1b, web developers can obtain deeper understanding of how users interact with the contents and can improve their websites. This paper proposes a set of web-content-based user engagement metrics that are adapted from existing web-page-based engagement metrics. In addition, the proposed metrics are accompanied by an analytics tool which the web developers can install on their websites to acquire deeper user engagement information.

The rest of the paper is organized as follows. Section 2 discusses related work. Section 3 explains the web contents model and content-based user engagement measurement method. In Sect. 4, user engagement metrics based on web contents are presented together with a case study. Section 5 describes the content-based user engagement analytics tool, and the paper concludes in Sect. 6.

2 Related Work

We focus the discussion about related work on user engagement metrics and a popular web analytics tool.

2.1 Web User Engagement Metrics: A Web Analytics Approach

Lehmann et al. [6] study models of user engagement with web applications which can vary as the way users engage with different applications can be very different. A set of metrics, grouped into popularity, activity, or loyalty categories, is used in their study as listed in Table 1. These metrics are widely used to measure how users engage with webpages or websites. This paper adapts from these metrics to devise a set of web-content-based engagement metrics.

Table 1 User engagement metrics based on web analytics [6]
Full size table

2.2 Web User Engagement Metrics: A Physiological Approach

A physiological approach to user engagement measures the users involuntary body responses, e.g. eye tracking, mouse tracking, and facial expression analysis [7]. The measurement may involve specific hardware device, e.g. eye tracking, or only software, e.g. mouse movement detection. In this paper, we consider only the metrics that can be measured by software, i.e. mouse gestures, in particular. Huang et al. [8] examine the users’ mouse cursor behavior on search engine results pages to help better design effective search systems. The mouse cursor behavior includes clicks, cursor movements, and hovers over different page regions. We adapt from the metrics in Table 2 which are used in their work.

Table 2 User engagement metrics based on physiological measure [8]
Full size table

2.3 Web Analytics Tool

We discuss Google Analytics [9] as an example of the tools which can be installed on a website to track how users engage with the website. Table 3 shows metrics that are used.

Google Analytics visualizes the measurements as graphs and tables. Figure 2 depicts how Google Analytics, for example, shows pageviews of a single page in a graph and others in a table. This paper will use similar visualization techniques to report web-content-based user engagement information.

Table 3 User engagement metrics used by Google Analytics [9]
Full size table
Fig. 2
figure 2

Example of Google Analytics visualization

Full size image

3 Web Contents Model and Content-Based Measurement

In this section, we give a definition of the web contents model, characteristics of web contents, and how user engagement is measured in this model.

3.1 Web Contents Model

The web contents model is depicted in Fig. 3. A website is a collection of webpages, where each webpage is composed of multiple containers. A container is a specific area on a particular webpage and can be nested inside another container. A web content is contained in a container, and can be seen on a web browser, e.g. text, image, sound, video etc. It can also be part of another content, e.g. text on an image.

Fig. 3
figure 3

Web contents model

Full size image
Fig. 4
figure 4

A web content in different containers at the same time

Full size image

3.2 Characteristics of Web Contents

Analyzing user engagement with web contents is different from that with webpages because of some unique characteristics of web contents as follows.

First, a web content can appear in different containers on different webpages at the same time. As shown in Fig. 4, a web content A can be in different containers on webpages 1 and 2 at the same time. We name a web content as cc or content in a container. That is, the same web content in different containers would be referred to differently, e.g. the web content A in Fig. 4 is referred to as c1 and c2 with regard to its container on a webpage 1 and another one on a webpage 2 respectively.

Fig. 5
figure 5

Different web contents in the same container at different time

Full size image
Fig. 6
figure 6

Measuring content and container visit

Full size image

Second, a container on a page can contain different web contents at different time. As shown in Fig. 5, a web content B contained in a container on a webpage 3 at time \(= 1\) is referred to as c3. Later at time \(= 2\), that container contains a different content C which will be referred to differently as c4.

Given these characteristics of web contents, we require a different set of metrics and tool for web-contents-based engagement analytics.

3.3 Measuring Content and Container Visit

Measuring visit to web contents and containers is different from measuring visit to webpages since the latter can be done when the page is loaded. However, for web contents and containers, they might be at the bottom of the page and cannot be seen by the user when the page is loaded. As shown in Fig. 6a, only the Content 1 is shown in the viewport but the Content 2 is not because it is off the screen. In the case of measuring the visit to web contents and containers, the viewport of the screen will be used to determine whether the web contents and containers are visited. For example, Content 2 will be considered as visited when the page is scrolled down until shown in the viewport as in Fig. 6b.

4 User Engagement Metrics Based on Web Contents

The proposed user engagement metrics based on web contents are classified into three levels, i.e. low-level, high- level, and overall-level metrics. The low-level metrics are used to measure directly the behavior of the users on web contents. High-level metrics then are derived from the low-level ones. The overall-level metrics are further derived from the high- level metrics to obtain the overall engagement information. These three classes of metrics are adapted from several of those widely used engagement metrics for webpages discussed earlier in Sect. 2. Before we describe the proposed metrics, we first introduce a case study to which the metrics will be applied as an example.

4.1 Case Study

The example in Fig. 1 is used as a case study and revised with some information added as shown in Fig. 7. There are five CC or content in a container. c1, c2, and c3 are on page 1. At time \(= 1\), c4 on page 2 has the same content (i.e. a music video A) as c2 on page 1. When time \(= 2\), the container of c4 has its content changed to a music video B, and that location is then identified as c5. Table 4 summarizes the web content and container of each CC in the case study.

Fig. 7
figure 7

Case study

Full size image

4.2 Low-Level User Engagement Metrics

Low-level user engagement metrics measure usage data directly from three sources, i.e. webpages, web contents, and containers. Since a content and a container share the same location on a webpage at the same time, we use the same low-level metrics for these two sources. Each metric is listed with its definition in Table 5.

Table 4 Content and container of each CC in case study
Full size table
Table 5 Low-level user engagement metrics
Full size table

Note that the metrics for webpage sources are taken from existing metrics but we enhance by proposing additional metrics for web contents and containers. Using the low-level metrics, we obtain the measurements of the case study in Tables 6 and 7.

Table 6 Low-level measurements for webpage sources of case study
Full size table
Table 7 Low-level measurements for content and container sources of case study
Full size table

4.3 High-Level User Engagement Metrics

High-level user engagement metrics are derived from low- level metrics and categorized by user behavior approaches, i.e. visit, click, and hover. Each metric is listed in Table 8 together with its definition. Table 9 shows the calculation results for the case study.

Table 8 High-level user engagement metrics
Full size table
Table 9 High-level measurements for content and container sources of case study
Full size table

High-level metrics can be used to determine user engagement with a web content in a container. However, to determine the overall user engagement with a web content that may appear in several containers as well as the overall user engagement with a container that may contain several contents over time, we need additional overall-level metrics.

4.4 Overall-Level User Engagement Metrics for Web Content

As depicted in Fig. 4, a web content may be contained in several containers across different webpages. To determine user engagement with this particular web content, we calculate an average engagement values of that web content over all containers in which it is contained:

$$\begin{aligned} Engagement_{content}=\frac{\sum _{i=1}^{n} Engagement_{content,i}}{n} \end{aligned}$$
(1)

where Engagement \(_{content}\) = overall user engagement with a web content at a particular time

n = number of containers in which that web content is contained, and

Engagement \(_{content,i} =\) user engagement with that web content in the container i (obtained by using high-level metrics).

In the case study, c2 and c4 have the same content 2 (music video A). Using the engagement measurements for c2 and c4 in Table 9 as Engagement \(_{content,i}\), we calculate the overall user engagement with the music video A at a particular time. In Table 10, the column “Content 2 at time \(= 1\)” lists the overall user engagement measurements for the music video A at time \(= 1\). For example, VSR\(_{content2} = (0.90\,+\,1.0)/2 = 0.95\) at time \(= 1\).

4.5 Overall-Level User Engagement Metrics for Web Container

As depicted in Fig. 5, a container may contain several web contents at different time. It is useful to get an insight into how a particular container is engaged in order to design a container better or select suitable content for a container. To determine user engagement with a particular container over time, we calculate an average engagement values of that container over all contents contained in it:

$$\begin{aligned} Engagement_{container}=\frac{\sum _{i=1}^{t} Engagement_{container,i}}{t} \end{aligned}$$
(2)

where Engagement \(_{container}\) = overall user engagement with a web container over a range of time

t = number of times that user engagement with that container is determined (at a regular interval) over the time range, and

Engagement \(_{container,i}\) = user engagement with that container measured at time i (obtained by using high-level metrics).

In the case study, c4 and c5 refer to the same container 4 (on webpage 2) with different contents (music video A and B) at different time. Using the engagement measurements for c4 and c5 in Table 9 as Engagement \(_{container,i}\), we calculate the overall user engagement with this container 4 over a range of time. In Table 10, the column Container 4 from time = 1 to time = 2 lists the overall user engagement measurements for this container 4. For example, VTR\(_{container4} = (1.0+0.4)/2= 0.7\) over that time range.

Table 10 Overall-level measurement of case study
Full size table

5 Web-Contents-Based Analytics Tool

We develop a web-contents-based analytics tool that collects user interaction data, determine user engagements using the three classes of metrics, and visualize engagement information. The tool is implemented in PHP, with MySQL database. To install the tool on a website, a web developer has to register the website and generates jQuery code to put on the website. Once the code is installed, the tool will automatically collect necessary data, send to the server to process, and visualize the analytical results. The tool provides a dashboard by which the web developer can select particular web contents or containers (based on how the developers design the pages) and the engagement metrics that are of interest. Figure 8 shows an example of a graph displaying the UCR\(_{C}\) (User Clickthrough Rate) for the content 4 (music video B) at different time.

Fig. 8
figure 8

Dashboard and visualization

Full size image

6 Conclusion

The paper proposes an enhancement to existing web user engagement metrics by introducing additional engagement metrics that can take into account engagement with particular contents and areas on webpages. The analytical result is aimed to give an insight into user behavior and to help determine a way to improve website design by placing the right web contents at the right location. We are conducting a test on a commercial website that has installed the web-contents-based analytics tool in order to see if the site can benefit from the analytical results and improve their website design. Other future work includes improving the visualization of the tool and extending the set of engagement metrics.