BACKGROUND

Health behaviour interventions that are delivered via online social network platforms represent an increasingly popular approach to addressing lifestyle-related public health problems such as physical inactivity [1] and diet behaviours [2]. Online social networks are widely used amongst the public, and the most popular network, Facebook, has at least 1.5 billion active users worldwide [3]. Interventions delivered via online social networking platforms can achieve excellent reach to large groups of people, relatively free from geographical restraints. Additionally, such interventions reflect the ways in which people use internet technology more broadly to connect with friends, which may lead to an enhanced uptake of the target behaviour and provide conditions associated with behaviour change, such as social support, visibility and friendly competition [4, 5].

Despite the significant advantages associated with delivering health behaviour interventions via online social networking platforms, research to date has only shown modest evidence for the efficacy of this approach [6]. A 2014 systematic review of 10 studies evaluating online social network-based health behaviour interventions found that, although nine studies reported a significant improvement in a primary outcome over time, effect sizes were general small and statistically non-significant, and the four studies that included a comparator group found no significant differences in outcomes between intervention and control conditions [6]. Of the six studies that reported user engagement, five reported engagement to be just 5–15% of that intended [1, 2, 7,8,9]. Only one study reported high user engagement, at 105% of that intended [10]. Findings suggest efficacy for behaviour change is related to intervention dosage, highlighting a need to further understand user engagement to enhance the efficacy of subsequent online social network-based interventions [6].

Program features suggested to enhance user engagement include gamification and the use of push notifications. Gamification refers to the inclusion of game-like feedback, goal setting or reward systems in non-game contexts [11] and, in a physical activity intervention, may involve users setting physical activity goals, providing awards for reaching predefined milestones and allowing for the provision of individualised feedback and normative influences (e.g. social visibility). Conversely, push notifications are automated reminders typically delivered by mobile phone notification, email or text message, designed to draw users back to a program [12]. Push notifications are thought to assist with the uptake up of intervention behaviours and help with habit formation [12].

To date, few studies of online health behaviour interventions have evaluated the relationships between use of specific intervention features and intervention efficacy, nor the relationship between users’ sociodemographic characteristics and program engagement [13, 14]. Therefore, the current study aims to examine user engagement with a gamified online social networking physical activity intervention, ‘Active Team’. A randomised controlled trial evaluating the Active Team Facebook app found that users significantly increased their weekly physical activity by 135 min, relative to waitlisted controls [15]. Basic engagement analyses revealed that participants’ physical activity behaviour change was positively associated with the number of days logged in the app. The current study set out to examine user engagement with the Active Team Facebook app in more detail and specifically examine

  1. 1.

    Associations between app engagement and participant sociodemographic (sex, age and education) and team characteristics (team size).

  2. 2.

    Rates of engagement, compliance and retention across the 50-day intervention period.

  3. 3.

    Temporal trends in participants’ daily and weekly engagement, including in relation to timing of push email notifications.

METHODS

This study received ethical approval from the University of South Australia Human Research Ethics Committee (application ID #31574).

Research design

The current study is a subgroup analysis of the intervention arm of a randomised controlled trial that tested the efficacy of the Active Team intervention in increasing participants’ performance of moderate-vigorous physical activity (MVPA) in comparison to a waitlisted control group. Recruitment took place between September 2013 and July 2014, via a paid Facebook advertising campaign, flyers posted to notice boards around the University of South Australia campuses and news stories published in local newspapers. Originally, 142 participants expressed interest in participating in the study; however, only 110 formed teams and enrolled in the study. Of these, 51 were randomised to the intervention arm, and 59 were randomised to the control group. Full details of the Active Team randomised controlled trial are published elsewhere [15].

The Active Team intervention

The Active Team 50-day intervention encourages users to meet two behavioural goals: (1) to take at least 10,000 steps per day and (2) to record or log their step counts for each day. Delivered via a Facebook application (app), users access Active Team by downloading the Facebook app and viewing it on either a desktop computer or personal device. The app contains a calendar that allows users to enter their daily step counts and track them across the time. Additionally, the app includes several gamification/social features such as virtual gifts, a team discussion board and a team tally board that ranks team members based on their step counts. The virtual gift feature follows an ‘unlockable’ structure, with new gifts unlocked as participants reach certain step count goals. In addition, users receive virtual medals or awards for achieving certain milestones that were developed in consultation with a comedian (consistent with Fun Theory), for example, one medal is awarded after participants’ step count reaches 300,000: “We’re hoping you haven’t walked all these steps in one direction. If so, turn around and walk home!” As part of the intervention, users receive a weekly email each Wednesday that includes an individualised summary of the users’ progress, a tailored motivational message and a link to the Active Team app.

Participants and procedures

Eligibility criteria required participants to be between 18 and 65 years old, performing less than 150 min of MVPA per week, proficient in English language and able to form or join a team with at least two of their existing Facebook friends.

The intervention condition included 51 participants organised in 12 teams (mean team size n = 4.4, SD 0.9, range 3–6). The majority of participants were female (73%) and had completed a university degree (69%). Teams were generally mixed in terms of sex and education levels, with the exception of four all-female teams and three teams in which all participants had a university degree. The mean age of participants was 35 (SD 12.5, range 19–65). All participants received a pedometer via post and instructions to download and install the Active Team Facebook app.

Outcomes

Each intervention participant was assigned a unique identifying number on registering for the app, and the servers automatically recorded data regarding participant engagement with the app, capturing the type of activity undertaken as well as a time and date stamp. The servers did not record occasions of passive engagement, such as when participants viewed the app but did not interact with it. Following completion of the Active Team study, the full server logs were downloaded into Microsoft Excel spreadsheets. Although the RCT included a follow-up time point at 20 weeks, the current study only considers data that was collected during the 8-week period in which the intervention was active.

Data were collapsed to determine the following four engagement metrics for each intervention participant: (1) number of step log occasions (primary engagement metric), (2) number of wall posts made to team discussion board, (3) number of virtual gifts sent to team mates and (4) gamification score (calculated as the sum of wall posts and gifts). In addition, two compliance variables were determined for each participant: (1) compliance with the daily 10,000 step count goal and (2) compliance with daily step logging.

Participant retention was assessed by counting the number of participants who continued to log steps on at least one occasion for each week of the intervention as a percentage of those who original enrolled in the study and were randomly allocated to the intervention (n = 51).

Statistical analysis

Data analysis was undertaken using the statistical program, IBM SPSS (v21). Associations between app engagement (number of step log occasions and gamification scores) and participant characteristics (sex, age and education) and team size were examined using the general linear model procedure in IBM SPSS (v21). Two models were constructed: the first with step log occasions as dependent variable and the second with gamification scores as dependent variable. In both models, sex, education, age and team size were entered as fixed effects, and participant and team entered as random effects. The minimum P value at which results were considered statistically significant was <0.05.

Trends in app engagement (step log occasions, wall posts and gifts) and compliance metrics (compliance with the 10,000 daily steps goal and compliance with the step logging goal) were analysed descriptively. For all metrics except compliance with the 10,000 steps goal, a daily mean frequency count was calculated at the per participant level. To investigate compliance to the step logging goal, mean daily step counts were calculated. Visual inspection of daily mean frequency counts was used to examine how engagement and compliance metrics changed across the 50-day intervention period. Following this, mean frequency counts were calculated for each metric and each week of the intervention. These scores were then analysed visually to examine how engagement and compliance metrics changed across each week of the intervention.

Daily and weekly temporal trends in step log occasions were analysed descriptively. To calculate frequency counts for each hour of the 24-h day, step log occasions were sorted according to time stamp and categorised into hourly intervals beginning at 24:00 h and ending at 23:59 h. For example, activity occurring between 07:00 and 07:59 was categorised as 07:00; activity occurring between 08:00 and 08:59 was categorised as 08:00. Additionally, step log occasions were summed and averaged for each day of the week to produce a frequency count representing the mean number of step log occasions occurring on each day of the week for the intervention. Following this, the frequency of step log occasions occurring in each hour was summed, and then averaged, to produce a count of the total number of step log occasions that occurred on any day of the intervention within each of the 24 h. It is important to note that because the intervention consisted of 7 weeks plus 1 day, the final day of the intervention was omitted from this analysis, as it represented an extra ‘Monday’ which may have biased results.

RESULTS

The relationships between step logging occasions and gamification scores and individual and team characteristics are presented in Table 1. Results indicated that step log occasions was significantly related to education (F 2, 145 = 9.29, p = <0.001), with participants who had completed some secondary training (trade/certificate/diploma) demonstrating the highest mean step log occasions (M = 21.3, SD = 15.0) and participants who had completed high school or less demonstrating the lowest mean step log occasions (M = 3.2, SD = 3.1). Analysis found no significant associations between step log occasions and sex, age or team size.

Table 1 Linear mixed model analysis of associations between engagement (operationalised as step log occasions and gamification scores) and participant and team characteristics
Full size table

Similarly, gamification scores were significantly associated with education (F 2, 147 = 3.36, p = 0.04), with participants with a university degree demonstrating the highest engagement (M = 7.4, SD = 7.3) and participants who had completed high school or less demonstrating the lowest engagement (M = 0.4, SD = 0.5). In addition, gamification scores were significantly associated with sex (F 1, 145 = 8.69, p = 0.004), with males demonstrating significantly higher gamification scores (M = 8.6, SD = 9.3) compared to females (M = 6.0, SD = 5.9). Analysis found no significant associations between gamification scores and age, nor team size.

Trends in engagement, compliance and retention

On average, participants logged in to the app to record their step counts on 17.0 occasions (SD 13.5, range 0–46) for the duration of the challenge, sent 4.1 virtual gifts (SD 5.5, range 0–24), and made 2.6 wall posts (SD 3.3, range 0–13). All measures of engagement including step log occasions, use of wall posts, and use of gifts declined across the 50-day intervention (see Fig. 1), with the most modest decline observed for step log occasions and the steepest decline for wall posts.

Fig. 1
figure 1

Trends in participants’ step count compliance, step log compliance and engagement metrics during the intervention

Full size image

Compliance with the step logging goal and the 10,000 steps goal was examined on a week-by-week basis across the intervention period (Fig. 1). This revealed that participants’ average daily step counts were maintained at a reasonably steady rate of approximately 8500 steps per participant, per day across the full 50-day intervention period. Similarly, participants logged step counts for 6–7 days per week for the duration of the challenge.

Of the 51 intervention participants, 35 (68.6%) logged step counts for the entirety of the intervention, whilst the remaining 31.4% ceased to use the app at various points throughout the intervention (see Supplementary Fig. 2) and three participants (representing 5.8%) did not log any step counts at all.

Daily and weekly usage trends

Analysis of weekly trends in step log occasions revealed that step logging activity peaked on Wednesdays (coinciding with the weekly push email) and was lowest on Fridays (see Supplementary Fig. 3).

When examined on a 24-h basis, a bi-modal pattern was seen (see Supplementary Fig. 4), with one peak in logins observed in the morning around 08:00 and another in the late evening around 23:00.

DISCUSSION

This study examined engagement, compliance and retention with a 50-day social physical activity intervention delivered via Facebook app. Analyses revealed that engagement differed on the basis of sociodemographic characteristics namely, engagement differed on the basis of education, which the strongest engagers from the middle and high education category, and engagement with gamification features differed by sex, with males the strongest engagers. Generally speaking, self-reported compliance with the physical activity goal (10,000 steps per day) was high, as was retention across the 50-day intervention period, based upon objective server data, though a downward trend in app engagement was apparent, and particularly with the app’s gamification features. Weekly usage peaked on Wednesdays, which coincided with the weekly push email.

Compared with previous research, the current study achieved relatively high rates of study participant retention, with only 7 out of 51 (14%) participants lost at the 20-week follow-up. This compares favourably with a systematic review of web-based health interventions which found that across 15 studies, attrition ranged from 7 to 69%, with a mean attrition rate of 27% [16]. In addition to this, participants showed relatively high levels of engagement with the app software and sustained physical activity behaviour compliance. This again compares favourably with previous research using online social media for health behaviour interventions [6]. A recent study that examined engagement with an incentivised, online health behaviour intervention found that a large proportion of users (34%) logged in to the intervention platform on just one occasion, and 48% used the platform for less than 4 weeks [17].

Interestingly, the intervention approach used in the current study was similar in a number of regards to a previous study which achieved high engagement rates (albeit over a shorter intervention period) [10]. Foster et al. [10] also created a Facebook app that encouraged users to compete with existing contacts in a pedometer-based challenge. It is possible that encouraging social interaction with existing contacts, rather than new contacts introduced during the study, contributed to the higher engagement. This approach also reflects how people use social media more broadly and may have greater potential to allow social contagion effects to spread behaviour [18]. Furthermore, both our study and Foster et al. provided pedometer to allow participants to self-monitor their physical activity. Systematic reviews and meta-analysis of physical activity interventions consistently identify self-monitoring as a potent mechanism underpinning intervention effectiveness [19, 20].

Consistent with other studies [21], there was a clear trend for engagement to decline across the intervention period, and in particular, there was a decline in the frequency of step log occasions and use of gamification features, whilst stepping behaviour itself appeared reasonably stable. On one hand, it may be argued that if physical activity can remain stable, a decline in engagement, per se, may not be problematic. On the other hand, the original RCT [15] found that overall engagement was significantly associated with physical activity, and previously systematic reviews have suggested likewise [16]. Essentially, intervention engagement is a proxy for intervention ‘dosage’. This would suggest that engagement is an important issue and that efforts to boost sustained engagement such as the inclusion of gamification features are warranted in order to maximise intervention efficacy. Further research is needed to determine which alternative gamification approaches may ameliorate the decline observed in this study.

Encouragingly, temporal data regarding users’ logging patterns suggest that the weekly push emails are likely to have increased user engagement with the app, as indicated by the weekly peak in app usage coinciding with the day on which the weekly emails were delivered. Whilst many online interventions incorporate push communications delivered via emails, SMS and increasingly mobile phone notifications, little is currently known about how effective they are and how the frequency, timing and content of such notifications affect intervention engagement. Further research experimentally manipulating these parameters is required. Based on the current findings, it is recommended that future interventions incorporate a weekly email reminder delivered via email to regularly draw users back to the intervention.

To our knowledge, this is the first study of a health intervention app to examine engagement in the context of gamification features. Other strengths include that engagement data were based upon date and time stamped server logs, rather than relying on self-report. Also, the study examined engagement in a number of novel ways, including examining the impact of push emails and temporal patterns in app usage. Limitations include the reasonably small sample size (n = 51), which increases the risk of type 2 errors (i.e. failing to detect associations which truly exist between engagement and individual and team characteristics). In addition, the lowest education category contained just five participants, and this small cell size increases the risk for type 1 error (i.e. falsely detecting a significant association between education and engagement). In the future, it would be pertinent to examine engagement with online-delivered health behaviour interventions in a larger sample, to explore further associations between engagement with education levels. Finally, server logs did not record app usage where users only viewed content without actively contributing content.

Looking to the future, this study has a number of implications. Previous research suggests that online physical activity interventions tend to attract high-educated women [6, 16, 22]. Whilst the sample characteristics from the current study reflect this, the actual usage patterns suggest that users of the middle education category were most engaged and, interestingly, that men used the gamification features at a higher rate. Given that higher engagement is generally sought after in the design of online health programs (e.g. one meta-analysis showed that higher engagement was associated with intervention efficacy [23]), these observations are encouraging, suggesting that once recruited, this online social networking approach to delivering a physical activity intervention was able to successful engage and retain population subgroups that are traditionally harder to reach. In particular, gamification appears to be useful for engaging males with health behaviour interventions. This highlights the need for future efforts concentrated on attracting a diverse user base. Facebook is acknowledged to have broad reach and appeal, with users from across the globe and from all sociodemographic backgrounds [24]. Targeted advertising on online social media advertising platforms may provide opportunity for attracting a diverse user base, and future work examining targeted advertising approaches to achieve this is warranted. It was beyond the scope of the current study to examine the significance of team members’ pre-existing relationships to each other in determining engagement. Future research could investigate whether intervention engagement and efficacy differs between teams comprised of members who are known or unknown to each other offline and, additionally, the effect of team demographic heterogeneity/homogeneity on engagement and other outcomes.

In addition, future work that experimentally investigates the implementation of gamification in health behaviour interventions is needed. In the current study, we observed a steep decline in the usage of gamification features across the 50-day intervention period. However, the gamification features employed in our software were relatively simplistic. More sophisticated execution of gamification may achieve better long-term engagement.

In conclusion, online social networks offer a promising platform on which to deliver effective and engaging health behaviour interventions. Future work should seek to incorporate push notifications and reminders, as these have the potential to draw participants back to the intervention. That engagement with the intervention was reasonably high is promising, and there is now a need to further examine the role of specific social and gamification elements in isolation, including how they can best be implemented to maximise engagement and efficacious outcomes.