Introduction

Approximately 1.2 million people are living with human immunodeficiency virus (HIV) in the United States (U.S.), with a stable incidence rate of an estimated 50,000 cases per year [1]. HIV rates in the U.S., for the recent years where data are available, have declined slightly from 13.1 new HIV diagnoses per 100,000 in 2012 to 11.8 in 2017 [1]. Despite decreasing HIV infection rates among injection drug users, women, and heterosexual men, infection rates among men who have sex with men (MSM) remain stable. In 2017, 70% of all new HIV infections in the US are attributed to male-to-male sexual contact (including 3% attributed to male-to-male sexual contact and injection drug use), with highest rates of new infections occurring among MSM between 25 and 29 years of age [1]. Black and Hispanic MSM are particularly impacted by HIV, accounting for 37% and 29% of new HIV diagnoses [1]. For these reasons, innovative strategies are needed to improve MSM’s engagement in the HIV prevention and treatment continuum.

HIV testing is often recognized as the first step on the continuum, precluding diagnosis, engagement in HIV care, prescribing antiretroviral therapy (ART), and achieving viral suppression for those living with HIV [2]. Testing for HIV is also a necessary first step in the continuum for uptake of pre-exposure prophylaxis (PrEP) for HIV prevention. The Centers for Disease Control and Prevention (CDC) recommends that sexually-active MSM get tested for HIV at least annually [3]. The same 2006 report states MSM might benefit from more frequent testing (e.g., every 3 or 6 months), depending on their engagement of other behavior(s) associated with being at high risk for HIV infection. A subsequent 2017 review of available evidence and expert consultation concluded that insufficient evidence was lacking to demonstrate the need for greater than annual HIV testing among MSM, despite the clear benefits of early HIV diagnosis and treatment and the continued trend of disproportionate burden in this population group [4]. However, the 2017 review also noted that more frequent HIV testing might be beneficial depending on individual risk factors, local HIV epidemiology, and local policies [4]. A study by An and colleagues showed the average HIV inter-test interval (ITI; i.e., the average number of months between HIV tests) shrunk between 2010 (10.5 months) and 2014 (7.7 months) for MSM, with similar trends across age and race/ethnicity groups, as well as for MSM with HIV risk factors (e.g., multiple sex partners, illicit drug use, or HIV-positive partners) [5]. An analysis of HIV testing in non-healthcare facilities in 20 jurisdictions in the southern US (i.e., 16 states and 5 cities in the southern U.S.) in 2016 showed 6% of 374,871 tests were among black MSM, despite accounting for 36% of new HIV diagnoses [6]. Among MSM, being unaware of their HIV status has consistently been shown to be associated with engagement in condomless anal sex (CAS) with serostatus-unknown or serodiscordant male partners [7]; a recent study showed that a higher proportion of MSM who were unaware of their HIV-positive status (25.9%) had CAS with a serodiscordant or serostatus-unknown male partner during their last anal sex encounter than men who were aware of their HIV status (18%) [8].

While the optimal HIV ITI (i.e., recommended test frequency) may be debated, it is clear consistent and repeat testing for HIV (i.e., testing for HIV at regular intervals) is needed to reduce onward transmission of HIV. Most recent CDC surveillance data show the vast majority (> 95%) of MSM have been tested for HIV in their lifetime, and over three-quarters of these men were tested for HIV in the past year (76.6%) [9]. Repeat HIV testing, however, appears to be less common. In a study of sexually-active HIV-negative MSM in concordant primary relationships, half of men tested for HIV at least annually (21% tested two or more times a year and 29% tested annually) [10]. The remaining percentage of men either tested for HIV less regularly (30%) or never tested for HIV while in their current relationship (20%). Interventions that promote repeat HIV testing are needed to substantially reduce the HIV burden among MSM in the U.S.

Technology-based HIV testing interventions have proliferated [11,12,13,14] due to the widespread adoption of technology, the ability to reach a broad audience, rapid scalability, their consistent and “real-time” delivery, and relatively low implementation costs [15, 16]. For these reasons, mobile health (mHealth) technologies may help overcome some of the limitations of in-person and clinic-based interventions [15]. A number of mHealth intervention approaches promoting HIV testing have been tested in recent years. McCoy and colleagues developed and pilot tested the “Stick To It” technology-based intervention to promote repeat HIV testing, defined as having two or more HIV tests in 6 or more months of follow-up, among MSM between 18 and 26 years of age living in Los Angeles [17]. The intervention consisted of an online dashboard with quizzes for men to take and a countdown timer until their next HIV test. Men could earn points both online and for in-clinic activities (e.g., HIV testing) for redeemable prizes. When restricted to the 31 participants who were recruited in the clinic, 48% of those men engaged in repeat HIV testing compared to historical controls [17]. The Get Connected! pilot study compared a tailored sexual health and HIV testing locator intervention to a non-tailored HIV testing locator-only intervention among young (15–24 years of age) MSM [18]. The majority (22 of the 30 participants) of young MSM who tested for HIV within 30 days after enrollment were assigned to the tailored intervention condition [18]. While not all mHealth HIV prevention interventions with MSM show effects on HIV testing behaviors [19], nearly all use any HIV testing during the follow-up period as the primary outcome with none specifically assessing repeat testing.

We developed and pilot tested the Status Update Project (SUP) mobile app intervention to promote repeat HIV testing among US MSM. The primary purpose of the SUP study was to assess the feasibility and acceptability of SUP to promote repeat HIV testing over an 8-month follow-up period among racially and ethnically diverse MSM. Secondarily, we assessed the preliminary impact of the SUP mobile app intervention on repeat HIV testing between baseline and 4 months and across the entire study period (baseline to month 8).

Methods

Recruitment, Enrollment, and Eligibility

A pilot randomized controlled trial of the SUP intervention was conducted between March 2017 and May 2018. A total of 113 participants were recruited and enrolled into the study through the use of targeted advertisements on dating and social networking platforms (Grindr, Scruff, Facebook). In order to obtain a geographically, racially and ethnically diverse cohort of MSM, we recruited in the metropolitan areas of Las Vegas, Miami, Minneapolis, and New Orleans from March to April 2017. Each targeted advertisement included a picture of a male using a smartphone, a brief study description, and a weblink to the study landing webpage.

Participants’ self-reported eligibility inclusion criteria included: (1) identifying their current gender as male; (2) being between 18 and 40 years of age (to reach sexually active and younger MSM); (3) having had sex with a man within the past year; (4) residing within the vicinity (i.e., 25-mile radius) of Minneapolis, Minnesota, Miami, Florida, New Orleans, Louisiana, or Las Vegas, Nevada; (5) not living with HIV (i.e., either HIV-negative or unknown serostatus); (6) not having been tested for HIV in the prior 6 months; and (7) having an iPhone or Android smartphone.

Persons interested in the study clicked on the advertisement to access the online screening survey and consent. Post consent, a multi-step enrollment process was used to help increase the probability that interested eligible and consented participants were aware and willing to participate in an 8-month study that included assessments at three time points, baseline and months four and eight. An 8-month observation period was purposely chosen to allow the opportunity to capture participants’ testing behavior via their recommended testing interval. For example, if a participant received a testing interval recommendation of every 3 months, then he could reasonably be tested repeatedly (i.e., twice) within the 8-month timeframe. Steps of the enrollment process included interacting with a study coordinator and having a scheduled phone interview; this process has been described in detail elsewhere [20]. Remuneration amounts for each survey was set at $50. All study procedures were approved by the University of Minnesota and University of Hawai’i Institutional Review Board.

Status Update Project Intervention Description

Men randomized to the control condition did not receive any intervention and were only asked to complete the baseline and follow-up assessments. We chose a no-treatment control since free HIV testing centers are available in all of the recruitment cities and because the design is optimal for detecting a between-group difference at this proof-of-concept phase. Men randomized to the SUP app had access to the following app components for 8 months. While a mobile-optimized website was considered, more men in the qualitative formative work expressed interest in a phone-based application rather than a mobile-optimized website [21, 22].

My Health Tab with HIV Test Date and Frequency Recommendation

Upon successfully creating a 4-digit personal identification number and logging into the app, men viewed a home screen with the study logo, study contact information, and a clickable tab directing them to take the My Health Survey (Fig. 1) on the My Health tab. The first time that participants completed the My Health Survey, they answered questions about the last time they were tested for HIV and their zip code, race, ethnicity, and age. An algorithm developed by the study team during an earlier phase of the study calculated and presented to participants the optimal date for their next HIV test and a recommendation for whether they should be tested every 3 or 6 months thereafter (Fig. 2). Men were given the option of setting a reminder on their default calendar for this test date, with a generic title “Check-Up.” The optimal test date was displayed on the participant’s app home screen until that particular date, after which it expired.

Fig. 1
figure 1

Status Update Project (SUP) home screen

Full size image
Fig. 2
figure 2

My health recommended HIV test date and testing interval

Full size image

Each month men received a notification on their homepage to retake the My Health Survey to provide them with an updated target/next HIV testing date. These follow-up check-ins asked a different set of questions from the first time the survey was taken to assess whether the recommended HIV testing intervals (with dates) needed to change (e.g., from 6 to 3 months), based on whether they started PrEP, the date of their last HIV test, the result of their last HIV test (if tested since last survey), their number of male anal sex partners, whether they felt the effects of drugs or alcohol during sex with male anal sex partners, having had CAS with male partners, and any newly diagnosed sexually transmitted infections (STIs) since their last check-in. As with the initial check-in, men were provided a recommended date for their next HIV test and guidance to test either 3 or 6 months after their next HIV test (depending on their risk). If positive, participants were followed-up by the study principal investigator to ensure that they were properly linked to care (which occurred once during the study).

Test Finder

Men were able to search for HIV testing centers in the four cities where recruitment occurred. Study staff assembled a list of HIV testing centers using data gathered from existing HIV test location databases (e.g., HIV.gov; aidsvu.org) and through targeted online searches in each of the recruitment cities. Men could either allow access to their location data to automatically show HIV testing centers in their area or search for testing centers by zip code. By clicking on a red “drop pin,” men were taken to a page for that testing center that provided basic information about it (e.g., name, address, languages that staff speak, hours of operation), get directions to the testing center from their current location, and visit the website for the testing center (Fig. 3).

Fig. 3
figure 3

Status Update Project (SUP) test finder screen

Full size image

Prevention 411

Men who navigated to the “Prevention 411” tab viewed information about PrEP, sexual pleasure, HIV and STI risk reduction, HIV testing, and basic HIV and STI information (Fig. 4). Men were allowed to freely navigate topics during the active intervention period.

Fig. 4
figure 4

Status Update Project (SUP) prevention 411 screen

Full size image

Resources

To engage men in the SUP app, the Resources tab contained information about topics relevant to gay, bisexual and other MSM, including dating and relationships, exercise and nutrition, finances, and fashion and grooming. Once a topic was chosen, men could scroll through articles within that topic, comment on it, and read other participant’s comments. Approximately 100 articles were created across the four categories.

Local Events

A list of local events was provided on a calendar for each of the recruitment cities (Fig. 5). Each week, study staff spent approximately 2 h searching and uploading local events happening in each of the cities identified through web-based searches. Men were given the option of adding the local event to their personal calendar located on their smartphone (outside of the SUP app).

Fig. 5
figure 5

Status Update Project (SUP) local events screen

Full size image

My Vote

In the “My Vote” tab, men were provided the option to answer a weekly poll question that was meant to be amusing (e.g., What was the reason for your WORST date ever?). The proportion of participants endorsing each response option in the weekly poll was displayed each Friday on the SUP app home screen.

Assessment Measures

Demographic Characteristics

Demographic characteristics included age (in years); race and ethnicity, relationship status (married, male or female partner, boyfriend; casually dating; single, and having sex with men; single, and not having sex with men), employment status (part-time, full-time, other), education (high school or less; some college, associates degree, technical college; college, graduate or professional school), and sexual orientation (homosexual/gay; bisexual; questioning/unsure; other) (Table 1).

Table 1 Status update demographics (baseline)
Full size table

Psychosocial Variables

Depressive symptoms were measured with the 10-item Center for Epidemiologic Studies-Depression Scale (CES-D), a widely used measure of depression in research studies, in which a score of 10 or higher (range 0–30) suggests that significant depressive symptoms may be evident [23]. Similarly, the Alcohol Use Disorders Identification Test (AUDIT) was used to determine whether participants were at risk for alcohol dependency or hazardous alcohol consumption [24]. In addition, using items from a prior study by the research team [25], participants were asked if they had used the following illicit drugs in their lifetime and in the past 4 months: marijuana, poppers, pain killers, downers, powder cocaine, crack cocaine, amphetamines, methamphetamines, GHB, ketamine, ecstasy, heroin, speedballs (heroin and cocaine mixed together), and hallucinogens.

HIV-Testing Intentions, Behavior, and Knowledge

In addition to demographic characteristics, participants were asked about their intentions, current behavior, and knowledge about HIV and HIV-testing. Participants were asked about whether they had ever been tested for HIV in their lifetime (yes; no), and subsequently if they had ever been tested their lifetime testing rates (1 time; 2 to 5 times; 6 to 10 times; 11 + times), and HIV-testing pattern (every 3 to 4 months; every 6 months; once a year; only if I feel I am at risk; no pattern of testing). For subsequent models, baseline testing pattern was dichotomized into those with an established pattern of testing (3-months, 6-months, once a year) and those without an established pattern (only if I feel I am at risk; not pattern of testing) or who had never been tested. Participants were asked how likely they were to test for HIV in the next 4 months (very unlikely; somewhat unlikely; undecided; somewhat likely; extremely likely). Intention to test for HIV in the next 4 months was dichotomized into unlikely/undecided (very unlikely; somewhat unlikely; undecided) and likely (somewhat likely; extremely likely). Knowledge of HIV was evaluated at baseline using the HIV Knowledge Questionnaire (HIV-KQ-18) [26], a self-reported 18-item measure of HIV-related knowledge (alpha = 0.95).

Self-reported Behavioral and Perceived Risk of HIV

Information was collected on participants’ perceived risk of HIV infection and their self-reported risk behaviors. Perceived risk was assessed using the Perceived Risk of HIV Scale, an 8-item measure of perceived risk of HIV infection (alpha = 0.79) [27]. Sexual encounters with main partners and casual partners was assessed at baseline and included frequency of condomless anal sex (CAS), insertive and receptive CAS, and CAS under the effects of alcohol and/or drugs. CAS with casual partners was defined as at least one encounter of receptive or insertive CAS with a casual partner.

Intervention Navigation and Acceptability

Participants who were randomized to the Status Update (SUP) intervention completed the 10-item System Usability Survey (SUS) [28], which asks respondents to rate on a five-point Likert scale (1—Strongly Disagree to 5—Strongly Agree) how much they agree with statements about how easy it was to navigate through the Status Update App. An average score of 68 has been found in over 500 studies to assess intervention usability [28, 29]. In addition, participants were asked to provide (1) three things they liked best about the SUP app; (2) three things they liked least about the SUP app; (3) any features that should be added to improve the SUP app. Major themes of the responses were identified and the comments were categorized.

HIV Testing Behaviors

At baseline and each follow-up, participants were asked to how many times they had been tested for HIV in the past 4 months (0 times; 1 time; 2 times; 3 times) as well as the dates and results of those HIV tests. At month 4, we characterized repeat HIV testers as those who had 2 or more HIV tests between the baseline and 4-month survey while repeat testers at month 8 were those who had 2 or more HIV tests between baseline and the 8-month survey.

A secondary analysis was conducted to assess the proportion of individuals who followed the recommendations of the HIV testing frequency survey. Those randomized to the SUP intervention arm had access to take the HIV testing frequency survey, which, based on their responses, provided either an every 3-month or every 6-month recommended testing frequency. Those individuals who had at least one HIV test within 3 months (90 days for 3-month recommendations) or 6 months (180 days for 6-month recommendations) were considered to have followed their baseline recommendation. Those who did not receive an HIV test or received an HIV test outside of the recommendation were considered to have not followed the recommendation.

Statistical Analysis

Recruitment and retention data, demographic characteristics, HIV-testing intentions and behaviors, and repeat HIV testing outcomes were summarized using descriptive statistics (means and standard deviations, counts and percentages). The acceptability of the Status Update intervention and utilization of the app were summarized using descriptive statistics. Differences in covariates at baseline were evaluated between the intervention and control group using t-tests and Chi square.

All available participant data were included in the analyses regardless of the participants’ level of engagement with the SUP app or assessment completion at each follow-up. Bivariate and multivariate generalized linear models with were run to assess repeat testing (yes vs. no) between (a) baseline and month 4 and (b) baseline and month 8. The binomial family and log link were utilized to describe increased risk (RR) of being a repeat tester. The demographic, behavioral, and HIV testing intention and behavior variables reaching p < 0.1 in univariate analyses were included in multivariate analyses. Secondary analyses were conducted to assess testing for HIV one or more times at month 4 and month 8 and, among only those in the SUP intervention arm, whether participants followed the HIV testing recommendation provided by the SUP mobile app.

Statistical analyses were performed using STATA version 15 (College Station, TX). Given the pilot nature of this study, statistical significance was set at p < 0.10 for the univariate models and p < 0.05 for the multivariate models. No adjustments were made for multiple comparisons.

Results

Retention

Study eligibility was accessed among 3057 individuals: 183 individuals met study criteria and gave consent, and 113 (62%) of them were enrolled and randomized to receive the SUP intervention (n = 57) or control condition (n = 56). The primary reason that persons who screened for the study were disqualified was because they had been tested for HIV in the past 6 months. Retention remained acceptable throughout the course of the 8-month study. Overall, 83% of participants were retained at the final 8-month assessment (n = 94); 45 (79%) completed the final assessment in the intervention arm and 49 (88%) completed the 8-month assessment in the control arm (Fig. 6). There was no differential loss to follow-up between the SUP intervention group and control group at either Month-4 or Month-8. However, those who were retained throughout the study were older compared to those who did not complete the SUP study (29 vs. 26 years old; p = 0.03). Approximately 40% of participants were recruited from the metropolitan areas of Las Vegas (n = 44), followed by Minneapolis (31%; n = 35), Miami (25%; n = 28) and New Orleans (14%; n = 16).

Fig. 6
figure 6

CONSORT diagram and retention for SUP. Note SUP utilization (downloaded and used SUP mobile app at least once): baseline: n = 45 (79%); 4-month: n = 38 (81%); 8-month: n = 37 (82%)

Full size image

Demographic Characteristics

Baseline demographics characteristics and HIV-testing behaviors for the total cohort and by study arm are shown in Table 1. On average, participants were 29 years of age and approximately half were considered racial or ethnic minorities (white, non-Hispanic: 49%; racial/ethnic minority: 51%). The majority of the study population were employed either full-time (63%) or part-time (21%), and were highly educated: 42% had some college or an associate’s degree or technical college while 40% had graduated from college, graduate, or professional school. Many of the participants (85%) identified as homosexual/gay. Almost two-thirds (63%) were single and having sex with men. There were no demographic differences between participants randomized to the SUP intervention and control groups at baseline.

HIV Testing Behaviors, Knowledge, and Intentions

Nearly all participants had been tested for HIV at least once in their lifetime (93%; n = 105), and 79% (n = 89) had two or more HIV tests in their lifetime. Among those who had tested for HIV at least once in their lifetime (n = 105), more than half had an established testing pattern (Every 3 Months: n = 7 (7%); Every 6 Months: n = 19 (18%); Once a Year: n = 34 (32%)), while the remaining men did not have an established testing pattern (n = 26; 25%) or only tested when they felt they were at risk (n = 19; 18%). At baseline, more than 60% of men reported being likely to test for HIV during the next 4 months. On average, participants’ knowledge of HIV was high (mean = 16.6; range of scale: 0–18). There were no differences in HIV testing behaviors, knowledge, and intentions between the SUP intervention and control groups at baseline.

Self-reported Behavioral and Perceived Risk of HIV

All participants reported they had condomless anal sex with another man in the past year. Overall, 80% (n = 92) of all study participants indicated a condomless anal sex encounter (either insertive or receptive) in the past 4 months, and half (n = 57) indicated a condomless anal sex encounter (either insertive or receptive) while feeling the effects of drugs and/or alcohol.

Among participants who had a main partner (n = 27), 21 indicated they had anal sex with their main partner in the past 4 months (84%). The majority of participants with a main partner reported receptive (n = 19; 90%) and/or insertive (n = 18; 81%) anal sex. The majority of participants who reported receptive anal sex had at least one condomless anal sex encounter with their main partner in the past 4 months (n = 17; 89%), of which 13 (76%) had at least one encounter while feeling the effects of alcohol and/or drugs. Similarly, among participants who had insertive anal sex, the majority had at least one condomless anal sex encounter with their main partner in the past 4 months (n = 15; 88%), of which 14 (93%) indicated that they had at least one encounter while feeling the effects of alcohol and/or drugs.

Nearly all (n = 105; 93%) of men reported that they had sex with one or more casual partners in the past 4 months. On average, participants had approximately 7 casual male sex partners in the past 4 months (median: 4.5; range 1–34). Ninety-two participants (88%) indicated that had anal sex with a casual male partner in the past 4 months. Of those, 71 (77%) reported receptive anal sex and 62 (67%) reported condomless receptive anal sex. Thirty-four men who had condomless receptive anal sex with one or more casual sex partners reported feeling the effects of drugs and/or alcohol. Of the 92 participants who had anal sex with a casual partner, 73 (79%) had insertive anal sex and 62 (67%) had condomless insertive anal sex. Thirty-nine men who had condomless insertive anal sex with one or more casual sex partners reported feeling the effects of drugs and/or alcohol.

Men reported a moderate level of perceived risk for HIV, with an average score of 25.6 (range of scale 0–40).

Use and Acceptability of Status Update Project Intervention

Participants who were randomized to SUP had acceptable utilization of the app over the course of the 8-month intervention period. Of the 57 participants randomized to receive the SUP app intervention, 49 (86%) participants downloaded it and entered their log-in information. Among the 49 users who logged into the app, 45 (92%) had opened and used it during the 8-month timeframe of the intervention whereas the remaining 8% never opened the SUP app. On average, the 45 men who opened and used the SUP app had done so approximately 10.2 times (sd = 9.2, median: 7; range 2–53) during the 8 months. Overall, the SUP app was opened 460 times during the intervention.

Acceptability of the SUP app intervention was accessed using the System Usability Scale (SUS) [24] for the participants who were randomized to the SUP intervention, downloaded the app, opened the SUP app at least once during the intervention, and completed the 4-month intervention survey (n = 38; 67% of SUP mobile app participants). Overall, the mean SUS score for the SUP intervention at month-4 was 68.5, which is considered average according to an analysis of SUS scores of 500 technology-based interventions [24]. Nearly three-quarters of SUP users agreed or strongly agreed that the SUP app was easy to use (n = 28, 74%) and felt very confident using the SUP app (n = 27, 71%), while 84% (n = 32) agreed or strongly agreed that most people could learn to use the SUP app very quickly. Over three-quarters of SUP users disagreed or strongly disagreed (n = 30, 79%) that they would need the support of a technical person to be able to use the SUP app and 76% (n = 29) disagreed or strongly disagreed they would need to learn a lot of things before they could get going with the SUP app. However, approximately half of participants agreed or strongly agreed that they would use the system frequently and the functions of the app were well integrated.

At the 4-month follow-up, participants were asked to identify up to three features of the SUP app that they liked best and least, and three features they would change or add to improve the app. For features they liked best, 25% of men reported they liked the ease of use and simplicity of the SUP app; 20% liked the health information it provided; 15% liked the feature of finding nearby locations for HIV testing centers; and 13% liked receiving reminders to be tested for HIV and STIs. While many men appreciated the ease of use and simplicity of the SUP app, one-quarter of the men reported that the what they liked least about the app was that it was too simple. Other features participants liked least included technical problems (20% reported the SUP app crashing or having slow responsiveness); the ‘repetitive nature’ of the app (13%), problems with the notifications (13%), and they thought the information presented was limited (13%). With respect to suggestions for changes or additions to future versions of the app, 23% of SUP participants suggested updating its content; 16% wanted the app to be more user friendly; 10% wanted to have other reasons to use the app such as including news or weather; and 10% suggested including additional information about the HIV and STI testing centers.

HIV Testing

At both month-4 and month-8 follow ups, participants were asked to self-report if they had tested for HIV in the previous 4 months, and if yes, to indicate the date of their last HIV test. Participants were able to enter up to three separate testing dates for the previous 4 months. Participants’ HIV testing behaviors at Month-4 and Month-8 are shown in Table 2 and Fig. 7a and b. More than half of the participants had at least one HIV test during the first 4 months of the intervention: 10% of participants were considered repeat testers, 47% had a single HIV test, and 43% did not test for HIV. Between the beginning of the intervention (baseline) and month-8, nearly 70% were tested for HIV: 45% of participants were considered repeat testers and 24% of participants had a single HIV test. Frequency of those who never tested for HIV decreased from 43% (n = 43) at month 4 to 31% (n = 28) at the end of the 8-month follow-up.

Table 2 Longitudinal HIV-testing rates
Full size table
Fig. 7
figure 7

The percentage of Status Update Project (SUP) participants who self-reported testing for HIV 0, 1, or 2 or more times in the study follow-up period. a Baseline to month 4. b Baseline to month 8

Full size image

Predictors of HIV Testing

Results from the generalized liner models conducted to evaluate predictors of repeat HIV testing at both month-4 and month-8 time points are described in Table 3. In univariate models at month-4, those randomized to the SUP intervention were more likely to be repeat testers (i.e., tested 2 or more times) compared to those in the control arm (RR = 4.4; 95% CI 0.9, 19.8). Those with a higher perceived risk of HIV were more likely to be repeat testers compared to those with lower perceived risk (RR = 1.2: 95% CI 1.0, 1.3). In multivariate analyses, neither group assignment nor perceived risk of HIV remained predictors of repeat HIV testing.

Table 3 Univariate and multivariate generalized linear models for repeat testing (> 1 Test vs. 0/1 Test) between baseline and month 4 and baseline and month 8
Full size table

Predictors of repeat HIV testing over the 8-month period of the intervention were also evaluated. Unlike at month-4, the SUP app intervention was not a predictor of being a repeat tester over the entire 8-month time period. Having condomless sex with casual partners (RR = 2.2; CI 1.0, 4.8), increased HIV knowledge (RR = 1.2; 95% CI 1.0, 1.5), intention to test for HIV in the next 4-months (RR = 1.9; 95% CI 1.1, 3.3), and having an HIV testing pattern (RR = 1.7; 95% CI 1.0, 2.6) were predictors of repeat HIV-testing at the 8-month follow-up. In multivariate models, having an established testing pattern remained a significant predictor of repeat testing (RR = 1.6; 95% CI 1.0, 2.5) after adjusting for intention to test for HIV and HIV knowledge.

As a secondary analysis (not shown in the table), we used generalized liner models to evaluate predictors of any HIV testing at the month-4 and month-8 time points. In the univariate model at month-4, having a baseline testing pattern was associated with testing for HIV one or more times in the past 4 months (RR = 1.4; 95% CI 1.0, 2.0). No other variables were associated with testing for HIV one or more times in the first 4 months. Univariate models at month 8 showed that no variables were associated with testing for HIV one or more times during the course of the 8-month intervention.

HIV Testing Frequency: My Health Check-In

All participants in the SUP intervention arm were given access to the My Health check-in, which provided them with a recommended HIV testing frequency based on their responses to the check-in items via the algorithm. Among the 57 intervention participants, 77% (n = 44) took the check-in at least once during the 8 months, most of whom (n = 36; 82%) completed their first check-in within one week after their baseline assessment (M = 3 days, sd = 2.3). On average, participants took the check-in 3.3 times during the course of the 8-month intervention (sd = 3.8; range 1–23 times).

Based on their initial responses to the My Health Survey, 73% (n = 32) of participants received the recommendation to test for HIV every 3 months while 12 (27%) participants were recommended an HIV-testing pattern of every 6 months (Fig. 8). Among participants who self-reported that they had an established testing pattern (quarterly, biyearly, or yearly) at baseline, less than half (48%) followed their baseline testing pattern recommendation. However, among those who reported a quarterly or biyearly testing pattern, 71% followed their baseline testing pattern recommendation (3–4 Month: 66% followed recommendation; 6-Month: 75% followed recommendation). In contrast, only 11% of men who tested yearly followed their baseline testing pattern recommendation. Among those who did not have an established testing pattern, 28% followed their recommendation (Only if I feel at risk: 50%; No Pattern: 27%; No Baseline Testing Response: 0%). There were no differences in demographics or HIV testing-intentions or behavior among those who followed their testing recommendation and those who did not. Lastly, of those who were recommended an every 3-month testing frequency, 22% were repeat testers over the first 4-months compared to 10% of those who were recommended an every 6-month testing frequency. Similarly, at 8-months, 60% of those recommended a 3-month testing frequency and 30% of recommended the 6-month testing frequency were considered repeat testers over the course of the study.

Fig. 8
figure 8

Proportion of participants who followed the HIV testing frequency recommendation provided by the Status Update Project (SUP) mobile app, by their baseline HIV testing patterns

Full size image

Discussion

The overall purpose of this study was to assess the feasibility, acceptability, and preliminary impact of the SUP mobile app intervention to improve regular HIV testing among at-risk HIV-negative sexual minority men. The study is unique in that most prior HIV testing mHealth studies only assess whether participants tested during the intervention and follow-up period, while we assessed repeat testing over the follow-up period. Overall, the results showed high feasibility, average acceptability, and potential promise for this approach; each of these are discussed in turn below.

Among the 113 men recruited for this study, we retained 87% and 83% of participants at the 4- and 8-month assessments, respectively. A prior examination of the enrollment processes for SUP showed that only 6% of over 3000 men screened were eligible for the study; of those eligible, 62% enrolled in the study [20]. We used a number of processes to improve retention, including asking that men complete a brief telephone call with study staff to review the features of the app and using multiple contact methods (text messaging, email, and telephone calls) to remind men to complete the assessments [20]. Retention in the SUP study is comparable or slightly better than that reported by other mHealth intervention studies, which have been reported to be 80% at a 30-day follow-up in one pilot trial [18] and 81% at a 4-month follow up in another pilot study [29]. Obtaining high engagement and retention in mHealth HIV prevention and treatment intervention studies is challenging, but necessary to both maintain study integrity and for long-term sustainability of effective mHealth interventions. Understanding best practices for optimizing study recruitment, enrollment and engagement in mHealth intervention studies, such as SUP, will be important to fully realize the potential widescale impact that is often suggested as a benefit of this approach.

Men found SUP to be acceptable, with an average SUS score and noting a number of strengths of the app. A prior HIV prevention mobile app study reported an SUS score of 73 [29], which is just slightly higher than that found in the current study. The most liked features of the app include its simplicity, the health information on the app, and reminders to be tested and a map to find testing locations. Encouragingly, 44 men randomized to the SUP intervention arm completed the My Health check-in where they were given a recommendation for the best date to be tested for HIV from their self-reported demographic and behavioral profile. That said, based on the average SUS score and men’s reports for what they would like improved, the app would benefit from enhanced features to make it more complex and engaging, with some men reporting that it was repetitive. Technological glitches occurred during the course of the pilot, and a small number of men reported problems receiving notifications. Overall, the SUP app was acceptable and men appeared to appreciate the primary features of the app, including the self-assessment for when to get tested and the map feature to locate HIV testing sites. However, the app would benefit for continued refinement to address some of the shortcomings identified by men.

Although assessing feasibility and acceptability of the app was the primary purpose of the study, we also examined the potential impact of the SUP app on repeat HIV testing. Lifetime HIV testing among this sample of men was high, and just over half had an established HIV testing pattern that included one-third of men testing once a year. Annual testing is recommended by the CDC for MSM, and more frequent testing may be warranted depending on individual risk factors, the local HIV epidemic, and other factors [4]. While this is encouraging, nearly one quarter of men had no testing pattern and a small number of men (n = 8) had never been tested for HIV. Having an established testing pattern at baseline was predictive of testing for HIV two more times in the 8-month study period in this sample of men, suggesting that establishing a regular HIV testing pattern may be an important step in the testing continuum. Although caution is warranted given the small sample size, a higher proportion of men who reported that they regularly test for HIV every 3–4 months or every 6 months at baseline followed the testing recommendation provided in the My Health Survey component of the SUP app (71%) than men who reported testing for HIV once a year (11%) or not having a regular testing pattern (29%). Among men who report testing once a year at baseline, there may be a mismatch between their perceived risk and actual risk for contracting HIV. Therefore, a component of future mobile-based HIV testing interventions may be to assess current HIV testing patterns and, for those persons who report testing for HIV once a year or less, provide supports and motivation to establish a regular testing pattern or a more frequent testing pattern.

A higher proportion of men assigned to the SUP mobile app reported repeat HIV testing in the first 4 months of the study than men assigned to the control condition (17% vs. 4%). However, this difference diminished over time, with 50% the SUP intervention arm and 40% of the control arm reporting having two or more HIV tests in the past 8 months. It is difficult to directly compare these findings to those of prior mHealth HIV testing pilot studies since previous studies assessed any HIV testing rather than repeat HIV testing and because the duration of follow-up varied across studies [18, 29]. However, the Get Connected study of adolescent (15–24 years of age) sexual minority men showed that 26% of men in the intervention arm reported any testing at 30-day follow-up compared to 18% of those in the control arm [18]. In the SUP study, 51% of men in the intervention group and 48% of men in the control group tested at least once by month 4. The proportion of men who tested for HIV once or more by month 8 only slightly improved, with approximately 55% of men in both arms testing at least one time in SUP. Taken together, the findings of the current study suggests that the SUP mobile app may have some impact on short-term testing behaviors among this sample of MSM. Longer-term impact of the SUP mobile app on repeat HIV testing was not shown, although reporting CAS with one or more casual sex partners, intention to test for HIV, and having a regular HIV testing pattern at baseline were significantly associated with repeat testing at the 8-month follow-up. The finding that short-term testing behavior may have been impacted by the SUP intervention, but reduced in the long term, is not unique and raises a number of important issues. First, HIV testing was very high in this sample of MSM and nationally three-quarters of MSM report testing for HIV in the past year [9]. Therefore, interventions with the goal of improving HIV testing behaviors are primarily targeting a relatively small proportion of MSM who could improve. Second, these results raise questions about how to optimize mHealth intervention for HIV testing, including: (a) how long and how frequently do HIV testing mHealth apps need to be used by MSM to have an impact?; (b) do MSM only need to access such interventions during periods of risk or are they more impactful when used consistently over time?; and (c) if mHealth HIV testing apps should be conceptualized as being always available and accessible, how can they be maintained over time and how do we engage or re-engage men with these apps at times of greatest need? We are at an important point where greater investment into understanding how mHealth interventions, such as the SUP app, can be optimized to address the needs of MSM and improve HIV testing is needed.

This study has several important limitations. First, we assessed HIV testing by self-report and, until a larger trial that utilizes more objective measures of testing behavior is conducted, the results of this pilot study should be interpreted with caution. Second, we recruited a relatively small number of participants for this pilot study, and not all men randomized to the SUP intervention arm downloaded the app. Although we did not observe any appreciable differences in baseline covariates between the intervention and control arms or in the estimates of the effect of SUP on repeat HIV testing among those who downloaded the app compared to those who did not, we recommend for future mHealth-based intervention trials to expand efforts to help increase the proportion of participants who download the app along with tech support to enhance engagement. Third, we did not require that men engage in CAS to be eligible for this study, which means that some men may not be at high risk for HIV. That said, the majority of men (80%) reported CAS in the past 4 months. That, and the recommendation by the CDC that all MSM test for HIV at least once a year [4], demonstrate that we were able to recruit a high risk sample of men for this study even though we did not specifically require CAS to enroll. Fourth, future mHealth research aimed to encourage repeat HIV testing among at-risk MSM ought to capture and use data about their intravenous illicit drug use to better understand their risk for HIV acquisition. Finally, participants were well educated, and therefore these results may not represent sexual minority men with less formal education.

Conclusion

Despite these limitations, findings from this pilot study add to our understanding of how mHealth interventions may play an important role in encouraging repeat HIV testing among MSM. The finding that baseline HIV testing pattern was potentially associated with repeat HIV testing in this sample of men suggests that the use of an adaptive study design (e.g., SMART) [30] in a future trial may be warranted. The first-stage intervention could, for example, provide a recommended HIV testing interval based on men’s HIV risk profile and prior HIV testing history. Men who respond (by testing at their recommended interval) could be praised for testing and provided simple reflection activities to affirm the importance of testing for their sexual health. Non-responders, in contrast, could receive additional video- or telephone-based supports to address their barriers to establishing a regular HIV testing pattern. Adaptive interventions may provide opportunities to respond to men’s testing behaviors to deliver tailored interventions that promote regular HIV testing for MSM.

Finally, a more comprehensive trial of the SUP mobile app intervention would provide more conclusive evidence as to how the intervention may impact longer-term repeat HIV testing behaviors, although enhancements to the app should be undertaken and more objective HIV testing outcomes are needed. Understanding how this and other innovative HIV testing mHealth interventions [16] may contribute to larger efforts to optimize HIV testing among MSM is warranted.