Introduction

HIV/AIDS-related research and interventions require recruitment of diverse samples of men who have sex with men (MSM). It is well known that MSM are disproportionately affected by human immunodeficiency virus (HIV) infection [1]. As a result, there is a significant need for prevention-related research and interventions, both of which require the recruitment of MSM. Historically, recruiting MSM has been challenging, with MSM characterized as a type of hidden population. Recruitment challenges have been especially pronounced for MSM of color, due to racism and stigma concerning sexual orientation and HIV status [2]. Given these challenges, a body of research has emerged regarding recruitment strategies for MSM, with prior identification of demographic and behavioral differences between samples recruited via different venues and methods [3, 4]. Accordingly, there is an important need to develop effective recruitment methods for MSM [5]; doing so requires understanding the strengths and limitations of different recruitment approaches.

Given the movement of many MSM communities to online spaces [6], and the reduction in LGBT-focused offline spaces [7], there is significant interest in Internet-based recruitment methods [2,3,4, 8,9,10]. This is partly facilitated by the fact that smartphone ownership rates amongst MSM are high, with a 2016 study of MSM in New York City finding that 72% currently owned one and 20% had plans to purchase one in the next year [11]. The internet is also widely used by MSM to meet others via online dating applications and social media [12]. Moreover, the use of location-based social networking applications to find sexual partners is prevalent, with a study of MSM in Washington, DC finding that 63.6% had used a smartphone application for this purpose in the past year [11]. Single case studies have examined specific online venues, such as Grindr, Scruff, or Craigslist, but there have been few comprehensive comparisons of the performance of different Internet-based venues in terms of yield and demographics. Moreover, comparisons between online and offline methods of recruitment have been limited. Additionally, most prior research on recruitment of MSM has focused on online research activities (e.g., recruitment on social media sites such as Facebook, MySpace, and Craigslist) [3, 4, 13]; therefore, we know little about use of online and other recruitment venues for face-to-face research activities such as focus groups.

Furthermore, one difficulty in recruiting MSM for research or services is achieving representation of the larger population. Numerous studies have demonstrated that the recruitment venue is associated with the demographics and risk behaviors of the yielded subjects [2, 14]. Previous studies suggest that recruitment for HIV-related research, especially via the Internet, results in the enrollment of predominantly white MSM, suggesting a need to learn the most effective strategies for reaching Black and Latino MSM [2, 12, 14]. Community-based participatory research is one strategy which may perform better in terms of recruiting racially diverse samples of MSM [15]; however, these approaches have rarely been evaluated alongside other methods.

There is also a need to understand the implications of using different recruitment efforts in terms of both cost and yield. While cost–benefit analyses have been developed to compare recruitment strategies for academic research in other populations [16], such analyses have been rarely performed for recruitment efforts among MSM. However, there is a need to understand the resource-related implications of different recruitment methods, so that researchers and practitioners can use limited resources effectively. We define comparative yield as the number of focus group participants that were identified via each recruitment venue.

This study compared recruitment venues for a focus group study that took place in Southeast Michigan between 2016 and 2017. The study was a collaboration between the University of Michigan and Unified—HIV Health and Beyond (Unified). Unified is a nonprofit HIV advocacy and outreach organization that works in Southeast Michigan. Unified staff members helped to plan the study, implemented recruitment efforts, and facilitated focus groups along with UM researchers. The purpose of this study was threefold. First, we compared the yield of online and offline recruitment strategies in Metropolitan Detroit. Second, we compared the demographics of participants recruited from each of our recruitment venues in order to understand the methods that best reach these groups. Third, we estimated the marginal costs associated with each method of recruitment.

Methods

This study was a part of a larger study whose goal was to conduct a series of focus groups with MSM in Southeast Michigan to determine attitudes of MSM towards HIV testing and pre-exposure prophylaxis (PrEP). The focus groups were intended to inform the development of a stigma reduction intervention focused on increasing the uptake of HIV testing and PrEP in this population. This study explored the yield of each of the recruitment venues used in the larger study. In order to recruit MSM in this region, activities were conducted in Wayne, Washtenaw, Oakland and Macomb Counties, which included the MSM-populous cities of Detroit, Ferndale, Royal Oak, Ypsilanti, and Ann Arbor. Recruitment was implemented via a variety of media and channels, and records were kept regarding the time, duration, and cost of each activity completed. Recruitment efforts linked respondents to either: 1) the phone number and email address of a study team member; or 2) in digital recruitment methods, an online screening survey which screened for eligibility and gathered contact information. Each electronic recruitment method had a separate but identical online survey to allow for accurate determination of the recruitment method that attracted each interested volunteer. Each person who telephoned the study team was also asked how they learned about the study, and their responses were recorded in a table.

The online screening survey featured two pages: the first served as a screening tool to determine eligibility for our study and the second as a collection tool for the contact information of eligible participants (Fig. 1). Eligibility criteria for our study included being 18 years of age or older, being a self-identified male, having hooked up with men, and living in Wayne, Macomb, Oakland, or Washtenaw Counties. Ineligible recruits were directed to a page which ended the survey and thanked them for interest. Eligible users were directed to the second page to leave contact information for the study team. The study was approved by the Health Sciences and Behavioral Sciences Institutional Review Board at the University of Michigan.

Fig. 1
figure 1

Online survey instrument

Full size image

Recruitment Venues

We describe each of the recruitment venues in detail below, and methods are summarized below in Table 1.

Table 1 Description of recruitment venues
Full size table

Distributing flyers and palm xards: Flyers and palm cards (Figs. 2 and 3) were created to provide information about the study and contact information through which potential participants could express interest in participating. The study team distributed 139 flyers to 23 businesses in the cities of Detroit, Ferndale, Royal Oak, Ypsilanti, and Ann Arbor. In addition, whenever possible a small stack of palm cards was placed at each business location. Businesses included cafes, restaurants, bars, clubs, LGBT community centers, colleges, and recreation centers. Flyers were placed on walls, bulletin boards, bathroom stalls and mirrors at these locations as permitted by venue owners. Additionally, in some locations, flyers and/or palm cards containing study information were left on tables in public venues, allowing interested individuals to conveniently and discreetly obtain study information.

Fig. 2
figure 2

Recruitment flyer

Full size image
Fig. 3
figure 3

Recruitment palm card

Full size image

Bar outreach: Study recruitment was performed as part of bar outreach services offered on a regular basis by Unified. Outreach was completed at an LGBT-friendly bar and at a local nightclub on Friday nights during the club’s Pride Night for the LGBT community. Recruitment was done for 2 h at each venue for 9 weeks. Unified’s staff conducted HIV outreach at the club by providing informative materials and safer sex supplies at a table in the main area of the club. Additionally, staff members walked around the venue and discussed the study with patrons. If interested, patrons were given a card (shown in Figs. 1 and 2) with contact information for the study team, allowing them to reach out if interested in participating.

Recruitment at Unified: Unified staff members advertised our study to 176 clients using the organization’s services, such as HIV testing and support groups. Staff members handed out palm cards with study information to clients after receiving testing in addition to contacting clients through emails, phone calls, and SMS messages. Clients approached had used their support group, outreach and case management services.

Posts on social media: Messages were posted to Facebook through a number of channels. In each instance, information about the study and a link to the online survey were included in a personal status post by study staff members or on a wall post in a group. Recruitment posts were made by members of the research team and members of Unified’s staff. Posts were placed once on the walls of three LGBT Facebook groups following permission by the group administrator. Posts were also made twice on the official Facebook and Twitter pages of local LGBT-focused organizations, including Unified, the Necto night club (which has a weekly night for MSM), and Spectrum Center, an organization focused on the LGBT student population at the University of Michigan. In each of these posts, links were provided to an online recruitment survey.

Email groups: Single emails about the study were sent to university students affiliated with 10 student organization email groups at the University of Michigan, including the University of Michigan Spectrum Center’s newsletter. Emails were only sent after receiving permission from the email group administrator. Additionally, emails were sent by staff members at Unified to internal email lists of clients. In each of these emails, links were provided to a separate online recruitment survey.

Advertisements on Grindr and Scruff: Advertisements were placed in the geo-located dating/hook up mobile applications Grindr and Scruff. Grindr and Scruff were chosen as a result of recommendations made by our community partner, Unified, as these platforms represented what they felt were the most widely used applications for networking in Metropolitan Detroit. At the time of this study, Grindr was the most popular geosocial-networking application for MSM [17]. Five advertisement broadcasts were purchased for Grindr’s markets in Wayne, Washtenaw, Oakland and Macomb counties for $500 USD ($100 USD per broadcast). This represents a discount from the normal price of $210 USD per broadcast. Advertisements on Grindr popped up on users’ screens when first they first opened the mobile app. Scruff advertisements (Figs. 4 and 5) were displayed to users from Detroit, Ann Arbor, Sterling Heights, and Warren, Michigan. Scruff was selected for this study due to its Benevolads program, which allows nonprofit organizations and researchers to place free advertisements on its mobile application. Advertisements placed on Scruff appeared as banner advertisements that invited users to click for more information. Scruff provided 2,042,191 banner ad impressions across all four counties, resulting in 2376 pop-up displays (0.12% pop-up display rate) valued at $25,180.22. Advertisement text for both applications included the following: “Paid research opportunity! Are you a man who hooks up with men? Click to learn how to meet others & help your community through paid research!” When clicked, the ad directed users to a unique online screening instrument as described above.

Fig. 4
figure 4

Scruff banner ad

Full size image
Fig. 5
figure 5

Scruff pop up

Full size image

Facebook and Instagram advertisements: Advertisements were placed through Facebook’s advertising program (Fig. 6); they were set to display on Facebook and Instagram for all men who indicated that they were interested in men in the study counties. The advertisements appeared in user timelines and as banner advertisements. The ads ran for 3 days and resulted in 1407 clicks. The ads linked to the online screening instrument.

Fig. 6
figure 6

Facebook ad image

Full size image

Follow Up with Recruits

Participants who completed the Qualtrics screening instrument were called by members of the research team. If the respondent answered, the study team member would discuss participation in the study using an IRB-approved oral recruitment script. The script provided details about the focus groups, compensation, and logistics such as transportation. Additionally, the script included a screening question, which asked if the participant was a man who has sex with men and who was 18 years of age or older. Respondents were offered transportation to a focus group if needed. If interested in participating in a focus group, participants were added to a secure database.

When speaking with an interested recruit, the study team member recorded the following information: name, phone number, email address, desired focus group date, where the respondent heard about the study, transportation details (when needed to be provided by the research staff), and any dietary restrictions. Surveys conducted at the beginning of each focus group gathered demographic data on each participant, including age, gender identity, sexual orientation, race, ethnicity, education attainment, employment status, monthly income, and HIV status.

Non-identifying voice messages were left for respondents who did not answer calls from the study team. Those who did not answer the first phone call were phoned one additional time.

Reminders were sent via SMS message to participants 1–2 days before the scheduled focus group discussion. Reminder messages included a question asking participants to confirm their intention of attending the focus group to provide the research team with estimated turnout counts.

Data Collection

Recruitment activities were tracked with a designated Qualtrics data collection instrument. The instrument was used by the study team and by the staff of Unified to record each recruitment activity. Data stored included the amount of time spent, which recruits were called, any costs incurred, the type and location of the activity, the number of flyers distributed or contacts connected, and the number of potential people reached by the recruitment activity.

Data Analysis

Demographics were tabulated from each subjects’ survey responses. We calculated conversion rates per venue as the number of participants from a specific venue who attended a focus group divided by those who completed the online screening instrument or contacted the research team directly by phone. Pearson Chi square tests of association were conducted to explore associations between recruitment venue and each of the demographic variables collected. All tests were conducted with an alpha level of 0.05. Participants who indicated multiple racial identities were assigned to a multiracial category in order to conduct Chi square tests. A common data file containing all participants and their demographics was created in SPSS for analysis. Separate data files containing responses to the online screening instrument were maintained for each recruitment venue.

Marginal costs were calculated for each recruitment venue. Costs were found to be one of two types: material costs (advertisement purchases, printing and fuel expenses) and labor costs (staff person hours spent). Labor cost was determined by multiplying the number of person hours spent by $20.79, which was the average hourly wage of all research team members and Unified staff members involved in recruitment efforts. Cost per participant was calculated by dividing the sum of the material and labor costs of each venue by the number of subjects yielded by that venue.

Results

Our recruitment efforts reached 40299 individuals. A total of 441 initiated contact with the research team: 407 started the online screening instrument and 34 contacted the study team by phone. Two hundred and twenty-eight individuals ultimately completed the instrument or contacted the study team directly by phone. One-hundred and forty-two individuals who had completed the screening instrument or contacted the research team were ultimately reached and 97 indicated their intention to participate in a focus group. Ultimately nine focus groups were conducted with a total of 64 participants (66% yield based on those who intended to participate; 15% based on initial prospects).

Relative yield of recruitment venues

Recruitment conducted by Unified’s staff yielded the greatest number of focus group participants (24/64, 37.5% of total focus group participants), followed by advertisements on Grindr (20/64, 31.25%) (Table 2, Fig. 7). Posts sent to email groups resulted in seven recruits (10.94%), while flyers and palm cards placed at businesses and nonprofit organizations yielded four participants (6.25%). Scruff advertisements resulted in three focus group attendees (4.69%). Facebook posts from the study’s partnering organizations, including Unified, Necto nightclub, and the University of Michigan Spectrum Center, and personal networking efforts each yielded two participants (3.13% each). Additionally, advertisements placed on Facebook and weekly outreach conducted at a local gay bar and club each generated one attendee (1.56% each). Finally, advertisements placed in newsletters and posted to LGBT-related Facebook groups yielded no focus group participants.

Table 2 Recruitment performance by venue
Full size table
Fig. 7
figure 7

Number of focus group participants from each venue

Full size image

A perfect conversion rate was identified for participants located via messages sent out to student email groups (7/7, 100%), posts on partnering organizations’ Facebook pages (2/2, 100%), and Facebook advertisements (1/1, 100%). Recruits located by Unified staff members had a high attendance rate (24/25, 96%). Publicly posted flyers and personal networking each generated good attendance rates (4/6, 66.7% and 2/3, 66.7%, respectively). Lower attendance rates were observed for Grindr advertisements (20/173, 11.6%), Scruff advertisements (3/8, 37.5%), and bar outreach (1/2, 50%). Finally, posts on Facebook groups resulted in no participation (0/1, 0%), while articles published in newsletters resulted in zero stated intentions to participate in a focus group.

Demographics of Participants Yielded from Each Venue

Focus group participants represented a range of social identities, though each recruitment venue yielded a different distribution of individuals (Table 3). Recruitment through Unified staff members yielded the largest number of African American MSM, unemployed respondents, and individuals with a monthly income between $0 and $1000. Additionally, 70.83% (17/24) of participants recruited by Unified had an educational attainment of some college or less. Moreover, Unified represented the largest source of HIV positive individuals in our study sample (21/24, 87.5%).

Table 3 Demographics by venue
Full size table

Grindr yielded a different demographic of participants relative to Unified recruitment efforts. Subjects from Grindr had higher proportions of white MSM (17/20, 85%), college completion (13/20, 65% had a bachelor’s or graduate degree), full time employment (14/20, 70%), and high income (10/20, 50% reported monthly income of $3001 or greater). Participants from email groups were also overwhelmingly white (6/6, 100%), college educated (6/6, 100% had a bachelor’s or graduate degree), and full-time students (5/6, 83.3%).

Additionally, Pearson Chi square tests of association identified statistically significant associations between recruitment venue type and a variety of demographic variables, including race, ethnicity, educational attainment, sexual orientation, employment status, income, and HIV status (Table 4).

Table 4 Chi square tests of association for recruitment venues and demographic variables
Full size table

Cost of Each Venue

Grindr advertisements had the largest material cost of all recruitment methods ($500 for 5 ad campaigns in Metropolitan Detroit). The large number of prospects generated by Grindr required much staff time for follow up, contributing to the high total cost. The three-day Facebook ad campaign used in our study cost $149.90. Conversely, Scruff offers free advertising through its Benevolads program, though it valued the advertisements run in our study at $25,180.22 and staff time was needed for follow up with prospects. Outreach at a local gay bar and club involved a large number of person hours, while posting palm cards and flyers necessitated printing costs. Expenses for lower-cost recruitment venues (email groups, personal networking, publishing an article in local newsletters, posting to LGBTQ Facebook groups, and posting on partnering organizations’ Facebook pages) were entirely paid staff hours.

Finally, recruitment efforts by Unified staff members were formally estimated to be 35 h of work for a total cost of $727.65. This estimate represented the formal process of recruitment for this study, including talking to clients during HIV tests and reaching out to clients directly by phone, SMS, or email.

Costs per participant were calculated for each recruitment venue to allow for direct comparison of the resources required for each venue (Table 5). Bar outreach had the highest cost per participant due to a yield of only a single attendee, followed by the Facebook advertisement, which also only yielded one recruit. Grindr had the next highest cost per participant, followed by flyers/palm cards, Scruff, and Unified staff recruitment. Email groups, personal networking, and organization Facebook posts had the lowest costs per participant.

Table 5 Cost per venue
Full size table

Discussion

We believe that this is the first study to compare the yield, diversity, and costs of a variety of recruitment venues for identifying MSM to participate in face-to-face focus groups. We found that Grindr and Unified staff recruitment yielded the greatest number of participants for our study. Unified recruitment had a high conversion rate, as did other recruitment venues with lower overall yield such as email groups, organization Facebook posts, personal networking, and bar outreach. There was a low rate of participant turnout (11.6%) among Grindr recruits, despite the large number of prospects generated. Unified staff recruitment reached the greatest number of socio-economically marginalized individuals, while Grindr and email groups yielded a greater proportion of white, highly educated and high-income MSM. Grindr was the most expensive recruitment venue utilized in our study due to both a high material cost and a large number of person-hours needed for follow up with prospective participants. Unified, Email lists and personal networking represented a good benefit relative to cost, but overall yield meant that they were not alone sufficient. Facebook ads, flyers/cards, and bar outreach represented greater costs than benefits.

Previous work has shown that field and online recruitment strategies yield differences in sample size, demographic characteristics, and risk behaviors [3]. We found that Unified staff recruitment efforts were most successful at identifying socio-economically marginalized individuals, including people who were African American, HIV positive, unemployed, and who had low incomes, and low educational attainment. The proportion of Unified participants who identified as African American, attended some college or received an associate degree, were unemployed, had a monthly income between $0 and $1000, or who were HIV positive was higher than expected given the Chi square distribution, while the proportion who identified as white, Spanish, Hispanic, or Latino, were either full-time employees or full-time students, held bachelor’s or graduate degree, or were HIV negative was lower than expected. Because MSM recruited from Unified were mainly existing clients, we intuitively expected Unified’s recruitment sample to have a higher proportion of HIV positive individuals.

Grindr allowed recruitment of individuals who were predominantly white, full-time employees, highly educated, gay, and HIV negative, which parallels previous work characterizing Grindr recruits [12]. The proportion of Grindr recruits who identified as white, multiracial, Spanish, Hispanic, or Latino, held a bachelor’s degree or a graduate degree, were full-time employees, had a monthly income of $3001 +, or were HIV negative was higher than expected given the Chi square distribution, while the proportion who identified as African American, full time students or unemployed, had a monthly income of $0–$2000, or were HIV positive was lower than expected.

Participants identified by email lists were predominantly white, HIV negative, full time students with at least a bachelors or graduate degree; this is likely due to the fact that the emails were for LGBT University of Michigan students in different programs. Previous work has shown that online recruitment venues yield fewer African American and Latino than white MSM, which has been attributed to differences in the quality of internet usage, such as a greater likelihood of accessing the internet using a smartphone [14, 18]. However, smartphone ownership and use is likely also influenced by income and educational attainment.

Because the demographics of recruits differed greatly between each recruitment venue, findings suggest that it is necessary to consider the use of multiple recruitment methods when planning a research or outreach effort with MSM. Additionally, choices of recruitment messaging and venues must consider their relevance to all subpopulations of MSM, such as racial groups, age groups, and LGBT groups, including the Bear and Leather subgroups. Future work may consider directed messaging and targeted selection of venues to reach these specific subpopulations. Moreover, online recruitment methods alone were insufficient for identifying prospects from marginalized populations, indicating the need to work with community organizations such as Unified, which has established relationships and trust with disadvantaged groups.

Additionally, by beginning our partnership with Unified before recruitment started, we were able to coordinate with Unified and leverage the organization’s expertise to identify the best recruitment strategies for our purpose and ensure that recruitment materials were acceptable to the intended audience. However, despite the success of Unified’s recruiting efforts in drawing underrepresented groups to our sample, it is possible that participants identified by Unified represented only those connected to the organization’s services. Individuals not connected to Unified or other HIV service providers may face greater need for, and barriers to, resources for sexual health [19]. Conversely, it is possible that MSM not connected to Unified are regularly accessing necessary sexual health resources, such as testing and PrEP. Moreover, many MSM find information about sexual health online [20, 21].

Findings revealed low rates of conversion of Grindr- and Scruff-recruited prospects into actual focus group participants. Non-participants provided several reasons for their failure to attend the focus group after committing to do so, including an inability to travel to a focus group due to inclement weather and a personal emergency. Other non-participants simply did not attend, without contacting the study team. The framing of recruitment messaging is an important factor in motivating men to participate in research, and subsequently other HIV/AIDS outreach programs. The different responses of subpopulations to the recruitment messaging and the survey experience may impact yield as much as the venue itself. Future recruitment efforts must make sure that messaging is engaging and trust-inspiring. We believe that some recruits may not have felt comfortable participating in an in-person focus group due to not knowing other participants and due to the inherently public nature of a focus group. Thus, subsequent studies may explore online research modalities to increase engagement with MSM without raising fear of visibility. Online recruitment methods may be most effective when used for online research such as surveys or focus groups conducted over the internet. Future studies might consider recruitment yield specifically with the use of online surveys or online focus groups to determine if results differ for in-person methods.

Additionally, the technological affordances related to each of the venues must be considered. Ads placed on Grindr were displayed in a pop-up when users opened the application. Users were provided with the ad text as well as buttons offering the option to continue to the screening survey or to dismiss the ad. This is different from ads placed on Facebook and Scruff, which were featured in banners that displayed persistently on the screen as the user interacted with the application. Thus, an ad viewed on Grindr may be more likely to have resulted in the user reading and considering the message than ads placed on Scruff and Facebook. Venues that do not feature pop up ads may benefit from messaging, in the form of words or pictures, that is brief and of wide appeal to increase the likelihood of being read by users. Finally, many calls made by the study team to prospects were not answered or were to non-working phone numbers. It is likely that prospects who completed the screening instrument did not want, remember, or expect a call from the study team, leading them to ignore the ringing phone or to input an invalid phone number. Previous work has shown that subject loss in focus groups can range from 25 to 40% [22, 23], though our observed attrition rate for Grindr recruits was much greater than this, and for Unified and email lists, far exceeded this.

Marginal costs were calculated for each recruitment venue used in this study, with an average cost of $67.66 per participant. We observed a poor cost–benefit tradeoff for a number of high-cost/low-yield recruitment venues, including bar outreach, Facebook advertisements, Scruff advertisements, and the placement of flyers and palm cards at local businesses. Each of these venues had significant costs in both material or person time despite yielding few prospective participants or focus group attendees. Based on observation, it may have been that bar outreach was less effective due to the incompatibility between the leisure goals of participants at the bar, and their alcohol consumption, and the task of talking to a member of the research team about the study. Additionally, the Grindr advertisement may have garnered more initial responses than Scruff, Facebook and Instagram because Grindr generated a pop-up window which required an action on the part of the users, whereas the others used more passive banners which could be easily ignored. Moreover, we recognize that each recruitment venue has a limited reach into the larger MSM community based on the number of people who engage with it (for example, only clients of Unified were in the coverage range of Unified’s recruitment efforts). Additionally, different combinations of recruitment methods might be used to optimize cost and benefit based on available resources and needs. Thus, a multi-venue approach is ideal for reaching as much of the MSM community as possible.

The types of costs differed for each recruitment venue. Grindr, the most expensive venue ($1747.40), included $500 spent on advertisement purchases and $1,247.40 spent on staff time for follow up. Much of the follow-up time required for Grindr was spent trying to reach non-respondents. Bar outreach, the second most expensive venue ($1621.62), was made up of entirely of staff time. The cost of Unified staff recruitment efforts was also entirely staff time, though this calculation fails to account for the resources expended by Unified over the length of the organization’s history to establish itself in the Southeast Michigan community as a welcoming source of high-quality HIV services. Direct contact with clients for recruitment relied heavily on the relationships and trust between Unified and its clients, which represents inestimable amounts of time and effort in Metropolitan Detroit’s MSM community. For example, Unified has likely established itself as a trustworthy resource in the community by hiring African American MSM staff members, offering more than 1100 HIV tests annually, and providing other services such as an HIV/STI hotline, outreach, and a food pantry.

Additionally, it is likely due to the trusted position Unified holds in the community that its case workers were able to identify so many socio-economically marginalized individuals for participation in our study. Thus, we posit that Unified’s recruitment efforts hold an enormous trust-related value that is difficult to empirically measure. There is a need to determine the value of trust in organizations [24], especially in those that provide healthcare and resources [25]. Therefore, future research would benefit from efforts to measure the complete costs of MSM recruitment, including both marginal and fixed costs associated with the infrastructure of relationships and trust that support these efforts.

This study has several limitations that should be kept in mind. First, in-person focus groups demand that participants be present, and thus, identifiable to everyone else who is there. We recognize that this may have created a barrier to participation for some members of the MSM community. Additionally, aside from reported age and county of residence, we have no additional demographic data for respondents who completed the screening instrument but did not attend a focus group. Moreover, because we have limited information about reasons for not attending a focus group after a positive RSVP was returned, we cannot determine how many non-participating respondents actually intended to participate in the study, but were otherwise prevented from doing so. We recognize that scheduling conflicts contributed to non-participation, as nine focus groups were conducted across four cities, creating only limited opportunities for each respondent to participate. Moreover, focus groups were held at specific times that conflicted with some participants’ work schedules. Furthermore, many phone numbers provided in the screening instrument were invalid or not available; it is possible that there is a need to clarify on future screening instruments that a valid phone number is necessary to be contacted by a research staff member for study scheduling purposes. Finally, this study was conducted in Southeastern Michigan, and regional characteristics may have influenced the performance of each of the chosen recruitment venues. Characteristics such as racial and socio-economic composition may impact the usage rates of specific social networking applications, which impacts the relative yield of online recruitment efforts. Grindr and Scruff were chosen as popular location-based social networking applications in Michigan, though other states may feature a different distribution of usage of these applications. Regional characteristics should be considered when selecting applications in other markets.

Conclusion

We found that a mix of online and offline recruitment venues is necessary to generate representative samples of MSM for research and HIV outreach efforts. Our study reinforces the need for partnerships with community organizations to reach marginalized groups within the population. Moreover, we find that online recruitment venues helped to increase the size of our sample, though at great expense and with limited contribution to the diversity of the sample. Consideration of the ability of different recruitment venues to yield diverse samples is of utmost importance in current research, especially in studies from the HIV Prevention Trials Network recruiting large samples of African American MSM [26]. Critical analysis of the performance of recruitment venues for research and outreach in the MSM community is necessary to ensure that diverse samples are generated and that the most marginalized individuals are reached.