1 Introduction

1.1 LGBT health

A recent request by the National Institutes of Health (NIH) to the Institute of Medicine (IOM) resulted in the IOM consensus report in March of 2011 entitled The Health of Lesbian, Gay, Bisexual, and Transgendered People: Building a Foundation for Better Understanding [1] that outlines what little is known regarding the specific health issues of LGBT groups. Among the research priorities identified is addressing the gap in knowledge regarding the general mental and physical health needs of LGBT groups both nationwide and within the many types of diversity within LGBT populations (e.g., geographic diversity). While a significant amount of funding is directed toward improving health associated with HIV/AIDS and substance use in the LGBT community, very little research examines the many other potential health disparities within this group (which on the whole is much more likely to face a diagnosis of cancer, depression, diabetes, or heart disease than HIV/AIDS) or the other factors involved in establishing a broader state of health and wellbeing.

A core issue faced with health promotion within the LGBT community is an experience-based fear and mistrust of the medical community. Homosexuality was considered a diagnosable illness in the United States until the late 1970s [2, 3], directly stigmatizing and ostracizing the LGBT community. However, fears are not simply historical. For example, the “right” of physicians to refuse medical services to an individual based on their sexual orientation remains the law in over half of states (only 21 US states have anti-discrimination laws protecting provision of care regardless of sexual orientation, and only 14 US states have laws protecting gender identity [4]). This leaves LGBT groups with patchwork protection against medical discrimination. This culture has resulted in homophobia (defined as fear and/or dislike of LGBT individuals) and heterosexism (defined as the assumption that all individuals are heterosexual) being two of the most common forms of discrimination faced in the health care system [5–8]. Even today, frequently reported negative experiences following disclosure include ostracism, fear for safety, invasive questioning, shock, embarrassment, unfriendliness, pity, condescension, provider fear, and sexist remarks [5]. These factors combine to create a complex dynamic regarding health promotion within LGBT communities that has impacted not only the ability to serve the needs of LGBT individuals, but also the ability to even engage LGBT individuals in health-related research.

1.2 Rural health

As within the LGBT community, rural areas have unique health problems, resource shortages, demographic characteristics, cultural behaviors, and economic concerns that combine to impact the health of its residents in systematic and persistent ways [9]. Unfortunately, rural areas are remarkably understudied—particularly given the fact that approximately one out of five Americans lives in a rural area [10] and 75 % of the nation’s counties are rural [11]. Although limited, the literature does agree that rural areas have unique health considerations that ultimately result in persistent health disparities in outcomes ranging from diabetes to suicide [12]. Despite the recognition of the breadth of challenges faced in rural health, there has been remarkably little progress in eliminating rural health disparities.

When considering the source of rural health disparities, rural areas have a unique cultural background and heritage that can impact health behaviors and outcomes in strong and surprising ways. This culture is shaped by many key factors, including remoteness and isolation, economic conditions, religion, behavioral norms, healthcare stigma, and distance to care [13]. These factors combine to impact not only their potential need for health care, but also the ways in which residents will seek out care (or avoid it). Taking these factors into consideration is essential in the planning and execution of rural health programs, but rarely considered.

Much health promotion research and action makes the assumption that theories, practices, and programs developed in urban settings will be, for the most part, translatable into rural settings, leaving a distinct lack of community-driven, rural-focused research to improve the health of rural individuals. This lack of rural consideration unfortunately impacts not only rural residents at large, but particularly impacts minority groups within rural areas (which remain a virtually unstudied branch of rural health). One such unstudied area is the health needs of rural LGBT individuals.

1.3 Rural LGBT health

Because of the history of negative experiences within the healthcare system, LGBT populations are generally hesitant to disclose their sexual orientation or gender identity to health professionals [14] and may also be more hesitant to disclose their sexual orientation in order to participate in LGBT-focused research studies. Within rural areas, this becomes even more problematic—having an open discussion regarding sexual orientation or gender identity with what may be the only primary care provider or health promotion agency in a person’s hometown (or even county) could have significant implications. Such hesitance to disclose can significantly impact both the quality of care they receive and even health outcomes [5]—this may be contributing substantially to health disparities within rural LGBT populations. Despite this fact, we are aware of no rigorous, quantitative research studies examining the health needs of rural LGBT groups (who must navigate a dual-minority context that increases the risk of health risk factors and outcomes including diabetes, hypertension, depression, reproductive cancers, substance use, STDs, reproductive cancers, smoking, alcohol abuse, poor diet, and physical inactivity [15–17]).

The few qualitative studies that have been conducted reveal that rural LGBT often avoid disclosing their sexual orientation in personal, professional, and medical situations [18]. For instance, rural LGBT express concerns that if their sexual orientation is disclosed to their medical providers they may be rejected or discriminated against, leaving them unable to access safe and appropriate care given the scarcity of providers in rural areas [19]. This places an extreme amount of stress upon rural LGBT in their decision to be open with health-related contacts, and can serve as a barrier to health-related research within this vulnerable community.

1.4 Challenges in recruiting rural LGBT

A core challenge when conducting research with rural LGBT groups is that they are considered a “doubly” hidden population—there is no method by which to draw a representative, probability-based sample of either rural Americans or of LGBT Americans, much less the intersection of the two. In addition, given the stigma associated with being a member of the LGBT community within rural settings [20], recruitment using traditional convenience sampling methods is very challenging. Innovative sampling methods, such as those that utilize online recruitment, hold much promise for accessing rural LGBT groups as they can help allay concerns regarding confidentiality and safety; however, current implementation strategies are not able to draw in a national sample to ensure a breadth of participation across rural LGBT individuals.

Preliminary investigations of online recruitment of LGBT groups has been promising, ranging from accessing substance-using minority gay men through chat rooms [21] to use of Craigslist to access rural LGBT individuals [22, 23]. Prior research has demonstrated, however, that individuals recruited in-person are frequently very different from individuals recruited online; for instance, gay men recruited online have been shown to be less identified with the overall LGBT community when compared to individuals recruited in person [21]. Given that rural sexual minorities are less likely to be connected to an organized LGBT community organization [24], online methods may therefore be particularly well-suited to recruiting rural LGBT into research studies. It is unclear, however, if differing online recruitment methods (such as email vs. Craigslist) also yield samples that are qualitatively different from each other, and how these differences can be informative for development of LGBT health promotion strategies.

The purpose of the current study was to explore the preliminary effectiveness of both email and Craigslist recruitment methods to access rural sexual and gender minorities, and to examine if individuals recruited by the two methods differ significantly from each other with regards to demographic and psychosocial characteristics.

2 Methods

2.1 Participants

Eligibility criteria for the study included being 18 years of age or older and self-identifying as a sexual or gender minority (e.g., LGBT, genderqueer, intersex, questioning, nonbinary). No other entry criteria were utilized.

Participants for the study were recruited through two sequentially-implemented online convenience sampling recruitment methods: listserv based and Craigslist based. Each recruitment method utilized the same recruitment message to minimize method variance. For listserv recruitment, email invitations were sent out to various staff, leaders, members, etc. involved with LGBT-related organizations across the country (e.g., university Gay-Straight Alliances, Rainbow Connection, LGBT Community Centers, APA Division 44). Individuals were asked for their anonymous participation, in addition to asking the email recipients to consider passing the link along to other LGBT individuals in order to facilitate a snowballing technique for recruitment. Approximately 5000 recruitment emails were sent.

The second recruitment method implemented utilized Craigslist, whereby classified advertisements were placed in the “Community Volunteer” section of each municipality in the United States (currently 477 municipalities in total) recruiting individuals to participate in the online study. Craigslist ads were placed a total of 3 times each, as they automatically cycle off of the website 30 days after posting.

2.2 Procedures

The email/classified advertisement provided potential participants with the link to a SurveyMonkey website (a private, secure data collection system—www.surveymonkey.com) that contained the survey. Separate recruitment links were used to allow for tracking of recruitment method (i.e., listserv or Craigslist). Once interested participants reached the survey website, they were automatically directed to an informed consent page that detailed the purpose, nature, risks, benefits, confidentiality, administrators’ contact information, and ethical parameters of participation in the study. If after reading the informed consent the individual wished to participate in the study, they gave their consent by clicking on a yes/no question based upon their desire to voluntarily participate in the survey (described below).

After the questionnaires were completed, the participant was directed to a debriefing page that contained information on LGBT support organizations. Once the participants read through the debriefing page, they were redirected to a separately archived survey which prompted them to enter their email address if they wished to be entered into a drawing to win 1 of 30 gift cards from Amazon.com worth $50 each. All research procedures were reviewed and approved by the Institutional Review Board at Georgia Southern University.

2.3 Measures

The survey consisted of 75 demographic and health related questions created by the investigators and 7 formal questionnaires. The battery was designed to measure experiences related to disclosure of sexual orientation to medical providers and significant others, mental and physical health risk factors, substance use, and other health-related behavioral risk factors.

2.3.1 Demographics and health outcomes questionnaire

A 75-item demographic and health-related outcomes questionnaire was constructed by the research team after a thorough review of the literature pertaining to barriers to care for sexual minorities, unique health risks, and parenting/fertility practices. Factors assessed included basic demographic characteristics such as age and relationship status, concealment and apparentness of sexual orientation/gender identity, geography of residence, height, weight, exercise and eating patterns, and medical history. Various questions related to accessibility, availability, and acceptability of care were also asked, as well as an inventory of substance use. Rurality was assessed using participant disclosed zip code, in addition to subjective responses such as “which best describes the area you currently live in?” (i.e., rural or non-rural), “have you ever lived in a rural area?”, and “do you consider yourself to be from a rural background?” (answer choices: yes or no). This allows for a more comprehensive view of rurality given its fluid and nonbinary nature [9]. Race and ethnicity were assessed in a manner that allowed participants to select multiple racial and ethnic groups.

2.3.2 Multidimensional disclosure to health care providers scale (MD-HCPS)

The Multidimensional Disclosure to Health Care Providers Scale (MD-HCPS) is a 62-item scale designed to assess sexual minority patients’ beliefs and experiences related to disclosing their sexual orientation to their physical health care providers [25]. The MD-HCPS is broken into four sections of questions: Disclosure Attitudes and Beliefs, Health Care Visits, Disclosure Communication, and Disclosure Acknowledgment and Reactions. Validation specifically with gay and lesbian samples revealed five factors with coefficient alphas greater than 0.80 [25].

2.3.3 Depression anxiety stress scale 21 (DASS21)

The DASS21 is a 21-item scale designed to screen for emotional distress [26]. The DASS21 provides an overall score and three subscale scores: depression, anxiety, and stress. The DASS21 is a brief version of the DASS, a 42-item instrument, with strong internal consistency and validity in both clinical and nonclinical samples [26–28]. Higher scores indicate greater levels of stress.

2.3.4 Alcohol use disorders identification test (AUDIT)

The Alcohol Use Disorders Identification Test (AUDIT) is a 10-item screening instrument developed by the World Health Organization (WHO) in order to assess for risky or problematic alcohol consumption and identify individuals at risk for alcohol abuse or dependence [29]. Total scores range from 0 to 40 with a cut-off score of 8 indicating problematic or hazardous alcohol consumption. The measure has been well validated cross-culturally and internationally [30].

2.3.5 Multidimensional scale of perceived social support (MSPSS)

Social support was measured using the MSPSS [31], a well-validated 12-item inventory that examines support from family, friends, and significant others. Higher scores indicate a higher degree of perceived social support.

2.3.6 Health risk questionnaire (HRQ)

The Health Risk Questionnaire (HRQ) is a 28-item instrument assessing self-reported frequency of engagement in a variety of health risk behaviors, including diet, exercise, substance use, motor vehicle risks, sexual behaviors, violence, and medical risk-taking (e.g., not taking prescribed medication). Items are scored on a five-point frequency scale (Never, Rarely, Sometimes, Most of the Time, Always or Almost Always). The scale was developed by the research team and has previously been used in rural samples [32].

2.3.7 Rosenberg self-esteem scale (RSS)

The RSES is one of the most widely used measures of self-esteem, validated in dozens of studies and translated into multiple languages. It is composed of 10 items answered on a four-point Likert scale, with higher scores indicating a higher degree of self-esteem [33].

2.4 Data analysis

In order to examine potential differences between the samples recruited using the two methods, a series of chi-square tests (for categorical variables) and t-tests (for continuous variables) was performed. Because race and ethnicity were assessed as individual variables (allowing participants to select multiple racial/ethnic groups), separate chi-square tests were conducted for each racial group.

3 Results

3.1 Geographic distribution

A total of 3279 LGBT-identified individuals were recruited. Participants came from all 50 states and the District of Columbia. Table 1 shows the relative geographic distribution of participants in comparison to the population size of their respective states – only 2 states had representation below 50 % of the expected number of recruits based upon state population; 45 states had representation between half and double the expected number of recruits; and 4 states had more than double the expected number of recruits.

Table 1 Geographic distribution of participants
Full size table

3.2 Recruitment yield

Listserv recruitment occurred from November 29, 2012 through June 23, 2013 and Craigslist recruitment occurred from April 4, 2013 through July 1, 2013. Of the final sample, 980 (29.9 %) were recruited via the listserv method, and 2299 (70.1 %) were recruited via Craigslist. Listserv recruitment achieved a sample size of 500 within just 6 days, whereas Craigslist recruitment took 14 days to achieve a sample size of 500. However, listserv recruitment plateaued quickly, with only moderate additional gains after 11 days of recruitment (taking 60 days to move from 75 % of final sample to 95 % of final sample), whereas Craigslist recruitment continued at nearly a constant pace until recruitment was closed (taking only 18 days to move from 75 % of final sample to 95 % of final sample, and achieving a final sample more than double that of listserv recruitment). Craigslist recruitment surpassed listserv recruitment’s total of 980 participants after only 34 days of recruitment. See Table 2 and Figs. 1 and 2 for further details regarding speed of recruitment.

Table 2 Recruitment measures
Full size table
Fig. 1
figure 1

Recruitment yield over time: Listserv

Full size image
Fig. 2
figure 2

Recruitment yield over time: Craigslist

Full size image

3.3 Demographic characteristics

Average age of the final sample was 29.8 years, with 2038 (62.2 %) identifying as female, 916 (27.9 %) as male, 82 (2.5 %) as transgender male, 126 (3.8 %) as transgender female, and 117 (3.6 %) as genderqueer, nonbinary, or other. In terms of sexual orientation, 29.0 % of the sample identified as lesbian, 22.9 % as gay, 27.1 % as bisexual, 9.0 % as queer, 9.1 % as pansexual or omnisexual, 1.5 % as straight, 1.1 % as asexual, and 0.2 % as questioning. The sample was predominantly Caucasian (68.9 %), with 10.3 % identifying as Hispanic, 8.1 % as African-American, 6.3 % as American Indian, and 3.4 % Asian. Over one-quarter (25.3 %) of participants were cohabiting, with an additional 14.3 % of participants in a legal marriage or other formalized partnership. The sample demonstrated remarkable diversity in terms of rural backgrounds, with 65.2 % identifying as having ever lived in a rural setting, 29.6 % identifying as currently living in a rural area, and 39.6 % considering themselves to be from a rural background. Other demographic characteristics, including education level, employment status, and insurance status are presented in Table 3.

Table 3 Demographic characteristics
Full size table

When comparing Listserv recruits to Craigslist recruits, the two groups differed on nearly every demographic factor examined (see Table 3). The largest differences were seen in gender (Craigslist more likely to identify as female), sexual orientation (Craigslist more than twice as likely to identify as bisexual), parenting (Craigslist more likely to be a parent), rural background (Craigslist more likely to identify as being from a rural background), and insurance status (Craigslist nearly three times as likely to be uninsured). Craigslist recruits were also more likely to be African American and Hispanic, but less likely to be of Jewish descent.

3.4 Psychosocial characteristics

In terms of psychosocial characteristics, the sample considered itself to be moderately “invisible” in terms of sexual orientation and gender identity, with nearly half (44.6 %) of the sample indicating they felt others were aware of their sexual orientation or gender identity only sometimes, rarely, or never. However, participants also indicated a low level of concealment, with only 19.9 % of the sample reporting that they often or always conceal their sexual orientation or gender identity from others. In terms of disclosure, participants had the lowest percentage of disclosure to their primary medical provider, with only 58.1 % of the sample indicating they had disclosed their sexual orientation or gender identity to the PCP. The highest levels of disclosure (other than to friends) was to sisters, with 74.2 % reporting they had disclosed to their sisters. Other psychosocial characteristics, including social support, distress, and self-esteem, can be seen in Table 4.

Table 4 Psychosocial characteristics
Full size table

When examining psychosocial differences between the two recruitment groups, Listserv and Craigslist recruits differed significantly on every psychosocial characteristic assessed (see Table 4). The largest differences were observed in concealment, where Craigslist recruits were more likely to often or always conceal their sexual orientation/gender identity (10.2 % Listserv vs. 24.1 % Craigslist), as well as in disclosure patterns (with 81.2 % of Listserv recruits having disclosed to their mothers and 70.5 % to their fathers, contrasted with only 70.5 % of Craigslist recruits having disclosed to their mothers and 58.6 % to their fathers). Craigslist recruits also reported lower social support, higher levels of overall distress, higher depression, anxiety, and stress, and lower levels of self-esteem (see Table 4 for details).

4 Discussion

Our results indicate that not only are online recruitment strategies feasible for accessing both rural and urban LGBT populations, they are remarkably effective at doing so. In less than three months of net recruiting (much of which was passive), we were able to garner a sample of over 3000 participants from an exceptionally hard-to-reach population, including remarkable diversity geographically, demographically, and psychosocially.

When considering the geographic diversity of the sample, we were able to readily access a national sample, with participants from all 50 US states. While some states were over- and under-represented, overall our recruitment roughly followed the distribution you would expect in terms of relative ranking of the size of the sample recruited from each state. We made no special efforts to ensure that a national sample was recruited (e.g., we did not target certain states when they were not appearing in the sample)—by simply targeting a national audience through the methods themselves we were able to gain access to it. Future studies should investigate if other online recruitment modalities are similarly able to reach a nationally diverse group.

In addition, our sampling approaches were remarkably effective at reaching rural LGBT individuals, which represent a doubly-hidden population. While both Listserv and Craigslist methods accessed higher than expected rural samples (greater than the estimated 20 % of Americans who reside in a rural setting), Craigslist was particularly effective at recruiting rural LGBT—nearly 70 % of respondents reported ever having lived in a rural setting, and nearly one-third reported that they currently lived in a rural area. This might indicate that online recruitment methods are particularly well-suited to the recruitment of rural LGBT groups and should be further explored as a method for obtaining much-needed health information regarding their needs. This is consistent with our prior research that demonstrated higher than expected levels of access to technology even among highly underserved rural residents [34].

The two recruitment modalities examined had differing overall strengths and differing abilities to reach diverse LGBT individuals. In terms of recruitment speed, Listserv recruitment yielded staggering initial yields, with recruitment rates nearly reaching 100 participants per day in the early days of the project. Recruitment quickly plateaued, however, and the average yield per recruitment day steadily declined until new participants were no longer coming in. While Craigslist was somewhat slower at first, averaging fewer recruits per day, its recruitment rate remained steady nearly throughout the entire enrollment window. In fact, we were still receiving participation from up to 20 participants per day when we closed enrollment; we could have readily enrolled more individuals into the study through our Craigslist methods. Listserv recruitment was therefore effective at reaching a quicker sample, but Craigslist was less limited in its ability to recruit a larger sample. In addition, Craigslist recruits exhibited more diversity in backgrounds, yielding greater racial/ethnic diversity, greater diversity in relationship status and parenting background, greater proportion of individuals from a rural background, and greater rates of being uninsured. This could be related to the fact that Listserv recruits either had to be directly or indirectly tied to a local LGBT-identified organization; less-identified LGBT individuals are likely under-represented in the corresponding sample. Craigslist recruits, on the other hand, may not have ever engaged directly with an LGBT organization or anyone connected to an LGBT organization, allowing for more diversity of background.

Another important aspect of this study’s results was its ability to reach notoriously under-represented groups within the LGBT community. In the combined sample, we were able to accrue a substantial sample size of transgender and genderqueer individuals (n = 208 and n = 117, respectively). This is particularly important as research into health needs and health promotion strategies for gender minorities has lagged significantly behind even the limited knowledge base for sexual minorities. In addition, Craigslist recruitment in particular was effective at reaching bisexual participants, with nearly one-third of Craigslist recruits identifying as bisexual. The study was also particularly effective at reaching LGBT parents (with nearly one-quarter of the sample identifying as a current parent), indicating online recruitment may be an ideal way to expand our current understanding of the lives and needs of LGBT parents and their children.

When examining the psychosocial characteristics of the sample, Craigslist recruits interestingly were more likely to indicate both that they felt their sexual orientation/gender identity was “never” and “always” visible to others (14.7 % Craigslist vs 9.9 % Listserv and 6.8 % vs 4.8 %, respectively). The same effect was seen within concealment, where Craigslist recruits were more likely to both “never” and “always” conceal their sexual orientation (22.2 % Craigslist vs 19.2 % Listserv and 8.6 % vs 1.5 %, respectively). This indicates that Craigslist recruits were more likely to be at an extreme on the visibility and concealment spectrums, and that overall the Listserv recruits were more clustered in the middle ranges of both factors. This was somewhat unexpected, as we had anticipated that Craigslist recruits would consider themselves both less visible with their sexual orientation/gender identity and more likely to conceal their sexual orientation/gender identity, related to our supposition that Craigslist recruits overall would be less connected to the broader LGBT community. This may be the product of other demographic differences between the samples (e.g., Craigslist recruits being more likely to be rural and more likely to be bisexual), and further research is needed to determine what underlying processes are impacting the visibility and concealment behaviors of LGBT individuals. This will have direct implications for the design and delivery of health promotion programs.

In line with our expectations, disclosure to every assessed individual was lower among the Craigslist recruits, suggesting that while they may not overall engage in more concealment behaviors in their everyday lives (as discussed in the previous paragraph), there is still overall greater concealment from significant others within Craigslist recruits’ lives. The largest difference seen was with respect to disclosure to coworkers, where 82.4 % of Listserv recruits reported having disclosed, but only 66.3 % of Craigslist recruits having disclosed. This may be a function of the rural representation differences seen between the two groups, or of the greater likelihood of Craigslist recruits to be a racial/ethnic minority. Regardless, however, these disclosure differences may be directly related to the co-occurring differences in social support, depression, anxiety, stress, and self-esteem between the two recruitment groups. Future research should investigate the impact of disclosure upon the psychosocial wellbeing of LGBT individuals, particularly with regard to geographic background and minority status.

When considering the overall demographic profiles of the two sampling methods and their comparison to in-person recruitment, our results indicate that, similar to comparisons between in-person and online recruitment, samples recruited through different online methods significantly differ from each other in meaningful ways. Interestingly, the differences between the two online recruitment methods did not vary in the same way as previous research comparing online to in-person recruitment; that is, neither Craigslist nor Listserv recruits appeared to be more aligned with in-person recruits. For instance, previous research [21] has shown that online recruits are more likely to be bisexual, have a higher level of education, and have higher levels of psychological distress, with no significant differences in employment status or in age. While our Craigslist recruits were more likely to be bisexual than the Listserv recruits and demonstrated higher levels of psychological distress, they had an overall lower level of education. In addition, our Craigslist sample differed from the Listserv sample on many other measures, including age and employment status. While direct comparisons between our sample and previous in-person sampling approaches are not possible because national, in-person sampling of LGBT individuals is unfeasible, it appears that each type of recruitment (i.e., traditional in-person, Listserv, and Craigslist) is reaching distinct groups that do not align on a clear demographic gradient, with each group having similarities and differences to each other.

The results of this study should be interpreted in light of its design and methodology. First, while our sample was composed of individuals from all 50 states, it is a two-approach convenience sample and we therefore cannot conclude that it is nationally representative; however, this is one of the most scalable and replicable models of recruitment that we are aware of that allows for capturing a national sample of LGBT individuals (be they rural or urban). While the results may not be inherently nationally representative, they do represent an important advancement in ensuring that LGBT research studies adequately represent the significant diversity within the LGBT community by including more than a geographically-limited, typically urban sample. Second, our sample was under-representative of racial and ethnic minorities, indicating that future research should identify ways of ensuring more minority representation within online sampling methodologies. Finally, our sampling approach only examined two potential online recruitment methodologies; additional research is needed to examine how social networking sites and other technology-enabled recruitment methods may be able to yield even more representative samples.

Overall, online recruitment methods seem particularly well suited for recruitment of under-represented groups within the LGBT community (such as rural LGBT, gender minorities, and LGBT parents), and present an important avenue for assembling samples that represent the geographic and demographic diversity of LGBT individuals nationwide. These methods present an important opportunity for advancing our current understanding of the health needs and desires of this highly-understudied group, and may also present the method whereby to engage them in research and outreach that is more directly driven by the community itself.