Introduction

Individuals from minority groups in the United States have been found to be less likely to participate in research compared to non-Hispanic whites [1]. Numerous factors have been identified that appear to reduce the ability and/or willingness of ethnic minority individuals to participate in health research including limited economic resources, past negative experiences with health professionals, and a socioeconomic or ethnic status that differs from that of the researcher [2]. In order to bolster investigator efforts to recruit and retain minority individuals in research, and to make the benefits of research participation more widely available across groups, the National Institutes of Health issued guidelines to promote the inclusion of minorities in research [3].

The recruitment and retention of mentally ill persons for participation in research has also been challenging. Greater than 25% of mentally ill research participants may be lost to follow-up during the course of a longitudinal study [4]. Even lower retention rates have been reported among mentally ill individuals experiencing periods of homelessness. In one study of homeless chronically mentally ill veterans, only 37.9% remained available for follow-up [5].

More recent research suggests, however, that the use of appropriate recruitment and retention strategies can promote high rates of recruitment and retention of mentally ill minority individuals [6]. In this article, we describe the challenges we encountered in recruiting and retaining severely mentally ill women of Mexican and Puerto Rican ethnicity for a qualitative, observational study of the context of HIV risk behavior. In doing so, we situate each challenge at the level of its occurrence and trace the use of our procedural, methodological, and personnel strategies to meet these various challenges through the various stages of our research protocol, from pre-recruitment through follow-up. We use the model outlined by Levkoff and Sanchez, which frames barriers to involvement in research for minority populations in the context of macro, mediator, and micro (individual) level barriers [7]. We conclude with a discussion of the reasons underlying our differential retention rates, 81.1% in our Puerto Rican cohort versus 26.7% in our Mexican cohort over a period of four years, and the implications of our findings for both research and clinical practice.

Challenges to Recruitment and Retention

Eligibility for study participation required a diagnosis of major depression, bipolar disorder, or schizophrenia; Puerto Rican ethnicity if residing in any of six enumerated counties of northeastern Ohio or Mexican ethnicity if residing in San Diego County, California; and age between 18 and 50 years at the time of enrollment. Following verification of eligibility and enrollment into the study, the protocol required that participants present for an additional two-part baseline interview, a follow-up interview each year for two years, and 100 h of observation (“shadowing”) by an assigned interviewer.

We identified the challenges to recruitment and retention at the macro, mediator, and micro levels [7]. We anticipated that, although relatively fewer challenges would be encountered at the macro and mediator levels, these would be significant. We devised procedural, methodological, and personnel strategies to address each of these challenges during multiple phases of the study.

Macro Level Challenges

The one macro level challenge we faced related to the immigration status of many of our Mexican participants (See Table 1). A large proportion of our Mexican participants were not legally present in the United States and expressed concern about the possibility of deportation. In order to maximize the likelihood that we could continue to find them, even if they were to go further into hiding, we obtained at the time of enrollment and at each subsequent contact the participant’s complete contact information, a listing of her usual “hangouts,” the names and complete contact information for a minimum of three persons, including care providers, who would always know of her whereabouts, and a signed consent form permitting us to contact these individuals for information about the participant’s whereabouts if we became unable to locate her. These “anchoring” strategies would allow us to more easily “track” participants who became lost due to unstable living situations, homelessness, periodic hospitalizations and/or incarcerations, and disrupted or violent relationships [8]. Although other commentators have suggested that updating such information every two years is adequate to track study participants [9], we found that our participants moved quite frequently and, if we had updated this information at lengthier intervals, we would have lost them to follow-up. Personnel strategies during these stages focused on the identification of staff familiar with the Mexican community in San Diego and with staff training around issues of immigration and cultural sensitivity.

Table 1 Recruitment and Retention Challenges and Strategies
Full size table

Mediator Level Challenges

Mediator level challenges included the identification of eligible participants, community-level distrust and fear of research and/or researchers, and language differences, since much of the Latino community from which we would draw participants was monolingual in Spanish or preferred to speak in Spanish. The formation of a community advisory board (CAB) during the pre-recruitment phase helped us to identify culturally-sensitive strategies that would enable us to identify potentially eligible participants in the community. The CAB was comprised of representatives from various social service agencies that provided services to mentally ill persons and/or to individuals within the Latino communities, law enforcement personnel, clergy, and participant-representatives. Participant-representatives were individuals who had received a diagnosis of mental illness and, although interested in the study, were not eligible to participate, most often due to age or because they were male. The CAB also assisted staff in the development of appropriate study instruments and translations and highlighted areas of particular concern to be addressed in training.

Based on the advice that we received, we disseminated information about the study within the Puerto Rican and Mexican communities through presentations conducted in diverse settings, such as language classes, vocational classes, churches, and support groups. Flyers were also distributed to a broad spectrum of specified venues, including churches, nightclubs, government assistance offices, social service organizations, laundromats, social clubs, beauty salons, restaurants, grocery stores, and other locales. We also contacted clinicians and counselors serving mentally ill Puerto Rican and Mexican women to apprise them of the study.

We used various strategies to address the community-level distrust of research and researchers. First, our “sponsored” appearance through known community organizations and venues, which served as gatekeepers to our target communities, promoted trust in the study and its team [3]. Second, we utilized passive methods of recruitment, such as flyers, which require the prospective participants to initiate contact with the researcher after receiving some basic information through flyers, posters, etc. This method does not compromise the confidentiality that the prospective participants may have established at other venues with other persons, such as clinicians, members of social organizations, by asking them to reveal the identity of eligible persons. Too, we believed that if we had utilized an active system of recruitment, through which we would have ascertained eligible persons’ identities and contacted them directly, we would have alienated some of the individuals who we wished to recruit because our knowledge of their identity might have exacerbated symptoms of paranoia and/or led to further distrust of researchers. Additionally, passive strategies of recruitment have been found to be more efficacious than active strategies [10].

Third, our presentations and flyers avoided using the words “mental illness,” which would have been stigmatizing and embarrassing, and instead advised individuals that we were conducting a study to better understand how to reduce HIV risk among Puerto Rican and Mexican women who were deprimida (depressed), who had suffered from ataques de nervios (nervous or panic attacks), or who had emotional troubles.

Interested individuals were provided with the names and telephone numbers of various staff members for additional information about the study. Those who contacted study personnel for more information were advised that the study focused on Puerto Rican and Mexican women who had been diagnosed with major depression, bipolar disorder, or schizophrenia.

Our attention to language also required that we convey the high level of respect that we have for individuals choosing to volunteer in the study and our perception of research as a joint venture between the researchers and the research volunteers, recognizing that a power differential continues to exist because of the nature of the venture [11]. Accordingly, we refer to research volunteers as participants, not as subjects, in order to underscore their importance in the process to foster mutual trust and respect.

Fourth, at each of the presentations and subsequent inquiries, we utilized a participant-centered approach [12, 13], making it known to individuals that interviews could be conducted at an hour and location of their choosing, in order to better accommodate their schedules and concerns; that, once assigned to an ethnographer for shadowing, a participant would continue to interact with the same individual, barring any unforeseen circumstances or difficulties; and that information about the participant would be maintained with the highest degree of confidentiality possible. We also provided detailed information about the study protocol.

Finally, the selection of staff and staff training were critical elements of our approach to identify, recruit, enroll, and retain participants. All of the project personnel hired for the study team had had extensive experience working in the relevant Latino communities and were familiar with the social and political organizations and hierarchies that existed within those communities, which maximized their effectiveness as recruiters [14, 15]. It was critical in selecting study staff that individuals be fluent in both English and Spanish. The PI and all members of the study team maintained involvement with the communities outside of the context of the research study, which allowed community members to feel that they knew the people responsible for the study.

Staff training focused on the development and enhancement of five competencies: detailed knowledge of the study protocol, public speaking ability, the ability to respond to spontaneous questioning, a comprehensive knowledge of community resources and venues for recruitment, and a knowledge and understanding of mental illness and its symptomatology. Staff engaged in role playing exercises designed to maximize their comfort level with public speaking and responding to questions from both providers and prospective participants, some of whom might be experiencing psychotic symptoms. These role playing sessions were conducted in English and in Spanish in order to ensure that all staff were consistent in their use of specific terms (e.g., “participant”) and geared their choice of words to the educational levels of their various audiences.

Micro Level Challenges

The vast majority of the challenges or barriers to recruitment and retention for this study occurred at the micro level, or the level of the individual participant. These included participant isolation; distrust and fear of the researchers and research; high levels of psychopathology and/or use of substances; a fear of being stigmatized as a result of study participation or being recognized as mentally ill; competing obligations such as attendance at family or church functions; financial considerations that impacted the individuals’ ability to pay for transportation costs, child care, etc.; relationship issues, such as partner violence; unstable living situations due to intermittent homelessness, partner violence, reduced income, and other factors; and migration out of the study area.

We addressed these challenges using a variety of strategies during multiple phases of the study (See Table 1). For instance, we used each encounter with a participant as an additional opportunity to explain the study and answer questions, to have the participant interact with study staff, and to emphasize both the confidentiality protections that we had developed and the flexibility of scheduling study activities. Confidentiality protections included the use of unique numeric identifiers on records pertaining to each participant, passwords on computers, limited access to listings of the study participants by name, signed agreements by all study staff to maintain confidentiality, and the receipt of a certificate of confidentiality from the National Institute of Mental Health, which could be used to contest access to the collected data for use in legal proceedings [16, 17]. Each repetition of these procedures was intended to mitigate participant distrust and fear.

We employed several mechanisms to address the challenges presented by participants’ psychopathology and substance use. For instance, we implemented procedures to allay participants’ concerns about their own potential failure should they forget to present at a preset appointment and to reduce potential attrition associated with varying levels of symptomatology. Increased severity of illness [9, 10]. Such as increasing paranoia, could interfere with individuals’ willingness and ability to present at a less familiar location and result in their withdrawal from the study. In contrast, individuals with reduced severity of mental illness could be concerned about the meeting location because of fears that their diagnosis might become known and, absent adequate study flexibility to address such concerns, choose to withdraw from the study. Either scenario could introduce additional bias [9]. Accordingly, we telephoned those participants with phones one week, one day, and one hour prior to established times for interviews to reconfirm the times and to establish that they were still at the same address. We sent such reconfirmations by mail in those instances in which the participant did not have a telephone. If we were unable to reach the participant by phone or mail, we relied on our list of contacts that had been provided by each participant to ascertain her whereabouts.

Based upon the advice of our study team members from the Latino communities and the membership of our CAB, we devised incentives that were culturally appropriate, that demonstrated the value that we placed on individual’s continuing participation, and that would reduce financial obstacles to participation and foster retention [17–19]. In providing information about the study to prospective participants during the recruitment and enrollment processes, we also explained the incentives. We did not offer any incentive for participation in the eligibility screening. After being found eligible, a participant would receive $20 for completion of each of the two parts of the baseline interview. At each of the follow-up interviews, spaced one and two years after the baseline interviews, the participant would again receive $20. We also provided small gifts at predetermined points during the shadowing period as a token of appreciation for the participant’s time and as a reminder to the participant that this was a research study, rather than a friendship. In this way, we hoped to minimize the possibility that the participant would experience trauma upon leaving the study and severing the relationship with the research team. Tokens included a small vinyl change purse, a magnetic refrigerator clip, a small fabric lunch bag and, upon conclusion of study participation, a T-shirt. All items carried the study logo, which consists of a royal purple, boomerang-shaped design with four ovals of various sizes in gold and royal purple situated around it, and the name, Center for Minority Public Health. This logo, which was designed together with a graphic deign artist and tested in focus groups in Ohio and California, signifies that “There is a place for everyone at the table.”

Practical inconveniences, such as lack of child care or transportation, could potentially affect retention [20]. We addressed this possibility by conducting interviews at a time and place that was convenient for the participant. This required significant flexibility from our staff with respect to their work schedules. Because some of the participants had living situations that were of questionable safety due to their location and/or their relationships, interviews and shadowing were sometimes conducted by a team of two study staff. We also utilized this strategy in instances in which the participant had a history of violence and the meeting was to occur in a non-public setting, such as the participant’s home.

In addition to constantly updating the contact list for each participant, previously described, we maintained frequent contact with participants to minimize their loss to follow-up [19]. We asked each participant to let us know which holidays she celebrated or observed, so that we could recognize these special days and her birthday by sending her personalized cards throughout each year [21]. Our cards were sent much more frequently than the two per year that other researchers have suggested [9], in order to develop a greater sense of “connectedness” between the participant and the study and to enable us to better track participants for follow-up. If mailed cards were returned to us, it would serve as an advisory that the participant had moved on and that we would need to find her through other strategies. We did not indicate the source of the card on the envelope, in order to maintain participant confidentiality and reduce any distrust of the researchers that might have been provoked by such a disclosure. This was particularly important as not all of the women had disclosed to their family members their participation in the study. All of the cards were personalized, were in English or Spanish depending upon the language preference of the participant, and were signed by all members of the study team. The “Season’s Greeting” cards that we developed in recognition of the holiday season and the new year were typically photograph cards of the entire study team. Participants consistently indicated that they enjoyed these holiday photo cards, which served to foster a greater sense of connectedness to the staff. A number of our participants displayed them prominently in their homes and were quite proud that they received them.

In order to maintain fidelity to tracking and other follow-up procedures, our study team held “participant conferences” on a weekly or biweekly basis. At each such meeting, we reviewed the status of one or two specified participants, including issues related to follow-up. Where follow-up was particularly difficult due to an inability to find the participant, we attempted to identify alternative strategies and venues where the participant might be found. In such instances, it was not unusual for study staff to literally drive up and down streets in particular neighborhoods to look for the participant, or to attend a church service at a particular church in hopes of finding the missing participant. In most such instances, we were highly successful.

Situations in which a participant is lost to follow-up may raise ethical issues. A participant may be “lost” due to circumstances in her life, increased severity of symptomatology, or because she does not wish to participate any longer. A persistent search for a missing participant could be interpreted by the participant as caring or, conversely, as undue pressure to continue with an unwanted activity. Accordingly, we re-verified individuals’ willingness to continue their participation in the study.

Results

Participants were enrolled and their study activities terminated at differing points in time over the course of the five-year study. Measuring retention from the time of the participant’s enrollment to the completion of data collection from that participant, we were able to retain 81.1% of the Puerto Rican cohort and 26.7% of the Mexican cohort over the course of the five-year study period. The underlying reasons for this differential loss to follow-up are displayed in Table 2.

Table 2 Reasons for Loss to Follow-Up, by Level
Full size table

Of the 11 Mexican participants who did not complete the study, we believe that 7 were lost to follow-up due to immigration-related issues at the macro level. Each of these 7 individuals was undocumented. During the course of the study, immigration enforcement along the southern border of the US escalated significantly, with US Immigration and Customs Enforcement (ICE) agents frequently boarding buses, trolleys, and trains to verify the legality of passengers’ presence in this country. In addition, individuals suspected of being undocumented increasingly became the targets of civilian paramilitary groups intent on effectuating the immigrants’ departure from the US [22–25]. Each of these 7 participants who were eventually lost to follow-up had communicated to us their fears of detection and deportation. Consequently, it is likely that these individuals hid themselves so far underground that we became unable to locate them or their contacts, or they were discovered and deported from the US.

Five of our Puerto Rican participants were lost to follow-up due to migration outside of the study’s geographic area, to locations that included Pittsburgh, Philadelphia, New York, and Puerto Rico. However, unlike the Mexican participants who migrated involuntarily due to fear and/or law enforcement efforts, the decision of our Puerto Rican participants to migrate was a function of individual-level factors, such as better employment opportunities or a decision to move in with family members.

Three of the Puerto Rican women and two of the Mexican women were lost to follow-up due to the severity of their psychopathology or their substance use. In one instance, the study team was consistently unable to meet with the participant due to her unreliability stemming from increased drug use. In four instances, the level of symptomatology and/or substance use increased to such a degree as to call into question the individuals’ continuing capacity to participate in the study, thereby requiring their withdrawal from the study.

Discussion

An analysis of the reasons underlying loss to follow-up within each subgroup is critical to the development of future research protocols.

Although migration was a major factor in the loss to follow-up among both Puerto Rican and Mexican participants, the context of that migration differed greatly. Accordingly, very different approaches would be required to reduce the loss to follow-up due to migration within each group in future studies. As an example, in a future study that would similarly require direct observation of the participants in their living situations, a greater proportion of our Puerto Rican participants could be retained by expanding the geographic coverage of the study to include nearby states. In contrast, retention of a greater number of our Mexican participants may demand that we exclude from participation individuals who are undocumented or whose family members are undocumented. This course of action, however, would raise ethical issues relating to the principle of justice. Recent proposed changes in immigration laws to criminalize illegal presence in the US [26] would increase difficulties associated with follow-up procedures and the likelihood that participants would be lost during the course of a study.

On the mediator/culture of care level, our experience emphasizes the importance of outreach to the community, a mediator level that facilitates a culture of trust and acceptance. Empowering community stakeholders and utilizing their suggestions on a local level allows for enhanced contact with individuals with serious mental illness and identifies those who might not be included in traditional research studies. We believe that we were able to retain a relatively high proportion of our Puerto Rican participants because of the trust that was developed between the research team, key members of the Puerto Rican community, and the individual participants themselves. The establishment of trust and comfortable interpersonal relations between the research team and prospective participants has consistently been found to be critical to successful recruitment efforts, particularly in low income and minority communities [3,13,27–31].

On a micro or personal level regarding the interaction between study staff and individuals with serious mental illness, our strategies appeared to effectively reduce or dispel distrust and fear that individuals may have felt and to establish a foundation of trust and respect between the members of the study team and prospective participants. Our avoidance of stigmatizing diagnostic classifications and our focus on “research participants” rather than “research subjects” in public forums also suggested to individuals that we valued their participation and would treat them with respect and consideration and do our utmost to maintain their privacy and the confidentiality of their data. Similarly, some minority individuals with substantial problems with illness symptoms or with substance abuse were more likely to drop out of the study. These participants were initially eligible but their condition worsened and/or they resumed substance use during the course of the study. In traditional research trials, these are often reasons for exclusion from study [32] and, for this reason, findings may not be generalizable to non-research trial populations. Our approach is to continue to encourage individuals with severe psychopathology/substance abuse to follow-up with their mental health providers and/or receive treatment for their addictions, rather than to exclude initially, which perpetuates the non-inclusion in clinical trials of minorities with more severe illness or with comorbidities.

The lessons learned in recruitment and retention of severely mentally ill minorities have multiple implications for clinical practice. Community engagement and outreach are critical to setting the stage for successful recruitment of seriously ill minority populations, and have been noted to be strong enhancers of care for individuals. Local stakeholders such as minority organizations, consumer mental health organizations, and community mental health clinics (CMHCs) are crucial in increasing awareness of SMI, reducing stigma, and providing appropriate links for specialty mental health care. Individuals of minority ethnicity may be particularly reliant on non-formal/nonprofessional therapeutic adjuncts or supports and care for minorities is likely to be enhanced by utilizing these largely community-based supports [33,34]. For example, a study by Tonigan suggested that minority individuals may be more likely to use resources in the community as a component of illness recovery [34].

One strength of our study team in conducting our project involving seriously mentally ill Latinas was ability to communicate well in the participant’s preferred language. The issue of communication between the provider/care network staff and the patient with serious mental illness is a potentially overwhelming barrier in many clinical settings [35]. About 40% of Latinos report that they do not speak English “well.”35 Additionally, ethnic match of providers and patients has been demonstrated to be important in treatment adherence [36]. Ethnic clients who attend ethnic-specific services have lower dropout rates and stay in programs longer than those using mainstream care [37]. Among individuals with SMI, there is evidence to suggest that those attending ethnic-specific services show less later use of emergency /crisis care that those who only use mainstream services [38]. However, there is a shortage of ethnic minority providers [35]. There are 173 White providers for every 100,000 Latinos and only 29 Latino providers for every 100,000 providers [39]. Increases in the Spanish-speaking clinical work-force are clearly needed to enhance clinical care of minorities with serious mental illness.

Finally, a key feature of our approach to study participants was regular and frequent contact using a variety of formats including home visits, mailings and frequent telephone calls. Outreach-type/highly personalized interventions are known to be helpful in improving illness outcomes for general populations of individuals with serious mental illness [40,41], and in some ways mirror the approach taken by our study teams in engaging and retaining minority study participants. In the case of minorities with serious mental illness, the need for flexible, non-clinic based care may be particularly great, and it is possible that greater access to personalized/flexible can lead to more enduring recovery for these disadvantaged populations.

In conclusion, a multi-level approach that addresses macro, mediating and micro barriers can optimize recruitment and retention of severely mentally ill Latinas in a research study. Successful strategies to engage these individuals in research may also be useful strategies in clinical care settings, but must be balanced with the realities of inadequate community resources, a limited multilingual clinical work force and limited access to personalized/case management services in a typical community mental health clinic.