Savin-Williams and Joyner (2014a, b) suggested that mischievous responders who provided untruthful responses about their romantic attractions in Wave 1 of the National Longitudinal Study of Adolescent Health (Add Health) might have led researchers to misidentify sexual minority youth in that sample. They further warned that such misidentifications might have contributed to erroneous conclusions that “sexual-minority youth are more problematic than heterosexual youth in terms of physical, mental, and social health” (Savin-Williams & Joyner, 2014b, p. 413). They also suggested that our critique (Li, Katz-Wise, & Calzo, 2014) was an attempt to promulgate a political agenda focused on portraying sexual minority youth as victims rather than focusing on their resilience and evidence of positive youth development.

We agree with Savin-Williams and Joyner that some adolescents in Wave 1 of Add Health might have lied about their romantic attractions, yet we do not think such responses were prevailing or largely biased conclusions from Wave 1 data on the health disparities of sexual minority youth. We also have different interpretations of the data that Savin-Williams and Joyner provided. Finally, we contend that research focused on understanding and eliminating health disparities is not a form of opposition to research focused on resilience—denying either risk or resilience would contribute to an incomplete understanding of the lives of sexual minority youth.

How Many Mischievous Responders Were There?

Savin-Williams and Joyner (2014a, b) raised a valid point that a common source of bias rooted in self-administered survey research is participants’ untruthful responses. It is therefore possible that all survey research is biased to some extent. Using data from a large anonymous online survey study conducted in Wisconsin high schools, Robinson-Cimpian (2014) screened for participants who simultaneously selected too many low-frequency responses (e.g., being deaf and blind and in a gang), whom he called “mischievous responders.” He estimated that 11.7 % of self-identified lesbian, gay, bisexual, or questioning participants were mischievous responders, whereas the corresponding percentage for heterosexual youth was 1.5 %. Although it would be interesting to apply Robinson-Cimpian’s technique to Wave 1 data of Add Health, we believe it is highly unlikely that mischievous responders would comprise the over 70 % (estimated by Savin-Williams & Joyner, 2014b) of adolescents who reported same-sex romantic attractions in Wave 1 but later self-identified as exclusively heterosexual in Wave 4.

In fact, we think that Savin-Williams and Joyner’s (2014a, b) approach to identifying “dubious” sexual minority youth is inherently flawed. Romantic attraction and sexual orientation identity are two distinct dimensions of sexual orientation that may not be concordant, even at a single time point (Igartua, Thombs, Burgos, & Montoro, 2009; Vrangalova & Savin-Williams, 2012). Even if Add Health had assessed the same facets of sexual orientation at all waves, it would still be incorrect to infer “dubious” sexual minorities from changes on the same dimension of sexual orientation, because these changes may reflect sexual fluidity (Diamond, 2008; Katz-Wise, 2014). Based on current data from Add Health, one cannot conclude that adolescents who experienced same-sex romantic attractions, but later identified as exclusively heterosexual, are “dubious” sexual minority youth, thereby justifying their exclusion from analyses. One could, however, as Robinson-Cimpian (2014) suggested, flag these mysterious responders (we call them “mysterious” to avoid premature judgments) and compare health disparities with and without them. Notably, excluding these responders did not change the finding that sexual minority youth suffer from more adverse mental and behavioral outcomes than heterosexual youth (Needham, 2012; Robinson-Cimpian, 2014).

We draw particular attention to the following issues: (1) no one has examined exactly how many of the mysterious responders in Add Health truly lied about their romantic attractions, (2) removing these mysterious participants, as identified by Savin-Williams and Joyner (2014b), does not eliminate the presence of sexual orientation health disparities, and (3) a great majority of other studies have observed similar sexual orientation health disparities as those found using data from Add Health. We do believe that Savin-Williams and Joyner’s (2014a) critique underscores the importance of precision in discussing the prevalence of attractions, sexual orientation identity, and sexual orientation developmental trajectories. Furthermore, an understanding of change in sexual orientation across time and across generational cohorts would be enhanced by including assessments of all facets of sexual orientation (primarily attractions, sex/gender of sexual partners, and sexual orientation identity) in longitudinal studies. We encourage researchers to explore, as in Austin, Conron, Patel, and Freedner (2007), what sexual orientation-related questions make the most sense to adolescents and implement these questions in federally funded studies (Institute of Medicine, 2011).

Discussing Competing Critiques

Savin-Williams and Joyner (2014a) pointed out “several inaccuracies” in our critical comment (Li et al., 2014) that invite rebuttal and further discussion. First, one critique we put forth regarding Savin-Williams and Joyner’s (2014b) original essay was their selective review of the Add Health literature. Although we appreciate their comment that essays are not meant to comprehensively represent the literature as would be expected in a peer-reviewed literature review or meta-analysis, we believe it is a critical oversight for Savin-Williams and Joyner (2014b) to exclude a study such as Needham (2012), which used Add Health data to present evidence that was contrary to their original thesis.

Second, we recommended that a more conservative statistical approach should be applied to the multiple tests comparing different types of respondents in Savin-Williams and Joyner’s (2014b) original essay, which Savin-Williams and Joyner (2014a) now provide. After applying a Bonferroni correction, significant differences in purported indicators of being a “jokester” existed only among males. Savin-Williams and Joyner interpreted the persistence of significant differences among males after the Bonferroni correction as evidence of greater confidence in their findings that boys who reported same-sex romantic attractions in Wave 1 but later self-identified as heterosexuals in Wave 4 were “by-and-large, heterosexual adolescents who were either confused and did not understand the measure of romantic attraction or jokesters who decided, for reasons we are not able to detect, to dishonestly report their sexuality” (Savin-Williams & Joyner, 2014b, p. 420). Curiously, the disappearance of most of the significant differences among the females after the Bonferroni correction was completely ignored.

Upon revisiting the interpretation of the original results and the updated statistical tests, we see at least two issues that require further discussion. We contend that the disappearance of significant differences among females should raise the question of whether the “jokester” hypothesis holds true across gender groups and thus whether this should change the conclusions made in Savin-Williams and Joyner’s (2014b) original essay. We also take issue with Savin-Williams and Joyner’s potentially problematic interpretation of gender differences in the profile of “inconsistent” responders in their original essay. Why would the endorsement of delinquent behaviors on a survey among inconsistent girls be tied to bullying, but the presence of a similar response pattern among inconsistent boys be due to mischief? Could bullying also underlie the response patterns observed in males? Could girls also be “mischievous?” Savin-Williams and Joyner themselves noted that they were not able to detect the reasons why youth would provide incongruent responses to attractions at Wave 1 and sexual orientation identity at Wave 4. How can they then be sure that “inconsistent” males provided their responses due to misunderstanding the question or dishonesty?

Third, Savin-Williams and Joyner (2014a) questioned our estimate of the prevalence of same-sex orientated youth by proposing that our interpretation of sexual orientation identity data from the Growing Up Today Study was a “sleight of hand,” because we included individuals who identified as “mostly heterosexual.” We note that the majority of analyses using data from the Growing Up Today Study to investigate sexual orientation health disparities have treated mostly heterosexuals as a sexual minority subgroup (e.g., Austin et al., 2009, Corliss et al., 2013). This is an appropriate strategy because mostly heterosexuals experience same-sex sexual and romantic attractions and same-sex sexual behavior (Savin-Williams & Vrangalova, 2013; Thompson & Morgan, 2008). In addition, similar to other sexual minority groups, mostly heterosexuals experience elevated health risks compared to heterosexuals (Vrangalova & Savin-Williams, 2014) and likely also experience some degree of minority stress. Simply put, research on sexual orientation health disparities should consider mostly heterosexuals and the current research should include mostly heterosexuals in prevalence estimates of same-sex-oriented youth, even if their inclusion results in a higher prevalence of sexual minorities than was previously found when they were not counted. Savin-Williams and Joyner (2014a) noted that the total prevalence of sexual minority youth estimated from the 1995 Massachusetts Youth Risk Behavior Survey was 2.5 % (Garofalo, Wolf, Kessel, Palfrey, & DuRant, 1998). However, Garofalo et al. did not report or include the number of youth who identified as mostly heterosexual in their estimate of the prevalence of sexual minority youth, suggesting that 2.5 % may be an underestimate.

Fourth, Savin-Williams and Joyner (2014a) criticized our discussion of the viability of the “back into the closet” hypothesis to describe the experience of “some” inconsistent youth. Frankly, we never suggested that the “vast majority” of non-heterosexuals go back into the closet (as Savin-Williams and Joyner purported we do). However, we do believe that shifting from one context (e.g., middle or high school for Wave 1 adolescents) into another (e.g., workplace for Wave 4 young adults) could lead some adolescents and young adults to identify as heterosexual after reporting same-sex attractions earlier in adolescence. We recognize a dearth of empirical support for this account (hence, our discussion of how adherence to gender role norms in adolescence may repress acknowledgement of same-sex emotional intimacy in adolescent boys); yet, we attribute this to a lack of empirical studies rather than to null findings. Case report data support plausible pathways for the “back into the closet” hypothesis, such as in the personal narratives of new professionals who go back into the closet as they enter the workforce (Feintzeig, 2014) or the experiences of lesbian, gay, and bisexual elders who go back into the closet upon moving into assisted living environments (Movement Advance Project, 2010). We deem it a potential future research direction to investigate how key life transitions impact the expression of sexual orientation identities.

Fifth, Savin-Williams and Joyner (2014a) defended their exclusion of Needham’s (2012) research from their original essay and their argument that sexual orientation health disparities in Add Health are exaggerated, by suggesting that including bisexuals with lesbians and gay males overestimates sexual orientation-based health disparities because bisexuals account for many of these disparities. As they claim, by excluding bisexuals, lesbians and gay males look more similar to heterosexuals on health outcomes. We argue that excluding subgroups of sexual minorities from an analysis of sexual orientation-based health disparities is problematic for at least two reasons. First, intentionally excluding bisexuals is unethical, particularly when data on this subgroup are available. It is important to note that researchers sometimes exclude bisexuals or merge bisexual participants with other sexual minority subgroups in an analysis due to issues of statistical power (which is often acknowledged as a limitation). However, excluding bisexuals from a larger discussion of sexual orientation health disparities is tantamount to erasing their experiences or treating them as a non-valid sexual minority subgroup, thereby contributing to bisexual invisibility. Second, meta-analyses suggest that Savin-Williams and Joyner’s (2014a) assertion simply is not true; health disparities between lesbian/gay individuals and exclusively heterosexuals exist independently from disparities between bisexuals and heterosexuals (King et al., 2008; Marshal et al., 2011; for an exception, see Marshal et al., 2008).

Misrepresentation of Our Critique

What was most astonishing in Savin-Williams and Joyner’s (2014a) response was that they characterized our critique as having a “political agenda.” If they are implying that we took a moral stance in writing our Letter, then they are not incorrect. Indeed, a moral stance of supporting sexual minority youth by acknowledging both risk and resilience is necessary to this work. We never argued that Savin-Williams and Joyner’s research or questioning of Add Health data is harmful to sexual minority youth, but we instead critiqued their disregard of the extensive evidence of health disparities found in other datasets (even undermining their own work in these areas). We provided empirical support for our critique whenever available.

Savin-Williams and Joyner (2014a) described our agenda and the agenda of health disparities researchers more generally as focused on characterizing sexual minority youth as mentally ill in order to secure publications and grant money. Although we agree that federally funded grants for research on sexual minority populations tend to focus on health disparities (most notably HIV/AIDS, not mental health) (Coulter, Kenst, Bowen, & Scout, 2014), we believe Savin-Williams and Joyner’s (2014a) sweeping characterization of health disparities research is destructive. From an epidemiologic perspective, detecting health disparities and their underlying mechanisms is fundamental to promoting health equity. Savin-Williams and Joyner’s (2014a, b) essay and response highlighted the flaws of one notable epidemiologic dataset as a jumping point for questioning the magnitude of sexual orientation health disparities in general. Their position ignored the totality of evidence from multiple studies indicating that sexual minority youth and adults face numerous challenges due to their stigmatized identities which may adversely affect their health. Instead of placing health disparities and resilience research in opposition to each other, a more constructive approach would be to recommend that they be conducted in conjunction with one another. Researchers predominantly focused on sexual orientation health disparities should also assess indicators of thriving and resilience as these may yield important information for health promotion. Likewise, researchers predominantly focused on resilience of sexual minority youth should also assess risk factors and negative experiences to validate any adversity the youth may have overcome and to avoid the pitfalls of assuming that sexual minority youth do not face oppression.

What Is at Stake?

Although we disagree with many of Savin-Williams and Joyner’s (2014a) critiques, we draw attention to the usefulness of engaging in this debate. Discussing the quality of existing research on sexual minority youth and consequently the conclusions and implications that can be drawn is the only way to truly move the field forward and encourage the production of better-designed studies to enable us to increase knowledge about this still-understudied population. When we wrote our critique of Savin-Williams and Joyner’s (2014b) essay, we were primarily concerned that the use of terms such as “national dataset,” “jokester responders,” “overestimate,” and “exaggerated health disparities” in conjunction with research on sexual minority youth would result in other researchers not trusting well-established findings regarding sexual orientation health disparities. While we agree that Add Health has a number of flaws, studies that have used this data to demonstrate health disparities between sexual minority and heterosexual youth yield findings that are consistent with other surveys and samples. In addition, detecting and removing “mischievous responders” from Add Health did not level the health disparities between sexual minority and heterosexual youth (Needham, 2012; Robinson-Cimpian, 2014). Thus, we must not disregard all Add Health research on sexual minority youth.

Considering both the limitations of Add Health and robust findings across studies regarding sexual orientation health disparities, we offer a vision for the future of this field. First, we recommend better measurement of sexual orientation, including well-defined measures assessing multiple dimensions of sexual orientation. However, it is not enough just to measure multiple dimensions of sexual orientation; researchers must also be mindful of not conflating these dimensions in their analyses or interpretation of the data. Second, we recommend an integration of health disparities and resilience research in studying sexual minority youth. These two approaches need not exist in opposition; rather, they can be complementary in providing a more holistic picture of sexual minority youths’ lived experiences. This point is particularly relevant in reference to the “illness” perspective that Savin-Williams and Joyner (2014a) attributed to health disparities researchers. Savin-Williams and Joyner argued that sexual minority youth should be considered “normal” and “ordinary” adolescents. We certainly agree that positive portrayals of sexual minority youth are important, but we also argue that it is important to acknowledge the unique experiences and contextual factors impacting these youth. The statement “sexual minority youth are normal” is similar to saccharine terms such as “colorblind” (which ignores racism) or statements such as “men and women are equal” (which ignores sexism). Sexual minority youth do not necessarily lead ordinary lives; they lead extraordinary ones, often in the face of extreme prejudice and injustice. We must recognize these youth’s incredible strengths just as much as we must understand and seek to address their vulnerabilities.