Interpretation-Driven Guidelines for Designing and Evaluating Grounded Theory Research: A Constructivist-Social Justice Approach

Abstract

This chapter is meant as guidance for not only investigators designing qualitative research projects but for reviewers assessing manuscripts using qualitative methods. An interpretation-driven approach to design and review of qualitative research is proposed as an alternative to the approach of setting in stone rules that consist of procedure-driven prescriptions. It walks the reader through the process of considering the design of an individual study and its coherence with the epistemology of the researcher(s). This approach emphasizes the centrality of the role of interpretation that is best evaluated in relation to an epistemology, within the context of the specific study characteristics, and in service of the scientific, practice, and/or social justice goals at hand. It presents context-sensitive guidelines for researchers and reviewers to use in designing and evaluating qualitative research studies. Within the chapter, there is a specific focus on grounded theory (e.g., Glaser and Strauss, The discovery of grounded theory. Aldine, Chicago, 1967); however, many of the principles put forward to guide study design and research review may be relevant across qualitative methods.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Developing Interpretation-Driven Guidelines for Constructivist-Social Justice Qualitative Research

The purpose of this chapter is to outline an approach to method design and assessment that upholds the scientific integrity of the interpretive process within qualitative approaches to research—with a specific focus on grounded theory method (e.g., Glaser and Strauss 1967). I use the term interpretation-driven to describe the approach because I am advocating that privileging the interpretive process requires consideration of the coherence between methods and epistemology with reference to the specific characteristics and goals within an individual project—as will become clear. This initiative is a response to the development of qualitative approaches that are procedure-driven—that is, guided by rules that privilege the integrity of a method when assessing research.

Interpretive judgments are important across both qualitative and quantitative approaches in ensuring that methods selected are appropriate for the questions being asked and in deciding upon the meanings of findings. In quantitative research, however, the process of identifying patterns across numeric data is relinquished to mathematical procedures. This mathematical aspect of analysis renders procedure-driven rules necessary to preserve the integrity of the calculations so that the resultant findings are valid. For instance, a statistical method requires a certain number of participants to have the power required to be informative—regardless of the researchers, participants, or the study context at hand.

In contrast, qualitative methods have their process of pattern identification located in the subjective interpretation of data—and so these procedure-driven assessments may be inappropriate when applied to qualitative methods because they mistake the nature of the process of pattern identification. Instead, study rigor should more accurately be assessed with a central focus upon the interpretive process that can best be understood as it is located within a given method, question, research goals, study characteristics, and epistemologies at play. The integrity of qualitative research then depends upon adapting the procedures within a method so as to best enable the interpretive process.

The task of developing interpretation-driven guidelines is complicated for a number of reasons. In a field where qualitative research (i.e., “human science” as distinguished by their epistemology from “natural science”; see Rennie 1997) is still new to many editors and reviewers, it can be helpful to have some direction on assessing these methods. Procedural-driven guidelines can be easier to apply because researchers and reviewers do not need to consider theoretical and contextual factors and do not need to develop sophistication in their thinking about methods. They can seem friendly and accessible. At first blush it may seem reasonable that good rules developed for one qualitative method should generalize to all.

For the process of initially learning qualitative research approaches, procedural guidelines can be helpful as a deeper understanding of epistemology might only emerge after they are engaged in analysis. For instance, I have written papers myself that include recommendations for teaching grounded theory to graduate students (Levitt et al. 2013). Some sets of guidelines are written with a stronger understanding of the epistemological underpinnings of qualitative research practice and address either qualitative research on whole (e.g., Elliott et al. 1999) or grounded theory methods more specifically (e.g., Fassinger 2005; Dourdouma and Mörtl 2012). In general, these guidelines allow for the development of professional standards, create a common language, communicate practices, and expedite assessments of research.

Although useful for those purposes, if taken as procedural prescriptions, these approaches can dictate practices rather than generate sets of considerations that need to be weighed together to best meet the multiple goals of research within an epistemological context. Their danger is not only in discouraging complex thinking in reviewers but in discouraging the submission and acceptance of qualitative research that uses nontraditional epistemologies and/or constructivist-feminist methods (see Frieze 2008 for an example of such a journal policy). In this paper, I argue that procedural recommendations or requirements encourage incoherent research that is judged superficially, constrain the evolution of methods in the field, and, thus, work against the goals of empirical research itself.

In this chapter, I put forward an alternative way of assessing research design—as an expression of an epistemological stance in relation to study characteristics and aims. The epistemology is constructivist as it views people as forming the meanings that make their world intelligible via methods like language, narrative, and personal constructs (e.g., Neimeyer 2009; Raskin and Bridges 2008). Also in this perspective, method is thought to serve scientific goals (as opposed to science being defined by sets of procedure-related decisions) as well as other goals, such as clinical, didactic, or social justice goals. My approach to research tends to have a social justice orientation that needs to be understood as having implications for research methods. For instance, social justice (i.e., feminist or multicultural) research tends to investigate and shift into focus marginalized experiences and so research may be used to serve both the goals of generating knowledge and of liberation—which at times might be at odds. Stiles (see Chap. 8) in this book offers a complimentary discussion in which he divides research goals differently (i.e., as developing theories, enriching understandings, or collecting evidence) but describes how each purpose can influence the strategies of research and frameworks of understanding that work. Although these considerations are focused upon the use of grounded theory within a constructivist-social justice framework, many can be adapted for use with other methods or epistemologies. I will begin by describing my journey as a qualitative researcher and my own epistemology to provide some context for what follows.

2 Situating the Interpretation-Driven Approach

2.1 Methodophily Versus Methodolatry

In qualitative research methods, it is incumbent upon researchers to provide information on their relevant personal perspectives, histories, and philosophies as a framework within which readers can assess their inquiry. In that tradition, I am providing some historical context to frame the development of my own expertise in qualitative methods. I first began using qualitative methods in graduate school (e.g., Levitt 1999). It was in the not-too-distant time before qualitative method courses existed in many North American psychology programs, but I learned about these methods from my mentors in the York University Psychology Department. They were pioneers in these approaches—Lynne Angus (e.g., Angus et al. 1999), Leslie Greenberg (e.g., Greenberg 2007), and David Rennie (e.g., Rennie 2000)—who introduced me to a variety of qualitative methods as well as developing my appreciation for psychotherapy process research (see Rennie 2010 for a history of this particularly innovative department).

The lingering impact of David Bakan’s (1967) early work at York was clear. His insightful perspective on statistics and research methods contributed to a climate in which alternative methods could be explored and developed. In particular, his cautions about “methodolatry” in our field have become classic—that is, our becoming so attached to a method (e.g., the clinical trial) that it begins to dictate research agenda instead of being one of many tools that can be used to answer theory-driven questions dictated by researchers (see Gelo 2012; Slife and Gantt 1999 for critiques of the former process). When set down in procedural terms, methods can be reified and take on a life of their own that can become disconnected from goals of science—to know and learn. Instead, aspects of the methods can be tailored to the questions, resources, the epistemology of the investigators, and purposes at hand.

While I continue to be interested in exploring many qualitative methods [see Rennie and Frommer (2015) and Mörtl and Gelo (2012)], I have become particularly attached to grounded theory—perhaps because I have found it flexible enough to be used across many topics of interest to me. It is a method that has allowed me to explore other people’s intimate experiences of struggle and making meaning of their world. I have had peak intellectual moments using this method—asking monks about their experiences of developing wisdom in India (Levitt 1999), learning about psychotherapy clients’ unspoken experiences in sessions and eminent therapists’ experiences of guiding therapy (e.g., Levitt 2001; Levitt and Williams 2010), and working to understand the ways gender is constructed within gay, lesbian, and transgender cultures (e.g., Levitt and Hiestand 2004). Each time I conduct a study, I feel privileged to talk with participants about topics that fascinate me and to learn from their experiences. Whenever I lecture on qualitative methods, I like to impress upon students how joyful and stimulating this research can be. Really—what could be better than being engaged in understanding the inner workings of a compelling subject and then having the opportunity to generate new understandings from those discussions?

I have used grounded theory in the context of three programs of study. My work on domestic violence and faith has included studies that examined the perspectives of perpetrators, survivors, and religious leaders (e.g., Levitt and Ware 2006). My research on gender has focused on sexual orientation identities and minority stressors (e.g., Levitt and Hiestand 2004). And my psychotherapy work has explored a variety of common factors from both clients’ and therapists’ perspectives (e.g., Levitt and Williams 2010). Over the last 15 years, I have authored or coauthored approximately 50 publications rooted in qualitative analyses, including over 30 projects using grounded theory methods, and have supervised many others. The methods have included content analysis, narrative analysis, metaphor analysis, task analysis, hermeneutic analysis, and grounded theory.

As well, I have taught a variety of methods in qualitative research courses for graduate students (e.g., phenomenology, Wertz 2005; discourse analysis, Gergen and McNamee 2000; consensual qualitative research, Hill et al. 2005). In my course, I survey different qualitative methods, but I guide the students through an intensive grounded theory class project that begins in the first class and runs through the semester. Together, we publish our joint project at the end of the semester (see Levitt et al. 2013 for a detailed description of this course).

This chapter has provided the opportunity to reflect upon my use of grounded theory and the ways I have conceptualized study design. When reviewing my work, I find that I have made quite different interpretation-driven decisions, depending on qualities of the participants, research context, or study topic. In the chapter, I consider the points where in using the method I have made decisions about my method design, and unpack those choices. I proceed to explicate this decisional process—both to stimulate consideration around the elements of the method and to interrogate my constructivist-social justice epistemological approach and its expression in relation to my study goals. Although my research has a scientific purpose broadly, often my psychotherapy research also is directed by the goal of developing clinical recommendations, whereas my domestic violence and gender research also is driven by the goal of advocacy.

2.2 Ontological and Epistemological Considerations

It is my ambition that this essay can aid other researchers in tailoring grounded theory designs that are best suited for their own studies and can guide reviewers and consumers of research to consider the role of their own ontological and epistemological beliefs, aims, and study characteristics when assessing research. There are a variety of approaches from which researchers have used grounded theory methods from post-positivist to critical constructivist approaches (e.g., Fassinger 2005). While Morrow (2005) provides a concise and insightful overview of the way criteria shift when moving between postpositive, interpretive constructivist, and critical ideological paradigms, the present paper describes specific ways that method can be adjusted within an epistemological framework given different study characteristics and research aims. It is toward that end that I present a description of my own constructivist-social justice, ontological, and epistemological approach.

Most often in psychology, researchers categorize these positions in terms of competing perspectives envisioned on continua (e.g., Fletcher 1996; Gelo 2012; Guba and Lincoln 1994; Ponterotto 2005). On a spectrum from naïve realism, in which one true fixed reality is directly apprehendable, and relativism, in which reality is grasped through our own understandings and meanings, I adopt a constructivist-social justice approach in which I study understandings of experience as shaped by social and cultural processes and values, often in the form of structural and systemic forces. To some extent, I have developed this ontological approach through my research on gender and sexual orientation and learning from participants how they experienced parts of their own identities as given and inalterable but as having expressions shaped by their context (e.g., Levitt and Hiestand 2004).

On a spectrum in which on one end inquiry is seen as objective and dualist, in which scientists intellectually can observe the truth, and on the other it is understood as subjective and transactional, in which embodied researchers engage with participants to cocreate findings, I see the epistemic task as inherently subjective, interactive, and embodied. By this I mean that our subjectivity, along with our values, blind spots, and biases, is the unavoidable lens through which data are interpreted. The investigator helps participants to articulate their experience, but this cocreation is limited as the exploration is rooted in the participants’ sense of the phenomenon of interest, which might be experienced as entirely created or as having roots in historical, social, or biological realms.

In this approach to methodology, the interpretive task is central and the assessment of the reflexive process of analysis is valued. For instance, the guidelines I put forth convey more faith in analyses that are conducted by researchers who become deeply immersed in a data set (e.g., lived experiences, intensive study) than in analyses conducted by less-engaged observers who can arrive at agreement. At the same time, the data communicate the participants’ engagement and experience of their reality. As a result, the principles I put forth in the guidelines emphasize obtaining data that are as clear and useful as possible and keeping analyses grounded to maximize the returns from the information therein. Maintaining the balance between these two tensions then becomes the central concern during method design and evaluation for investigators and reviewers. The approach I have adopted is congruent with the methodical hermeneutic approach that Rennie (2000) argued situates grounded theory as a synthesis of relative and realist epistemologies—as an interpretive analysis (relativist aspect) of empirical participants’ reports of their experiences in the world (realist aspect).

In addition, the principles reflect social justice and pragmatist approaches to method. A social justice epistemology is present in that I am using my inquiry with an objective to bring into focus marginalized experiences with the aim of raising consciousness and repairing practices that maintain social biases (e.g., Fine 2012; Harding 1986) and an awareness that my process of inquiry itself can be fundamentally influenced by these biases. In my own approach to research, I incorporate insights from several different feminist and multicultural epistemologies: (1) I have been influenced by feminist standpoint epistemologies (e.g., Harding 1998, 2011) in that I tend to start my inquiry process in research focused upon the standpoint of those who have lesser power with the understanding that marginalized groups may have perspectives that usefully can lead to understanding (e.g., clients in therapy, marginalized sexual orientation groups, victims of violence). (2) At the same time, I do not assign epistemic privilege to minority groups (e.g., Bar On 1993; Longino 1993) by confining my research to their perspectives or assigning them sole authority on a subject, but seek to study how those experiences interact with those who have more power and with systems (e.g., therapists, dominant sexual orientation groups, religious authorities). (3) While I am interested in examining and situating my own position as a researcher as it interacts with my reflexive process (e.g., Code 1996), I tend to be interested in understanding communities as generators of understanding in relationship to their own experiences and needs. I see individual participants as members of communities that are shaped by privileging and oppressive forces and that are not monolithic themselves and are in flux over time (see Diamond 2006; Levitt 2006).

A pragmatic framework toward study design is at play because the constraints of the study characteristics (i.e., qualities of the phenomenon, participants, and researchers) are considered in terms of their implications for method, and the effects of methods, in turn, are evaluated in terms of their implications for praxis in light of the research goals (e.g., Peirce 1958). That is, throughout the principles described in this paper, I consider how different method-related decisions influence the processes of gathering clearly articulated useful data, training analysts who are positioned to conduct a thorough analysis, and conducting assessments of research that do not compromise a grounded process of theory creation. These decisions are made to maximize both the epistemic coherence and the practical returns of each study. For instance, I recognize that method-related ideals may need to be altered to give voice to participants who are marginalized and face barriers to research participation. By generating research that is not well represented in the literature, these studies can serve both research and advocacy goals. This is one example of one way in which epistemological beliefs can guide method. The following principles are articulations of rationales for making such decisions.

3 Developing Principles for Grounded Theory Research

To make the principles that follow easier to locate, they are organized within the context of the components of a method section in a research paper (see Table 22.1). Within each component, common questions from graduate students or investigators new to grounded theory are addressed, and responses are presented to highlight the ways epistemological and study-specific factors influence design. In the course of answering these questions, I will provide a description of how and when I conduct grounded theory (as opposed to other qualitative or quantitative methods) and formulate the principles that have driven some of my own study design decisions.

Table 22.1 Principles of interpretation-driven research design and evaluation

Before describing principles, however, it can be helpful to provide a brief overview of grounded theory methods and purposes. There are a number of variants of grounded theory method (e.g., Charmaz 2006; Dourdouma and Mörtl 2012; Glaser and Strauss 1967; Rennie and Fergus 2006; Rennie et al. 1988; Corbin and Strauss 2008). They can be based within multiple epistemologies and use different procedures and terms.

Across grounded theory approaches, however, are the following typical procedures: (1) At first, the interview text is divided into units that capture meanings being conveyed (Rennie et al. 1988, recommend using meaning units, a construct borrowed from phenomenology for this process; Giorgi 2009). (2) As the units are created, labels are assigned to each unit to reflect the meaning therein. (3) Using a process of constant comparison, the researchers then compare the meaning units to one another and create categories to reflect the commonalities identified (i.e., open coding). As the data can be sorted into as many categories as are relevant to its content during this analysis, the categories are not independent of one another. (4) As initial level categories are formed, they are compared to one another and higher-order categories reflect the commonalities therein (i.e., axial coding). By repeating this process with each layer of categories and incorporating new data (i.e., selective coding), a hierarchical structure of categories is developed. Finally, a core category is conceptualized, representing a central connection between categories at the apex of this hierarchy. (5) Throughout this process of data analysis, participant recruitment is being conducted strategically to enrich the developing theory (this process being called “theoretical sampling”). (6) When new categories no longer appear to be forthcoming from the addition of new interviews into the hierarchy, the analysis is thought to have reached the point of saturation signifying that the data collection is comprehensive. Grounded theory analyses usually require a smaller number of participants than quantitative studies—often saturating with approximately 6–20 interviews. These studies can analyze a wealth of data, however, as interviews typically range from 1 to 2 h and can easily result in 30–80 transcript pages of data per interview, replete with rich description about a phenomenon containing meanings and distinctions generated by the participant. (7) Throughout this process of analysis, memoing or note-taking is used to record shifts in hypotheses and conceptualizations in an attempt to record and restrict the influence of a priori ideas upon the analytic process as well the influence of theories that develop during the analysis.

Increasingly, these grounded theory procedures are imported into other methods—perhaps to add clarity to qualitative methods that have been described within vague or conflicting approaches—even when the purpose of the method is not to generate a theory. Because there are so very many different qualitative methods and versions thereof, it may be that it is easier for researchers to appeal to grounded theory methods that are familiar to many readers and reviewers and are often clearly defined. In particular, processes of constant comparison, of developing lower- and upper-level categories (although usually not a complete hierarchy), and of saturation are combined with other qualitative methods to strengthen the process of data analysis. For instance, a content analysis might be conducted in a way that uses a process of constant comparison to create a structure of initial and higher-order categories to answer multiple questions even though a central theory is not being formed (e.g., Kannan and Levitt 2009).

4 Method

4.1 Deciding Upon a Method of Analysis

4.1.1 Is My Question Consistent with a Grounded Theory Method? (Question 1)

Answer: Advisors often spend a good deal of time helping their students decide upon a method to use in a given project. Although grounded theory might demand too much effort in some instances, it can also be too limiting, depending on one’s objectives. Grounded theory focuses on identifying the commonalities among participants. That is, although differences can be noted between subsets of participants, the method functions by creating categories that are based upon commonalities within these subsets. Grounded theory also tends to be focused more upon the content of participants’ speech and what they can report experiencing than focused upon the structure or context of their reports, although this is not a hard line [see Mörtl and Gelo (2015)]. There are four main considerations that I have found helpful in identifying the primary research goals and selecting appropriate method for given projects.

• 1. It might be preferable to conduct a quantitative statistical analysis at times when the primary research question is to compare mean or modal responses across groups or to verify specific a priori hypotheses within groups. Generally, I prefer to begin a program of study with a qualitative analysis to discover meanings intrinsic to a group and then assess the theory developed using quantitative methods (see Gelo et al. 2008, 2009 for other arguments for mixed-methods research). There are times, however, when a theory is already well developed and it makes sense to begin by evaluating or validating an existing theory with a quantitative method. As a corollary of this approach, there are times when a quantitative data analysis leads to questions that can best be resolved through a qualitative inquiry.

• 2. In contrast, when the primary purpose of an analysis is to collect or compare responses to separate main questions, it might be preferable to conduct a content or thematic analysis (e.g., Braun and Clarke 2006). In the process of conducting such an analysis, methods might be imported from grounded theory (i.e., open coding, constant comparison)—for instance one might decide to create a few levels of categories for each response set without creating a complete hierarchy. The development of an extensive hierarchy and a core category would not make sense in this case because the research goal is not the development of a theory of one phenomenon. (To see an example, I have incorporated grounded theory methods within a content analysis when wanting to compare developing feminist therapists’ training needs with their program experiences; Kannan and Levitt 2009.)

• 3. If the primary goal is to develop in-depth theories about one question from different perspectives, it might be necessary to conduct complete qualitative analyses with multiple sets of participants. These analyses could result in separate hierarchies with core categories and then the hierarchies being subjected to a secondary analysis to compare and contrast the similarities and differences between them. For instance, I have used this approach when comparing clients’ and therapists’ experiences of challenges to clients’ beliefs in therapy sessions (Williams and Levitt 2008b) or when conducting research on how victims, perpetrators, and faith leaders experience faith as influencing domestic violence (e.g., Knickmeyer et al. 2003; Levitt and Ware 2006; Levitt et al. 2008).

• 4. Finally, when a purpose of the research is to identify patterns that extend beyond the participants’ experiences (but are about one question and within one perspective), it can be helpful to conduct a secondary analysis within a hierarchy to glean trends and differences—an analysis of the analysis. This goal usually occurs when researchers are interested in the data for a secondary purpose, such as developing guidelines for practice.

To give an example, in conducting research on eminent therapists’ experiences of directing the process of therapy, we developed a hierarchy grounded in our participants’ experiences but then wanted to develop principles for practice that focused clinicians on key decision-making points in their sessions (e.g., Levitt and Williams 2010). Some of these decision points were not described in any of the therapists’ experiences directly because they were derived by our noticing the different types of description between groups of therapists. For instance, CBT therapists were more likely to position the change moment outside of the therapy exchange (e.g., in homework exercises, experiments) and psychodynamic and humanistic therapists tended to locate the change moment within the therapy exchange. This structural difference allowed us to make sense of the comparably stronger focus on the relationship by the humanistic and psychodynamic and the comparably stronger focus on designating homework and motivating clients to complete it by CBT therapists. While it allowed us to develop multiple principles for treatment, they were not grounded in the experiences in either group of therapists but the comparison of groups within the hierarchy for the purpose of identifying practice-relevant decisions.

Our importing of a hermeneutic analysis of the differences across these approaches allowed us to conduct this secondary work due to its attunement to both contextual and covert factors (see Rennie 2000 on the view that grounded theory is a form of methodical hermeneutics itself). Hermeneutic analyses can allow for exploration across both content and structure of an identified pattern in data, with a particular attention to the pre-understandings and philosophies underlying that pattern (e.g., Packer and Addison 1989). We have used this method when looking at conflicting client experiences in therapy as well (e.g., Levitt et al. 2006). When clients’ responses conflicted, the context and assumptions within the interview text were examined closely to provide differential guidance on when or under what conditions an intervention might be helpful. The driving consideration across these points is: To decide if a method (e.g., grounded theory) is appropriate, the researchers consider whether the question aims to develop a model of common experiences of one experience (might use grounded theory alone) versus to conduct a comparison of experiences between groups of participants or for purposes beyond understanding participants’ experiences (might use other methods or adapt grounded theory methods to meet this aim).

4.1.2 Is a Grounded Theory Analysis Too Intensive Considering the Data I Will Collect? (Question 2)

Answer: There are times when the intensive analyses in grounded theory are not necessary. The time that goes into building a hierarchy is worthwhile when the results that are produced are complex, rich, and shed light on processes that would otherwise be difficult to conceptualize or to do so in a manner that has fidelity to the experience of the event. Sometimes, however, this level of focus is simply not necessary to obtain useful results and would create undue labor. In my advising on method, I especially caution students when: (1) the study question is relatively simple to answer (for instance, studying participants’ reactions to different course structures); (2) the participants have not had a long or in-depth experience that would have complex enduring associations or meanings (for instance, in a study on participants’ reactions to a one-session intervention); (3) the participants do not have a great deal of insight or cannot communicate that insight in detail (e.g., if conducting a study on young children’s experiences of rewards or punishments); or (4) results in the form of a detailed hierarchy are not necessary for the purpose of the analysis (e.g., detailed hierarchies may not be necessary for intervention development or for generating dialogues between groups of people).

Typically, I have an estimated length of time for the interviews, which I revise as they are conducted. That said, it is hard for me to imagine that interviews shorter than an hour in length can produce new data that might result in a useful analysis. Similarly, when considering conducting this form of analysis with written text, it is important to have not only a sufficient quantity to find overlapping meanings but a quality of depth that would make an intensive analysis worthwhile. If the participants can communicate the entirety of their experience in 15 min or in a paragraph, you likely will not have the depth of information to make this analysis worthwhile. The principle in this case is: The scientific goal of the analysis (i.e., an in-depth understanding of the components of an experience) and the characteristics of both the phenomenon (i.e., complexity) and the participants (i.e., ability to clearly articulate their experience) should be considered when justifying a grounded theory analysis. This being said, grounded theory methods can be used when these conditions are not met, but other methods (e.g., phenomenology, narrative analysis, or discourse analysis) [see Mörtl and Gelo (2015)] might produce similar results with greater ease and in a more applicable format.

4.2 Participants

4.2.1 Which Types of Difference Are Most Important in Participant Recruitment? (Question 3)

Answer: Differences among participants are seen as a strength in grounded theory approaches as researchers seek to diversify sources of information to develop results that are as rich and encompassing as possible. The method of theoretical sampling in grounded theory asks researchers to review their emerging theory within the developing hierarchy to identify gaps and then seek to recruit participants whose differences in perspective can best enrich the theory (Glaser and Strauss 1967; see Chap. 8). The logic underlying this participant sampling method is different than in quantitative studies, which have as a goal the estimation of probability and so tend to use larger numbers of participants and procedures like random or representational. I find it helps for me to consider the pool of participants that I wish to understand and the different perspectives therein that would result in a useful theory.

I typically find that I am in a difficult position when I explain to reviewers of my work not only why this diversity is important in this method, but how choices are made in the recruitment process. A challenge is that, given that grounded theory requires a relatively small set of participants, it is often impossible to have a reasonable number of participants from every type of diversity in one’s sample (e.g., race, gender, sexual orientation, ability, age, ethnicity, sex, therapeutic orientation, therapeutic issues, time since therapy ended, length of time in therapy). As a result, choices have to be made in each study. By consulting the research and theoretical literature on the question at hand, I often can identify some factors that might be most likely to influence the question being investigated. For instance, if I am conducting a study on clients’ experience of differences from their therapists, then forms of differences between the clients and therapists likely would be a form of diversity that I would work hard to obtain in my participants (e.g., Williams and Levitt 2008a). In contrast, if my goal is to understand how therapists guide clients through sadness, then differences in psychotherapy orientation might be a primary form of diversity that is sought, as therapists most often understand their methods of delivering treatment via their psychotherapy orientations. That being said, it still may be hard for me, as a white, middle class, able-bodied, Jewish, femme lesbian to conceptualize how other cultural backgrounds might affect the experience I am researching, and because the literatures often do not include discussions of cultural factors, there typically are secondary forms of diversity that I would seek. In the course of interviewing participants, I continue to consider which types of cultural factors (or other factors) might be important and then can seek to recruit those participants (in keeping with the concept of theoretical sampling; Glaser and Strauss 1967). The principle at hand here is: Diversity within participants is sought out to clarify likely differences in experiencing that appear most relevant to the subject at hand. Typically, this includes considering how cultural factors may be influential throughout the analytic process, as their impact might be difficult for investigators to assess, and seeking out participants with relevant cultural diversity characteristics. This check stems from a social justice approach to research in which there is an effort to represent and include perspectives that might be marginalized if not deliberately considered and integrated into the theories put forward by psychologists.

4.3 Investigators/Interviewers

At times, research may benefit from having one primary or sole interviewer. In other studies, interviewing may be shared between a few researchers or across a large research group. The interview is the most important moment of a qualitative research study. If it does not go well and rich data are not obtained that shed light on the subject, the analysis will not be fruitful no matter how wonderful the methods of analysis. Researchers are required to think on the spot within an interview context to clarify data and direct the interview focus. For each study then, decisions need to be made on who will be interacting with the participants and gathering data.

4.3.1 How Does One Decide Who Should Be Conducting the Interviews? (Question 4)

Answer: The following three factors may be useful to weigh together when deciding who should conduct interviews within a given study:

• 1. Interviewing skills. All interviewers who I work with undergo qualitative interview training in which they observe and perform role-plays, and receive feedback, before they conduct their own interviews. Usually, their interviewing skills improve dramatically after obtaining feedback on their first few interviews, so I ask students to transcribe and critique their first interviews within 3 days of conducting them and then we review them together. When the participants are highly verbal and self-reflective and are likely to have already talked with others about the experience under investigation, it is not as important to have interviewers who are as highly skilled. In contrast, interviewing skills become particularly important either when conducting interviews on topics that are difficult to discuss or when participants have difficulty articulating their experience. For instance, using interviewers with a higher level of interviewing skills might be more important when asking depressed clients about their experiences of sadness in psychotherapy (e.g., Henretty et al. 2008) than when asking people about their experiences of curiosity (Levitt et al. 2009).

• 2. Knowledge about the subject. There are different advantages and costs when interviewers are either naïve or expert about a subject at hand. I prefer to have investigators who tend to be naïve about the phenomenon under focus when I think it will prompt interviewees to unpack their experience of it more. For instance, an expert psychotherapist might be more likely to explicate change processes when talking to a graduate student than to another experienced psychotherapist (e.g., Levitt and Williams 2010). In contrast, expertise is particularly important when the question at hand requires subtle differentiations to be made and when the participants need more guidance to focus on a less familiar topic. For instance, I was the primary interviewer in a study of psychotherapeutic wisdom, as I had more therapy experience than my coinvestigator and felt better able to guide participants to clarify the nature of this ambiguous concept (e.g., Levitt and Piazza-Bonin 2011).

• 3. Power (Im)balance. Another factor I consider is how power differentials might influence the interview. The impact of cultural oppression and disparity on participants’ comfort with disclosure can be important to consider. For instance, when interviewing men who had committed domestic violence, I chose to have the graduate student investigator act as the primary interviewer (although she observed me conducting the initial interviews and I attended most of the interviews; Levitt et al. 2008). I reasoned that it could be difficult for these men (some of whom were men of color and all of whom had lower socioeconomic means) to talk about perpetrating abuse to a white person of professional and economic privilege, but, alternatively, talking to a younger female student might allow them to speak more openly. In another study, I conducted the interviews of butch and femme lesbians on their gender experience because I was part of that community and was trusted. Participants repeatedly told me that they would not have confided in an interviewer who was outside of their community.

The study-level principle that can be distilled is: Decide who should conduct interviews after weighing together the need for interviewing skills, the knowledge needed on a subject matter, and the influence of cultural oppression and disparity upon the participants’ ability to disclose and articulate information clearly. After considering all three factors, we decide upon the types of training that are necessary, and who is best equipped to perform the interviews. These decisions can change as investigators receive more training and as the circumstances shift. For instance, in the wisdom project mentioned, two of the interviews I had conducted were accidentally deleted and had to be repeated. Having conducted the first interview, I worried that it would be difficult for participants to elaborate on their thoughts again with me and so the graduate student coinvestigator conducted the second interview as a more naïve interviewer could better gather these data.

4.4 Data Collection

4.4.1 Recruitment

The main question I have encountered relating to participant recruitment is if screening should be conducted before an interview.

4.4.1.1 Should a Screening Be Conducted Before the Interview? (Question 5)

Answer: A screening process to decide whether or not to include participants in a project before an interview can be important to implement when (1) the criteria for participation might be unclear (e.g., what does it mean for domestic violence to be “resolved”); (2) participants might be opting in who are unqualified to participate (e.g., wanting to get course credit in a subject pool); or (3) when there are issues around safety that needed to be addressed (e.g., interviewing women who had experienced domestic violence and needing to ensure that they would be safe; Knickmeyer et al. 2003). For instance, when studying psychotherapy phenomena that I am confident routinely occur (e.g., silences), I will not conduct screening if I am prepared to compensate participants for their time on the chance that the phenomena do not occur in the given session that we are examining (in which case I might ask instead about the lack of silence). In this case, the principle is: Conduct screening interviews if it is difficult to know if the participants are able to describe usefully the topic of interest, because the topic is unclear or uncertain, the participants are suspect, or if their safety needs to be secured.

4.4.2 Data Collection Procedure

There are two questions I most often encounter that relate to data collection.

4.4.2.1 Should I Analyze Interview or Written Data? (Question 6)

Answer: Sometimes an analyst only has access to written data—as is the case with analyses of historical documents or archived data. Although grounded theory usually entails a semi-structured interview protocol that is organized around one central question (the question whose answer is the theory being developed), written analyses can lead to productive research as well. Although the analysis written data can be wonderful for certain purposes (e.g., item development, analysis of historical documents), there are disadvantages to be kept in mind. Since the purpose of the interview typically is to help the investigators to develop an understanding and accurate interpretation of a complex individual’s experience, there are often idiosyncratic clarifying questions asked in each interview that are difficult to ask within a written format. Also, interview questions can be changed across and within interviews as the researchers’ understandings become more complex, and they notice gaps in their understanding. As a result, I prefer to conduct interview research when possible. Here the principle is: Grounded theory methods can be adapted for the analysis of written data, but analysts should keep in mind the benefits and costs of this choice when designing studies.

4.4.2.2 How Do I Structure My Interview? Do I Have a Main Question? (Question 7)

Answer: I consider the results I am seeking. If I am looking to develop a theory of how episodic disengagement in psychotherapy is experienced (Frankel and Levitt 2008), my main question will be “What is the experience of disengagement in psychotherapy for clients?” and the sub-questions will be variations of this main question (e.g., their experiences, before, during, and after these moments). In contrast, if the goal of my project is not to produce a singular theory but to answer a set of questions, I would use another method that is meant to explore multiple themes—like content or theme analysis. In this process, I might use some grounded theory procedures within those analytic approaches. For instance, I might divide the data and conduct separate analyses to answer the different questions and develop only a couple of levels of a hierarchy (e.g., how do feminists understand psychotherapy, what training do they receive, how satisfied are they with their training; Kannan and Levitt 2009). I would structure my interview protocol in this case to provide thorough answers to each main question and my results would be written in a corresponding format. Here, the principle being used is: When writing your interview protocol, consider the scientific goal of your analysis. If your goal is to develop a theoretical model of one phenomenon, design your question protocol to elaborate a central question and use a method of analysis designed for this purpose (such as grounded theory or phenomenology), but if your goal is to shed light on a number of discrete subtopics, create separate groups of questions and conduct an analysis designed for that purpose (such as content analysis or theme analysis).

4.5 Measures

At times, participants are asked to complete measures within a qualitative research paradigm. Because the number of participants is necessarily small, valid statistical analyses or comparisons cannot be conducted because the analyses would have little power. Reviewers become confused at times about this practice and often researchers have to defend this procedure.

4.5.1 When Should I Give Measures to Participants? (Question 8)

Answer: Measures for descriptive purposes most often are given in order to situate a given sample of participants. Giving measures, whether they are given within a mixed-methods data set or collected as part of a solely qualitative analysis, can provide a better sense of a sample’s characteristics and can contribute toward the interpretation of the data and theoretical sampling—the process by which additional participants are recruited to flesh out a theory under development (Glaser and Strauss 1967). For instance, if I am conducting a study on psychotherapy clients’ experiences, I might be interested to know if I have variation in my sample in the clients’ experiences of alliances or therapy outcome (e.g., Levitt et al. 2006). I have found that I often face criticism when including measures, however. Some reviewers have criticized my use of these measures as catering to quantitative psychology and others have wanted statistical analyses conducted which do not make sense because of the small number of participants in most qualitative analyses. The principle at hand is: Within the confines of the qualitative research, measures likely will rarely have the power to provide useful statistical knowledge due to the small data sets used. They can be useful, however, to provide information that can better describe your participants to aid in generalization (i.e., transferability), theoretical sampling, and data interpretation.

4.6 Data Analysis: Adaptations of Grounded Theory

The method of analysis that I use is sourced, with variation, in that developed by David Rennie and his colleagues (e.g., Rennie et al. 1988). I base my work in this approach mainly because of the unmatched depth of its philosophical framework (Rennie 2006; Rennie and Frommer 2015). Although this is the prototypical design that I use, there are variations that come into play depending on different design features. [For a description of common problems that I see when supervising the work of new investigators to grounded theory and tips on how to troubleshoot them, see Levitt et al. (2013)]. Questions I often encounter when consulting about method design are whether grounded theory would be the best method to use and how to go about the process of analysis.

4.6.1 Do I Really Need So Many Categories or Hierarchy Levels? (Question 9)

Answer: There are many approaches to qualitative analyses, such as phenomenology (Giorgi 2009), content analyses (e.g., Schilling 2006), theme analysis (Braun and Clarke 2006), and versions of grounded theory (see Rennie and Frommer 2015), that do not entail the development of extensive hierarchies. In contrast, I favor the development of comprehensive hierarchies in most cases but in particular for graduate students and new investigators (i.e., emerging out of typically between 30 and 80 lowest-level categories, depending on the complexity of the phenomenon under study and culminating in 2–8 categories at the level below the core category). The reasons I recommend this process are that: (1) The process of moving between category levels seems to encourage creativity and prevents novel ideas from becoming lost in a large analysis. I ask students not to create categories that simply restate the participants’ words, but initial categories that stick close to the language of the participants but emphasize the novel, interesting, or the metaphoric concepts in that language—and then moving gradually into more abstract categorization. (2) I find that the process of creating initial categories and moving up slowly helps graduate students learn to think in a complex way about their topic and to have confidence in their thinking. It can be challenging to learn what a strong theory might sound like in the absence of this process. Often I find that if students begin creating higher-order categories too quickly, the categories end up reflecting their questions rather than the answer provided. The category titles (e.g., “Types of Client Disengagement”) are not as creative and do not provide answers to the questions being explored (cf. “Disengagement as Moderating Distress Toward Continued Exploration of Sensitive Experiences: Constructive Affect Regulation”; Frankel and Levitt 2008). (3) I find that I can better defend my analyses if I can explain clearly their foundation when I send my work out for review. For instance, when a reviewer asks about a concept, I can easily describe the concepts that led to its development. (4) The results can translate more easily into future analyses when a multilevel hierarchy is formed. For instance, the detail can help in the process of item development for a quantitative measure and can be helpful in the development of a manual to guide raters for coding qualitative variables (e.g., Levitt and Frankel 2004). Because it can be hard to tell how a program of research may develop over time, the process of creating a hierarchy can allow for many options after the initial project is completed. (5) Because I tend to approach my research with feminist and social justice aims, the development of more gradual and complex hierarchies makes it more likely that my findings will remain true to the experiences of my participants. If I jump to an abstract level too quickly, it is more likely that my own cultural biases will hold sway. That said, the more familiar investigators are with both grounded theory and with the subject at hand, the more likely they will be able to move more quickly between levels of analysis and still produce creative meaningful findings. (6) Although having many lower-level categories can be helpful for all these reasons, I find that having more than seven or eight categories at the level below the core category makes the resultant theory unwieldy, difficult to communicate, and obscures the central or dominant features at play. As a result, I work toward having this number become as small as possible while remaining true to the structure of the data. When considering the creation of a hierarchy, the study-level principle I offer is that: Detailed hierarchies typically are useful when researchers are new to grounded theory, when the data is complex and findings are hard to organize or defend, and when results are intended to be used as the basis for future analyses.

4.6.2 Should I Have a Core Category? (Question 10)

Answer: The core category is formed at the very top of the hierarchy and is the key category in the analysis. In general, a core category is recommended as it articulates the theory that is being put forth and distilling this understanding really is the point of the analytic process. At the same time, there are analyses where a core category might be counterproductive. It may be that within the process of analysis, it becomes clear that commonalities do not exist beyond a certain point. For instance, in my analysis of silences in psychotherapy (Levitt 2001), the central finding was that, although our field had tended to lump silences together in research studies, my analysis suggested there were seven quite distinct processes that led to silences. To develop a core category did not make sense as it would only obfuscate this finding of difference. There may be times as well when, even if a core category is developed, the more important level is the next level of the hierarchy which might outline different processes or types in a phenomenon (e.g., distinct clinical interpretations of disengagement; Frankel and Levitt 2008) that could have theoretical or clinical utility (Dourdouma and Mörtl 2012). The principle at play is: A core category can be developed when it furthers the understanding of the phenomenon being studied, but is not useful when it creates a level of commonality that distracts from a more meaningful plurality in the findings.

4.6.3 Should I Use Multiple Analysts? If So, How Should I Assign Epistemic Privilege? (Question 11)

Answer: Traditionally, grounded theory is conducted by one investigator (Glaser and Strauss 1967), although increasingly multiple investigators or research teams collaborate in projects. In most of my research, I prefer to have a single investigator or primary and secondary analyst, but I also have conducted research within large teams (e.g., Levitt et al. 2009). The advantage of having a smaller number of investigators is twofold. First, a primary or sole investigator conducts all the interviews and so has not only the experience of hearing the participants’ words, but their attitudes, self-presentation, and a host of other meanings that can be lost when only transcripts are used. Also, the process of interviewing can lead an interviewer to care about participants and to become invested in safeguarding their stories—which I find increases his/her commitment to a highly attuned analysis and, interestingly, to separating out his/her biases from the analysis. In this way, an interviewer can develop an intimate connection with the data, be well positioned to conduct an analysis with high fidelity to the participants’ experiences, and be more likely to advance the understanding of the experience.

On the other hand, there are times when having multiple perspectives on a data set is inherently worthwhile, but only if they are committed to meeting regularly together and focusing intensively on the research over a stretch of time. There are strategies that can be used to improve communication across large teams and to build a sense of caring and investment together. Also, training is an important part of academic research so often I am working with groups of student-investigators who do not have a great deal of experience in either psychotherapy, gender studies, or in qualitative analysis, and working closely together is crucial. Although I use a consensus model, I am careful to consider and discuss the role of epistemic privilege in relation to these factors:

• 1. Data analysis experience. The process of learning to conduct interviews and divide them into units and develop a hierarchy is one that requires close supervision. Typically, I indicate to students when to seek feedback, and, at each point I work closely with them, review initial efforts until I believe that they can continue independently. I give detailed feedback on my students’ first interviews, first unitized transcripts, initial categories, and the level of categories below the core category. I usually meet with the student each week to review his/her progress. This level of supervision is the minimal level that students receive. If the only area of expertise that I bring is method of training, my reviews of their work are focused more on the method decisions and less upon the interpretations. I am not so much seeking consensus in interpretation as helping them to make distinctions in the data and to represent or code their data in a way that will result in a useful analysis. To the degree that my experience is greater than theirs, I am more likely to engage in more co-analysis of the data (see point 3 below).

• 2. Interview experience. As can be seen in the preceding section on interview considerations, it may be that the person conducting the analysis is not the best choice as interviewer but there may be insights that the interviewer still has to contribute to the analysis. In this case, I will work with the students to impart to them the meanings and reactions that were communicated in the interview process. These reactions do not necessarily override other interpretations of the data but we look at the data together and work to find interpretations that make sense to us both given our different experiences of the data. In other words, we use a process of consensus to aid our analysis in this case and I work to add my experience of the interviewing to the interpretation (or ask the student to do this when the student was the interviewer). When conducting class analyses, this guideline is explicit—that the student who conducted an interview gains priority in interpretation because of that lived experience.

• 3. Experience with the subject. More typically I am confident that the students can divide the text into the meaning units with supervision, but have some concerns about their ability to interpret the data and draw out all the important distinctions therein. In these cases, I review the data being coded each week and then conduct intensive reviews of the analysis periodically, acting as a co-analyst. I save the prior version of the analysis and then make note of changes I make during my review. When I meet with the students again, we discuss the changes together and if they disagree with a change, we discuss our interpretations with an eye toward reaching consensus. Typically, this process takes the form of representing both of our understandings as there can be aspects of the data that we are attending to differentially but are both important. If we have a strong disagreement and can see the rationale for each other’s perspectives, we can inevitably find a way to give voice to the pieces that are important to us both—often with a statement that includes a caveat or a “when-if” clause.

• 4. Different cultural experiences. It can enrich a data set when the interviewers bring to the analysis lived experiences that can refine their interpretation of the data. For instance, when I was conducting research on lesbian gender, having a co-analyst who identified as a butch lesbian allowed us to have discussions that were helpful in developing a more highly attuned analysis (e.g., Levitt and Hiestand 2004), and when conducting analyses on gay male communities, it was similarly useful to have gay male coinvestigators (Manley et al. 2007). Here the idea is to invite coinvestigators who have specific experiences relevant to a phenomenon. I do not think that having investigators of the same cultural background as a participant group is necessary (or often possible as people may have many different cultural identities), but it can be helpful especially when the group is one that is systemically oppressed in ways that are difficult for the participants to express and/or the investigator to understand. A process of co-analysis can be particularly useful as well in thinking through how to make the results of an analysis applicable in the real life context of different groups of people.

In integrating these points, the study-level principle that emerges is: Multiple analysts are not necessary but are especially helpful when they provide method, topic, or culturally based experiences that will allow for a more complex interpretation of the data at hand. These different factors all are considered and weighed together. Ideally, one ends up with an analysis in which one is confident in the meaning units created, the ability of the investigators to conduct the method and interpret the data, and the depth of analysis and its applicability.

5 Assessments of Research Checks

Grounded theory entails an empirical process of gathering data from sources that are knowledgeable on the topic and who are able to shed light on the subjective experience of a given phenomenon. Rennie (2000) argued that this method along with all forms of qualitative research (Rennie 2012) is best understood in terms of a methodical hermeneutic methodology (theory of method). He drew upon and modifies Peirce’s (1965) theory of inference when proposing a logic of interpretation involving the cycling of education and conceptualization of meaning, abduction, deduction, and induction (cf. Rennie 2000, 2012), wherein in the latter moment evidence is recruited from the text in support of a given conceptualization, whether a category, theme, or structure, etc. In Rennie’s view, the cycling of these logical moments makes qualitative research sufficient unto itself. Thus, it can be derived from this formulation that when other checks on rigor are used, these should be understood as supplemental rather than necessary.

Over time, criteria have been recommended for assessing rigor in qualitative research that are congruent with the epistemological paradigm at hand (see Morrow’s 2005 review). For instance, assessing the “trustworthiness” or “credibility” (e.g., Elliott et al. 1999) of the research emphasizes the role of the researchers’ and readers’ faith in an interpretive analysis, rather than the capacity of a method to apprehend an existing truth. Often cited are Lincoln and Guba’s (1985) criteria to assess trustworthiness that parallel natural science criteria: transferability (like external validity) to indicate the applicability of findings across contexts, credibility (like internal validity) to refer to readers’ confidence in the truth of the findings by demonstrating depth of engagement and convincing interpretations, dependability (like reliability) to suggest whether similar themes could be found by other analysts, and confirmability (like objectivity) to indicate the degree to which the analysis is grounded in the data and unaffected by bias.

In addition, other criteria have been developed for assessing trustworthiness within nonrealist research paradigms (see Guba and Lincoln 2005; Morrow 2005). Among others, these included assessing historical situatedness and erosion of ignorance in critical (e.g., feminist) research, assessing trustworthiness and authenticity within constructivist research, and assessing congruence and practical knowing in participatory research. Across all three of these paradigms, they also recommend the criteria of assessing the potential of the research to stimulate action in response to the new understandings developed. Although I will not review all the criteria of assessing research quality across paradigms, I will discuss how I adapt methods for studies in relation to criteria that are relevant for my research (i.e., within a constructivist-social justice framework).

5.1 Should I Use Inter-rater Checks or External Auditor Checks on Coding? (Question 12)

Answer: In the quest to strengthen qualitative research processes, researchers often seek to augment their methods to establish the credibility of their analyses. Two methods that are sometimes utilized to assess intersubjective agreement are the quantitative documentation of inter-rater reliability on the process of coding or unitizing text and the incorporation of qualitative checks from auditors who are external to an analysis and do not engage in the inductive process. There are a number of reasons why typically I am reluctant to utilize either of these methods:

• 1. Quantitative inter-rater agreement is rarely possible or desirable. Quantitative indices of inter-rater reliability may be especially useful in a quantitative (or natural science) epistemology because the logic of a deductive comparison requires that the data be coded in a similar fashion for statistical analyses. Typically this coding is comprised of a limited set of responses—such as a scale from 1 to 7 or a set of qualitative labels [see Pokorny (2015) and Gelo and Manzo (2015)]. This reduction of complexity is necessary and useful because it allows for the identification of trends across average experiences. The purpose of this coding is to capture a process within a limited set of possibilities for statistical trend identification and hypothesis verification. [And I use these methods myself in my psychotherapy process measure research, e.g., Stringer et al. (2010).]

  In contrast, inter-rater reliability of induction-based coding is virtually impossible, however, when using traditional forms of grounded theory (and many other qualitative methods) as they use large numbers of categories with units that can be assigned to multiple categories (e.g., Glaser and Strauss 1967). (To make this more concrete—some of my analyses have had over 1,600 meaning units with separate labels, over 75 lowest-level categories, and included over 13 hierarchy levels.) Quantitative indices of inter-rater agreement are applicable to a vastly smaller number of categories than these and have no place in such complex categorizing where the ontological commitment is to the production of an interpretation based upon the understanding of complex and contextualized subjective processes.

• 2. Inter-rater reliability or external auditor checks within this context could compromise the integrity of the analyses. A danger of these methods is that they could result in the watering down of analyses to make fine distinctions more accessible to someone less intimate with the data at hand. Qualitative analyses result from an intensive engagement with data, and investigators often take a year to design a study and complete interviewing and then a second in analysis and writing to develop the necessary level of understanding. It could compromise the strengths of the research method—that is, attunement, especially to ambiguity, context, and complexity (sacrificing authenticity) for the sake of a form of rigor that is intrinsic to the logic of a quantitative context.

  Since the logic of this approach is sufficient to itself, I am reluctant to include supplemental checks that may compromise the trustworthiness of an interpretation by asking an investigator with a high level of commitment and understanding of their data to adjust interpretations for the sake of obtaining agreement with an investigator who may not share the same investment in or knowledge. It would be inconsistent with a constructivist epistemology that prioritizes the development of attuned interpretation (e.g., authenticity), as well as a feminist epistemology that is concerned with reducing biases stemming from superficial understandings (e.g., an erosion of ignorance).

• 3. The need for these checks is not coherent with a constructivist-social justice ontology. External audits and inter-rater checks may hinder the scientific goal of these analyses, which is not to produce one theory that is replicable by every analyst, but one that is trustworthy. Qualitative analyses have a different scientific goal than quantitative ones. They are used to shed light upon data that contain multiple meanings, contradictions, ambiguities, and subjective complexities and to create understandings that accurately represent these qualities. These understandings or theories then can be evaluated and subjected to quantitative evaluation but that typically entails separate studies—as developing a model of a phenomenon itself is a substantial scientific contribution that requires considerable work, and it is rarely possible to present both analyses credibly in one journal article (e.g., Levitt 2001; Frankel et al. 2006).

  To elaborate a bit on that point, a core premise of grounded theory is that many different valid understandings can emerge from different perspectives on the same data (e.g., Charmaz 2006; Fassinger 2005; Glaser and Strauss 1967). For instance, the same set of data could lead to a theory explaining clients’ rationale for withholding information from therapists or explaining the ways clients develop trust. Both might be grounded in the data and be valid and productive contributions. Feminist approaches also hold that multiple perspectives may be valid—for instance, people may have very different experiences of an event depending on their position in terms of power and privilege (e.g., Code 2006; Harding 1998). Being able to position oneself within the standpoint of participants is key for interpretation. This understanding also means that qualitative analysis is not a completely relative process in which any interpretation could be valid. There are definite limits to the theories that can be produced from any one piece of text as it needs to be interpreted in relation to the concepts, perspectives, and meanings that are contained therein. And for an idea to become a dominant theme in an analysis, it would need to be repeated across sections of texts and participants, further limiting possibilities. The purpose of the coding is to interpret and articulate patterns so that a useful understanding can be developed.

  This said, I have used external reviewers to shed light on the limits of an analysis or provide perspectives on how analyses can best be useful within a context—especially one with which I am less familiar. For instance, in research on legal wisdom (e.g., Levitt and Dunnavant 2014), two legal consultants advised us on how our findings could be used by or presented to lawyers and judges. They shared an external source of expertise that we lacked and educated us but did not directly evaluate or alter our analysis. If reviewers or auditors are used to strategically provide advice or context for the researchers to consider alongside of their analyses (and to accept or reject as it fits with their interpretation of the data), this process would be more in keeping with my approach. The use of an internal auditor when researchers are seeking another check is another innovative possibility (see Hill et al. 2005 on this evolution in consensual qualitative research; see also Chap. 23).

  In making decisions about the type of co-analysis and supervision to provide, the principle distilled from this discussion is: Within a grounded theory analysis of a complex topic, the use of inter-rater reliability and external auditor checks is not desirable when it hinders the scientific integrity of the research—that is, to create fine-tuned categories that represent complex and contextualized data. The researchers can demonstrate to their readers the rigor of their methods by describing that qualitative methods tend to contain intrinsic checks and by supplementing these with additional checks that are consistent with the epistemology at hand.

5.2 How Many Participants Do I Need to Interview to Reach Saturation? (Question 13)

Answer: In grounded theory method, data collection continues until the categories are “saturated,” that is, until further categories that add to or change the meaning of the analysis do not appear to be forthcoming (Glaser and Strauss 1967). Achieving saturation enhances rigor and trustworthiness by suggesting that the theory is comprehensive and thereby develops a basis for generalization of the theory. Typically, I like to collect at least two interviews that have not produced new categories in the hierarchy. As a general rule, I expect that the number of interviews should be related in some way to the level of complexity that might be expected in the data. For instance, I might be satisfied that saturation was reached within a data set of five interviews that examined the ways that clients interpreted therapists’ minimal encouragers (e.g., “Mhm” and “Hm”) within sessions. I would be unlikely to be satisfied with a claim that saturation was reached within a data set of eight interviews if the question being analyzed was all the processes by which psychotherapeutic change occurred. I would be suspicious that the last participant might just have been someone who was not very verbal or adept at explaining his/her experience and might want more sessions to be added to see if saturation holds. Usually grounded theory analyses seem to contain a minimum of about five interviews for this reason. The study-level principle is: Saturation can be demonstrated by showing that new meaningful categories are not generated when adding a new interview; however, when the data is complex, it is recommended that saturation be tested further.

5.3 What Credibility Checks Should I Use? How Should I Conduct Participant Checks? (Question 14)

Answer: Increasingly, grounded theory researchers are incorporating a variety of checks to assess the trustworthiness of their analysis and establish its rigor. These checks should be selected with consideration to the purpose and features of the study at hand. Typically, I use four kinds of checks on my credibility:

• 1. Interview check. I conduct a check on my interview process. Usually participants are asked a series of questions to determine whether or not their experience was fully represented at the end of each interview (e.g., Was there anything that wasn’t asked about that feels significant about your therapy experience?) and to assess the effects of any cultural or interviewer-participant differences on the interview (e.g., Is there any way that my being a white woman might have influenced the interview?). This process provided the opportunity to collect information that might have been omitted. The principle here is: Providing an interview check helps investigators to assess both the comprehensiveness of the interview and the influences of cultural or interpersonal differences on it.

• 2. Consensus. Usually but not always, I work with coinvestigators to conduct analyses and we use a method of researcher consensus. I seek consensus only with investigators who have some level of intimacy with the data and hierarchy (see the previous section on the use of external auditors). We typically meet weekly throughout the entire study to talk about and review together the interviewing and the analysis. Throughout these discussions, I keep in mind the level of and type of expertise being brought by each researcher through that process (see section 22.4.6.3 for more discussion).

  Also, because of power differences between graduate student collaborators and myself, I encourage differences of opinion overtly and seek to include different perspectives within the hierarchy as opposed to representing only one interpretation of a unit. For instance, a segment of text might be coded as representing both the importance of connection and emotion. Irreconcilable conflicts have not occurred yet and I believe that this is not simply a result of my holding more power but a process of ensuring that all perspectives on the analysis are considered in light of the data and incorporated in a way that respects the investigators’ sources of interpretative and methodological expertise. In this process, the principle is: Consensus should be conducted in a way assigns epistemic privilege to the differing forms of methodological and interpretative expertise of the investigators, is sensitive to differences in power between investigators, and is open to incorporating multiple perspectives on a dataset.

• 3. Memoing. Memoing, a form of note-taking in grounded theory (Glaser and Strauss 1967) sometimes used in other qualitative methods, is sometimes seen as a fourth type of check but sometimes seen as part of the process of data analysis. In this process, researchers act to take notes to record the hypotheses they are creating, the coding they are engaged in, and the methods they are using. It allows them to self-reflect upon the process of making meaning of the data and upon any consensus process. It acts as a form of “fallible bracketing” (Rennie 2000) as, although researchers realize that their perspectives unavoidably influence their analysis, they become aware of biases they hold and decide upon ways of restricting their influence on the data analysis increasing the credibility of the analysis. Memoing can be used to assist with the guideline for qualitative research offered by Elliott et al. (1999) called “owning your perspective.” The principle at play is: Use memoing to record thoughts, theories, and method decisions and to recognize and limit the influence of investigators’ biases and processes upon the data.

• 4. Participant feedback. The fourth check I tend to use is a check on the analysis and its fidelity to participants’ experiences. This process can take different forms, such as follow-up interviews or mailing summaries of the results to participants and requesting written feedback. At other times, both methods might be used or something in-between. Often this decision is made based upon the researchers’ evaluation of competing goals and restraints of a given project. The following are issues I consider when deciding how to seek and use feedback:

  • (a) Efficiency in obtaining written feedback. When I am conducting an analysis as part of a classroom didactic experience, however, I often ask students to email written summaries of the main categories and request quantitative and qualitative feedback from the participants whom they interviewed. This process allows us to obtain quick responses from participants before the semester ends and is influenced by the didactic aim to provide the students with the experience of seeking feedback and writing results. It maximizes the chance that participants will respond but at the expense of the resulting feedback usually being more concise. On the other hand, when seeking feedback from eminent therapists who are very busy but also very adept at providing written descriptions of their work, emailing summaries of findings for feedback might maximize the return without much cost. It may be necessary to contact individuals whose feedback is unclear for additional discussion. The principle at hand is: Written feedback can maximize the response of participants because a second interview is not required, but can limit the ability to receive detailed responses—especially if the participants find written expression challenging.

  • (b) Depth of feedback. When analyses are complex and researchers have many remaining questions about a hierarchy, asking a subset of the participants to engage in longer second interviews or feedback discussions is more useful—providing a greater depth to feedback. For instance, when I conducted two companion grounded theory projects on butch and femme lesbians—I had a butch, a femme, and an androgynous-identified lesbian provide feedback on all three categories in intensive interviews (1–2 h each). Although the butch and femme women were participants in the study, the androgynous woman provided an external perspective (that provided more contexts on the community but was not a check on the analysis). Having an in-depth conversation with women who see these identities from different perspectives was useful in answering some of my questions and helping me to make sense of some of the differences and commonalities across the separate hierarchies. I would frame the principle here as: Seek feedback from participants or others who you think might help you shed light on the questions that remain about your findings, given the purpose of the research at hand (see Sect. 22.5.1 for more on how I might use nonparticipant reviewers’ feedback).

  • (c) Social justice goals. In my research, I always seek participant feedback (i.e., member checking) as it provides me with helpful feedback on my interpretation of data and, in keeping with my feminist concerns, helps me move beyond the limits of my own cultural understandings. Although desirable, often participants’ feedback from all participants is neither possible nor necessary and may not serve feminist ideals (cf. Frieze 2008, 2013). A complicating problem is that a grounded theory study can take a length of time to complete and it may be impossible to track down all the participants without invading participants’ privacy—so then ironically the strength of the analysis may lead to fewer participants who can give feedback in the end. In other words, it is more coherent within a constructivist epistemology to prioritize the depth of interpretation over agreement in feedback from participants who have engaged in the analysis of the entire set of data under analysis—as the assessment of the analysis is in a strong inductive process rather than in supplemental feedback checks.

    In particular, it is harder to obtain feedback when studying populations that have fewer resources, protect their identities more (e.g., may not give out contact information as easily), are reticent to participate in research, are transient, or are members of minority groups with stressors in their own lives that limit their time for research participation. It is coherent with a social justice perspective to have the voices from these groups presented in the literature rather than to insist upon feedback from all participants (see Fine 2011).

    Also, experienced researchers realize that it is rare that grounded theory findings result in stark disagreements with participants because the analysis is grounded so entirely in participants’ interviews. In my experience the vast majority of feedback responses take the form of affirmations, clarifications of minor points being made, or suggestions on how ideas can be framed. While these responses are helpful, the main results of a study are rarely questioned. The principle here is: Obtaining feedback from participants is ideal; however, researchers committed to a constructivist-social justice framework should recognize that all participants typically are not able and should not be expected to provide feedback and that this feedback is supplementary. Because the central form of rigor and trustworthiness in grounded theory is the strength of the induction-based analysis, supplemented as judged helpful by credibility checks, member checking is only one of the many ways in which the research can be assessed.

  • (d) Conflicts between participants’ feedback and your interpretation. Typically, we use the feedback from participants to fine-tune the hierarchy or expand our understanding of the phenomenon. We often make adjustments in our analysis after receiving participant feedback but ultimately we are using the feedback to enrich my understanding rather than to veto my interpretation. Although the participant may have authority over his or her own experiences, we have access to the experiences of all the data across the participants and have conducted a study looking for patterns within it that privileges our interpretation of those patterns. Still, as is the process of a hermeneutic circle, the new piece of information can influence our understanding of the whole analysis, and our understanding of the analysis can influence our understanding of the feedback. The principle here is that Feedback from participants can enrich investigators’ understanding of data; however, feedback needs to be reconciled with the investigators’ interpretations of patterns from across the participants and the hierarchy. If we cannot reconcile the feedback with our interpretation, we typically present the feedback alongside of our interpretation so that readers can assess this discrepancy themselves.

6 Meta-principles of Interpretation-Driven Research Design and Evaluation

Reviewing these guidelines, there are many ways in which the specific qualities within a project can radically influence the design of that study. Still, I am certain that there are principles that are missing from the guidelines because they have not been relevant to my program of research as of yet. From a review of the principles created, the following four meta-principles were created to guide research design and evaluation for grounded theory research, but can be extended to other forms of research as well (see Table 22.2):

Table 22.2 Meta-principles of interpretation-driven research design and evaluation

• 1. The qualities of phenomena under study need to be considered particularly in terms of: seeking diversity within participants, deciding upon the degree of detailed coding needed within a hierarchical structure, deciding if a core category is helpful, selecting the number of transcripts used to establish saturation, and choosing procedures.

• 2. The qualities of investigators involved need to be evaluated, especially in terms of selecting interviewers and analysts, deciding how to respond to the limits of their cultural perspectives, deciding how to involve external reviewers, and structuring of a process of consensus and assigning epistemic privilege.

• 3. The qualities of the research participants need to be appraised when deciding upon the need for screening, the necessity and structure of feedback checks, and the methods with which they might engage (e.g., grounded theory).

• 4. Scientific, clinical, and social justice goals of a given analysis (e.g., are the goals to produce a theory, develop an intervention, and/or to give voice to an underrepresented or marginalized group) need to be considered when selecting a process of analysis, making decisions about measures, designing credibility checks, and deciding upon the necessity and structure of feedback.

From this perspective, creating a set of rules for all qualitative research can be seen as problematic (see Levitt et al. 2005, on the function of principles versus rules). While rules can be inflexible and focused on behaviors across settings, principles allow for flexibility and focus more on intentionality and rationales for adapting decisions across contexts. It does not recognize that qualitative research designs tend to be situated within epistemologies that require understandings of rigor and trustworthiness that are relevant to the qualities of the participants, researchers, phenomena, and research goals. In contrast, interpretation-driven considerations of research design, research consumption, and review, such as those developed in this chapter, allow for a flexibility that can best serve clinical, advocacy, and scientific aims.

Conclusion

In summary, conceptualizing the design and evaluation from interpretation-driven approach has a number of benefits for qualitative research. First, it allows for an appropriate assessment of qualitative studies in which procedures within methods are adapted to serve a contextualized interpretive process, rather than using a procedure-driven approach that privileges rules associated with decontextualized methods. Second, an interpretation-driven framework can allow one to adopt a pragmatic approach to the characteristics and constraints of a study while considering the effects of method decisions upon scientific, clinical, and social justice goals. As well, it can allow researchers to consider how best to generate results that lead to a depth of understanding that also have fidelity to the experiences of participants. Although this chapter is focused upon grounded theory, researchers can extend the principles to other methods of qualitative research. Instead of solidifying sets of procedural rules that are insensitive to the processes and components at play, interpretation-driven principles guide researchers and reviewers to an understanding of method as an expression of epistemology that is shaped as it serves scientific, practice, and social justice goals. Researchers then are not reduced to technicians who blindly serve a method but become advocates of understanding, developers of treatments, and scientists.