Abstract
Beyond entrenched methodological disputes, Mixed Methods attempt to promote a combination of qualitative and quantitative materialities in a single study, suitable to study the subject of research and aiming at understanding social phenomena in a more comprehensive way than it would be possible with a monomethodological approach. This article focuses on the meaning and relevance of criteria for evaluating the quality of Mixed Methods Research. A complex and ambivalent state of discussion is presented based on current research literature, and the applicability of criteria for research projects is discussed.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
However, problems and difficulties are found not only in ontological and epistemological perspectivation and the combination of qualitative and quantitative data, but the integrative aspect of relevance in corresponding studies is also discussed in detail. There are also many challenges linked to the integration concept and its irrefutable centrality in Mixed Methods research. Thus, for example, Uprichard and Dawney (2016) question whether the integration of qualitative and quantitative data is always possible and/or also appropriate, and whether it results from a coherent representation and reconstruction of the subject of research and the social reality/realities. Instead of integration, empirical social research, particularly that used in Mixed Methods, would have to focus far more on breaks and contradictions in the data material and less on an orthodox and enforced combination that is as coherent as possible (cf. Ibid, also Fielding 2012, p. 127).
- 3.
At the same time, the argument against standards is also that they are guidelines issued by scientists or entire communities, who develop such standards, while remaining unclear, however, as to who is judging the quality of these decisions, whereby for Creswell (2015), power issues are also directly involved. It is also critically added that standards certainly have the potential to establish structures and boundaries of what is permissible and/or impermissible, and that this can mean a narrowing of creative potential, which ultimately forces epistemological progress into a pre-specified corset.
- 4.
Here, a differentiation between different facets of quality evaluation is essential, whereby criteria and standards, as well as indicators and their discursive negotiation, are of importance. Criteria are characterised accordingly as features that permit statements regarding the quality of empirical studies on a scientific theory basis, and which are supported in particular by knowledge obtained through empirical studies. Standards are in turn regarded as being normative specifications, “the characterisation of which must be inherent at least to the respective quality indicators, so that it can be assumed that quality […] is poor, adequate or good” (Döring and Bortz 2016, p. 83, translated by author). Indicators permit statements to be made regarding the quality of a scientific study and can be measured and reconstructed (cf. Ibid.).
- 5.
These include (a) using and/or implementing quantitative and qualitative components in accordance with their respective standards, (b) the necessity of transparency in the research process, (c) a reference to the research question with regard to the necessity of using Mixed Methods, (d) presenting the research design and discussing it with regard to the appropriateness in relation to the research questions, (e) a convincing justification for the Mixed Methods use, and (f) the necessity for integrating the qualitative and quantitative data (Bryman 2014).
- 6.
Here, a differentiation is made between: (a) sample integration: this means the benefits gained by integrating qualitative and quantitative materialities with regard to meta-inferences; (b) the inside-outside problem describes the extent to which researchers appropriately and in a differentiated way reflect perspectivations on investigation materialities, such as between description and explanations. (c) Weakness minimisation means a reflection as to the extent to which the weaknesses of one approach are compensated by the strengths of the other; (d) sequential series is the extent to which the potentiality is given for minimising meta-inference through the inversion of the sequence of the quantitative and qualitative phases. (e) Conversion in turn means the extent to which integrative conclusions can be reached accordingly through quantification or qualification. (f) Paradigmatic mixing the extent to which epistemological, ontological, axiological and methodological shared beliefs, which form the basis for the respective methods, have been combined; (g) commensurability means “the extent to which the meta-inferences made reflect a mixed worldview based on the cognitive process of Gestalt switching and integration” (Onwuegbuzi and Johnson 2006, p. 57). (h) Multiple validity means the extent to which the legitimisation of the quantitative and qualitative components result from its extensive elements and the mixing of the results. (i) Finally, Onwuegbuzi and Johnson refer to a political legitimisation dimension: “The extent to which the consumers of Mixed Methods research value the meta-inferences stemming from both the quantitative and qualitative components of a study” (Ibid.).
- 7.
These in turn include different subdimensions, which are of relevance in the evaluation of the quality of Mixed Methods studies.
References
Adorno, Theodor W., Else Frenkel-Brunswik, Daniel Levinson, und Nevitt Sanford. 1950. The authoritarian personality. https://is.muni.cz/el/1423/jaro2017/SOC286/um/Adorno_et_al._1950_-_Authoritarian_Personality.pdf. Zugegriffen: 3. Dez. 2017.
Baur, Nina. 2005. Verlaufsmusteranalyse. Wiesbaden: VS Verlag.
Baur, Nina, und Jörg Blasius, Hrsg. 2015. Handbuch Methoden der empirischen Sozialforschung. Wiesbaden: VS Verlag.
Baur, Nina, Udo Kelle, und Udo Kuckartz. 2017. Mixed Methods – Stand der Debatte und aktuelle Problemlagen. In Mixed Methods. Kölner Zeitschrift für Soziologie und Sozialpsychologie, Sonderheft 57, Hrsg. Nina Baur et al., 1–37. Wiesbaden: VS Verlag.
Bazeley, Pat. 2003. Teaching mixed methods. Qualitative Research Journal 3:117–126.
Behnke, Joachim, Thomas Gschwend, Delia Schindler, und Kai-Uwe Schnapp, Hrsg. 2006. Methoden der Politikwissenschaft. Neuere qualitative und quantitative Analyseverfahren. Baden-Baden: Nomos.
Bergmann, Manfred Max. 2008. Advances in mixed method research. Thousand Oaks: Sage.
Bergmann, Manfred Max. 2011. The good, the bad, and the ugly in mixed methods research and design. Journal of Mixed Methods Research 5 (4): 271–275.
Betzner, Anne E. 2008. Pragmatic and dialectic mixed method approaches: An empirical comparison. Dissertation, University of Minnesota.
Biesta, Gert. 2010. Pragmatism and the philosophical foundations of mixed methods research. In Handbook of mixed methods in social & behavioral research, Hrsg. Abbas Tashakkori und Charles Teddlie, 95–118. Thousand Oaks: Sage.
Brake, Anna. 2011. Kombinieren, mixen, verbinden: Integration als konstitutives Element methodentriangulierender Zugänge. In Methodentriangulation in der qualitativen Bildungsforschung, Hrsg. Jutta Ecarius und Ingrid Miethe, 41–64. Leverkusen: Budrich.
Breuer, Franz, und Jo Reichertz. 2001. Wissenschafts-Kriterien: Eine Moderation [40 Absätze]. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research 2 (3), Art. 24. http://nbn-resolving.de/urn:nbn:de:0114-fqs0103245. Zugegriffen: 6. Apr. 2018.
Breuer, Franz, Arnulf Deppermann, Udo Kuckartz, Günter Mey, Katja Mruck, und Jo Reichertz. 2009. All is data – Qualitative Forschung und ihre Daten. In Qualitative Forschung. Analysen und Diskussionen – 10 Jahre Berliner Methodentreffen, Hrsg. Günter Mey und Katja Mruck, 261–290. Wiesbaden: VS Verlag.
Bryman, Alan. 2006. Integrating quantitative and qualitative research: How is it done? Qualitative Research 6:97–113.
Bryman, Alan. 2014. June 1989 and beyond: Julia Brannen’s contribution to mixed methods research. International Journal of Social Research Methodology 17 (2): 121–131.
Bryman, Alan, Saul Becker, und Joe Sempik. 2008. Qualitity criteria for quantitative, qualitative and mixed methods research: A view from social policy. International Journal for Social Research Methodology 11 (4): 261–276.
Burzan, Nicole. 2016. Methodenplurale Forschung: Chancen und Probleme von Mixed Methods. Weinheim: Juventa.
Cherryholmes, Cleo. 1999. Reading pragmatism. New York: Teachers College Press.
Creswell, John W. 2003. Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks: Sage.
Creswell, John W. 2015. A concise introduction to mixed methods research. Thousand Oaks: Sage.
Creswell, John W., und Vicki L. Plano Clark. 2011. Designing and conducting mixed methods research. Thousand Oaks: Sage.
Creswell, John W., Ann Carroll Klassen, Vicki L. Plano Clark, und Katherine Clegg Smith. 2011. Best practices for mixed methods research in the health sciences. https://www2.jabsom.hawaii.edu/native/docs/tsudocs/Best_Practices_for_Mixed_Methods_Research_Aug2011.pdf. Zugegriffen: 4. Sept. 2017.
Dellinger, Amy, und Nancy Leech. 2007. Toward a unified validation framework in mixed methods research. Journal of Mixed Methods Research 1 (4): 309–332.
Denscombe, Martyn. 2008. Communities and practice: A research paradigm for mixed methods approach. Journal of Mixed Methods Research 2:270–283.
Denzin, Norman. 1970. The research act. New York: McGraw Hill.
Denzin, Norman. 1989. The research act. A theoretical introduction to sociological methods, 3. Aufl. Englewood Cliffs: Prentice Hall.
Denzin, Norman. 2010. Moments, mixed methods, and paradigm dialogs. Qualitative Inquiry 16 (6): 419–427.
Denzin, Norman. 2012. Triangulation 2.0. Journal of Mixed Methods Research 6 (2): 80–88.
Döring, Nicola, und Jürgen Bortz. 2016. Qualitätskriterien in der empirischen Sozialforschung. In Forschungsmethoden und Evaluation in den Sozial- und Humanwissenschaften, Hrsg. Nicola Döring und Jürgen Bortz, 81–119. Wiesbaden: VS Verlag.
Ecarius, Jutta, und Ingrid Miethe, Hrsg. 2011. Methodentriangulation in der qualitativen Bildungsforschung. Leverkusen: Budrich.
Erzberger, Christian. 1998. Zahlen und Wörter. Die Verbindung quantitativer und qualitativer Daten und Methoden im Forschungsprozess. Weinheim: Deutscher Studien Verlag.
Fielding, Nigel. 2012. Triangulation and mixed methods design: Data integration with new research technologies. Journal of Mixed Methods Research 6 (2): 124–136.
Fiedling, Nigel, und Jane Fielding. 1986. Linking data: The articulation of qualitative and quantitative methods in social research. Beverly Hills: Sage.
Flick, Uwe. 2008. Triangulation. Wiesbaden: VS Verlag.
Flick, Uwe. 2011. Triangulation. In Empirische Forschung und Soziale Arbeit, Hrsg. Gertrud Oelrich und Hans-Uwe Otto, 323–328. Wiesbaden: VS Verlag.
Gage, Nathaniel S. 1989. The paradigm wars and their aftermath. Educational Research 18:4–10.
Gläser-Zikuda, Michaela, Tina Seidel, Carsten Rohlfs, Alexander Gröschner, und Sascha Ziegelbauer. 2012. Mixed Methods in der empirischen Bildungsforschung. Münster: Waxmann.
Grecu, Alyssa, und Matthias Völcker. 2017. Mixed-Method: Die Integration standardisierter und nichtstandardisierter Forschungs-methoden: Grenzen, Herausforderungen und Potenziale. In Qualitative Bildungsforschung. Methodische und methodologische Herausforderungen in der Forschungspraxis, Hrsg. Maja S. Maier et al., 229–246. Wiesbaden: VS Verlag.
Hammersley, Martyn. 2002. The relationship between qualitative and quantitative research: Paradigm loyalty versus methodological ecleticism. In Handbook of qualitative research methods for psychology and the social sciences, Hrsg. John Richardson, 159–174. Oxford: Blackwell.
Hammersley, Martyn. 2007. The issue of quality in qualitative research. International Journal of Research & Method in Education 30:287–305.
Hammersley, Martyn. 2010. Reproducing or constructing? Some questions about transcription in social research. Qualitative Research 10 (5): 1–17.
Hathcoat, John, und Cara Meixner. 2015. Pragmatism, factor analysis, and the conditional incompatibility thesis in mixed methods research. Journal of Mixed Methods Research 11:1–14.
Helsper, Werner, Helga Kelle, und Hans-Christoph Koller. 2016. Quaitätskriterien der Begutachtung qualitativer Forschungsvorhaben in der Erziehungswissenschaft. Ergebnisse eines Roundtable. Zeitschrift für Pädagogik 62 (5): 738–748.
Hesse-Biber, Sharlene. 2015. The problems and prospects in the teaching of mixed methods research. International Journal of Social Research Methodology 18:463–477.
Hesse-Biber, Sharlene, und Burke Johnson. 2015. The Oxford handbook of multimethod and mixed methods research inquiry. Oxford: Oxford University Press.
Howe, Kenneth. 1988. Against the quantitative-qualitative incompatibility thesis or dogmas die hard. Educational Researcher 17 (8): 10–16.
Jahoda, Marie, Paul Lazarsfeld, und Hans Zeisel. 1975. Die Arbeitslosen von Marienthal. Ein soziographischer Versuch über die Wirkungen langandauernder Arbeitslosigkeit. Frankfurt a. M.: Suhrkamp.
Johnson, Burke, und Larry Christensen. 2014. Educational research: Quantitative, qualitative, and mixed approaches. Thousand Oaks: Sage.
Johnson, Burke, und Anthony Onwuegbuzie. 2004. Mixed methods research: A research paradigm whose time has come. Educational Researcher 33 (7): 14–26.
Johnson, Burke, Anthony Onwuegbuzie, und Turner Lisa. 2007. Toward a definition of mixed methods research. Journal of Mixed Methods Research 1 (2): 112–133.
Kelle, Udo. 2008. Die Integration qualitativer und quantitativer Methoden in der empirischen Sozialforschung: Theoretische Grundlagen und methodologische Konzepte. Wiesbaden: VS Verlag.
Kelle, Udo. 2017. Die Integration qualitativer und quantitativer Forschung – Theoretische Grundlagen von “Mixed Methods”. In Mixed Methods. Kölner Zeitschrift für Soziologie und Sozialpsychologie, Sonderheft 57, Hrsg. Nina Baur et al., 39–61. Wiesbaden: VS Verlag.
Kluge, Susann, und Udo Kelle. 2001. Methodeninnovation in der Lebenslaufforschung. Weinheim: Juventa-Verlag.
Krüger, Heinz-Hermann. 2006. Einführung in Theorien und Methoden der Erziehungswissenschaft. Springer VS.
Krüger, Heinz-Hermann, und Winfried Marotzki, Hrsg. 2006. Handbuch erziehungswissenschaftlicher Biographieforschung. Wiesbaden: VS Verlag.
Kuckartz, Udo. 2014. Mixed Methods. Methodologie, Forschungsdesigns und Analyseverfahren. Wiesbaden: VS Verlag.
Laucken, Uwe. 2002. Qualitätskriterien als wissenschaftspolitische Lenkinstrumente. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research 3 (1), Art. 6. http://www.qualitative-research.net/index.php/fqs/article/view/888/1938. Zugegriffen: 7. Apr. 2018.
Leech, Nancy, Amy Dellinger, Kim Brannagan, und Hideyuki Tanaka. 2010. Evaluating mixed research studies: A mixed methods approach. Journal of Mixed Methods Research 4 (1): 17–31.
Ludwig, Peter. 2012. Thesen zur Debatte um Gütestandards in der qualitativen Bildungsforschung – Eine integrative Position. In Mixed Methods in der empirischen Bildungsforschung, Hrsg. Michaela Gläser-Zikuda et al., 79–89. Münster: Waxmann.
Maier, Maja S., Catharina Keßler, Ulrike Deppe, Anca Leuthold-Wergin, und Sabine Sandering, Hrsg. 2017. Qualitative Bildungsforschung. Methodische und methodologische Herausforderungen in der Forschungspraxis. Wiesbaden: VS Verlag.
Marotzki, Winfried. 2006. Forschungsmethoden der erziehungswissenschaftlichen Biographieforschung. In Handbuch erziehungswissenschaftlicher Biographieforschung, Hrsg. Heinz-Hermann Krüger und Winfried Marotzki, 111–136. Wiesbaden: VS Verlag.
Maschke, Sabine, und Ludwig Stecher. 2012. Strategien einer integrativen Sozialforschung am Beispiel der beruflichen Entscheidungsfindung. In Qualitative Bildungs- und Arbeitsmarktforschung. Theoretische Grundlagen und Methoden, Hrsg. Karin Schittenhelm, 379–406. Wiesbaden: VS Verlag.
McKim, Courtney. 2015. The value of mixed methods research: A mixed methods study. Journal of Mixed Methods Research 11 (2): 202–222.
Mertens, Donna, Pat Bazeley, Lisa Bowleg, Nigel Fielding, Joseph Maxwell, Jose Molina-Azorin, und Katrin Niglas. 2016. MMIRA task force report. The future of mixed methods: A five year projection to 2020. https://mmira.wildapricot.org/resources/Documents/MMIRA%20task%20force%20report%20Jan2016%20final.pdf. Zugegriffen: 27. Nov. 2017.
Mey, Günter, und Katja Mruck, Hrsg. 2009. Qualitative Forschung. Analysen und Diskussionen – 10 Jahre Berliner Methodentreffen. Wiesbaden: VS Verlag.
Morgan, David. 2007. Paradigms lost and pragmatism regained. Methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research 1 (1): 48–76.
Morgan, David. 2014. Pragmatism as a paradigm for social research. Qualitative Inquiry 20 (8): 1045–1053.
Myrick, Jane. 2006. What is good qualitative research? A first step towards a comprehensive approach to judging rigour/quality. Journal of Health Psychology 11 (5): 799–808.
O’Cathain, Alicia. 2010. Assessing the qualitity of mixed methods research: Toward a comprehensive Framework. In Handbook of mixed methods in social & behavioral research, Hrsg. Abbas Tashakkori und Charles Teddlie, 531–555. Thousand Oaks: Sage.
O’Cathain, Alicia, Elisabeth Murphy, und John Nicholl. 2008. The quality of mixed methods studies in health services research. Journal of Health Service Research and Policy 13 (2): 92–98.
Oelrich, Gertrud, und Hans-Uwe Otto, Hrsg. 2011. Empirische Forschung und Soziale Arbeit. Wiesbaden: VS Verlag.
Onwuegbuzie, Anthony, und Burke Johnson. 2006. The validity issue in mixed research. Research in the Schools 13 (1): 48–63.
Onwuegbuzie, Anthony J., und Nancy L. Leech. 2005. On becoming a pragmatic researcher: The importance of combining quan-titative and qualitative research methodologies. International Journal of Social Research Methodology 8 (5): 375–387.
Reichertz, Jo. 2007. Qualitative Sozialforschung – Ansprüche, Prämissen, Probleme. Erwägen – Wissen – Ethik 18 (2): 1–14.
Reichertz, Jo. 2009. Die Konjunktur der qualitativen Sozialforschung und Konjunkturen innerhalb der qualitativen Sozialforschung [48 Absätze]. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research 10 (3), Art. 30. http://nbn-resolving.de/urn:nbn:de:0114-fqs0903291. Zugegriffen: 14. Juni 2018.
Reichertz, Jo. 2015. Empirische Sozialforschung und soziologische Theorie. In Handbuch Methoden der empirischen Sozialforschung, Hrsg. Nina Baur und Jörg Blasius, 65–80. Wiesbaden: VS Verlag.
Richardson, John, Hrsg. 2002. Handbook of qualitative research methods for psychology and the social sciences. Oxford: Blackwell.
Roethlisberger, Fritz, und William Dickson. 1939. Management and the worker. Cambridge: Harvard University Press.
Schifferdecker, Karen, und Virginia Reed. 2009. Using mixed methods research in medical education: Basic guidelines for researchers. Medical Education 43 (7): 637–644.
Schittenhelm, Karin, Hrsg. 2012. Qualitative Bildungs- und Arbeitsmarktforschung. Theoretische Grundlagen und Methoden. Wiesbaden: VS Verlag.
Schnapp, Kai-Uwe, Delia Schindler, Thomas Gschwend, und Joachim Behnke. 2006. Qualitative und quantitative Zugänge: Eine integrative Perspektive. In Methoden der Politikwissenschaft. Neuere qualitative und quantitative Analyseverfahren, Hrsg. Joachim Behnke et al., 11–26. Baden-Baden: Nomos.
Schoonenboom, Judith. 2017. A performative paradigm for mixed methods research. Journal of Mixed Methods Research 1–17.
Steinke, Ines. 1999. Kriterien qualitativer Forschung. Ansätze zur Bewertung qualitativ-empirischer Sozialforschung. Weinheim: Juventa.
Strauss, Anselm, und Juliet Corbin. 2010. Grounded Theory: Grundlagen qualitativer Sozialforschung. Weinheim: Beltz.
Strübing, Jörg. 2008. Grounded Theory: Zur sozialtheoretischen und epistemologischen Fundierung des Verfahrens der empirisch begründeten Theoriebildung. Wiesbaden: VS Verlag.
Symonds, Jenny, und Stephen Gorard. 2009. The death of mixed methods: Research labels and their casualties. The British Educational Research Association. Annual Conference, Heriot Watt University, Edinburgh, September 3–6 (Konferenzpapier). https://www.leeds.ac.uk/educol/documents/174130.pdf. Zugegriffen: 26. Dez. 2017.
Tashakkori, Abbas, und Charles Teddlie. 2003. The past and future of mixed methods research: From data triangulation to mixed model designs. In Handbook of mixed methods in social & behavioral research, Hrsg. Abbas Tashakkori und Charles Teddlie, 671–701. Thousand Oaks: Sage.
Tashakkori, Abbas, und Charles Teddlie. 2008. Quality of inferences in mixed methods research: Calling for an integrative framework. In Advances in mixed method research, Hrsg. Manfred Max Bergmann, 101–119. Thousand Oaks: Sage.
Tashakkori, Abbas, und Charles Teddlie. 2010. The past and future of mixed methods research: From data triangulation to mixed model designs. In Handbook of mixed methods in social & behavioral research, Hrsg. Abbas Tashakkori und Charles Teddlie, 671–702. Thousand Oaks: Sage.
Tashakkorie, Abbas, und John Creswell. 2007. Editorial: The new era of mixed methods. Journal of Mixed Methods Research 1 (1): 3–7.
Teddlie, Charles, und Abbas Tashakkorie. 2009. Foundations of mixed methods research. Integrating quantitative and qualitative approaches in the social and behavioral sciences. Thousand Oaks: Sage.
Uprichard, Emma, und Leila Dawney. 2016. Data diffraction: Challenging data integration in mixed methods research. Journal of Mixed Methods Research 1–14.
Whittemore, Robin, Susan Chase, und Carol Mandle. 2001. Validity in qualitative research. Qualitative Health Research 11 (4): 522–537.
Wohlrab-Sahr, Monika, und Aglaja Przyborski. 2008. Qualitative Sozialforschung: Ein Arbeitsbuch. Oldenburg: Oldenburg Wissenschaftsverlag.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Fachmedien Wiesbaden GmbH, ein Teil von Springer Nature
About this chapter
Cite this chapter
Völcker, M. (2019). The Quality of ‘Good’ Mixed Methods Research: Development and Discussion of an Orientation Framework. In: Lüdemann, J., Otto, A. (eds) Triangulation und Mixed-Methods. Studien zur Schul- und Bildungsforschung, vol 76. Springer VS, Wiesbaden. https://doi.org/10.1007/978-3-658-24225-1_4
Download citation
DOI: https://doi.org/10.1007/978-3-658-24225-1_4
Published:
Publisher Name: Springer VS, Wiesbaden
Print ISBN: 978-3-658-24224-4
Online ISBN: 978-3-658-24225-1
eBook Packages: Education and Social Work (German Language)