1 Introduction

More often than not, causal relations in the social sciences are complex. A significant body of literature has put forward contextual theories in consequence. For instance, the responsiveness of democratically elected heads of government to public opinion has been hypothesized, and found to vary, with the stage of the electoral cycle as well as public approval ratings (e.g., Canes-Wrone and Shotts 2004; Nincic 1990).

In a recent contribution to this journal, Denk and Lehtinen (abbr. D & L; 2014) present Comparative Multilevel Analysis (CMA) as an innovative method whereby the effects of contexts on outcomes of interest can be studied configurationally if complemented with Qualitative Comparative Analysis (QCA). After a previous version of CMA had been introduced in Denk (2010) but was shown by Rohlfing (2012) to hold no methodological potential beyond crisp-set (csQCA) and fuzzy-set QCA (fsQCA), in the follow-up article, D & L seek to breath new life into their method by generalizing the concept of context to multivalent factors that are beyond the direct reach of csQCA or fsQCA alone. Such multivalent factors may, for instance, be given by the dominant religion in a country with Buddhist, Hindu and Christian as its three levels. As csQCA and fsQCA only allow bivalent factors, their functionality is insufficient and the application of CMA warranted, so the authors argue.

In this comment, I show that CMA, also in its latest guise, is neither innovative in nor necessary for ascertaining the influence of context in a configurational-comparative manner. The reason is that QCA is appreciably more powerful than D & L acknowledge. More specifically, its third well-known variant called multi-value Qualitative Comparative Analysis (mvQCA; Berg-Schlosser and Cronqvist 2005; Cronqvist 2004; Cronqvist and Berg-Schlosser 2009) provides all required functionality.Footnote 1 Because of the fact that even multivalent contexts are handled just like any other condition factor within the factor frame of mvQCA, separate sub-group analyses—the distinguishing feature of CMA—is made redundant. What is more, all recent methodological advances in csQCA and fsQCA such as counterfactual analysis (Ragin and Sonnett 2005) and probabilistic set relations (Ragin 2006), for example, are equally available in mvQCA.Footnote 2 In summary, QCA more generally, and mvQCA in particular, are vastly superior to CMA in every respect and need thus not be augmented in this direction.

The article is structured as follows. In Sect. 2, I recapitulate D & L’s argument and briefly describe the core features of CMA. Although the authors present their method in combination with csQCA as well as fsQCA, the fact that they concentrate on the former justifies the same focus in this comment. Subsequently, I explain the logic behind mvQCA in Sect. 3, whereby the search aims of CMA can be realized more effectively. In this connection, I also demonstrate mvQCA’s capabilities in handling imperfect set relations, limited empirical diversity and model ambiguities. I conclude that CMA offers no advantages. Researchers interested in the contextual analysis of configurational data are well-served by the existing toolbox of QCA.

2 Comparative multilevel analysis: a summary

According to D & L (pp. 6f.), CMA consists of four steps: (1) the grouping of cases in relation to their similarities on the context level, (2) the systematic comparison of cases within each context group by means of QCA, (3) the comparison of within-context results across contexts, and (4) the drawing of conclusions about the effects of context on the set of outcomes in question (cf. Denk 2010, 32f.). How CMA is employed is shown by D & L with a simple empirical example in which the effect of a minority’s relative size and wealth on its mobilization are analyzed as a function of religious context.Footnote 3

The upper part of Table 1, cases 1–12, presents the data set constructed by D & L in a slightly modified notation, where the conditions and the outcome are denoted by bold upper-case letters—the condition/outcome factors—to which indicators for the respective factor levels are appended in superscript. For example, \(\mathbf {C}^{\{0\}}\) denotes a context in which the majority facing the minority is Buddhist, \(\mathbf {W}^{\{0\}}\) the absence of wealth, and \(\mathbf {M}^{\{1\}}\) a case in which the minority has been mobilized.

Table 1 Extended data set from Denk and Lehtinen (2014)
Full size table

On the basis of these data, D & L cluster the cases according to their value on the context condition factor \(\mathbf {C}\) (CMA’s step 1) and perform three separate csQCA (CMA’s step 2). Their results show that the conjunction of small size and the absence of wealth forms a sufficient condition for mobilization in a Buddhist context, while it is large size and the presence of wealth in a Hindu context, and small size and the presence of wealth in a Christian context. The authors summarize this result in CMA’s notation as given in expression (1).

$$\begin{aligned}&\mathbf {C}^{\{0\}}\left[ \mathbf {S}^{\{1\}}\mathbf {W}^{\{0\}} = \mathbf {M}^{\{1\}}\right] + \mathbf {C}^{\{1\}}\left[ \mathbf {S}^{\{0\}}\mathbf {W}^{\{1\}} = \mathbf {M}^{\{1\}}\right] \quad +\mathbf {C}^{\{2\}}\left[ \mathbf {S}^{\{1\}}\mathbf {W}^{\{1\}} = \mathbf {M}^{\{1\}}\right] \end{aligned}$$
(1)

These steps are repeated for the other level of the outcome factor—the absence of mobilization—for which the result is restated in expression (2).

$$\begin{aligned} \mathbf {C}^{\{0\}}\left[ \mathbf {S}^{\{0\}}\mathbf {W}^{\{0\}} = \mathbf {M}^{\{0\}}\right] + \mathbf {C}^{\{1\}}\left[ \mathbf {S}^{\{0\}}\mathbf {W}^{\{0\}} = \mathbf {M}^{\{0\}}\right] +\mathbf {C}^{\{2\}}\left[ \mathbf {S}^{\{0\}}\mathbf {W}^{\{1\}} = \mathbf {M}^{\{0\}}\right] \end{aligned}$$
(2)

It says that the conjunction of large size and the absence of wealth forms a sufficient condition for the absence of mobilization in a Buddhist as well as a Hindu context, whereas it is large size and the presence of wealth in a Christian context. Unfortunately, this data set and D & L’s analytic strategy are ill-chosen to demonstrate the combined strength of CMA and csQCA since no Boolean minimization—the central analytical feature of QCA—is possible within any context. In addition, the usage of the equal sign “=” is ambiguous but must be construed as denoting logical equivalence since each configuration is sufficient and necessary for the outcome within its context. More generally, since the data are ideal with respect to the measure of consistency (Ragin 2006), and case frequencies are inconsequential, each configuration could have been represented by a single case instead of two without altering the findings from D & L’s CMA. Irrespective of these shortcomings, however, the clustering of cases according to their value on the context factor, which represents CMA’s distinguishing feature, is unnecessary for analyzing its effects configurationally as I show in the next section.

3 A straightforward procedure for contextual analysis

After having summarized CMA, I argue in this section that the method of mvQCA, which has been introduced by Berg-Schlosser and Cronqvist (2005), Cronqvist (2004) and Cronqvist and Berg-Schlosser (2009) in a basic version, and has subsequently been extended by Thiem (2013, 2014a), is superior to CMA in every respect. In preparation of the argument, a short description of QCA’s analytical core is in order.

The systematic minimization of the function that describes the configurational pattern in the data is the core procedure that unites all QCA variants. It is predicated on the principle that if a disjunction of at least two unique conjunctions, all of which exhibit the same output function value, agree on the levels of all condition factors but one, and if all the levels of this one condition factor are represented in the disjunction, then it is redundant with respect to the level of the outcome factor on which the common output function value is based. A generic example serves to illustrate this principle. Consider the following scenario with three condition factors \(\mathbf {X}_1\), \(\mathbf {X}_2\) and \(\mathbf {X}_3\). The former two possess two levels: \(\{0\}\) and \(\{1\}\); the latter three levels: \(\{0\}\), \(\{1\}\) and \(\{2\}\). The outcome factor \(\mathbf {O}\) also has three levels: \(\{0\}\), \(\{1\}\) and \(\{2\}\).Footnote 4 By convention, the conjunction operator “\(\wedge \)” between individual factor levels will not be written out. An output function with respect to \(\mathbf {O}^{\{1\}}\) could then be given by implication (3).

$$\begin{aligned} \mathbf {X}_1^{\{0\}}\mathbf {X}_2^{\{1\}}\mathbf {X}_3^{\{0\}} \vee \mathbf {X}_1^{\{0\}}\mathbf {X}_2^{\{1\}}\mathbf {X}_3^{\{1\}} \vee \mathbf {X}_1^{\{0\}}\mathbf {X}_2^{\{1\}}\mathbf {X}_3^{\{2\}} \Rightarrow \mathbf {O}^{\{1\}} \end{aligned}$$
(3)

Implication (3) can be simplified according to the principle described above since all levels of \(\mathbf {X}_3\) are present and the respective levels of \(\mathbf {X}_1\) and \(\mathbf {X}_2\) are constant across all conjunctions in this disjunction. More precisely, it can be reduced successively from implication (4) to implication (7).

$$\begin{aligned} \mathbf {X}_1^{\{0\}}\mathbf {X}_2^{\{1\}}\mathbf {X}_3^{\{0\}} \vee \mathbf {X}_1^{\{0\}}\mathbf {X}_2^{\{1\}}\mathbf {X}_3^{\{1\}} \vee \mathbf {X}_1^{\{0\}}\mathbf {X}_2^{\{1\}}\mathbf {X}_3^{\{2\}}&\Rightarrow \mathbf {O}^{\{1\}}\end{aligned}$$
(4)
$$\begin{aligned} \mathbf {X}_1^{\{0\}}\mathbf {X}_2^{\{1\}}\left( \mathbf {X}_3^{\{0\}}\vee \mathbf {X}_3^{\{1\}}\vee \mathbf {X}_3^{\{2\}}\right)&\Rightarrow \end{aligned}$$
(5)
$$\begin{aligned} \mathbf {X}_1^{\{0\}}\mathbf {X}_2^{\{1\}}\left( 1\right)&\Rightarrow \end{aligned}$$
(6)
$$\begin{aligned} \mathbf {X}_1^{\{0\}}\mathbf {X}_2^{\{1\}}&\Rightarrow \mathbf {O}^{\{1\}} \end{aligned}$$
(7)

Factor \(\mathbf {X}_3\) is redundant and can therefore be eliminated. If not all levels of \(\mathbf {X}_3\) had been present in (3), no reduction would have been possible. Reduction in mvQCA thus generalizes the Boolean minimization procedure in csQCA and fsQCA, in which both condition and outcome factors always comprise only two levels.Footnote 5

After having laid out the central principle of configurational data analysis with QCA, in the following I replicate D & L’s findings and subsequently extend their analysis to demonstrate the capabilities of mvQCA using the QCA package (Duşa and Thiem 2014; Thiem and Duşa 2013a, b, c) for the R environment for statistical computing and graphics (R Development Core Team 2014).Footnote 6 This package is currently the most versatile and powerful software available for performing QCA.

The first step in analyzing configurational data with mvQCA is to create a truth table on the basis of the data given in Table 1. Truth tables feature exactly \(d = \prod _{j=1}^{k}{p_{j}}\) configurations (rows) with \(k\) condition factors of \(p_{j}\) levels each. As context factor \(\mathbf {C}\) has three levels in D & L’s example while \(\mathbf {S}\) and \(\mathbf {W}\) have two levels each, there will be \(d = 3\cdot 2\cdot 2 = 12\) configurations. These configurations are denoted by \(\mathcal {C}\) and are given in Table 2. Truth tables additionally require an output function value for each configuration, which is determined in QCA by what Ragin (2006) has introduced as consistency. The same statistic is called inclusion in the QCA package in recognition of its earlier use in fuzzy-set theory.

Table 2 Truth table resulting from data in Table  1
Full size table

Contrary to what D & L suggest (p. 9), inclusion is not peculiar to fsQCA, but applies to all QCA variants. If, and only if, a configuration’s inclusion score meets a user-defined cut-off, the configuration is assigned a positive function (“1”). If it does not meet this cut-off, it receives a negative function value, (“0”). If the configuration has not been instantiated, in other words, if it lacks empirical cases, it cannot receive an inclusion score and is designated as a logical remainder by a question mark (“?”). Output function values and inclusion scores for each outcome are presented in Table 2 under the columns labelled “OUT” and “\(Incl\)”, respectively.

Since there is no case of two observations showing the same configuration yet a different outcome, all inclusion scores leave no doubt as to the configurations’ output function values. The minimally sufficient and necessary conditions for each outcome are given in expressions (8) and (9).

$$\begin{aligned} \mathbf {C}^{\{0\}}\mathbf {S}^{\{1\}}\mathbf {W}^{\{0\}} \vee \mathbf {C}^{\{1\}}\mathbf {S}^{\{0\}}\mathbf {W}^{\{1\}} \vee \mathbf {C}^{\{2\}}\mathbf {S}^{\{1\}}\mathbf {W}^{\{1\}} \Leftrightarrow \mathbf {M}^{\{1\}}\end{aligned}$$
(8)
$$\begin{aligned} \mathbf {C}^{\{0\}}\mathbf {S}^{\{0\}}\mathbf {W}^{\{0\}} \vee \mathbf {C}^{\{1\}}\mathbf {S}^{\{0\}}\mathbf {W}^{\{0\}} \vee \mathbf {C}^{\{2\}}\mathbf {S}^{\{0\}}\mathbf {W}^{\{1\}} \Leftrightarrow \mathbf {M}^{\{0\}} \end{aligned}$$
(9)

The two expressions are presented as logical equivalences because the disjunctions to the left of the operator are both necessary and sufficient for their outcome. That they are necessary can be read off the information provided in Table 3, which presents all solution details. More specifically, the measure of raw coverage indicates that each prime implicant covers exactly 1/3 (\(n\) = 2) of the six cases of each outcome (\(Cov_r\)). Moreover, no case is simultaneously covered by more than one prime implicant, which means that each prime implicant’s unique coverage (\(Cov_u\)) equals its raw coverage. The disjunction of all prime implicants thus covers all cases of the outcome and is therefore also necessary.

Table 3 Prime implicants and parameters of fit
Full size table

In essence, then, equivalences (8) and (9) formulate in a more concise way exactly the same as expressions (1) and (2). However, in conjunction with Table 3, mvQCA delivers much more without requiring sub-group analyses at any stage. It is able to accommodate contexts in the same way as any other condition factor within its factor frame.

Strictly speaking, however, mvQCA proper has not been applied because, in point of fact, no minimization has taken place. The data chosen by D & L are not amenable to simplification. In order to demonstrate this essential capability of mvQCA nonetheless, I let the algorithm of QCA’s minimization function determine the most parsimonious solutions.Footnote 7 They are presented in equivalences (10) and (11).

$$\begin{aligned}&\mathbf {S}^{\{1\}} \vee \mathbf {C}^{\{1\}}\mathbf {W}^{\{1\}} \Leftrightarrow \mathbf {M}^{\{1\}}\end{aligned}$$
(10)
$$\begin{aligned}&\mathbf {C}^{\{2\}}\mathbf {S}^{\{0\}} \vee \mathbf {S}^{\{0\}}\mathbf {W}^{\{0\}} \Leftrightarrow \mathbf {M}^{\{0\}} \end{aligned}$$
(11)

The model for \(\mathbf {M}^{\{1\}}\) includes the logical remainder configurations \(\mathcal {C}_{4}\), \(\mathcal {C}_{7}\), \(\mathcal {C}_{8}\) and \(\mathcal {C}_{11}\) as simplifying assumptions, which means that these configurations have been implicitly assigned a positive function value so that they do not block the process of minimization. For example, equivalence (10) says that relatively small size alone is sufficient for mobilization, irrespective of religious context, whereas the presence of wealth is only effective within a Hindu context.

There should remain no doubts at this stage that CMA offers nothing that mvQCA has not been capable of achieving. In fact, mvQCA has proven superior to CMA in every respect. To offer a last reason for giving preference to mvQCA, I briefly show how the method can even reveal model ambiguities in configurational data with multivalent context factors (cf. Thiem 2014c). Without loss of generality, I focus on outcome \(\mathbf {M}^{\{1\}}\).

Imagine case 13 is added to the data in Table  1 so that Table  2 changes in the way indicated after the forward slashes under the affected columns. As a result, configuration \(\mathcal {C}_{12}\) is assigned a negative function value and excluded from the minimization process in consequence. When the parsimonious solution is now generated on the basis of this truth table, the two alternative models in implications (12) and (13) emerge. The prime implicant \(\mathbf {C}^{\{1\}}\mathbf {W}^{\{1\}}\) is essential and must be part of any mvQCA solution, but both \(\mathbf {C}^{\{0\}}\mathbf {S}^{\{1\}}\) and \(\mathbf {S}^{\{1\}}\mathbf {W}^{\{0\}}\) are inessential and equally-matched alternatives for each other. It cannot be decided by means of mvQCA which model represents the more plausible data-generating process. Additional data or further complementary research would be required in such situations.

$$\begin{aligned} \mathbf {M}_{1}:\ \mathbf {C}^{\{1\}}\mathbf {W}^{\{1\}} \vee \left( \mathbf {C}^{\{0\}}\mathbf {S}^{\{1\}}\right) \Rightarrow \mathbf {M}^{\{1\}}\end{aligned}$$
(12)
$$\begin{aligned} \mathbf {M}_{2}:\ \mathbf {C}^{\{1\}}\mathbf {W}^{\{1\}} \vee \left( \mathbf {S}^{\{1\}}\mathbf {W}^{\{0\}}\right) \Rightarrow \mathbf {M}^{\{1\}} \end{aligned}$$
(13)

4 Conclusion

Since the first application of QCA in Ragin et al. (1984) and its detailed introduction by Ragin (1987), the method has advanced without breaking stride. Particularly during the last 10 years, numerous important innovations have broadened its functional scope. Despite these developments, Denk and Lehtinen (2014, p. 11) see ‘a new research area for the use of QCA-methods’ by embedding it in CMA, which the authors propose as an innovative method for the configurational-comparative analysis of contextual effects.

In this comment, I have argued that CMA is neither innovative in nor necessary for ascertaining contextual effects in a configurational-comparative manner. The method of mvQCA is vastly superior to CMA in every respect. It not only allows for probabilistic set relations, counterfactual analysis and model ambiguities, but also avoids the cumbersome and error-prone examination of contextual effects with numerous context factors and/or factor levels. In consequence, the established inventory of QCA need not be extended in the direction proposed by Denk and Lehtinen. Researchers interested in the contextual analysis of configurational data are well-served by the existing toolbox of QCA.