Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Two years after Greg Shaffer and Tom Ginsburg have proclaimed an ‘empirical turn in international legal scholarship’, empirical research is very much en vogue among international lawyers, but continues to be perceived as a ‘new frontier’. By contrast, scholarship on international courts and tribunals has been en vogue for a few decades now. It is not a new field by any means, and yet the focus of inquiry is changing—towards a fuller analysis of the functions and agendas of international courts and tribunals, which are no longer seen just as dispute settlers, but also (or even primarily) as law-enforcers, law-makers, norm entrepreneurs, review agencies, etc. Yuval Shany has been a key figure in this move towards such a fuller analysis; his research has done a lot to broaden our understanding of the many functions of international courts and tribunals. With his new book, he now seems to take his own empirical turn, and readers are encouraged to follow him on this path.

At the outset, however, Shany reveals to his readers that they should not judge a book by its title. Rather than assessing effectiveness according to one or more of the usual indicators—compliance with decisions, usage rates, impact on the conduct of parties—he puts forward a ‘conceptual framework to analyze questions about the effectiveness of international courts, which could serve as the basis for future research programs’ (p. 4). Put differently, the book’s focus is not on whether international courts are effective, but on how their effectiveness should be assessed.

In the first half of the book, Shany sets out his conceptual framework, which is indeed (as is noted early on) ‘sophisticated and complex’ (p. 6). In essence, Shany proposes to gauge effectiveness not against fixed variables (impact, compliance, etc.), but according to a more flexible criterion, viz. that of ‘attaining the goals set by the mandate provider’. Put differently, an international court is effective if, through its operation, it meets the expectations of States and organisations that set them up. This is, both, a surprising and refreshing methodological turn. It allows Shany to tap into a rich (and previously largely untapped) body of social sciences scholarship on ‘goal-based approaches’ to measuring the performance of institutions. As Shany is serious in engaging with this scholarship, his analysis centres around categories that may not be overly familiar to readers with a legal background—from ‘mandate providers’ to ‘goal ambiguity’ (which comes in four different versions) and the distinction between ‘ultimate’ and ‘intermediate goals’. All of this means that this book is not one for light reading. But the goal-based approach is indeed considerable more differentiated than other models of measuring effectiveness.

Most importantly, it suggests that effectiveness cannot be measured across the board, by applying one criterion for all courts, and that in fact, compliance with judgments may not be the most instructive indicator, if only because it favours ‘low-aiming’ courts rendering timid judgments. Instead, according to Shany, assessing effectiveness is a ‘meticulous, institution-specific endeavor [that] requires identification of the goals designated by a particular international court’s mandate providers’ (p. 37). While this makes for a  nuanced analysis, Shany to some extent levels the playing field by postulating four ‘generic goals’: in his view, all international courts are set up to support an existing set of norms, to facilitate the settlement of disputes, to strengthen the institutional regime of which they form part, and to legitimise the exercise of public authority. If nevertheless, assessing the effectiveness of international courts is a ‘meticulous and institution-specific endeavor’, then it is because the relative weight of the generic goals varies, and they are complemented by ‘idiosyncratic goals’ particular to a particular institution.

So how can it be assessed whether courts attain these differentiated goals? Outcomes of judicial activity—the impact of a court’s operation on the outside world—are no doubt crucial. But as they are difficult to measure, Shany also proposes to rely on certain proxy categories: the legal powers of courts and tribunals, their structure, the procedures available to them. This analytical move widens the range of indicators that can be used to gauge effectiveness: rather than outcomes as such, ‘proxies for outcomes’ such as jurisdiction, independence, resources are relied upon, and whilst outcomes are difficult to quantify, their proposed proxies may be easier to assess. At the same time, the use of proxy categories introduces an element of abstraction: a court possessing a wide margin of jurisdiction does not necessarily make use of it; or indeed, States setting up a court may not have wanted it to be all that independent, which a goal-based analysis would have to capture. Thus, proxy categories need to be used with caution.

The methodological re-orientation brought about by Shany’s goal-based approach has considerable potential, but comes with risks, too. As for its potential, it clearly points a way out of stale debates about compliance, which is very difficult to measure; and it comes with analytical tools that permit for a fuller, richer assessment of how courts perform. At the same time, it may over-emphasise the role of mandate providers: does it not turn courts into permanent hostages of States that set them up—can it capture the reality of bold courts that become assured in their jurisprudence and over time expand their influence? More pragmatically, there is a risk that, while moving beyond simplistic categories, the goal-based approach gets lost in sophistication. If the goals designated for a particular court are too many, if they conflict, and if the mandate providers do not indicate hierarchies between them—in short: if mandate providers set up courts either without clear mandates, or based on mixed motives—a conceptual framework relying on mandate providers may struggle to deliver reliable results.

To some extent, the second half of the book—comprising five chapters, applying the goal-based approach to existing international courts, co-authored by Shany and various associates involved in the research project—illustrates this risk. Five prominent dispute settlement systems (ICJ, WTO, ICC, ECtHR and ECJ) are scrutinised: their goals identified, proxy indicators assessed and outcomes evaluated. Each of these chapters is exciting in its own right, but for readers of this Yearbook, the treatment of the WTO dispute settlement system may be of particular interest. In it, Shany and Sivan Shlomo-Agon identify no less than seven goals for which the ‘mandate providers’ during the Uruguay Round decided to set up the WTO dispute settlement system in its present form: to provide security and predictability to the world trading system; to contribute to the functioning of the WTO regime; to help maintain (or restore) a balance of benefits between WTO members; to legitimise the WTO as an institution and the norms on which it is based; to induce compliance with WTO obligations; and to facilitate the positive settlement of disputes over unilateral sanctions. These goals are manifold and can conflict; but if judged by Shany’s proxy factors (structures, procedures), the WTO dispute settlement system seems rather well equipped to attain them. Its jurisdictional powers are considerable, especially compared to courts requiring parties to opt into their jurisdiction. While integrated into an institutional framework, panels and Appellate Body enjoy sufficient levels of independence and impartiality. Their legitimacy is often questioned—both by contracting parties wary of judicial activism and by a wider public concerned about the accommodation of public interests in world trade law,—but Shany and Shlomo-Agon point to satisfactory compliance rates and the gradual opening up of the dispute settlement system to amici curiae to put this criticism in perspective. As for the outcomes of the WTO dispute settlement system, Shany and Shlomo-Agon, unsurprisingly, discuss compliance rates; but they also flag concerns about the impact of dispute resolution on developing countries and highlight the reduction in unilateral trade sanctions. Overall, their assessment suggests that the WTO is relatively effective in attaining some of its goals. However, existing empirical studies ‘disclos[e] only a partial account of the [WTO dispute settlement system’s] effectiveness. Further research is needed in order to attain a broader understanding of its performance’ (p. 222). This is an honest assessment, but also a little anti-climactic.

As has hopefully become clear, if judged by its title, this book makes for a surprising read. But it is a welcome surprise, and an enriching one. The conceptual framework set out is indeed ‘sophisticated and complex’, and it is presented in a tightly argued and dense manner. In applying the model, the book illustrates the strengths and problems of the conceptual ‘goal-based’ framework. The analysis of particular courts is nuanced and differentiated. Yet as the mandate providers may have designated rather too many overlapping and competing goals, there is a risk of getting lost in complexity. Perhaps the model is too sophisticated to be applied in depth to five different courts. Shany and his research associates seem aware of this risk; hence frequent caveats (such as the one quoted above) and suggestions for further study. All things considered, perhaps this book should be seen as a prologue to a new, and higher, level of engagement with effectiveness. The conceptual framework is unfolded in remarkable detail. It can now be debated, and future studies may perhaps offer opportunities to test and to refine it, and to apply it in a fuller manner.