Coherence and Reliability in Judicial Reasoning

Abstract

Experimental studies of how juries reach their verdicts in court strongly suggest that coherence reasoning is ubiquitous in judicial reasoning. Under massive cognitive pressure to process large numbers of conflicting pieces of evidence and witness reports, jury members base their judgment on an assessment of the most coherent account of the events. From a normative perspective, the legitimacy of coherence reasoning in court hinges on the premise that such coherence is a plausible guide to justified belief. Unfortunately, this notion has been severely challenged by numerous recent studies in Bayesian formal epistemology. Bovens and Hartmann (2003) (Bayesian epistemology). New York/Oxford: Oxford University Press and Olsson (2005) (Against coherence: Truth, probability and justification). Oxford: Oxford University Press have shown that there is no way to measure coherence such that coherence is truth conducive in the sense that more coherence implies a higher likelihood of truth. This is so even under seemingly very weak boundary conditions. In previous work we have shown that (certain forms of) coherence can be reliability conducive in paradigmatic scenarios where such coherence fails to be truth conducive. In other words, more coherence can still be indicative of a higher probability that the witnesses are reliable. We have also argued that the connection between (certain forms of) coherence and probability of reliability may be what justifies our common reliance on coherence reasoning. While the link between coherence and reliability was found to be not completely general, our studies so far do support the contention that this link is stronger than that between coherence and truth. In this paper, we add credence to this conclusion by proving several new formal results connecting one prominent measure of coherence, the Shogenji measure, to witness reliability. The most striking of these results is that in a case where the witnesses’ degrees of reliability are maximally dependent of each other—i.e., where either all witnesses are reliable or all witnesses are unreliable—the Shogenji measure is reliability conducive. We also relate our approach to the Evidentiary Value tradition in Scandinavian legal theory.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

2.1 Introduction

Suppose that a robbery has taken place and that John has been accused of committing the crime. Imagine sitting on a jury as three witnesses take the stand. The first witness testifies that John was at the crime scene at the time of the crime, the second that John owns a weapon of the type used, and the third that John shortly after the robbery deposited a large sum of money in his bank account. This would be an example of a highly coherent set of testimonies, i.e. a set in which the individual elements hang together or are in agreement. The case would have been quite different had the first witness reported that she was having dinner with John at the time of the crime. That would have led to an incoherent set of testimonies.

Empirical research strongly indicates that people, and jurors in particular, are disposed to trust coherent sets of testimonies.¹ According to the influential story model of juror decision making (see, e.g., Pennington and Hastie 1993), jurors construct narratives in response to evidence in trials and then choose the one that scores best on Pennington and Hastie’s favored criteria or ‘certainty principles’—‘coverage’ and ‘coherence’. Jurors then determine the verdict on the basis of their chosen story. Similarly, Lagnado and Harvey (2008) argue that when performing complex reasoning tasks where not all evidence point in the same direction, people group the evidence into different coherent sets as a basis for further consideration. In related studies it has been concluded that, if an individual witness delivers inconsistent testimonies, then subjects will assign her a lower degree of credibility; and it has been observed that inconsistency leads to lower rates of conviction (see, e.g., Berman and Cutler 1996; Berman et al. 1995). Finally, Brewer et al. (1999) present evidence supporting the claim that perceived witness credibility is positively affected by consistency between reports from different witnesses, albeit to a lesser extent than intra-witness consistency.

Thus, much speaks in favor of coherence reasoning playing a fundamental role when jury members and judges evaluate evidence as presented before the court. The question, though, is whether this reliance on coherence can be motivated from a normative perspective. Given that A is more coherent than B, can we conclude that A is in some sense more appropriate to believe than B? Our first task (subsequent to having introduced the concept of coherence, as that concept is understood in the philosophical literature) will be to review some previous work on our normative question. Thereafter, we present our own preferred account of the normative basis of coherence reasoning, in terms of reliability conduciveness, a concept first proposed in (Olsson and Schubert 2007). We add further substance to that account by proving some formal results that reveal the intimate connection between a certain conception of coherence and the probability of reliability. Finally, we draw some parallels between our account and the Evidentiary Value tradition in Scandinavian legal philosophy.²

2.2 Coherence and Truth

Epistemologists have generally thought that coherence is an epistemically useful property. But exactly what is it that makes coherence so useful—or, in other words, what positive epistemic qualities do we obtain from a high degree of coherence? The common-sense answer, and the standard view among coherence theorists, is that coherence is related to truth. Coherence is, according to this view, evidence of a high probability of truth. In recent years, coherence theorists have spelled out this idea in terms of truth conduciveness. Thus we would expect that, if one set A is more coherent than another set B, then A is more likely to be true than B (Klein and Warfield 1994). The exact meaning of this claim has been the source of much controversy: both the notion of coherence and the notion of likelihood of truth have been heavily discussed. Let us start with the concept of coherence.

In his 1934 book on idealism, the Cambridge philosopher A. C. Ewing put forward a much cited definition of coherence. In his view, a coherent set is characterized partly by consistency and partly by the property that every belief in the set follows logically from the others taken together. On this picture, a set such as $\left\{ {p,q,p\wedge q} \right\}$ would, if consistent, be highly coherent, as each element follows by logical deduction from the rest in concert. While Ewing should be credited for having provided a precise definition of an intangible idea, his proposal must be rejected on the grounds that it defines coherence too narrowly. Few sets that occur naturally in everyday life satisfy the second part of his definition, i.e., the requirement that each element follow logically from the rest when combined. Consider, for instance, the set consisting of propositions A, B and C, where

• A = ‘John was at the crime scene at the time of the robbery’

• B = ‘John owns a gun of the type used by the robber’

• C = ‘John deposited a large sum of money in his bank account the next day’

Many of us would consider this set to be coherent, and yet it does not satisfy Ewing’s definition. A, for instance, does not follow logically from B and C taken together: that John owns a gun of the relevant type and deposited money in his bank the day after does not logically imply him being at the crime scene at the time of the crime.

From that perspective, C. I. Lewis’s (1946) definition of coherence is more promising. According to Lewis, whose proposal can be seen as a refinement of Ewing’s basic idea, a set is coherent just in case every element in the set is supported by all the other elements taken together, where ‘support’ is understood in a weak probabilistic sense: A supports B if and only if the probability of B is raised on the assumption that A is true. It is easy to see that Lewis’s definition is wider than Ewing’s, so that more sets will turn out to be coherent on the former than on the latter. (There are some uninteresting limiting cases for which this is not true. For instance, a set of tautologies will be coherent in Ewing’s but not in Lewis’s sense.)

To illustrate, let us go back to the example with John. Here one could argue that A, while not being logically entailed by B and C, is nevertheless supported by those propositions taken together. Assuming that John owns the relevant type of gun and deposited a large sum the next day serves to raise the probability that John did it and hence that he was at the crime scene when the robbery took place. Similarly, one could hold that each of B and C is supported, in the probabilistic sense, by the other elements of the set. If so, this set is not only coherent in an intuitive sense but also coherent according to Lewis’s definition.

It is worth noticing that the support the elements of a set obtain from each other need not be very strong for the set to be coherent in Lewis’s sense. It suffices that they support each other to some, however miniscule, degree. A second observation is that on Lewis’s account whether or not a set is coherent will presumably depend on empirical data that constrain what (conditional) probabilities we are willing to assign. This is not a feature of Ewing’s definition, which relies on purely logical notions.³ Another proposal for how to say something more definite about coherence originates from Laurence BonJour (1985), whose account of coherence is considerably more complex than earlier suggestions. While Ewing and Lewis proposed to define coherence in terms of one single concept—logical consequence and probability, respectively—BonJour thinks that coherence is a notion with a multitude of different aspects, corresponding to the following coherence criteria (ibid. 97–99):

• 1. A system of beliefs is coherent only if it is logically consistent.

• 2. A system of beliefs is coherent in proportion to its degree of probabilistic consistency.

• 3. The coherence of a system of beliefs is increased by the presence of inferential connections between its component beliefs and increased in proportion to the number and strength of such connections.

• 4. The coherence of a system of beliefs is diminished to the extent to which it is divided into subsystems of beliefs which are relatively unconnected to each other by inferential connections.

• 5. The coherence of a system of beliefs is decreased in proportion to the presence of unexplained anomalies in the believed content of the system.

These criteria are formulated in terms of beliefs, but they could just as well be applied to statements made in court. The first criterion, of logical consistency, is nothing new but was employed already by Ewing. The second criterion is somewhat more problematic, mainly due to the fact that Bonjour never clearly states what he means by ‘degree of probabilistic consistency’. Nevertheless, the idea seems to be that a system is probabilistically consistent if and only if it contains no belief that P such that ‘It is highly unlikely that P’ can be derived from the other beliefs in the system. The criterion then dictates that it is of importance to the degree of coherence to avoid this predicament for as many beliefs as possible.

Both the third and the fourth criterion make use of the idea of an ‘inferential connection’, which should here be interpreted in a wide sense as including all types of support between beliefs, such as logical or probabilistic support. The suggestion embodied in the third criterion is simply that the degree of coherence is increased in proportion to how much different beliefs support each other. According to the fourth criterion, the degree of coherence is decreased in proportion to the presence of relatively isolated subsystems within the system. As for an extreme case, a person suffering from multiple personality disorder would satisfy the fourth criterion to a very low degree. But there are of course many less spectacular examples of how we sometimes entertain various views without ever connecting them. A child may learn most things worth knowing about cats and dogs without wondering what is common between these two kinds of animal. Eventually, she acquires the concept of a mammal and learns that much of what is true of cats and dogs is true of mammals in general. In science, it often happens that two areas are pursued in isolation until someone discovers that they are but special cases of a more comprehensive theory. In both cases, the unification entails an increase in coherence, as that concept is understood by Bonjour. The last criterion dictates that the presence of anomalies is something that reduces the overall level of coherence. An anomaly is, roughly, an observation that cannot be explained from within the belief system of the person in question.

A difficulty pertaining to theories of coherence that construe coherence as a multifaceted concept is to specify how the different aspects are to be amalgamated into one overall coherence judgment. It could well happen that one system S is more coherent than another system T in one respect, whereas T is more coherent than S in another. Perhaps S contains more inferential connections than T, which in turn has less anomalies than S. If so, which system is more coherent in an overall sense? Bonjour’s theory remains silent on this important point.

Bonjour’s account also serves to illustrate another general difficulty. The third criterion stipulates that the degree of coherence increases with the number of inferential connections between different parts of the system. As a system grows larger the probability that there will be relatively many inferentially connected beliefs is increased. Hence, there will be a positive correlation between system size and the number of inferential connections. Taken literally, Bonjour’s third criterion implies, therefore, that there will be a positive correlation between system size and degree of coherence.

The general problem is to specify how the degree of coherence of a system should depend on its size. One possibility is that mere system size should have no impact on the degree of coherence, which should rather only depend on the system’s inferential density. Another possibility is that we also need to take into account the number of inferential connections, so that larger systems have a potential to be more coherent for the simple reason that there are more opportunities for inferential connections to arise. This seems to be more congruent with Bonjour’s way of looking at things.

Here is another general challenge for those wishing to give a clear-cut account of coherence. Suppose a number of eye witnesses are being questioned separately concerning a robbery that has recently taken place. The first two witnesses, Robert and Mary, give exactly the same detailed description of the robber as a red-headed man in his 40s of normal height wearing a blue leather jacket and green shoes. The next two witnesses, Steve and Karen, also give identical stories but only succeed in giving a very general description of the robber as a man wearing a blue leather jacket. So here we have two cases of exact agreement. In one case, the agreement concerns something very specific and detailed, while in the other case it concerns a more general proposition. This raises the question of which pair of reports is more coherent. Should we say that agreement on something specific gives rise to a higher degree of coherence, perhaps because such agreement seems more ‘striking’? Or should we rather maintain that the degree of coherence is the same, regardless of the specificity of the thing agreed upon?

The challenge is to specify how the degree of coherence of an agreeing system should depend on the specificity of the system’s informational content. Everything else being equal, should an agreeing system containing very specific, and therefore more informative, propositions be considered more coherent than a system of mainly general, and therefore less specific, propositions?

To illustrate these points about size and specificity consider the following recently proposed coherence measures:

$$\begin{aligned} {C_{Sh}}\left( {{A_1},\ldots,{A_n}} \right)&=\frac{{P\left( {{A_1}\wedge \ldots \wedge {A_n}} \right)}}{{P\left( {{A_1}} \right)\times \ldots \times P\left( {{A_n}} \right)}} \\ {C_{Ol}}\left( {{A_1},\ldots,{A_n}} \right)&=\frac{{P\left( {{A_1}\wedge \ldots \wedge {A_n}} \right)}}{{P\left( {{A_1}\vee \ldots \vee {A_n}} \right)}} \end{aligned} $$

Both measures assign a degree of coherence to a set of propositions in probabilistic terms, following Lewis, but they do it in slightly different ways. C _Sh was put forward in (Shogenji 1999) and is discussed for instance in (Olsson 2001). C _Ol was tentatively proposed in (Olsson 2002) and, independently, in (Glass 2002).

To illustrate the differences, suppose that A ₁,…, A _n are equivalent propositions. We first consider the probability of the conjunction which figures in the numerator of both measures. Since A ₁,…, A _n are equivalent, $P\left( {{A_1}\wedge \ldots \wedge {A_n}} \right) = P\left( {{A_1}} \right)$. For the same reason, the denominator in the definition of C _Sh equals P(A ₁)ⁿ. Hence, ${C_{Sh }}\left( {{A_1},\ldots,{A_n}} \right) = P\left( {{A_1}} \right)/P{{\left( {{A_1}} \right)}^n} = 1/P{{\left( {{A_1}} \right)}^{n-1 }}$. Now as more equivalent propositions are added, i.e., as n grows larger, the denominator will approach zero, making the degree of C _Sh -coherence approach infinity. The same is true if the propositions involved are substituted for more specific equivalent propositions or, equivalently, the initial probabilities are reassigned so that the same propositions become less probable. Then, too, the degree of C _Sh -coherence will tend towards infinity. Not so for C _Ol , which assigns a coherence degree of 1 to every set of equivalent propositions, regardless of size or specificity. On the basis of observations such as these, it has been suggested that these two measures actually measure two different things. While C _Ol captures the degree of agreement of the propositions in a set, C _Sh is more plausible as a measure of how striking the agreement is (Olsson 2002; see also Bovens and Olsson 2000 for a discussion of agreement vs. striking agreement). Since these two proposals were made, a large number of other measures have been suggested, many of which are studied in (Olsson and Schubert 2007).

Given what has been said so far, a case could be made for the special relevance of the Shogenji measure in legal contexts. We recall Pennington and Hastie’s observation that jurors deal with trial evidence by constructing narratives, whereby the best explanatory story is the one that conforms most convincingly to the two principles of coverage and (what they call) coherence by accounting, in a coherent manner, for a large subset of the available evidence. An attractive feature of the Shogenji measure, from this perspective, is that it treats ‘coverage’ (size) as part and parcel of the concept of coherence. Hence we do not need two measures—one measuring coherence, another that measures coverage—but can make do with one. This makes it particularly interesting from the point of view of the story model of juror decision making.

Now that we have a somewhat firmer grasp of the concept of coherence, how should we understand the claim that coherence implies ‘likelihood of truth’? Klein and Warfield (1994) claimed, in effect, that the conjunction of the statements in more coherent sets should always have a higher probability of truth (than the corresponding conjunctions of less coherent sets). This proposal was rejected in (Olsson 2001) in favor of an account in terms of the posterior probability of truth, i.e., the conditional probability of the statements given that they have been reported by the witnesses (see also Cross 1999, referring to Bonjour 1985). This account provided the foundation of much later work in this area (e.g., Olsson 2002, 2005; Bovens and Hartmann 2003).

A further source of controversy concerned the question under what circumstances we can reasonably expect coherence to be truth conducive. Already C. I. Lewis (1946) had observed that coherence does not seem to be interestingly related to truth unless the witnesses delivering the statements are independent, which means that they have not talked to, or otherwise influenced, each other beforehand. Also, Lewis claimed that each witness must be considered to be somewhat reliable for coherence to have confidence-boosting power. Later work has essentially proven Lewis right on both accounts (e.g., Olsson 2002, 2005; Bovens and Hartmann 2003), although there are also dissident voices (Shogenji 2005).

Equipped with precise accounts of coherence as well as likelihood of truth, philosophers and computer scientists set out to show that coherence is truth conducive at least under the conditions of independence and partial reliability. Contrary to the hopes and expectations of most coherence theorists, it was soon shown that coherence is not truth conducive (Bovens and Hartmann 2003; Olsson 2005). Importantly, this is so regardless of how coherence is measured.

To get a feel for what these so-called impossibility results entail, and the conditions under which they hold, we will review the impossibility theorem in Olsson (2005). This theorem was proved in the context of a so-called basic Lewis scenario—a scenario where two independent and partially reliable witnesses give equivalent testimonies. Since the testimonies are equivalent we may suppose that the witnesses in fact utter one and the same proposition. In the following, R _i expresses the proposition that the i:th witness is reliable, A is the proposition that the witnesses agree upon, and E _i expresses the proposition that the i:th witness asserts that A. Following epistemological tradition (Lewis 1946; BonJour 1985), we will restrict attention to a situation in which each witness is either fully reliable (truth teller) or fully unreliable (randomizer). We will model a basic Lewis scenario as a pair $\left\langle {\mathbf{S,P}} \right\rangle$ where $\mathbf{S} = \left\{ {\left\langle {{E_1},A} \right\rangle, \left\langle {{E_2},A} \right\rangle } \right\}$ and P is a class of probability distributions satisfying a number of conditions. We will state the conditions first and explain them afterwards. The following should hold (for any i):

• (a) $P\left( {{E_i}|A,\ {R_i}} \right)=1$

• (b) $P({E_i}|\neg A,\ {R_i})=0$

• (c) $P({E_i}|A,\neg {R_i})=P(A)$

• (d) $P({E_i}|\neg A,\neg {R_i})=P(A)$

• (e) $P\left( {{R_i}|A} \right)=P\left( {{R_i}} \right)$

• (f) $0<P(A)<1$

• (g) $0<P\left( {{R_i}} \right)<1$

• (h) $P\left( {{R_1}} \right)=P\left( {{R_2}} \right)$

The notions of reliability and unreliability are defined by conditions (a)–(d). Condition (a) and (b) state that a reliable witness will give a certain testimony if and only if it is true. Conditions (c) and (d) state firstly that the probability that an unreliable witness will give a certain testimony is independent of whether its content is true and secondly that it equals the prior probability that the content is in fact true. Obviously, these clauses do not hold in general, but they do hold in interesting cases.⁴ Under what circumstances would they be realistic? Here is one example: Let A be the proposition ‘Forbes committed the crime’, and let us imagine that a certain witness, Smith, is presented with a line-up comprising all and only the suspects of the case, Forbes included, among which he has to choose, and that the suspects are equally likely to be the criminal in question. Then the probability that Forbes did it is 1/n, where n is the number of suspects, and if Smith is completely unreliable, he will pick out Forbes with probability 1/n, regardless of whether Forbes is actually guilty or not. Condition (e) says that the probability that a given witness is reliable is independent of the truth of the content of the testimonies. Conditions (f) and (g) exclude certain uninteresting limiting cases. Condition (h), finally, expresses that the witnesses have the same prior probability of being reliable. This assumption is included in order to simplify calculations.

Let us by a coherence measure mean any function from ordered sets of testimonial contents to real numbers defined solely in terms of the probabilities of the testimonial contents and their Boolean combinations. It follows that a coherence measure, when restricted to a basic Lewis scenario, is a function of the probability of A. Let us furthermore say that a coherence measure C is informative in a basic Lewis scenario $\left\langle {\mathbf{S,P}} \right\rangle$ if and only if there are at least two probability distributions that give rise to different degrees of coherence, i.e., if there are P, P′ ∈ P such that ${C_P}\left( \mathbf{S} \right)\ne {C_{{P^{\prime}}}}\left( \mathbf{S} \right)$. We say that a coherence measure is truth conducive ceteris paribus in a basic Lewis scenario $\left\langle {\mathbf{S,P}} \right\rangle$ if and only if: if ${C_P}\left( \mathbf{S} \right) > {C_{{P^{\prime}}}}\left( \mathbf{S} \right)$, then $P\left( \mathbf{S} \right) > {P}^{\prime}\left( \mathbf{S} \right)$ for all P, P′ ∈ P such that $P\left( {{R_i}} \right) = {P}^{\prime}({{R^{\prime}}_i})$, for all i. Using these definitions, Olsson proved the following:

Theorem 1

(Olsson 2005): There are no informative coherence measures that are truth conducive ceteris paribus in a basic Lewis scenario.⁵

The impossibility results pose a major problem for the coherence theory as an epistemological framework for legal reasoning, shedding doubt, as they do, on the normative correctness of relying on coherence in court. Worried about these seemingly negative consequences of their deductions, coherence theorists have suggested various strategies for how to reconcile the troublesome findings with our reasoning practice.

2.3 Coherence as Conducive to Reliability

Some coherence theorists have argued that the impossibility results are not as consequential as they might seem because, they claim, the results are proved against the background of certain implausible assumptions (e.g., Mejis and Douven 2007; Schupbach 2008). In particular, it has been argued that we need to keep various further factors fixed when measuring the impact of coherence on probability of truth. One of us has argued that these rescue attempts fail: the proposed ceteris paribus-clauses do not deliver the goods (i.e., the impossibility theorems hold true anyway) and introducing new ceteris paribus-clauses sufficiently strong to save the truth conduciveness thesis would make it trivial (Schubert 2012b).

A second approach is to defend our reliance on coherence reasoning by arguing that coherence has some positive epistemic property other than truth conduciveness. For example, Staffan Angere (2007, 2008) has shown, by means of extensive computer simulations, that while a more coherent set is not always more likely to be true than a less coherent set, there is still a significant correlation between increased coherence and increased likelihood of truth. Thus, to the extent that assessing the coherence of a set is cognitively less demanding than assessing the truth of its content by other means, relying on coherence is a useful heuristic.

According to another proposal in this category due to Olsson and Schubert (2007), coherence can be reliability conducive even when it fails to be truth conducive. Roughly, a coherence measure is reliability conducive if more coherence implies a higher likelihood that the witnesses delivering the testimonies are reliable. More exactly, a coherence measure is reliability conducive ceteris paribus in a basic Lewis scenario $\left\langle {\mathbf{S,P}} \right\rangle$ if and only if: if ${C_P}\left( \mathbf{S} \right) > {C_{{P^{\prime}}}}\left( \mathbf{S} \right)$, then $P\left( {{R_i}|{E_i},\ldots,{E_n}} \right) > {P}^{\prime}\left( {{R_i}|\ {E_i},\ldots,{E_n}} \right)$ for all P, P′ ∈ P such that $P\left( {{R_i}|{E_i}} \right) = {P}^{\prime}\left( {{R_i}|{E_i}} \right)$, for all i. Olsson and Schubert showed that several measures of coherence are indeed reliability conducive under the same conditions which were used in Olsson’s impossibility result. Refinements and extensions of this result have been obtained for more elaborate witness scenarios, including situations with n equivalent testimonies or two non-overlapping testimonies. This research has focused on the Shogenji measure showing this measure to be reliability conducive in these other paradigm cases as well (Schubert 2011, 2012a).

However, it has also been shown that no measure of coherence is reliability conducive in the general case involving n non-equivalent testimonies:

Theorem 2

(Schubert 2012b): There are no informative coherence measures that are reliability conducive ceteris paribus in a scenario of n non-equivalent testimonies. ⁶

Let us take a closer look at the more striking and intuitive of the two proofs offered in (Schubert 2012b) of this theorem. This proof compares two sets of testimonies: ${S_1} = \left\{ {{A_1},\ {A_2},\ {A_3}} \right\}$, consisting of three pair-wise and jointly independent testimonies, and ${S_2} = \{{{A^{\prime}}_1},\ {{A^{\prime}}_2},\ {{A^{\prime}}_3}\}$, consisting of three jointly inconsistent but pair-wise positively relevant testimonies. If the prior probability of reliability is high, it will go down upon receiving the evidence in S ₂ because at most two witnesses can be reliable given the inconsistency of that set. If, by contrast, the prior probability of reliability is low, it will go up when receiving that same evidence, given that it is rather probable that two of the testimonies are true. Because S ₁ consists of independent propositions, the posterior probability of reliability of the sources delivering those reports will equal the prior probability of reliability. Hence, for some prior probabilities of reliability, the witnesses giving the information in S ₁ will have a higher posterior probability of reliability than the witnesses giving the information in S ₂, and for other prior probabilities of reliability, the converse will be true. But reliability conduciveness requires that more coherence implies a higher posterior probability of reliability for all prior probabilities of reliability. Hence, no coherence measure is reliability conducive in general.

Notwithstanding the impossibility theorem for reliability conduciveness, it should be remembered that coherence is reliability conducive in many cases in which it fails to be truth conducive. Thus, we have reason to believe that the link between coherence and reliability is stronger than that between coherence and truth. We now move on to uncover some further close ties between coherence and probability of reliability.

2.4 Further Connections Between Coherence and Reliability

Our next result shows that Shogenji coherence and witness reliability are even more closely related than previous work has shown: the probability that a witness is reliable given a set of testimonies is a function of the Shogenji coherence of the set and its subsets. After that, we will establish that even if Shogenji coherence falls short of being generally reliability conducive it still is reliability conducive in cases where either all witnesses are reliable or all witnesses are unreliable—i.e., where the witnesses’ levels of reliability are (maximally) positively dependent on each other. In the final section we ponder the normative significance of these results for judicial reasoning.

In order to prove the theorems below, we need to introduce the concept of a witness scenario with n witnesses which do not have to give equivalent reports. This version of the witness scenario, which we believe is an improvement in several respects to the earlier ones, was developed and used in (Schubert 2011, 2012a, b). For a discussion of the assumptions of the scenario, see, e.g., (Schubert 2011). In the following, R _i , A _i and E _i are propositional variables taking on the values R _i and $\neg$ R _i , A _i and $\neg$ A _i , and E _i and $\neg$ E _i , respectively. R _i and E _i have the same meaning as above, whereas A _i denotes the i:th witness’s testimonial content. Now a general witness scenario is a pair $\langle {{{\mathbf{S}}^{\prime}},\mathbf{P}} \rangle$ where $\mathbf{S} \mathbf{^{\prime}} = \left\{ {\left\langle {{E_1},{A_1}} \right\rangle,\ldots,\left\langle {{E_n},{A_n}} \right\rangle } \right\}$ and P is a class of probability distributions satisfying the following assumptions (for i, j = 1,…,n and i ≠ j):

• (i) $P\left( {{A_i}|{E_i},{R_i}} \right) = 1$

• (ii) $P({A_i}|{E_i},\neg {R_i}) = P({A_i}|\neg {R_i})$

• (iii) ${\boldsymbol{E}_{\mathbf{i}}}\bot {\boldsymbol{E}_{\mathbf{1}}},{\boldsymbol{R}_{\mathbf{1}}},{\boldsymbol{A}_{\mathbf{1}}},\ldots,{\boldsymbol{E}_{{i-\mathbf{1}}}},{\boldsymbol{R}_{{i-\mathbf{1}}}},{\boldsymbol{A}_{{i-\mathbf{1}}}},{\boldsymbol{E}_{{i+\mathbf{1}}}},{\boldsymbol{R}_{{i+\mathbf{1}}}},{\boldsymbol{A}_{{i+\mathbf{1}}}}, \ldots,{\boldsymbol{E}_n},{\boldsymbol{R}_n},\break {\boldsymbol{A}_n}|{\boldsymbol{R}_i},{\boldsymbol{A}_i}$

• (iv) ${{\boldsymbol{R}_i}\bot {\boldsymbol{R}_{\mathbf{1}}},\ldots,{\boldsymbol{R}_{{i-\mathbf{1}}}},{\boldsymbol{R}_{{i+\mathbf{1}}}},\ldots,{\boldsymbol{R}_n},{\boldsymbol{A}_{\mathbf{1}}},\ldots,{\boldsymbol{A}_n}}$

• (v) $0 < P\left( {{A_i}} \right) < 1$

• (vi) $0 < P\left( {{R_i}|{E_i}} \right) < 1$

• (vii) $P\left( {{R_i}|{E_i}} \right) = P\left( {{R_j}|{E_j}} \right)$

In this scenario, the conditions (i) and (ii) define the notions of reliability and unreliability. Condition (i) states that a reliable witness will only report true facts, but it does not state that if a certain fact is true, then a reliable witness will report it (in contrast to the corresponding conditions in the basic Lewis scenario). Condition (ii) says that unreliable testimonies do not affect the probability that the testimonial content is true, but does not, contrary to the basic Lewis scenario, assume that the probability that an unreliable witness will give a certain testimony equals the probability that the content of the testimony is true. Hence these conditions hold true in more real-world cases than the conditions (a)–(d) in the basic Lewis scenario do.

Conditions (iii) and (iv) define the important notion of independence. Condition (iii) states, roughly, that the probability that a witness will report a certain proposition is independent of what other witnesses have reported and of their reliability, conditional on her reliability and the truth value of the reported proposition. By condition (iv), the (un)reliability of one witness is independent of the (un)reliability of the other witnesses, as well as of the truth of the reported propositions. A condition corresponding to condition (v) was already included in the basic Lewis scenario (condition f). Condition (vi) says that the probability that a given witness is reliable conditional on her report is neither zero nor one. Condition (vii), finally, expresses that the probability that one witness is reliable, given her statement, is the same as the probability that another witness is reliable, given her statement. These two last conditions are slight variations of the conditions (g) and (h) in the basic Lewis scenario.

We are now in a position to prove our first new theorem. Let S _k be the sum of the degrees of Shogenji coherence of all subsets of {A ₁,…, A _n } with k members. (For example, if n = 3, then ${S_2} = {C_{Sh}}\left( {{A_1},{A_2}} \right) + {C_{Sh}}\left( {{A_1},{A_3}} \right) + {C_{Sh}}({A_2},{A_3})$.) Let ${S_{{{A_i}k}}}$ be the sum of the degrees of Shogenji coherence of all subsets of {A ₁,…, A _n } with k members having A _i as an element. Finally, let

$$ x=\left( {1-P\left( {R\left| {{E_i}} \right.} \right)/P\left( {R\left| {{E_i}} \right.} \right)} \right) $$

We can now show that:

Theorem 3:

$$ P({R_i}\left| {{E_1},\ldots,{E_n}} \right.)=\frac{{{S_{{{A_i}n}}}+x{S_{{{A_i}n-1}}}+\ldots +{x^{n-2 }}{S_{{{A_i}2}}}+{x^{n-1 }}}}{{{S_n}+x{S_{n-1 }}+\ldots +{x^{n-2 }}{S_2}+n{x^{n-1 }}+{x^n}}} $$

Proof:

See appendix.

This shows that the connection between reliability and coherence, in the sense of the Shogenji measure, is very close indeed. The following two observations bring out the full significance of our theorem.

Observation 1:

The posterior probability that a witness i is reliable is a strictly increasing function of the degrees of Shogenji coherence of all sets of testimonial contents that include the content of i’s testimony, and a strictly decreasing function of the degrees of Shogenji coherence of all sets of testimonial contents that do not include the content of i’s testimony.

Proof:

Follows directly from theorem 3.

In other words, given that we hold all other factors fixed, a higher degree of Shogenji coherence of a subset of {A ₁,…, A _n } which includes A _i implies a higher probability that i is reliable. Conversely, a higher degree of Shogenji coherence of a subset of {A ₁,…, A _n } which does not include A _i implies a lower probability that i is reliable.

Observation 2:

Two factors together determine the posterior probability that a witness i is reliable given a set of testimonies: the probabilities that the individual witnesses are reliable given their respective testimonies, and the degrees of Shogenji coherence of the reported set of propositions and its subsets with at least two members.

Proof:

Follows directly from theorem 3.

Observation 2 shows that there is no need for incorporating a third factor, such as (for example) the prior probability that the contents of the testimonies are true, when computing the probability of reliability given the testimonies. The Shogenji coherence of the set of testimonial contents and its subsets, and the probabilities of reliability of the individual witnesses, given their own testimonies, are the only factors needed to determine the probability of reliability given all the testimonies.

As previously mentioned, the Shogenji measure has been shown to be reliability conducive in a number of paradigmatic cases (Schubert 2011, 2012a). We will now extend these results to a further interesting case. In order to set the stage for what is to come we need to make a small digression. Condition (iv) in the definition of the general witness scenario states that for all witnesses, the fact that one witness is reliable (or not) does not directly affect the reliability of the other witnesses. In other words, R _i and R _j are assumed to be independent, for all i, j. But as Bovens and Hartmann (2003, 64) point out in an interesting section, this is often an unrealistic assumption. Sometimes the reliability of one witness positively affects the reliability of another witness, in which case R _i and R _j are positively dependent. For example, if we learn that one member of a group of indigenous people, whom we have very little knowledge of, is reliable, we are inclined to upgrade our beliefs in the reliability of other members of the group. Another important case is of course the case where all testimonies have been given by one and the same witness. In such a case, R _i and R _j should surely be strongly positively dependent.

Bovens and Hartmann construct a model where the witnesses are maximally positively dependent by using a variable R which can take on only two values corresponding to all witnesses being reliable or all witnesses being unreliable. If we replace R ₁ ,…,R _n by R in our definition of a witness scenario, it can be shown that the Shogenji measure is reliability conducive if we use a slightly revised definition of reliability conduciveness where both P(R|E _i ) and P(R) are kept fixed. In order to see this, let us first define this modified witness scenario formally.

The witness scenario with a single reliability variable is a pair 〈S ^*, P*〉 where $\mathbf{S}^*=\left\{ {\left\langle {{E_1},{A_1}} \right\rangle,\ldots,\left\langle {{E_n},{A_n}} \right\rangle } \right\}$ and P ^* a class of probability distributions satisfying the following conditions (for i, j = 1,…,n and i ≠ j):

• (i′.) $P\left( {{A_i}|{E_i},R} \right) = 1$

• (ii′.) $P\left( {{A_i}|{E_i},\neg R} \right) = P\left( {{A_i}|\neg R} \right)$

• (iii′.) ${\boldsymbol{E}_i}\bot {\boldsymbol{E}_{\mathbf{1}}},\ {\boldsymbol{A}_{\mathbf{1}}},\ldots,\ {\boldsymbol{E}_{{i-\mathbf{1}}}},{\boldsymbol{A}_{{i-\mathbf{1}}}},{\boldsymbol{E}_{{i+\mathbf{1}}}},{\boldsymbol{A}_{{i+\mathbf{1}}}},\ldots,{\boldsymbol{E}_n},{\boldsymbol{A}_n}|\boldsymbol{R},{\boldsymbol{A}_i}$

• (iv′.) ${\boldsymbol{R}\bot {\boldsymbol{A}_{\mathbf{1}}},\ldots,{\boldsymbol{A}_n}}$

• (v′.) $0 < P\left( {{A_i}} \right) < 1$

• (vi′.) $0 < P(R|{E_i}) < 1$

• (vii′.) $P\left( {R|{E_i}} \right) = P\left( {R|{E_j}} \right)$

We are now in a position to give a precise definition of reliability conduciveness in the case in question:

Definition 1:

A coherence measure C is reliability conducive ceteris paribus in the witness scenario $\left\langle {\mathbf{S}^*,\mathbf{P}^*} \right\rangle$ with a single reliability variable if and only if: if ${C_P}(\mathbf{S}^*)>{C_{{P^{\prime}}}}(\mathbf{S}^*)$, then $P(R|{E_1},\ldots,{E_n})>{P}^{\prime}({R}^{\prime}|{{E^{\prime}}_1},\ldots,{{E^{\prime}}_n})$ for all P, P′ ∈ P ^* such that $P(R)={P}^{\prime}({R}^{\prime})$ and $P(R|{E_i})={P}^{\prime}({R}^{\prime}|{{E^{\prime}}_i})$, for all i.

Theorem 4:

The Shogenji measure is reliability conducive ceteris paribus in the witness scenario with a single reliability variable.

Proof:

In appendix.

Thus, even though the Shogenji measure is not reliability conducive in the witness scenario where the witnesses’ degrees of reliability are independent, it is reliability conducive when the witnesses’ degrees of reliability are maximally dependent of each other. Theorem 4 shows, together with observations 1 and 2, that there are important further connections between the Shogenji measure of coherence and the posterior probability of reliability.

We will now prove some further observations which make the link between coherence and reliability still stronger. They will also serve to show why theorem 4 holds. We will consider the effect of adding the following two conditions (for i = 1,…,n). Together with conditions (i′) and (ii′) above, they correspond to (a)–(d) in the basic Lewis scenario:

• (a′) $P\left( {{E_i}|{A_i},\ R} \right) = 1$

• (b′) $P\left( {{E_i}|{A_i},\neg R} \right) = P\left( {{A_i}} \right) = P\left( {{E_i}|\neg {A_i},\neg R} \right)$

Under these extra assumptions we get a particularly simple formula for calculating the probability that a particular set of testimonies will be given conditional on the fact that all witnesses are reliable.

Observation 3:

In a witness scenario with a single reliability variable satisfying (a′) and (b′), $P({E_1},\ldots,{E_n}\left| R \right.)=P({A_1},\ldots,{A_n}).$

Proof:

In appendix.

It should be obvious why this is true. Given that all witnesses know the truth and are willing to share their knowledge, the chance that they will give a certain conjunction of testimonies should equal the probability that the conjunction is true.

Similarly, the probability of the evidence given that all witnesses are unreliable now simplifies to:

Observation 4:

In a witness scenario with a single reliability variable satisfying (a′) and (b′), $P\left( {{E_1},\ldots,{E_n}\left| {\neg R} \right.} \right)=P\left( {{A_1}} \right)\times \ldots \times P\left( {{A_n}} \right).$

Proof:

In appendix.

Thus we get the following elegant corollary:

Observation 5:

In a witness scenario satisfying (a′) and (b′), $P\left( {{E_1},\ldots,{E_n}\left| R \right.} \right)/\break P\left( {{E_1},\ldots,{E_n}\left| {\neg R} \right.} \right)={C_{Sh }}\left( {{A_1},\ldots,{A_{\mathrm{n}}}} \right)$.

Proof:

Follows directly from Observation 3 and Observation 4 using the definition of the Shogenji measure.

$P\left( {{E_1},\ldots,{E_n}\left| R \right.} \right)/P\left( {{E_1},\ldots,{E_n}\left| {\neg R} \right.} \right)$ is known as the likelihood ratio. In general, given evidence E and hypothesis H, the likelihood ratio equals $P\left( {E\left| H \right.} \right)/P\left( {E\left| {\neg H} \right.} \right)$. In this case, the hypothesis is obviously R and the evidence E ₁,…,E _n . The likelihood ratio is proposed as a measure of the degree to which evidence confirms a hypothesis by various authors (Kemeny and Oppenheim 1952; Good 1983).⁷ Thus, in the scenario with a single reliability variable which includes (a′) and (b′), the Shogenji measure of a set of propositions A ₁,…, A _n equals the degree to which E ₁,…,E _n confirm R, according to the likelihood measure.

Now using Bayes’ theorem, we may note that:

Formula 1:

$$ P\left( {R\left| {{E_1},\ldots,{E_n}} \right.} \right)=\frac{{P(R)\times P\left( {{E_1},\ldots,{E_n}\left| R \right.} \right)/P\left( {{E_1},\ldots,{E_n}\left| {\neg R} \right.} \right)}}{{P\left( {\neg R} \right)+P(R)\times P\left( {{E_1},\ldots,{E_n}\left| R \right.} \right)/P\left( {{E_1},\ldots,{E_n}\left| {\neg R} \right.} \right)}} $$

It follows immediately from observation 5 and formula 1 that the Shogenji measure is reliability conducive in the scenario with a single reliability variable which includes (a′) and (b′).

Let us now consider a scenario where (a′) and (b′) does not hold. Then:

Observation 6:

$$ P\left( {{E_1},\ldots,{E_n}\left| R \right.} \right)=P\left( {{A_1},\ldots,{A_n}} \right)\times \frac{{P\left( {{E_1}\left| R \right.} \right)}}{{P\left( {{A_1}} \right)}}\times \ldots \times \frac{{P\left( {{E_n}\left| R \right.} \right)}}{{P\left( {{A_n}} \right)}} $$

Observation 7:

$$ P\left( {{E_1},\ldots,{E_n}\left| {\neg R} \right.} \right)=P\left( {{E_1}\left| {\neg R} \right.} \right)\times \ldots \times P\left( {{E_n}\left| {\neg R} \right.} \right) $$

Proofs:

In appendix.

From those observations, the following observation can be made:

Observation 8:

$$ {C_{Sh }}\left( {{A_1},\ldots,{A_n}} \right)=\frac{{P\left( {{E_1},\ldots,{E_n}\left| R \right.} \right)/P\left( {{E_1},\ldots,{E_n}\left| {\neg R} \right.} \right)}}{{P\left( {{E_1}\left| R \right.} \right)/P\left( {{E_1}\left| {\neg R} \right.} \right)\times \ldots \times P\left( {{E_n}\left| R \right.} \right)/P\left( {{E_n}\left| {\neg R} \right.} \right)}} $$

This means that in the general case, the Shogenji measure is rather the ratio between the degree to which E ₁,…,E _n collectively supports R (as measured by the likelihood measure) and the product of the degrees to which E ₁,…,E _n individually supports R (as measured by the likelihood measure). Now we may note that:

$$ \frac{{P\left( {{E_i}\left| R \right.} \right)}}{{P\left( {{E_i}\left| {\neg R} \right.} \right)}}=\frac{{P\left( {R\left| {{E_i}} \right.} \right)/P(R)}}{{P\left( {\neg R\left| {{E_i}} \right.} \right)/P\left( {\neg R} \right)}}\mathrm{for}\ \mathrm{all}\ i. $$

Hence if P(R|E _i ) and P(R) are kept fixed (as demanded by definition 1), the degrees to which E ₁,…,E _n individually supports R (as measured by the likelihood measure) will be kept fixed. Hence, it follows from observation 8 and formula 1 that the Shogenji measure is reliability conducive in this case, too.

2.5 Comparison with the Evidentiary Value Model

A model of legal reasoning similar to that outlined above was developed in the 1970s by Swedish philosophers Martin Edman (1973) and Sören Halldén (1973), drawing on ideas from Per Olof Ekelöf, a legal theorist (1963/1982; 1983).⁸ The central concept of the Evidentiary Value Model (EVM) is that of an evidentiary mechanism (usually denoted by A, B, etc.) linking the hypothesis and the evidence. The evidentiary value is correlated with the (probability of) presence or absence of such evidentiary mechanisms. In the terminology of the theory, the evidentiary theme (usually denoted H) is the hypothesis to be proved. Various pieces of evidence called evidentiary facts (usually denoted by small letters: e, f, etc.) may either confirm or disconfirm the evidentiary theme. The original idea was to interpret such a mechanism as a causal link between an evidentiary theme and an evidentiary fact, although several researchers—among them Edman (1973) and Hansson (1983)—use the notion of an evidentiary mechanism in a more general sense without implying any causal connotations. Sahlin (2011) explains the concept of an evidentiary mechanism as follows:

One way to think of the evidentiary mechanism is to interpret it as a link between an evidentiary theme and an evidentiary fact which, if present, can be said to ‘prove’ the theme, given the evidentiary fact. Think of this mechanism (denoted M) as a triple consisting of an evidentiary theme, an evidentiary fact and an event such that, if we know that this event has occurred, and we have received the evidentiary fact, we have a proof of the hypothesis.

The EVM theorists now claim that legal examination of the evidence should focus on P(A|e) rather than on P(H|e). Hence, rather than assessing the probability of the hypothesis given the evidence, we should assess the probability that the evidence proves the hypothesis. Various reasons have been presented for the preoccupation with P(A|e). According to Halldén (1973), it is easier for a judge to assess P(A|e) than P(H|e), whereas Hansson (1983) notes that P(H|e) is not primarily what we are looking for since the evidentiary theme may be very probable for reasons that have nothing to do with the defendant. Thus, to take Hansson’s example, even if it is true that 98% of all habitual criminals are in fact found guilty when they are prosecuted for yet another crime, this fact alone is not sufficient for a conviction. This is so even if another defendant is convicted on evidence which indicates guilt with a probability of less than 98%, say 90%, provided that the evidence is directly connected to that person. Sahlin (2011), finally, suggests that what we wish to obtain in court is knowledge and not mere (true) belief and that this is the reason why we should be primarily concerned with assessing the probability of a reliable connection:

Assume that a judge is in the business of trying to reach an opinion as to whether the evidence gives knowledge of the hypothesis under consideration, rather than merely trying to form a belief as to whether the hypothesis is true. He or she is then trying to ascertain, on the basis of his evidence, the probability of the existence of a reliable link between the hypothesis and the evidentiary fact — trying to ascertain how probable it is that the evidentiary mechanism has worked, given the evidence at hand.

Our emphasis. ‘Knowledge’ should here be taken in the reliabilist sense of true belief acquired through a reliable process (Ramsey 1931; Goldman 1986).

It is generally assumed by the practitioners of EVM that P(H|Ae) = 1. Since H is the evidentiary theme (e.g., whether the suspect did in fact commit the crime), this means that each evidentiary fact is such that, if produced by a working evidentiary mechanism, it suffices to prove the case.^{9 , 10}

The interesting cases, from our perspective, are of course those involving several evidentiary facts that cohere. Suppose that there are two pieces of concurring evidence, e and f, both of which point to the truth of an evidentiary theme, H, via two independent evidentiary mechanisms A (concerning e) and B (concerning f). The EVM theorists now claim that the relevant probability to asses is P(A ∨ B|ef), i.e., the probability that at least one of the mechanisms worked. Why is that? Given the assumption that $P\left( {H|Ae} \right) = P\left( {H|Bf} \right)=1$, it suffices, for the purposes of proving the truth of H, to establish that at least one of A or B worked; it is not necessary that they both did.

EVM now offers the following attractive formula relating the probability that at least one mechanism worked, given the combined evidentiary facts, to the probability that the first and the second worked, respectively, given their corresponding evidentiary facts:

$$ \left( \mathrm{S} \right)P(A\vee B|ef)\geq P\left( {A|e} \right)+P\left( {B|f} \right)-P\left( {A|e} \right)P\left( {B|f} \right) $$

Halldén (1973) proves (S) using the independence assumption

$$ \left( \mathrm{I} \right)\ P\left( {A|e} \right) = P(A|ef\neg B) $$

together with the further principle

$$ \left( \mathrm{P} \right)\ P\left( {A|ef} \right)\geq P\left( {A|e} \right), $$

which in the EVM tradition is something of a cornerstone.

The independence assumption states that the evidentiary value of a piece of evidence is not altered by the presence of an evidentiary fact deriving from a malfunctioning evidentiary mechanism. According to (P), a further concurring evidentiary fact increases or leaves equal the probability that the first evidentiary mechanism was working.¹¹

As the reader has probably noticed, the principles mentioned above for evidentiary value bear strong resemblance to our principles for reliability. Rather than talking of functioning or malfunctioning evidentiary mechanisms we can talk about reliable or unreliable witnesses. Thus, (P) translates into $P\left( {{R_1}|{E_1},{E_2}} \right)\geq P\left( {{R_1}|{E_1}} \right)$: the probability that a given witness is reliable is not diminished by the appearance of a further witness giving a concurring testimony. The other principles can also be translated into our framework in obvious ways. It can be shown that that the translated versions of (S), (I) and (P) are derivable from our main scenario, the general witness scenario $\langle {{{\mathbf{S}}^{\prime}},\mathbf{P}} \rangle$.

Observation 9 (corresponding to S):

$$ P({R_1}\vee {R_2}|{E_1},{E_2})\geq P\left( {{R_1}|{E_1}} \right)+P\left( {{R_2}|{E_2}} \right)\hbox{--} P\left( {{R_1}|{E_1}} \right)P\left( {{R_2}|{E_2}} \right) $$

Observation 10 (I):

$$ P\left( {{R_1}|{E_1}} \right) = P({R_1}|{E_1},{E_2},\neg {R_2}) $$

Observation 11 (P):

$$ P\left( {{R_1}|{E_1},{E_2}} \right)\geq \left( {{R_1}|{E_1}} \right) $$

Proofs:

In appendix.

These formal parallels are no coincidence. Rather they reflect similar theoretical interests and goals. This means that the philosophical and legal motivations of the EVM framework carry over to our framework. In particular, we find attractive the view that the quest for reliable knowledge is, in a judicial context, more central than the quest for true belief, making reliability conduciveness a more fundamental property than truth conduciveness. Hence our proofs of close connections between the Shogenji measure and reliability constitute, we believe, an important vindication of coherence reasoning in judicial contexts.

Still, there is a salient difference in focus between our framework and the EVM framework. While our theory is chiefly concerned with assessing the probability that any given witness i is reliable, the EVM theorists were more interested in ascertaining the probability that at least one mechanism worked reliably. This preoccupation on the part of the EVM theorists with (S) (corresponding to our Observation 9) reveals a primary interest in cases in which each evidentiary fact is potentially sufficient for settling the matter under dispute, i.e. the evidentiary theme. A paradigm case would be one in which the fact to be demonstrated by the prosecutor is the guilt of the accused, whereby the evidentiary facts consist in several witnesses reporting, individually, something that, if correct, would be sufficient for convicting the defendant. The relevant probability to be ascertained in such cases is indeed the probability that at least one of these evidentiary mechanisms worked.

Yet, the normal case is surely one in which it is not the case that the evidentiary facts, taken by themselves, suffice to prove the evidentiary theme but rather one in which the evidentiary facts are only indirectly related to the evidentiary theme, as the case would be if one witness states that she saw the accused near the crime scene, another that he was told by someone else that the accused did it, and so on. If so, it would not be sufficient for a conviction that only one of the corresponding evidentiary mechanisms worked, making the assessment of the corresponding probability an idle task. Rather, we should be interested in having as many reliably formed testimonies as possible. If this is correct, then the more widely relevant task is to assess—in conformity with our account—the probability that any given witness is reliable.

2.6 Conclusion

We started out by referring to the ubiquity of coherence reasoning in court. When jurors assess the evidence presented before them, they try to construct the most coherent story based on the information at hand, selecting the verdict that they find most appropriate given this story. We then noticed that the impossibility results for coherence shed doubt on the normative correctness of this practice. Our response was to argue, as we have done in previous work, that coherence can still be conducive to reliability in the sense that more coherence implies a higher probability that the witnesses are reliable in several paradigm cases. In further support of this proposal we showed that there are several close connections that have hitherto gone unnoticed between the Shogenji measure of coherence and the degrees of witness reliability. One such observation stated that the probability that a witness is reliable given a set of testimonies is a function of the Shogenji coherence of the set and its (non-singleton) subsets; another that even if Shogenji coherence falls short of being generally reliability conducive, it is reliability conducive in cases where the witnesses’ degrees of reliability are maximally dependent on each other—i.e., where either all witnesses are reliable or all witnesses are unreliable. In addition, we proved that, under certain circumstances, the degree of Shogenji coherence of a set equals the degree of support that the testimonies in that set confers on the hypothesis that all witnesses are reliable. In the penultimate section we unraveled the intimate relationships between our framework and that of the Scandinavian School of Evidentiary Value. In particular, we found independent support in the writings of the Evidentiary Value theorists for thinking that assessing the probability that the witnesses are reliable is more fundamental a task than ascertaining the probability that what they are saying is true.¹²

Appendix

Proof of theorem 3:

$$ P\left( {{R_i}\left| {{E_1},\ldots,{E_n}} \right.} \right)=\frac{{{S_{{{A_i}n}}}+x{S_{{{A_i}n-1}}}+\ldots +{x^{n-2 }}{S_{{{A_i}2}}}+{x^{n-1 }}}}{{{S_n}+x{S_{n-1 }}+\ldots +{x^{n-2 }}{S_2}+n{x^{n-1 }}+{x^n}}} $$

Let ${\boldsymbol{R}_1},\ldots,{\boldsymbol{R}_n},{\boldsymbol{E}_1},\ldots,{\boldsymbol{E}_n},{\boldsymbol{A}_1},\ldots,{\boldsymbol{A}_n}$ be propositional variables. Then:

$$ P\left( {{\boldsymbol{R}_1},\ldots,{\boldsymbol{R}_n},{\boldsymbol{E}_1},\ldots,{\boldsymbol{E}_n},{\boldsymbol{A}_1},\ldots,{\boldsymbol{A}_n}} \right)= $$

$$ P\left( {{\boldsymbol{E}_1}\left| {{\boldsymbol{R}_1}}\right.,{\boldsymbol{A}_1}} \right)\times \ldots \times P\left( {{\boldsymbol{E}_n}\left| {{\boldsymbol{R}_n}}\right.,{\boldsymbol{A}_n}} \right)\times P\left( {{\boldsymbol{R}_1},\ldots,{\boldsymbol{R}_n},{\boldsymbol{A}_1},\ldots,{\boldsymbol{A}_n}} \right)= $$

(iii)

$$ P\left( {{\boldsymbol{E}_1},{\boldsymbol{R}_1},{\boldsymbol{A}_1}} \right)\times \ldots \times P\left( {{\boldsymbol{E}_n},{\boldsymbol{R}_n},{\boldsymbol{A}_n}} \right)\times \frac{{P\left( {{\boldsymbol{A}_1},\ldots,{\boldsymbol{A}_n}} \right)}}{{P\left( {{\boldsymbol{A}_1}} \right)\times \ldots \times P\left( {{\boldsymbol{A}_n}} \right)}}= $$

(iv)

$$ \begin{gathered} P\left( {{\boldsymbol{A}_1}\left| {{\boldsymbol{E}_1},{\boldsymbol{R}_1}} \right.} \right)\times \ldots \times P\left( {{\boldsymbol{A}_n}\left| {{\boldsymbol{E}_n},{\boldsymbol{R}_n}} \right.} \right)\times \frac{{P\left( {{\boldsymbol{A}_1},\ldots,{\boldsymbol{A}_n}} \right)}}{{P\left( {{\boldsymbol{A}_1}} \right)\times \ldots \times P\left( {{\boldsymbol{A}_n}} \right)}} \hfill \\ \times P\left( {{\boldsymbol{R}_1}\left| {{\boldsymbol{E}_1}} \right.} \right)\times P\left( {{\boldsymbol{E}_1}} \right)\times \ldots \times P\left( {{\boldsymbol{R}_n}\left| {{\boldsymbol{E}_n}} \right.} \right)\times P\left( {{\boldsymbol{E}_n}} \right) \hfill \\ \end{gathered} $$

Using this derivation, we calculate the probability of $P\left( {{R_1}\left| {{E_1},\ldots,{E_n}} \right.} \right)$, which equals $P\left( {{R_i}\left| {{E_1},\ldots,{E_n}} \right.} \right)$, for any i. Let:

$$ P\left( {{A_i}} \right)={a_i} $$

$$ P\left( {{A_i},\ldots,{A_j}} \right)={a_{{i,\ldots,j}}} $$

$$ P\left( {{E_i}} \right)={e_i} $$

$$ P\left( {{R_i}\left| {{E_i}} \right.} \right)=m $$

Then:

$$ P\left( {{\boldsymbol{R}_1}\left| {{\boldsymbol{E}_1},\ldots,{\boldsymbol{E}_n}} \right.} \right)=\frac{{P\left( {{\boldsymbol{R}_1},{\boldsymbol{E}_1},\ldots,{\boldsymbol{E}_n}} \right)}}{{P\left( {{\boldsymbol{E}_1},\ldots,{\boldsymbol{E}_n}} \right)}}= $$

$$ =\frac{{\sum\nolimits_{{{A_1},\ldots,{A_n},{R_2},\ldots,{R_n}}} {P\left( {{R_1},{R_2}\ldots, {R_n},{A_1},\ldots,{A_n},{E_1},\ldots,{E_n}} \right)} }}{{\sum\nolimits_{{{A_1},\ldots,{A_n},{R_1},\ldots,{R_n}}} {P\left( {{R_1},\ldots,{R_n},{A_1},\ldots,{A_n},{E_1},\ldots,{E_n}} \right)} }} $$

$$ P\left( {{R_1},{E_1},\ldots,{E_n}} \right)=\prod\limits_{k=1}^n {{e_k}} \times \sum\limits_{k=0}^{n-1 } {{m^{n-k }}{{{\left( {1-m} \right)}}^k}{b_k}} $$

(vii)

where ${b_k}=\sum\limits_{{1<{q_2}<\ldots <{q_r}\leq n,r=n-k}} {\frac{{{a_{{1,{q_2},\ldots,{q_r}}}}}}{{{a_1},{a_{{{q_2}}}},\ldots,{a_{{{q_r}}}}}}}$

$$ P\left( {{E_1},\ldots,{E_n}} \right)=\prod\limits_{k=1}^n {{e_k}} \times \sum\limits_{k=0}^n {{m^{n-k }}{{{\left( {1-m} \right)}}^k}{c_k}} $$

(vii)

where ${c_k}=\sum\limits_{{{q_1}<,\ldots,<{q_r}\leq n,r=n-k}} {\frac{{{a_{{{q_1},\ldots,{q_r}}}}}}{{{a_{{{q_1}}}},\ldots,{a_{{{q_r}}}}}}}$

Hence:

$$ \begin{gathered} P\left( {{R_1}\left| {{E_1},\ldots,{E_n}} \right.} \right) =\frac{{\prod\limits_{k=1}^n {{e_k}} \times \sum\limits_{k=0}^{n-1 } {{m^{n-k }}{{{\left( {1-m} \right)}}^k}{b_k}} }}{{\prod\limits_{k=1}^n {{e_k}} \times \sum\limits_{k=0}^n {{m^{n-k }}{{{\left( {1-m} \right)}}^k}{c_k}} }}= \end{gathered} $$

$$ \quad\quad\;\; =\frac{{\sum\limits_{k=0}^{n-1 } {{m^{n-k }}{{{\left( {1-m} \right)}}^k}{b_k}} }}{{\sum\limits_{k=0}^n {{m^{n-k }}{{{\left( {1-m} \right)}}^k}{c_k}} }} $$

$$ \quad\quad\;\; =\frac{{{S_{{{A_1}n}}}+x{S_{{{A_1}n-1}}}+\ldots +{x^{n-2 }}{S_{{{A_1}2}}}+{x^{n-1 }}}}{{{S_n}+x{S_{n-1 }}+\ldots +{x^{n-2 }}{S_2}+n{x^{n-1 }}+{x^n}}} $$

Proof of theorem 4:

Let $\boldsymbol{R},{\boldsymbol{E}}_{\textbf{1}},\ldots,{\boldsymbol{E}}_{\boldsymbol{n}},{\boldsymbol{A}}_{\textbf 1},\ldots,{\boldsymbol{A}}_{\boldsymbol{n}}$ be propositional variables. Then:

$$ P\left( {\boldsymbol{R}},{\boldsymbol{E_1}},\ldots,{\boldsymbol{E}}_{\boldsymbol{n}},\boldsymbol{A}_{\textbf 1},\ldots,{\boldsymbol{A}}_{\boldsymbol{n}} \right)= $$

$$ P\left( {\boldsymbol{E}}_{\textbf 1}\left| {\boldsymbol{R}} \right.,{\boldsymbol{A}}_{\textbf 1} \right)\times \ldots \times P\left( {\boldsymbol{E}}_{\boldsymbol{n}}\left| \boldsymbol{R} \right.,{\boldsymbol{A}}_{\boldsymbol{n}} \right)\times P\left( {\boldsymbol{R}},{\boldsymbol{A}}_{\textbf 1},\ldots,{\boldsymbol{A}}_{\boldsymbol{n}} \right)= $$

(iii′)

$$ \begin{gathered} P\left( {\boldsymbol{E}}_{\textbf 1},{\boldsymbol{R}},{\boldsymbol{A}}_{\textbf 1} \right)\times \ldots \times P\left( {\boldsymbol{E}}_{\boldsymbol{n}},{\boldsymbol{R}},{\boldsymbol{A}}_{\boldsymbol{n}} \right)\times \hfill \\ \frac{{P\left( {\boldsymbol{A}}_{\textbf 1},\ldots,{\boldsymbol{A}}_{\boldsymbol{n}} \right)}}{{P\left( {\boldsymbol{A}}_{\textbf 1} \right)\times \ldots \times P\left( {\boldsymbol{A}}_{\boldsymbol{n}} \right)}}\times \frac{1}{P{({\boldsymbol{R}})^{{\boldsymbol{n}}-{\textbf 1} }}}= \end{gathered} $$

(iv′)

$$ \begin{gathered} P\left( {\boldsymbol{A}}_{\textbf 1}\left| {{\boldsymbol{E}}_{\textbf 1},{\boldsymbol{R}}} \right. \right)\times \ldots \times P\left( {{\boldsymbol{A}}_{\boldsymbol{n}}\left| {{\boldsymbol{E}}_{\boldsymbol{n}},{\boldsymbol{R}}} \right.} \right)\times \frac{{P\left( {\boldsymbol{A}}_{\textbf 1},\ldots,{\boldsymbol{A}}_{\boldsymbol{n}} \right)}}{{P\left( {\boldsymbol{A}}_{\textbf 1} \right)\times \ldots \times P\left( {\boldsymbol{A}}_{\boldsymbol{n}} \right)}}\times \hfill \\ \frac{{P\left( {\boldsymbol{R}}\left| {\boldsymbol{E}}_{\textbf 1} \right. \right)\times P\left( {\boldsymbol{E}}_{\textbf 1} \right)}}{P({\boldsymbol{R}})}\times \ldots \times \frac{{P\left( {\boldsymbol{R}}\left| {\boldsymbol{E}}_{\boldsymbol{n}} \right. \right)\times P\left( {\boldsymbol{E}}_{\boldsymbol{n}} \right)}}{P({\boldsymbol{R}})}\times P({\boldsymbol{R}}) \hfill \\ \end{gathered} $$

Then:

$$ P\left( {{E_1},\ldots,{E_n}\left| R \right.} \right)=P\left( {{A_1},\ldots,{A_n}} \right)\times \frac{{P\left( {{E_1}\left| R \right.} \right)}}{{P\left( {{A_1}} \right)}}\times \ldots \times \frac{{P\left( {{E_n}\left| R \right.} \right)}}{{P\left( {{A_n}} \right)}} $$

(i′)

$$ P\left( {{E_1},\ldots,{E_n}\left| {\neg R} \right.} \right)=P\left( {{E_1}\left| {\neg R} \right.} \right)\times \ldots \times P\left( {{E_1}\left| {\neg R} \right.} \right) $$

(ii′)

Let:

$$ P(R)=r $$

$$ P\left( {R\left| {{E_i}} \right.} \right)=m $$

$$ \bar{p}=1-p, \hbox{for any variable}\ p. $$

Then:

$$ \begin{aligned}[b] &P\left( {R\left| {{E_1},\ldots,{E_n}} \right.} \right)=\\ &\frac{{{C_{Sh }}\left( {{A_1},\ldots,{A_n}} \right)\times \frac{{m{e_1}}}{r}\times \ldots \times \frac{{m{e_n}}}{r}\times r}}{{{C_{Sh }}\left( {{A_1},\ldots,{A_n}} \right)\times \frac{{m{e_1}}}{r}\times \ldots \times \frac{{m{e_n}}}{r}\times r+\frac{{\bar{m}{e_1}}}{\bar{r}}\times \ldots \times \frac{{\bar{m}{e_n}}}{\bar{r}}\times \bar{r}}} \\ &\quad =\frac{{{C_{Sh }}\left( {{A_1},\ldots,{A_n}} \right)\times \frac{m}{r}\times \ldots \times \frac{m}{r}\times r}}{{{C_{Sh }}\left( {{A_1},\ldots,{A_n}} \right)\times \frac{m}{r}\times \ldots \times \frac{m}{r}\times r+\frac{\bar{m}}{\bar{r}}\times \ldots \times \frac{\bar{m}}{\bar{r}}\times \bar{r}}} \end{aligned}$$

(vii′)

Thus P(R|E ₁,…,E _n ) is a strictly increasing function of C _Sh (A ₁,…, A _n ), given the assumptions of the scenario. Hence, the Shogenji measure is reliability conducive, in the present scenario.

Proofs of observation 3 and 4:

$$ {\boldsymbol{R}},{\boldsymbol{E}}_{\textbf 1},\ldots,{\boldsymbol{E}}_{\boldsymbol{n}},{\boldsymbol{A}}_{\textbf 1},\ldots,{\boldsymbol{A}}_{\boldsymbol{n}}=P\left( {\boldsymbol{E}}_{\textbf 1}\left| {\boldsymbol{R}} \right.,{\boldsymbol{A}}_{\textbf 1} \right)\times \ldots \times P\left( {\boldsymbol{E}}_{\boldsymbol{n}}\left| {\boldsymbol{R}} \right.,{\boldsymbol{A}}_{\textbf 1} \right)\\ \quad \times P\left( {\boldsymbol{R}},{\boldsymbol{A}}_{\textbf 1},\ldots,{\boldsymbol{A}}_{\boldsymbol{n}} \right) $$

(see proof of theorem 4)

Hence:

$$ P\left( {{E_1},\ldots,{E_n}\left| R \right.} \right)=P\left( {{A_1},\ldots,{A_n}} \right) $$

(a′)

$$ P\left( {{E_1},\ldots,{E_n}\left| {\neg R} \right.} \right)=P\left( {{A_1}} \right)\times \ldots \times P\left( {{A_n}} \right) $$

(b′)

Proofs of observation 6 and 7:

See proof of theorem 4.

Proof of observation 9:

Assume for reductio that:

$$ P\left( {{R_1}\vee {R_2}\left| {{E_1},{E_2}} \right.} \right)<P\left( {{R_1}\left| {{E_1}} \right.} \right)+P\left( {{R_2}\left| {{E_2}} \right.} \right)-P\left( {{R_1}\left| {{E_1}} \right.} \right)P\left( {{R_2}\left| {{E_2}} \right.} \right) $$

Let P(H) = h. P(H) is the probability of what the witnesses agree upon.

$$ \frac{{P\left( {{R_1}\vee {R_2},{E_1},{E_2}} \right)}}{{P\left( {{E_1},{E_2}} \right)}}=\frac{{{e_1}{e_2}{m^2}\frac{1}{h}+2{e_1}{e_2}m\left( {1-m} \right)}}{{{e_1}{e_2}{m^2}\frac{1}{h}+2{e_1}{e_2}m\left( {1-m} \right)+{e_1}{e_2}{{{\left( {1-m} \right)}}^2}}} $$

(Schubert 2011, 273) together with condition (vii)

$$ P\left( {{R_1}\left| {{E_1}} \right.} \right)+P\left( {{R_2}\left| {{E_2}} \right.} \right)-P\left( {{R_1}\left| {{E_1}} \right.} \right)P\left( {{R_2}\left| {{E_2}} \right.} \right)=2m-{m^2} $$

$$ \frac{{{e_1}{e_2}{m^2}\frac{1}{h}+2{e_1}{e_2}m\left( {1-m} \right)}}{{{e_1}{e_2}{m^2}\frac{1}{h}+2{e_1}{e_2}m\left( {1-m} \right)+{e_1}{e_2}{{{\left( {1-m} \right)}}^2}}}<2m-{m^2} $$

$$ \Leftrightarrow {m^2}\frac{1}{h}+2m\left( {1-m} \right)<\left( {2m-{m^2}} \right)\left( {{m^2}\frac{1}{h}+2m\left( {1-m} \right)+{{{\left( {1-m} \right)}}^2}} \right) $$

$$ \Leftrightarrow {{\left( {1-m} \right)}^2}\left( {{m^2}\frac{1}{h}+2m\left( {1-m} \right)} \right)<\left( {2m-{m^2}} \right){{\left( {1-m} \right)}^2} $$

$$ \Leftrightarrow 1<h $$

But this contradicts condition (v), which says that 0 < h < 1. Hence $P\left( {{R_1}\vee {R_2}\left| {{E_1},{E_2}} \right.} \right)\geq P\left( {{R_1}\left| {{E_1}} \right.} \right)+P\left( {{R_2}\left| {{E_2}} \right.} \right)-P\left( {{R_1}\left| {{E_1}} \right.} \right)P\left( {{R_2}\left| {{E_2}} \right.} \right)$.

Proof of observation 10:

$$ P\left( {{R_1}\left| {{E_1},{E_2}} \right.,\neg {R_2}} \right)=\frac{{{e_1}{e_2}m\left( {1-m} \right)}}{{{e_1}{e_2}\left( {1-m} \right)m+{e_1}{e_2}{{{\left( {1-m} \right)}}^2}}}=m $$

(Schubert 2011, 273) together with condition (vii)

Hence $P\left( {{R_1}\left| {{E_1},{E_2}} \right.,\neg {R_2}} \right)=P\left( {{R_1}\left| {{E_1}} \right.} \right)$

Proof of observation 11:

$$ \frac{{P\left( {{R_1},{E_1},{E_2}} \right)}}{{P\left( {{E_1},{E_2}} \right)}}=\frac{{{e_1}{e_2}{m^2}\frac{1}{h}+{e_1}{e_2}m\left( {1-m} \right)}}{{{e_1}{e_2}{m^2}\frac{1}{h}+2{e_1}{e_2}m\left( {1-m} \right)+{e_1}{e_2}{{{\left( {1-m} \right)}}^2}}} $$

(Schubert 2011, 273) together with condition (vii)

$$ =\frac{{{m^2}\frac{1}{h}+m\left( {1-m} \right)}}{{{m^2}\frac{1}{h}+2m\left( {1-m} \right)+{{{\left( {1-m} \right)}}^2}}} $$

Assume for reductio that $P\left( {{R_1}\left| {{E_1},{E_2}} \right.} \right)<P\left( {{R_1}\left| {{E_1}} \right.} \right)$

$$ \frac{{{m^2}\frac{1}{h}+m\left( {1-m} \right)}}{{{m^2}\frac{1}{h}+2m\left( {1-m} \right)+{{{\left( {1-m} \right)}}^2}}}<m $$

$$ \Leftrightarrow m\frac{1}{h}+\left( {1-m} \right)<{m^2}\frac{1}{h}+2m\left( {1-m} \right)+{{\left( {1-m} \right)}^2} $$

$$ \Leftrightarrow \frac{1}{h}-1<m\left( {\frac{1}{h}-1} \right) $$

But, since $1>m>0$ and $1>h>0$, this cannot hold. Hence $P\left( {{R_1}\left| {{E_1},{E_2}} \right.} \right)\geq P\left( {{R_1}\left| {{E_1}} \right.} \right)$.