The Optimal Choice of Hypothesis Is the Weakest, Not the Shortest

Abstract

If A and B are sets such that $A \subset B$, generalisation may be understood as the inference from A of a hypothesis sufficient to construct B. One might infer any number of hypotheses from A, yet only some of those may generalise to B. How can one know which are likely to generalise? One strategy is to choose the shortest, equating the ability to compress information with the ability to generalise (a “proxy for intelligence”). We examine this in the context of a mathematical formalism of enactive cognition. We show that compression is neither necessary nor sufficient to maximise performance (measured in terms of the probability of a hypothesis generalising). We formulate a proxy unrelated to length or simplicity, called weakness. We show that if tasks are uniformly distributed, then there is no choice of proxy that performs at least as well as weakness maximisation in all tasks while performing strictly better in at least one. In experiments comparing maximum weakness and minimum description length in the context of binary arithmetic, the former generalised at between 1.1 and 5 times the rate of the latter. We argue this demonstrates that weakness is a far better proxy, and explains why Deepmind’s Apperception Engine is able to generalise effectively.

1 Introduction

If A and B are sets such that $A \subset B$, generalisation may be understood as the inference from A of a hypothesis sufficient to construct B. One might infer any number of hypotheses from A, yet only some of those may generalise to B. How can one know which are likely to generalise? According to Ockham’s Razor, the simpler of two explanations is the more likely [2]. Simplicity is not itself a measurable property, so the minimum description length principle [3] relates simplicity to length. Shorter representations are considered to be simpler, and tend to generalise more effectively. This is often applied in the context of induction by comparing the length of programs that explain what is observed (to chose the shortest, all else being equal). The ability to identify shorter representations is compression, and the ability to generalise is arguably intelligence [4]. Hence the ability to compress information is often portrayed as a proxy for intelligence [5], even serving as the foundation [6,7,8] of the theoretical super-intelligence AIXI [9]. That compression is a good proxy seems to have gone unchallenged. The optimal choice of hypothesis is widely considered to be the shortest. We show that it is not¹. We present an alternative, unrelated to description length, called weakness. We prove that to maximise the probability that one’s hypotheses generalise, it is necessary and sufficient to infer the weakest valid hypotheses possible².

2 Background Definitions

To do so, we employ a formalism of enactive cognition [1, 10, 11], in which sets of declarative programs are related to one another in such a way as to form a lattice. This unusual representation is necessary to ensure that both the weakness and description length of a hypothesis are well defined³. This formalism can be understood in three steps.

• 1. The environment is represented as a set of declarative programs.

• 2. A finite subset of the environment is used to define a language with which to write statements that behave as logical formulae.

• 3. Finally, induction is formalised in terms of tasks made up of these statements.

Definition 1

(environment)

• We assume a set $\varPhi$ whose elements we call states, one of which we single out as the present state⁴.

• A declarative program is a function $f : \varPhi \rightarrow \{true, false\}$, and we write P for the set of all declarative programs. By an objective truth about a state $\phi$, we mean a declarative program f such that $f(\phi ) = true$.

Definition 2

(implementable language)

• $\mathfrak {V} = \{V \subset P : V \ is \ finite\}$ is a set whose elements we call vocabularies, one of which we single out as the vocabulary $\mathfrak {v}$ for an implementable language.

• ${L_\mathfrak {v}} = \{ l \subseteq \mathfrak {v} : \exists \phi \in \varPhi \ (\forall p \in l : p(\phi ) = true) \}$ is a set whose elements we call statements⁵. $L_\mathfrak {v}$ follows from $\varPhi$ and $\mathfrak {v}$. We call $L_\mathfrak {v}$ an implementable language.

• $l \in {L_\mathfrak {v}}$ is true iff the present state is $\phi$ and $\forall p \in l : p(\phi ) = true$.

• The extension of a statement $a \in {L_\mathfrak {v}}$ is $Z_a = \{b \in {L_\mathfrak {v}} : a \subseteq b\}$.

• The extension of a set of statements $A \subseteq {L_\mathfrak {v}}$ is $Z_A = \bigcup \limits _{a \in A} Z_a$.

(Notation). Z with a subscript is the extension of the subscript⁶. Lower case letters represent statements, and upper case represent sets of statements.

Definition 3

( $\mathfrak {v}$ -task). For a chosen $\mathfrak {v}$, a task $\alpha$ is $\langle {S}_\alpha , {D}_\alpha , {M}_\alpha \rangle$ where:

• ${S}_\alpha \subset L_\mathfrak {v}$ is a set whose elements we call situations of $\alpha$.

• ${S_\alpha }$ has the extension $Z_{S_\alpha }$, whose elements we call decisions of $\alpha$.

• ${D_\alpha } = \{z \in Z_{S_\alpha } : z \ is \ correct \}$ is the set of all decisions which complete $\alpha$.

• ${M_\alpha } = \{l \in L_\mathfrak {v} : {Z}_{S_\alpha } \cap Z_{l} = {D_\alpha }\}$ whose elements we call models of $\alpha$.

$\varGamma _\mathfrak {v}$ is the set of all tasks⁷.

(Notation). If $\omega \in \varGamma _\mathfrak {v}$, then we will use subscript $\omega$ to signify parts of $\omega$, meaning one should assume $\omega = \langle {S}_\omega , {D}_\omega , {M}_\omega \rangle$ even if that isn’t written.

(How a task is completed). Assume we’ve a $\mathfrak {v}$-task $\omega$ and a hypothesis ${\textbf {h}} \in L_\mathfrak {v}$ s.t.

• 1. we are presented with a situation ${s} \in {S}_\omega$, and

• 2. we must select a decision $z \in Z_{s} \cap Z_{\textbf {h}}$.

• 3. If $z \in {D}_\omega$, then z is correct and the task is complete. This occurs if ${\textbf {h}} \in {M}_\omega$.

3 Formalising Induction

Definition 4

(probability). We assume a uniform distribution over $\varGamma _\mathfrak {v}$.

Definition 5

(generalisation). A statement l generalises to $\alpha \in \varGamma _\mathfrak {v}$ iff $l \in M_\alpha$. We say l generalises from $\alpha$ to $\mathfrak {v}$-task $\omega$ if we first obtain l from ${M}_\alpha$ and then find it generalises to $\omega$.

Definition 6

(child and parent). A $\mathfrak {v}$-task $\alpha$ is a child of $\mathfrak {v}$-task $\omega$ if ${S}_\alpha \subset {S}_\omega$ and ${D}_\alpha \subseteq {D}_\omega$. This is written as $\alpha \sqsubset \omega$. If $\alpha \sqsubset \omega$ then $\omega$ is then a parent of $\alpha$.

A proxy is meant to estimate one thing by measuring another. In this case, if intelligence is the ability to generalise [4, 10], then a greater proxy value is meant to indicate that a statement is more likely to generalise. Not all proxies are effective (most will be useless). We focus on two in particular.

Definition 7

(proxy for intelligence). A proxy is a function parameterized by a choice of $\mathfrak {v}$ such that $q_\mathfrak {v} : L_\mathfrak {v} \rightarrow \mathbb {N}$. The set of all proxies is Q.

(Weakness). The weakness of a statement l is the cardinality of its extension $|Z_{l} |$. There exists $q_\mathfrak {v} \in Q$ such that $q_\mathfrak {v}(l) = |Z_{l} |$.

(Description length). The description length of a statement l is its cardinality $|{l} |$. Longer logical formulae are considered less likely to generalise [3], and a proxy is something to be maximised, so description length as a proxy is $q_\mathfrak {v} \in Q$ such that $q_\mathfrak {v}(l) = \frac{1}{|l |}$.

A child task may serve as an ostensive definition [14] of its parent, meaning one can generalise from child to parent.

Definition 8

(induction). $\alpha$ and $\omega$ are $\mathfrak {v}$-tasks such that $\alpha \sqsubset \omega$. Assume we are given a proxy $q_\mathfrak {v} \in Q$, the complete definition of $\alpha$ and the knowledge that $\alpha \sqsubset \omega$. We are not given the definition of $\omega$. The process of induction would proceed as follows:

• 1. Obtain a hypothesis by computing a model $\textbf{h} \in \underset{{m} \in {M}_\alpha }{\arg \max } \ q_\mathfrak {v}(m)$.

• 2. If $\textbf{h} \in {M}_\omega$, then we have generalised from $\alpha$ to $\omega$.

4 Proofs

Proposition 1

(sufficiency). Weakness is a proxy sufficient to maximise the probability that induction generalises from $\alpha$ to $\omega$.

Proof:

You’re given the definition of $\mathfrak {v}$-task $\alpha$ from which you infer a hypothesis $\textbf{h} \in {M}_\alpha$. $\mathfrak {v}$-task $\omega$ is a parent of $\alpha$ to which we wish to generalise:

• 1. The set of statements which might be decisions addressing situations in ${S}_\omega$ and not ${S}_\alpha$, is $\overline{Z_{{S}_\alpha }} = \{ l \in L_\mathfrak {v} : l \notin Z_{{S}_\alpha } \}$.

• 2. For any given $\textbf{h} \in {M}_\alpha$, the extension $Z_\textbf{h}$ of $\textbf{h}$ is the set of decisions $\textbf{h}$ implies. The subset of $Z_\textbf{h}$ which fall outside the scope of what is required for the known task $\alpha$ is $\overline{Z_{{S}_\alpha }} \cap Z_{\textbf{h}}$ (because $Z_{{S}_\alpha }$ is the set of all decisions we might make when attempting $\alpha$, and so the set of all decisions that can’t be made when undertaking $\alpha$ is $\overline{Z_{{S}_\alpha }}$ because those decisions occur in situations that aren’t part of ${S}_\alpha$).

• 3. $|\overline{Z_{{S}_\alpha }} \cap Z_{\textbf{h}} |$ increases monotonically with $|Z_\textbf{h} |$, because $\forall z \in Z_m : z \notin \overline{Z_{{S}_\alpha } } \rightarrow z \in Z_{{S}_\alpha }$.

• 4. $2^{|\overline{Z_{{S}_\alpha }} |}$ is the number of tasks which fall outside of what it is necessary for a model of $\alpha$ to generalise to (this is just the powerset of $\overline{Z_{{S}_\alpha }}$ defined in step 2), and $2^{|\overline{Z_{{S}_\alpha }} \cap Z_{\textbf{h}} |}$ is the number of those tasks to which a given $\textbf{h} \in M_\alpha$ does generalise.

• 5. Therefore the probability that a given model $\textbf{h} \in {M}_\alpha$ generalises to the unknown parent task $\omega$ is

  $$p(\textbf{h} \in {M}_\omega \mid \textbf{h} \in {M}_\alpha , \alpha \sqsubset \omega ) = \frac{2^{|\overline{Z_{{S}_\alpha }} \cap Z_{\textbf{h}} |}}{2^{|\overline{Z_{{S}_\alpha }} |}}$$

$p(\textbf{h} \in {M}_\omega \mid \textbf{h} \in {M}_\alpha , \alpha \sqsubset \omega )$ is maximised when $|Z_\textbf{h} |$ is maximised.

Proposition 2

(necessity).To maximise the probability that induction generalises from $\alpha$ to $\omega$, it is necessary to use weakness as a proxy, or a function thereof⁸.

Proof:

Let $\alpha$ and $\omega$ be defined exactly as they were in the proof of Proposition 1.

• 1. If $\textbf{h} \in {M}_\alpha$ and $Z_{{S}_\omega } \cap Z_{\textbf{h}} = {D}_\omega$, then it must be he case that ${D}_\omega \subseteq Z_{\textbf{h}}$.

• 2. If $|Z_{\textbf{h}} |< |{D}_\omega |$ then generalisation cannot occur, because that would mean that ${D}_\omega \not \subseteq Z_{\textbf{h}}$.

• 3. Therefore generalisation is only possible if $|Z_{{m}} |\ge |{D}_\omega |$, meaning a sufficiently weak hypothesis is necessary to generalise from child to parent.

• 4. The probability that $|Z_{{m}} |\ge |{D}_\omega |$ is maximised when $|Z_{{m}} |$ is maximised. Therefore to maximise the probability induction results in generalisation, it is necessary to select the weakest hypothesis.

To select the weakest hypothesis, it is necessary to use weakness (or a function thereof) as a proxy.

Remark 1

(prior). The above describes inference from a child to a parent. However, it follows that increasing the weakness of a statement increases the probability that it will generalise to any task (not just a parent of some given child). As tasks are uniformly distributed, every statement in $L_\mathfrak {v}$ is a model to one or more tasks, and the number of tasks to which each statement $l \in L_\mathfrak {v}$ generalises is $2^{|Z_l|}$. Hence the probability of generalisation⁹ to $\omega$ is $p(\textbf{h} \in M_\omega \mid \textbf{h} \in {L}_\mathfrak {v}) = \frac{2^{|Z_\textbf{h} |}}{2^{|L_\mathfrak {v} |}}$. This assigns a probability to every statement $l \in L_\mathfrak {v}$ given an implementable language. It is a probability distribution in the sense that the probability of mutually exclusive statements sums to one¹⁰. This prior may be considered universal in the very limited sense that it assigns a probability to every conceivable hypothesis (where what is conceivable depends upon the implementable language) absent any parameters or specific assumptions about the task as with AIXI’s intelligence order relation [9, def. 5.14 pp. 147]¹¹. As the vocabulary $\mathfrak {v}$ is finite, $L_\mathfrak {v}$ must also be finite, and so p is computable.

We have shown that, if tasks are uniformly distributed, then weakness is a necessary and sufficient proxy to maximise the probability that induction generalises. It is important to note that another proxy may perform better given cherry-picked combinations of child and parent task for which that proxy is suitable. However, such a proxy would necessarily perform worse given the uniform distribution of all tasks. Can the same be said of description length?

Proposition 3

Description length is neither a necessary nor sufficient proxy for the purposes of maximising the probability that induction generalises.

Proof:

In Propositions 1 and 2 we proved that weakness is a necessary and sufficient choice of proxy to maximise the probability of generalisation. It follows that either maximising $\frac{1}{|m |}$ (minimising description length) maximises $|Z_{m} |$ (weakness), or minimisation of description length is unnecessary to maximise the probability of generalisation. Assume the former, and we’ll construct a counterexample with $\mathfrak {v} = \{a,b,c,d,e,f,g,h,j,k,z \}$ s.t. $L_\mathfrak {v} = \{ \{a,b,c,d,j,k,z\}, \{e,b,c,d,k\}$, $\{a,f,c,d,j\}, \{e,b,g,d,j,k,z\}, \{a,f,c,h,j,k\}, \{e,f,g,h,j,k\} \}$ and a task $\alpha$ where

• ${S}_\alpha = \{ \{a,b\}, \{e,b\} \}$

• ${D}_\alpha = \{ \{a,b,c,d,j,k,z\}, \{e,b,g,d,j,k,z\} \}$

• ${M}_\alpha = \{\{z\}, \{j, k \} \}$

Weakness as a proxy selects $\{j,k\}$, while description length as a proxy selects $\{z\}$. This demonstrates the minimising description length does not necessarily maximise weakness, and maximising weakness does not minimise description length. As weakness is necessary and sufficient to maximise the probability of generalisation, it follows that minimising description length is neither.

5 Experiments

Included with this paper is a Python script to perform two experiments using PyTorch with CUDA, SymPy and $A^*$ [15,16,17,18] (see technical appendix for details). In these two experiments, a toy program computes models to 8-bit string prediction tasks (binary addition and multiplication). The purpose of these experiments was to compare weakness and description length as proxies.

5.1 Setup

To specify tasks with which the experiments would be conducted, we needed a vocabulary $\mathfrak {v}$ with which to describe simple 8-bit string prediction problems. There were 256 states in $\varPhi$, one for every possible 8-bit string. The possible statements were then all the expressions regarding those 8 bits that could be written in propositional logic (the simple connectives $\lnot$, $\wedge$ and $\vee$ needed to perform binary arithmetic – a written example of how propositional logic can be used in to specify $\mathfrak {v}$ is also included in the appendix). In other words, for each statement in $L_\mathfrak {v}$ there existed an equivalent expression in propositional logic. For efficiency, these statements were implemented as either PyTorch tensors or SymPy expressions in different parts of the program, and converted back and forth as needed (basic set and logical operations on these propositional tensor representations were implemented for the same reason). A $\mathfrak {v}$-task was specified by choosing ${D}_n \subset L_\mathfrak {v}$ such that all ${d} \in {D}_n$ conformed to the rules of either binary addition or multiplication with 4-bits of input, followed by 4-bits of output.

5.2 Trials

Each experiment had parameters were “operation” and “number_of_trials”. For each trial the number $|{D}_k |$ of examples ranged from 4 to 14. A trial had 2 phases.

Training Phase

• 1. A task n (referred to in code as ${T}_n$) was generated:

  • (a) First, every possible 4-bit input for the chosen binary operation was used to generate an 8-bit string. These 16 strings then formed ${D}_n$.

  • (b) A bit between 0 and 7 was then chosen, and ${S}_n$ created by cloning ${D}_n$ and deleting the chosen bit from every string (${S}_n$ contained 16 different 7-bit strings, each of which was a sub-string of an element of ${D}_n$).

• 2. A child-task $k = \langle {S}_k, {D}_k, {M}_k \rangle$ (referred to in code as ${T}_k$) was sampled (assuming a uniform distribution over children) from the parent task ${T}_n$. Recall, $|{D}_k |$ was determined as a parameter of the trial.

• 3. From ${T}_k$ two models were then generated; a weakest $c_w$, and a MDL $c_{mdl}$.

Testing Phase: For each model $c \in \{c_w, c_{mdl}\}$, the testing phase was as follows:

• 1. The extension $Z_c$ of c was then generated.

• 2. A prediction ${D}_{recon}$ was made s.t. ${D}_{recon} = \{z \in Z_c : \exists {s} \in {S}_n \ ({s} \subset z ) \}$.

• 3. ${D}_{recon}$ was then compared to the ground truth ${D}_n$, and results recorded.

Between 75 and 256 trials were run for each value of the parameter $|{D}_k |$. Fewer trials were run for larger values of $|{D}_k |$ as these took longer to process. The results of these trails were then averaged for each value of $|{D}_k |$.

5.3 Results

Two sorts of measurements were taken for each trial. The first was the rate at generalisation occurred. Generalisation was deemed to have occurred where ${D}_{recon} = {D}_n$. The number of trials in which generalisation occurred was measured, and divided by n to obtain the rate of generalisation for $c_w$ and $c_{mdl}$. Error was computed as a Wald 95$\%$ confidence interval. The second measurement was the average extent to which models generalised. Even where ${D}_{recon} \ne {D}_n$, the extent to which models generalised could be ascertained. $\frac{|{D}_{recon} \cap {D}_n|}{|{D}_n|}$ was measured and averaged for each value of $|{D}_k|$, and the standard error computed. The results (see Tables 1 and 2) demonstrate that weakness is a better proxy for intelligence than description length. The generalisation rate for $c_w$ was between 110–500% of $c_{mdl}$, and the extent was between $103-156\%$.

Table 1. Results for Binary Addition

Table 2. Results for Binary Multiplication

6 Concluding Remarks

We have shown that, if tasks are uniformly distributed, then weakness maximisation is necessary and sufficient to maximise the probability that induction will produce a hypothesis that generalises. It follows that there is no choice of proxy that performs at least as well as weakness maximisation across all possible combinations of child and parent task while performing strictly better in at least one. We’ve also shown that the minimisation of description length is neither necessary nor sufficient. This calls into question the relationship between compression and intelligence [5, 19, 20], at least in the context of enactive cognition. This is supported by our experimental results, which demonstrate that weakness is a far better predictor of whether a hypothesis will generalise, than description length. Weakness should not be conflated with Ockham’s Razor. A simple statement need not be weak, for example “all things are blue crabs”. Likewise, a complex utterance can assert nothing. Weakness is a consequence of extension, not form. If weakness is to be understood as an epistemological razor, it is this (which we humbly suggest naming “Bennett’s Razor”):

Explanations should be no more specific than necessary.¹²

The Apperception Engine: The Apperception Engine [21,22,23] (Evans et al. of Deepmind) is an inference engine that generates hypotheses that generalise often. To achieve this, Evans formalised Kant’s philosophy to give the engine a “strong inductive bias”. The engine forms hypotheses from only very general assertions, meaning logical formulae which are universally quantified. That is possible because the engine uses language specifically tailored to efficiently represent the sort of sequences to which it is applied. Our results suggest a simpler and more general explanation of why the engine’s hypotheses generalise so well. The tailoring of logical formulae to represent certain sequences amounts to a choice of $\mathfrak {v}$, and the use of only universally quantified logical formulae maximises the weakness of the resulting hypothesis. To apply this approach to induction from child $\mathfrak {v}$-task $\alpha$ to parent $\omega$ would mean we only entertain a model ${m} \in {M}_\alpha$ if $p({m} \in {M}_\omega \mid {m} \in {M}_\alpha ) = 1$. Obviously this can work well, but only for the subset of possible tasks that the vocabulary is able to describe in this way (anything else will not be able to be represented as a universally quantified rule, and so will not be represented at all [24]). This illustrates how future research may explore choices of $\mathfrak {v}$ in aid of more efficient induction in particular sorts of task, such as the inference of linguistic meaning and intent (see appendix).

Neural Networks: How might a task be represented in the context of conventional machine learning? Though we use continuous real values in base 10 to formalise neural networks, all computation still takes place in a discrete, finite and binary system. A finite number of imperative programs composed a finite number of times may be represented by a finite set of declarative programs. Likewise, activations within a network given an input can be represented as a finite set of declarative programs, expressing a decision. The choice of architecture specifies the vocabulary in which this is written, determining what sort of relations can be described according to the Chomsky Hierarchy [25]. The reason why LLMs are so prone to fabrication and inconsistency may be because they are optimised only to minimise loss, rather than maximise weakness [10]. Perhaps grokking [26] can be induced by optimising for weakness. Future research should investigate means by which weakness can be maximised in the context of neural networks.