On the Grammatical Complexity of Finite Languages

Abstract

We study the grammatical production complexity of finite languages w.r.t. (i) different interpretations of approximation, i.e., equivalence, cover, and scattered cover, and (ii) whether the underlying grammar generates a finite or infinite language. In case the generated language is infinite, the intersection with all words up to a certain length has to be considered in order to obtain the finite language under consideration. In this way, we obtain six different measures for regular, linear context-free, and context-free grammars. We compare these measures according to the taxonomy introduced in [J. Dassow, Gh. Păun: Regulated Rewriting in Formal Language Theory, 1989] with each other by fixing the grammar type and varying the complexity measure and the other way around, that is, by fixing the complexity measure and varying the grammar type. In both of these cases, we develop an almost complete picture, which gives new and interesting insights into the old topic of grammatical production complexity.

This research was completed while the author was on leave at the Institut für Informatik, Universität Giessen, Arndtstr. 2, 35392 Giessen, Germany, in 2017 and is supported by the Vienna Science Fund (WWTF) project VRG12-004.

1 Introduction

Measures of descriptional complexity or cost functions have a long and fruitful history. Most approaches to defining descriptional complexity measures are based on quantifying the ability of a device—automaton or grammar—to approximate languages (by finite subsets). The interesting quantities in the case of automata and grammars are, e.g., the number of states or transitions and the number of nonterminals or productions, respectively. For instance, finite languages can be represented by ordinary deterministic finite automata ($\mathsf {DFA}$s) or by cover finite automata ($\mathsf {CFA}$s)—roughly speaking a $\mathsf {CFA}$ is a $\mathsf {DFA}$ A and a natural number $\ell$ whose accepted language is defined as $L(A)\cap \varSigma ^{\le \ell }$. For a precise definition of $\mathsf {CFA}$s, we refer the reader to [5]. This gives rise to two different complexity measures: for a finite language $L\subseteq \varSigma ^{\le \ell }$, one defines the $\mathsf {DFA}$ state and the $\mathsf {CFA}$ state complexity as¹

$$\begin{aligned} \mathsf {DFAc}(L)&=\min \{\,|A|\mid \,\, A\,\, \text{ is } \text{ a }\,\, \mathsf {DFA}\ \,\,\text {and}\,\, L=L(A)\,\} \\ \nonumber \text {and} \\ \nonumber \mathsf {DFAc}_\infty (L)&=\min \{\,|A|\mid \,\,A \,\,\text {is a}\,\, \mathsf {DFA}\ \text {and}\,\, L=L(A)\cap \varSigma ^{\le \ell }\,\}, \end{aligned}$$

(1)

respectively, where |A| refers to the number of states of the automaton A. Obviously, $\mathsf {DFAc}_\infty (L)\le \mathsf {DFAc}(L)$, for every finite language L. Note that equality is possible for certain finite languages. It is worth mentioning that although these measures look very similar in their definitions, they can differ quite immensely when applied to the same language. For instance, there is a finite language L such that $\mathsf {DFAc}_\infty (L)=1$, but $\mathsf {DFAc}(L)=n$, for every $n\ge 1$. Hence, the gap between both measures can be arbitrarily large. In general, complexity gaps can be classified according to the different growth rates of the complexity measures. To this end, a notion was introduced in [6], which defines three categories $\le ^1$, $\le ^2$, and $\le ^3$ of increasing complexity gaps; a precise definition is given later. The aforementioned complexity gap of arbitrary size between $\mathsf {DFA}$s and $\mathsf {CFA}$s is a gap of highest type, which is simply written as $\mathsf {c}_\infty \le ^3_{\mathsf {DFA}}\mathsf {c}$. This is one example of a complexity measure, but one can find legions of other automata-based descriptional complexity measures in the literature.

In this paper, we study the grammatical production complexity of finite languages. This topic is not new, and already in [2,3,4], measures similar in definition to (1), for regular ($\mathsf {REG}$), linear context-free ($\mathsf {LIN}$), and context-free grammars ($\mathsf {CFG}$) have been investigated; they are named $\mathsf {Xc}$, for $X\in \{\mathsf {REG},\mathsf {LIN},\mathsf {CF}\}$. For instance, there it has been shown that there are incompressible finite languages for each grammar type mentioned above, i.e., languages that need at least as many productions of a certain type as there are words in that language. To the best of our knowledge, a classification of these grammatical measures in the sense of [6] has not been done yet. We close this gap and, moreover, also consider natural variants of $\mathsf {Xc}$ by varying the equivalence condition $L=L(A)$ in (1) to $L\subseteq L(A)$ and L(A) finite (cover) or even $L\le L(A)$ and L(A) finite (scattered cover), where $\le$ refers to the scattered subword relation—similarly this can be done for $L=L(A)\cap \varSigma ^{\le \ell }$ in $\mathsf {Xc}_\infty$, too. This leads to the additional grammatical measures (i) $\mathsf {Xcc}$ and $\mathsf {Xcc}_\infty$ (cover) and (ii) $\mathsf {Xsc}$ and $\mathsf {Xsc}_\infty$ (scattered cover), for $X\in \{\mathsf {REG},\mathsf {LIN},\mathsf {CF}\}$. The variation $\mathsf {Xcc}$ is inspired by recent results on proof complexity in first-order logic, a research topic, which, from a first glance, seems completely unrelated to grammatical complexity, that connects the number of certain inference rules used in a specific logical calculus with the number of productions needed to cover a certain finite language [7]. For further results on the cover complexity of finite languages, see also [8]. We compare these measures according to the taxonomy introduced in [6] with each other by (i) fixing the grammar type and varying the complexity measure and (ii) by fixing the complexity measure and varying the grammar type. In both of these cases, we develop an almost complete picture. As a byproduct, we also show that there are finite languages with large complexity. More precisely, the language of even length palindromes $P_n=\{\,w\$w^R\mid w\in \{a,b\}^{\le n}\,\}$ requires at least $\varOmega (2^n)$ productions to be generated by a regular grammar. Moreover, the triple language $T_n=\{\,w\$w\#w\mid w\in \{a,b\}^n\,\}$ can only be generated by grammars of type X, for $X\in \{\mathsf {REG},\mathsf {LIN},\mathsf {CF}\}$, by simply listing all words in $T_n$, i.e., $\mathsf {Xc}(T_n)=\varOmega (2^n)$.

2 Preliminaries

We assume that the reader is familiar with the basic notions of formal language theory as contained in [10]. Nevertheless, to fix notation and terminology, we introduce the basic notions and results relevant to this paper in this section.

Let $\varSigma$ be a finite alphabet. Then $\varSigma ^*$ denotes the set of all words over the finite alphabet $\varSigma$ including the empty word $\varepsilon$ and we write $\varSigma ^+$ for $\varSigma ^* \setminus \{ \varepsilon \}$. The length of a word w in $\varSigma ^*$ is denoted by |w|. In particular, the length of the empty word $\varepsilon$ is 0, i.e., $|\varepsilon |=0$. The reversal of a word is defined as follows: $\varepsilon ^R = \varepsilon$ and $(wa)^R = aw^R$, for $w \in \varSigma ^*$ and $a \in \varSigma \cup \{ \varepsilon \}$. Let $\ell \ge 0$. Then $\varSigma ^\ell$ and $\varSigma ^{\le \ell }$ refers to the set of all words over $\varSigma$ of length exactly $\ell$ and at most $\ell$, respectively. A subset L of $\varSigma ^*$ is called a language. Any language $L \subseteq \varSigma ^{\le \ell }$, for $\ell \ge 0$, is called finite and, unless stated otherwise, we always assume $\ell = \max \{\, |w| \mid w \in L \,\}$. If L is a subset of $\varSigma ^\ell$, for $\ell \ge 0$, then L is called a uniform language. This means that in a uniform language all words have the same length.

A context-free grammar ($\mathsf {CFG}$) is a quadruple $G=(N,\varSigma ,P,S)$, where N and $\varSigma$ are disjoint alphabets of nonterminals and terminals, respectively, $S\in N$ is the start symbol, and P is a finite set of productions of the form $A\rightarrow \alpha$, where $A\in N$ and $\alpha \in (N\cup \varSigma )^*$. As usual, the derivation relation of G is denoted by $\Rightarrow _{G}$ and the reflexive and transitive closure of $\Rightarrow _{G}$ is written as $\Rightarrow _{G}^*$. The language generated by G is defined as $L(G)=\{\,w\in \varSigma ^*\mid S\Rightarrow _{G}^* w\,\}$. We also consider the following restrictions of context-free grammars: (i) a context-free grammar is said to be linear context-free ($\mathsf {LIN}$) if the productions are of the form $A\rightarrow \alpha$, where $A\in N$ and $\alpha \in \varSigma ^*(N\cup \{\varepsilon \})\varSigma ^*$, and (ii) a context-free grammar is said to be right-linear or regular ($\mathsf {REG}$) if the productions are of the form $A\rightarrow \alpha$, where $A\in N$ and $\alpha \in \varSigma ^*(N\cup \{\varepsilon \})$. Furthermore, $\varGamma$ will denote the set of grammar types in the sequel, that is, $\varGamma =\{\mathsf {REG},\mathsf {LIN},\mathsf {CF}\}$.

Let $G=(N,\varSigma ,P,S)$ be a context-free grammar. By |G|, we denote the number of productions of G, i.e., the cardinality of P. Then the (exact) X-complexity (or X-complexity for short) of a finite language L w.r.t. X-grammars, for $X\in \varGamma$, is defined as

$$\mathsf {Xc}(L) = \min \{\, |G| \mid \,\, G\,\, \text {is an}\,\, X \text {-grammar with}\,\, L = L(G)~ \text {and}~L(G)~ \text {finite}\,\}.$$

The additional condition that L(G) is finite is redundant, but becomes important whenever we replace $L=L(G)$ by $L\subseteq L(G)$ or some other language-relating property. Similarly, the infinite X-complexity of a finite language $L\subseteq \varSigma ^{\le \ell }$ is defined as

$$\mathsf {Xc_{\infty }}(L) = \min \{\, |G| \mid \,\,G\,\, \text{ is } \text{ an }\,\, X\text {-}\text{ grammar } \text{ with }\,\, L = L(G) \cap \varSigma ^{\le \ell } \,\}.$$

Note that in the definition of $\mathsf {Xc}_{\infty }$, the grammar G is allowed to generate an infinite language. If we replace $L = L(G)$ and $L = L(G) \cap \varSigma ^{\le \ell }$ in the definitions of $\mathsf {Xc}(L)$ and $\mathsf {Xc_{\infty }}(L)$, respectively, by $L \subseteq L(G)$ and $L \subseteq L(G) \cap \varSigma ^{\le \ell }$, respectively, then we get the definitions for the X-cover-complexity $\mathsf {Xcc}(L)$ and the infinite X-cover-complexity $\mathsf {Xcc}_{\infty }(L)$, respectively. The scattered subword relation $\le$ is defined as follows: let $w = w_1 u_1 w_2 u_2 \ldots u_{n-1} w_n$ be a word with $w_i, u_j \in \varSigma ^*$, for $1 \le i \le n$ and $1 \le j \le n-1$. Then the word $w' = w_1 w_2 \ldots w_n$ is called a scattered subword of w and we write $w' \le w$ in this case. We extend the relation $\le$ from words to languages $L_1$ and $L_2$ as follows: $L_1 \le L_2$ if for all words $w_1 \in L_1$, there is a word $w_2 \in L_2$ such that $w_1 \le w_2$. If $L_1 \le L_2$ holds, we say that $L_1$ is a scattered sublanguage of $L_2$. We, obtain the definitions for $\mathsf {Xsc}(L)$ and $\mathsf {Xsc}_{\infty }(L)$ if we replace $L = L(G)$ and $L = L(G) \cap \varSigma ^{\le \ell }$ by $L \le L(G)$ and $L \le L(G) \cap \varSigma ^{\le \ell }$, respectively, in the definitions of $\mathsf {Xc}(L)$ and $\mathsf {Xc_{\infty }}(L)$, respectively. This results in the X-scattered-complexity and the infinite X-scattered-complexity, respectively.

Note that the definitions of $\mathsf {Xc}$, $\mathsf {Xcc}$, and $\mathsf {Xsc}$ also have the additional requirement that L(G) is a finite language. In the following, $\mathcal {M}$ will denote the set of measure types, i.e., $\mathcal {M} = \{ \mathsf {c},\mathsf {cc},\mathsf {sc}, \mathsf {c}_\infty , \mathsf {cc}_\infty ,\mathsf {sc}_\infty \}$. By definition, for $\tau \in \mathcal {M}$, the following relations hold:

$$\begin{aligned} \mathsf {CFG}\le _\tau \mathsf {LIN}\le _\tau \mathsf {REG}, \end{aligned}$$

(2)

where, for $X,Y\in \varGamma$, we define $X \le _\tau Y$ if and only if $\mathsf {X}\tau (L) \le \mathsf {Y}\tau (L)$, for all finite languages L. In case that $X \le _{\tau } Y$, we say that X is more succinct than Y (w.r.t. the complexity measure $\tau$).

We say that G is a minimal X-grammar, for $X \in \varGamma$, w.r.t. the measure $\mathsf {X}\tau$ with $\tau \in \mathcal {M}$, if $|G|=\mathsf {X}\tau (L)$. In the case of the measure $\mathsf {Xc}$, we speak of a minimal X-grammar generating a finite language.

Finally, we show that grammars that generate non-trivial uniform languages do not contain $\varepsilon$-productions. To this end, we first need the following result on the length of words generated by some nonterminal from a grammar that describes a finite uniform language.

Lemma 1

Let $X \in \varGamma$ and $G = (N, \varSigma , P, S)$ be a minimal X-grammar generating a finite and uniform language. Then, for all $A \in N$, all words occurring in the set $\{\, w \in \varSigma ^* \mid A \Rightarrow _{G}^* w \,\}$ have the same length.

An easy consequence of the previous lemma is the next theorem.

Theorem 2

Let $X \in \varGamma$ and $G = (N, \varSigma , P, S)$ be a minimal X-grammar generating a finite and uniform language satisfying $L(G) \ne \{ \varepsilon \}$. Then G is $\varepsilon$-free, i.e., P does not contain any rule of the form $A\rightarrow \varepsilon$, for all $A\in N$.

3 Some Bounds on the Various X-Complexities

First, we prove some upper bounds for the finite variants of the introduced grammatical measures. We obtain the following results:

Theorem 3

Let $L\subseteq \varSigma ^{\le \ell }$ be a finite language over the alphabet $\varSigma$. Then, for $X\in \varGamma$ and $Y \in \{ \mathsf {REG},\mathsf {LIN}\}$, we have

• 1. $\mathsf {Xc}(L) \le \ell + 1$ if $|\varSigma | = 1$, and $\mathsf {Xc}(L)\le (|\varSigma |^{\ell +1}-1)/(|\varSigma |-1)$, otherwise,

• 2. $\mathsf {CFcc}(L) \le |\varSigma |+2$ and $\mathsf {Ycc}(L) \le \ell + 1$ if $|\varSigma | = 1$, and $\mathsf {Ycc}(L)\le (|\varSigma |+1)\cdot \ell$, otherwise, and

• 3. $\mathsf {Xsc}(L)=1$ if L is non-empty, and $\mathsf {Xsc}(L)=0$, otherwise.

Proof

We argue as follows:

• 1. Every finite language L can be generated by a grammar of type $X \in \varGamma$ by simply listing all words from L. Since there are at most $\sum _{i=0}^\ell |\varSigma |^\ell$ words of length at most $\ell$ in L, the upper bounds of $\ell + 1$ and $(|\varSigma |^{\ell +1}-1)/(|\varSigma |-1)$ follow for the cases $|\varSigma | = 1$ and $|\varSigma | \ge 2$, respectively.

• 2. Consider the context-free grammar $G' = (\{ A,S' \},\varSigma ,P',S')$, where $P'$ consists of the productions $S'\rightarrow A^\ell$ and $A\rightarrow a$, for $a\in \varSigma \cup \{\varepsilon \}$. Clearly, we have $L(G') = \varSigma ^{\le \ell }$ and $|G'| = |\varSigma | + 2$, i.e., $\mathsf {CFcc}(\varSigma ^{\le \ell }) \le |\varSigma | + 2$.

  By assumption, $L \subseteq \varSigma ^{\le \ell }$. Thus, every grammar generating $\varSigma ^{\le \ell }$ automatically covers the language L. For $|\varSigma | \ge 2$, consider the regular grammar $G = (N,\varSigma ,P,S)$, where $N = \{ A_1,A_2, \ldots , A_\ell \}$ with $S = A_1$ and

  $$\begin{aligned} P&= \{\, A_i \rightarrow a A_{i+1} \mid a \in \varSigma \text { and } 1 \le i \le \ell -1 \,\} \cup \{\, A_{\ell } \rightarrow a \mid a \in \varSigma \,\}\\&\qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \quad \qquad {}\cup \{\, A_i \rightarrow \varepsilon \mid 1 \le i \le \ell \,\}. \end{aligned}$$

  Obviously, $L(G) = \varSigma ^{\le \ell }$ and $|G| = (|\varSigma |+1) \cdot \ell$. In the case that $|\varSigma | = 1$, we simply list all $\ell +1$ words occurring in $\varSigma ^{\le \ell }$.

• 3. Assume that $\varSigma = \{ a_1, a_2, \ldots , a_n \}$ and consider the language $\{ (a_1a_2\ldots a_n)^\ell \}$, which is generated by $G = (\{ S \}, \varSigma , \{ S \rightarrow (a_1a_2\ldots a_n)^\ell \}, S)$, a regular grammar with a single production rule. Clearly, we have $L \le \{ (a_1a_2\ldots a_n)^\ell \}$, for all nonempty languages $L \subseteq \varSigma ^{\le \ell }$. Thus, $\mathsf {Xsc}(L) \le 1$. Since any grammar with empty production set can only generate the empty language, we also have $\mathsf {Xsc}(L) \ge 1$. In case $L=\emptyset$, we obviously have $\mathsf {Xsc}(L)=0$.   $\square$

What about lower bounds for these measures? Observe that $\mathsf {Xsc}(L)=1$ is already a lower bound result. In the seminal paper [4] on concise description of finite languages by different types of grammars, certain languages have been identified that require at least a polynomial number of productions. The proofs of these results are based on [4, Lemma 2.1] which states some easy facts about minimal context-free grammars:

Lemma 4

Let $G=(N,\varSigma ,P,S)$ be a minimal context-free grammar for the finite language L. Then, for every nonterminal $A\in N\setminus \{S\}$, there are strings $\alpha _1$ and $\alpha _2$ with $\alpha _1,\alpha _2 \in (N \cup \varSigma )^*$ and $\alpha _1\ne \alpha _2$ such that $A\rightarrow \alpha _1$ and $A\rightarrow \alpha _2$ are in P. Moreover, for every $A\in N\setminus \{S\}$, the set $L_A(G)=\{\,w\in \varSigma ^*\mid A\Rightarrow _G^* w\,\}$ contains at least two words and there is no derivation of the form $A\Rightarrow _G^+ \alpha A\beta$ with $\alpha ,\beta \in (N\cup \varSigma )^*$. Finally, for every $A\in N\setminus \{S\}$, there are $u_1,u_2,v_1,v_2\in \varSigma ^*$ such that $u_1Au_2\ne v_1Av_2$ as well as $S\Rightarrow _G^* u_1Au_2$ and $S\Rightarrow _G^* v_1Av_2$.

Our first candidate with a large X-complexity is the language of all even palindromes (with middle marker) $P_n = \{\, w\$w^R \mid w \in \{ a,b \}^{\le n} \,\}$. We show that any regular grammar generating this language needs at least an exponential number of productions.

Theorem 5

Let $n \ge 1$. Then $\mathsf {REGc}(P_n) \ge 2^n$.

Proof

In the proof, we will use the following result from [4, Lemma 2.2]: let $G = (N,\varSigma ,P,S)$ be a context-free grammar generating a finite language. Then there is a context-free grammar $G_{\max } = (N_{\max },\varSigma ,P_{\max },S)$ such that $N_{\max } \subseteq N$, $P_{\max } \subseteq P$, and $L(G_{\max }) = L_{\max }$, where $L_{\max }$ is the subset of L(G) consisting of the words of maximal length. In light of this result, it suffices to show that $\mathsf {REGc}(P'_n) \ge 2^n$, for the language $P_n' = \{\, w\$w^R \mid w \in \{a,b\}^n \,\}$. To this end, assume that $\varSigma = \{a,b,\$\}$ and $G = (N,\varSigma ,P,S)$ is a minimal regular grammar generating $P_n'$ that contains a nonterminal $A \in N\setminus \{S\}$. By Lemma 4, there are derivations $S \Rightarrow _{G}^* u_1 A$ and $S \Rightarrow _{G}^* u_2 A$ with $u_1,u_2 \in \varSigma ^*$ and $u_1 \ne u_2$ as well as $v_1,v_2 \in \varSigma ^*$ with $A \Rightarrow _{G}^* v_1$, $A \Rightarrow _{G}^* v_2$, and $v_1 \ne v_2$. Note that we must have both $|u_1| = |u_2|$ and $|v_1| = |v_2|$, for otherwise we would be able to derive words $w_1$ and $w_2$ with $|w_1| \ne |w_2|$, but $P_n'$ only contains words of the same length. Since $v_1 \ne v_2$ and $|v_1| = |v_2|$, it follows that both $v_1 \ne \varepsilon$ and $v_2 \ne \varepsilon$. Let $w \in \{a,b\}^n$ be arbitrary. We distinguish the following cases:

• 1. Suppose $u_1 \in \{w\$\}\{a,b\}^*$. Then we must have $u_1 = w_1w_2 \$ w_2^R$, where $w = w_1w_2$ and both $v_1 \in \{a,b\}^*$ and $v_2 \in \{a,b\}^*$ holds. Assume, w.l.o.g., that $v_1 = w_1^R$. Since $v_1 \ne v_2$, it follows that $v_2^R \ne w_1$. Thus, $u_1v_2 \not \in P_n'$. Contradiction.

• 2. Suppose $u_1 \in \{a,b\}^*$. Then we must have $v_1,v_2 \in \{a,b\}^*\{\$w^R\}$. Assume $w = u_1w_2$, for some $w_2 \in \{a,b\}^*$ and, w.l.o.g., $v_1 = w_2 \$ w_2^R u_1^R$. Since $v_1 \ne v_2$, it follows that $v_2 = w_2' \$ w_2^Ru_1^R$ with $w_2' \ne w_2$. Thus, $u_1 v_2 \not \in P_n'$. Contradiction.

Consequently, we have $N = \{ S \}$ and so the only way to generate the language $P_n'$ minimally with a regular grammar is to list all of its words using S.   $\square$

For linear context-free and context-free grammars, one observes that both measures $\mathsf {LINc}(P_n)$ and $\mathsf {CFc}(P_n)$ are at most linear, as witnessed by the linear context-free grammar $G=(N,\varSigma ,P,S)$ with $N=\{S_0,S_1,\ldots S_n\}$, $\varSigma =\{a,b,\$\}$, start symbol $S=S_0$, and the productions

$$P=\{\,S_i\rightarrow aS_{i+1}a, S_i\rightarrow bS_{i+1}b,S_i\rightarrow S_{i+1}\mid 0\le i\le n-1\,\}\cup \{S_n\rightarrow \$\},$$

satisfying $L(G)=P_n$, for $n\ge 1$.

Using similar arguments as in the proof of Theorem 5, one can show that the triple language $T_n=\{\,w\$w\#w\mid w\in \{a,b\}^n\,\}$ has large X-complexity, for all grammar types X with $X\in \varGamma$. A detailed proof of this fact can be found in [9].

Theorem 6

Let $X \in \varGamma$ and $n \ge 1$. Then $\mathsf {Xc}(T_n) = 2^n$.

4 Relating Finite and Infinite Complexity Measures

In this section, we will consider several different complexity measures on finite languages and relate them according to a group of relations that vary in strength. By the very nature of these relations, one can distinguish two main categories: the first category fixes the measure type $\tau \in \mathcal {M}$ and then compares the different grammar types in $\varGamma$ with each other w.r.t. the measure type $\tau$—see, e.g., (2). The second category swaps the roles of measure and grammar type, i.e., some grammar type $X\in \varGamma$ is fixed and then the different measure types in $\mathcal {M}$ are compared with each other w.r.t. the grammar type X.

4.1 Relating Grammar Types

Now, we define several different relations on the grammar types in $\varGamma$ w.r.t. some fixed measure type from $\mathcal {M}$. In this way, we classify the difference between different grammar types w.r.t. the same measure type. This is similar to the notion introduced in [6] for the nonterminal complexity and thus leads to a certain kind of production complexity hierarchy.

Let $X, Y \in \varGamma$ and $\tau \in \mathcal {M}$. Then we write

• $X \le _\tau Y$ if and only if $\mathsf {X}\tau (L) \le \mathsf {Y}\tau (L)$, for all finite languages L;

• $X \le _\tau ^1 Y$ if and only if there is a constant c such that $\mathsf {X}\tau (L) \le \mathsf {Y}\tau (L) + c$, for all finite languages L, and there is a sequence of finite languages $(L_i)_{i \ge 0}$ such that $\mathsf {Y} \tau (L_i) - \mathsf {X}\tau (L_i) \ge i$;

• $X \le _\tau ^2 Y$ if and only if there is a constant c such that $\mathsf {X}\tau (L) \le \mathsf {Y}\tau (L) + c$, for all finite languages L, and there is a sequence of finite languages $(L_i)_{i \ge 0}$ such that 

  $$\lim _{i \rightarrow \infty } \frac{\mathsf {X}\tau (L_i)}{\mathsf {Y}\tau (L_i)} = 0;$$

• $X \le _\tau ^3 Y$ if and only if there is a constant c such that $\mathsf {X}\tau (L) \le \mathsf {Y}\tau (L) + c$, for all finite languages L, and there is no function $f :\mathbb {N} \rightarrow \mathbb {N}$ such that $\mathsf {Y}\tau (L) \le f(\mathsf {X}\tau (L))$, for all finite languages L.

Clearly, $X \le _\tau ^3 Y$ implies $X \le _\tau ^2 Y$, which, in turn, implies $X \le _\tau ^1 Y$. Moreover, $X \le _\tau ^3 Y$ holds if the first condition of its definition is satisfied and there is a sequence $(L_i)_{i \ge 0}$ of finite languages such that $\mathsf {X}\tau (L_i) \le k$, for some constant k and $\mathsf {Y}\tau (L_i) \ge i$. We write $X =_\tau Y$ if both $X \le _\tau Y$ and $Y \le _\tau X$ hold.

Now, we are going to relate the different grammar types in $\varGamma$ w.r.t. the finite complexity measure types under investigation. As already mentioned earlier, the relation $\mathsf {CF}\le _\tau \mathsf {LIN}\le _\tau \mathsf {REG}$ holds by definition, for all $\tau \in \mathcal {M}$. The following result was shown in [4]:

Theorem 7

It holds that $\mathsf {CF}\le _c^2 \mathsf {LIN}\le _c^2 \mathsf {REG}$.

Next, we show that we can only obtain $\mathsf {CF}\le _\tau ^3 \mathsf {LIN}$ and $\mathsf {CF}\le _\tau ^3 \mathsf {REG}$, for $\tau \in \{ \mathsf {c}, \mathsf {cc}\}$. As a prerequisite, we need the following result from [4].

Lemma 8

Let G be a linear grammar generating a finite language with $|G| \ge 1$. Then $|L(G)| \le 2^{|G|-1}$ and $|G| \ge \log |L(G)| +1$.

Now, we are ready for the following theorem.

Theorem 9

Let $\tau \in \{ \mathsf {c}, \mathsf {cc}\}$. Then

• 1. $\mathsf {CF}\le _\tau ^3 X$, for $X \in \{\mathsf {REG},\mathsf {LIN}\}$, and

• 2. $\mathsf {LIN}\not \le _\tau ^3 \mathsf {REG}$ and $\mathsf {REG}\not \le _\tau ^3 \mathsf {LIN}$.

Proof

• 1. Let $L_n = \{a,b\}^{\le n}$, for $n \ge 0$. It was shown in [4], that $\mathsf {CFc}(L_n) \le 4$ and $\mathsf {LINc}(L_n) \ge n + 1$. From Lemma 8, it follows that also $\mathsf {LINcc}(L_n) \ge n + 1$. Moreover, it holds that $\mathsf {CFcc}(L_n) \le \mathsf {CFc}(L_n) \le 4$. Consequently, $\mathsf {CF}\le _\tau ^3 \mathsf {LIN}$. Since $\mathsf {REG}\tau (L_n) \ge \mathsf {LIN}\tau (L_n) \ge n+1$, we immediately get $\mathsf {CF}\le _\tau ^3 \mathsf {REG}$.

• 2. Let L be an arbitrary finite language and G be a minimal linear grammar with $L(G) = L$. Clearly, $\mathsf {REG}\tau (L) \le |L| = |L(G)|$. From Lemma 8, it follows that $|L(G)| \le 2^{\mathsf {LIN}\tau (L)-1}$. Thus, $\mathsf {REG}\tau (L) \le |L(G)| \le 2^{\mathsf {LIN}\tau (L)-1}$. The function $f :\mathbb {N} \rightarrow \mathbb {N}$ with $x \mapsto 2^{x-1}$ fulfills $\mathsf {REG}\tau (L') \le f(\mathsf {LIN}\tau (L'))$, for all finite languages $L'$, i.e., $\mathsf {LIN}\not \le _\tau ^3 \mathsf {REG}$. Since $\mathsf {LIN}\tau (L') \le \mathsf {REG}\tau (L')$, setting $f = id_{\mathbb {N}}$ yields $\mathsf {REG}\not \le _\tau ^3 \mathsf {LIN}$.   $\square$

For the X-scattered-complexity, we have the following situation:

Theorem 10

Let $X,Y\in \varGamma$. Then $\mathsf {REG}=_\mathsf {sc}\mathsf {LIN}=_\mathsf {sc}\mathsf {CF}$, but $X \not \le _\mathsf {sc}^i Y$, for $i \in \{1,2,3\}$.

It remains to relate the different grammar types in $\varGamma$ w.r.t. the infinite complexity measure types $\mathsf {c}_\infty , \mathsf {cc}_\infty , \mathsf {sc}_\infty$. For the infinite X-complexity we have:

Theorem 11

Let $n \ge 1$. Then $\mathsf {REGc}_{\infty }(P_n) = \varOmega (2^n)$.

Proof

The idea of the proof is as follows: let $G=(N,\varSigma ,P,S)$ be a regular grammar that is a witness for $\mathsf {REGc}_{\infty }(P_n)$. Then we construct a regular grammar generating the finite set $L(G)\cap \varSigma ^{\le 2n+1}$. Since this language is equal to $P_n$, we can apply Theorem 5 in order to obtain a lower bound on |G|. Although G may contain $\varepsilon$-productions, we can safely assume that G does not contain productions with a right hand-side longer than $2n+2$, since none of these productions generates a word of length less than or equal to $2n+1$ even with erasing rules.

To keep the presentation simple, assume for a moment that the grammar G is in 2-normal form.² Then we apply the triple construction—see, e.g., [10]—to the grammar G and the nondeterministic finite automaton $A=(Q,\varSigma ,\delta ,q_0,F)$ with $Q=\{0,1,\ldots ,2n+1\}$, initial state $q_0=0$, final state set $F=\{2n+1\}$, and the transition function $\delta (i,a)=\{i+1,2n+1\}$, for $0\le i<2n+1$ and $a\in \varSigma$. Then the regular grammar $G'=(N',\varSigma ,P',S')$ with $N'=Q\times N\times Q$, start symbol $S'=[0,S,2n+1]$, and the productions

$$\begin{aligned} P'&= \{\,[i,A,j]\rightarrow a\mid \,\,A\rightarrow a\in P,\ a\in \varSigma \cup \{\varepsilon \}, \,\text{ and }\,\, j\in \delta (i,a)\,\} \\&\qquad \cup \{\,[i,A,k]\rightarrow a[j,B,k]\mid \,\,A\rightarrow aB\in P,\ a\in \varSigma \cup \{\varepsilon \}, \text{ and }\,\, j\in \delta (i,a)\,\} \end{aligned}$$

generates the finite language $L(G)\cap \varSigma ^{\le 2n+1}$. Observe that $|G'|\le |G|\cdot (2n+2)^3$, because the productions of the form $A\rightarrow aB$ from P increase $|G'|$ the most.

Now, we are ready to prove the stated claim. Assume to the contrary that we have $|G| = \mathsf {REGc}_{\infty }(P_n)=o(2^n)$, for a regular grammar G. Then we transform G into an equivalent regular grammar in 2-normal form and apply the above described intersection construction. Let $G'$ refer to the result of these construction steps. The transformation into 2-normal form is done by simply splitting the right hand-side of each production into a sequence of productions of the appropriate form. This increases |G| by at most a factor of $2n+2$. Together with the increase of productions by the triple construction by a factor of at most $(2n+2)^3$, we conclude that $|G'|=o(2^n)$, because $o(2^n\cdot (2n+2)^4)=o(2^n)$. Since the regular grammar $G'$ with $|G'|=o(2^n)$ generates $P_n$, we get a contradiction to Theorem 5. Thus, we obtain $\mathsf {REGc}_{\infty }(P_n)=\varOmega (2^n)$.   $\square$

The taxonomy of the next theorem applies to the infinite exact X-complexity.

Theorem 12

It holds that $\mathsf {CF}\le _{\mathsf {c}_\infty }^i \mathsf {REG}$ and $\mathsf {LIN}\le _{\mathsf {c}_\infty }^i \mathsf {REG}$, for all $i \in \{ 1,2,3\}$.

Finally, both the infinite X-cover and the infinite X-scattered-complexity do not classify according to the used taxonomy.

Theorem 13

Let $X,Y \in \varGamma$ and $\tau \in \{ \mathsf {cc}_\infty , \mathsf {sc}_\infty \}$. Then we have $X \not \le _\tau ^i Y$, for all $i \in \{1,2,3\}$.

We have to leave open the question whether $\mathsf {LIN}\le _\mathsf {cc}^i \mathsf {REG}$ and $\mathsf {CF}\le _{\mathsf {c}_\infty }^j \mathsf {LIN}$, for $i \in \{ 1,2\}$ and $j \in \{1,2,3\}$, holds.

4.2 Relating Complexity Measure Types

Now, we introduce relations for measuring the difference between different measure types w.r.t. some fixed grammar type that we consider in this paper. Therefore, for $\tau ,\sigma \in \mathcal {M}$ and $X\in \varGamma$, we similarly define the relations $\tau \le _X\sigma$, $\tau \le _X^1\sigma$, $\tau \le _X^2\sigma$, and $\tau \le _X^3\sigma$ as in the beginning of the previous subsection. For instance, $\tau \le _X^1 \sigma$ if and only if there is a constant c such that $\mathsf {X}\tau (L) \le \mathsf {X}\sigma (L) + c$, for all finite languages L, and there is a sequence of finite languages $(L_i)_{i \ge 0}$ such that $\mathsf {X} \sigma (L_i) - \mathsf {X}\tau (L_i) \ge i$.

Clearly, the following chain of implications holds: $\tau \le _X^3 \sigma$ implies $\tau \le _X^2 \sigma$, which, in turn, implies $\tau \le _X^1 \sigma$. Moreover, $\tau \le _X^3 \sigma$ holds if the first condition of its definition is satisfied and there is a sequence $(L_i)_{i \ge 0}$ of finite languages such that $\mathsf {X}\tau (L_i) \le c$, for some constant c, and $\mathsf {X}\sigma (L_i) \ge i$.

We start with comparing the finite X- with the infinite X-measures. Except for the X-scattered-complexity, the infinite versions are more succinct than their finite counterparts.

Lemma 14

Let $X \in \varGamma$. Then (i) $\mathsf {c}_\infty \le _X \mathsf {c}$ and (ii) $\mathsf {cc}_\infty \le \mathsf {cc}$, but we have (iii) $\mathsf {sc}\le _X \mathsf {sc}_\infty$.

Proof

The first relation follows by definition. Let $L\subseteq \varSigma ^{\le \ell }$. Assume that G is a witness for $\mathsf {Xc}(L)$, i.e., G generates a finite language, $L=L(G)$, and $\mathsf {Xc}(L)=|G|$. But then also $L=L(G)\cap \varSigma ^{\le \ell }$, which implies $\mathsf {Xc_{\infty }}(L)\le \mathsf {Xc}(L)$. A similar argumentation applies to the second relation. For the third relation, we argue as follows: in Theorem 3, it was shown that $\mathsf {Xsc}(L)=1$ if L is non-empty and $\mathsf {Xsc}(L)=0$ if $L = \emptyset$. In the latter case, we also have $\mathsf {Xsc}_{\infty }(L)=0$. Thus, we conclude that $\mathsf {Xsc}_{\infty }(L)\le \mathsf {Xsc}(L)$, for every finite language L.   $\square$

It is worth mentioning that in the previous lemma, the argumentation used in the proof of the first two relations does not apply to the third one. This is seen as follows: consider the uniform finite language $L=\{a,b\}$. Then the regular grammar $G=(\{S\},\{a,b\},\{S\rightarrow ab\},S)$ witnesses $\mathsf {Xsc}(L)=1$, because L is a scattered sublanguage of $\{ab\}$, i.e., $L\le \{ab\}=L(G)$. But $L(G)\cap \{a,b\}^{\le 1}=\emptyset$. Thus, |G| cannot be used as an upper bound for $\mathsf {Xsc}_{\infty }(L)$ and hence $\mathsf {sc}_\infty \le _X\mathsf {sc}$ does not hold in general. Thus, we conclude:

Corollary 15

Let $X\in \varGamma$. Then $\mathsf {sc}_\infty \not \le _X\mathsf {sc}$.

Next, we compare the remaining X-complexities. Observe that $L_1=L_2$ implies $L_1\subseteq L_2$, which, in turn, implies $L_1\le L_2$. As an easy consequence, we deduce that the (infinite) X-scattered-complexity is more succinct than the (infinite) X-cover-complexity and it is also easy to see that the (infinite) X-cover-complexity is more succinct than the (infinite) X-complexity. We summarize:

Lemma 16

Let $X \in \varGamma$. Then (i) $\mathsf {sc}\le _X \mathsf {cc}\le _X \mathsf {c}$ and (ii) $\mathsf {sc}_\infty \le _X \mathsf {cc}_\infty \le _X \mathsf {c}_\infty$.

Fig. 1.

Relations between the different grammatical complexity measures on finite languages. In this figure, a $\le$-relation is of type (i, j, k), for $i,j,k\in \{1,2,3\}$, if (i) $\le _\mathsf {CFG}^i$, (ii) $\le _\mathsf {LIN}^j$, and (iii) $\le _\mathsf {REG}^k$ hold for the appropriate X-measures. All $\le$-relations are of type (3, 3, 3), except $\mathsf {X}\mathsf {cc}\le \mathsf {X}\mathsf {c}$, $\mathsf {X}\mathsf {cc}_\infty \le \mathsf {X}\mathsf {cc}$, $\mathsf {X}\mathsf {sc}\le \mathsf {X}\mathsf {sc}_\infty$, and $\mathsf {X}\mathsf {sc}_\infty \le \mathsf {X}\mathsf {cc}_\infty$, which are of types (3, 2, 2), $(-,3,3)$, $(-,-,-)$, and $(-,-,-)$, respectively; the − sign means that it cannot be classified by the taxonomy.

The obtained $\le$-relations are visualized in Fig. 1. For the measures $\mathsf {cc}$ and $\mathsf {c}_\infty$ we show incomparability w.r.t. the $\le$-relation. Before we can prove this, we need two prerequisites. The first one is a lower bound on $T_n$ w.r.t. the infinite X-complexity, which reads as follows—the proof is similar to the proof of Theorem 11 with the slight modification that we use 2-Greibach normal form³ instead of 2-normal form, since we cannot apply the reasoning regarding the absence of $\varepsilon$-productions to context-free grammars in case of the $\mathsf {c}_\infty$-measure. The change to 2-Greibach normal form induces the fourth root in the lower bound:

Theorem 17

Let $X \in \varGamma$. Then $\mathsf {Xc_{\infty }}(T_n) = \varOmega (2^{n/4})$.

The second prerequisite is an exact complexity bound for the language $\varSigma ^{\le \ell }$ w.r.t. the finite X-cover-complexity.

Lemma 18

Let $X \in \varGamma$ and $\ell \ge 2$. Then $\mathsf {CFcc}(\varSigma ^{\le \ell }) = |\varSigma | +2$.

Now, we are ready for the incomparability results:

Theorem 19

Let $X \in \varGamma$. Then (i) $\mathsf {cc}\not \le _X \mathsf {c}_\infty$ and (ii) $\mathsf {c}_\infty \not \le _X \mathsf {cc}$.

Proof

For the proof of (i), observe that the grammar $G = (N,\varSigma ,P,S)$ with the productions $P= \{ S \rightarrow aS, S \rightarrow bS, S \rightarrow \varepsilon \}$ shows that $\mathsf {Xc_{\infty }}(\{a,b\}^{\le n}) \le 3$. By Lemma 8, we have $n+1 \le \mathsf {Ycc}(\{a,b\}^{\le n})$, for $n \ge 3$ and $Y \in \{ \mathsf {REG},\mathsf {LIN}\}$. As a consequence, $\mathsf {Yc_{\infty }}(\{a,b\}^{\le n}) < \mathsf {Ycc}(\{a,b\}^{\le n})$. From Lemma 18, it follows that $\mathsf {CFcc}(\{a,b\}^{\le n}) \ge 4$, i.e., $\mathsf {CFc}_{\infty }(\{a,b\}^{\le n}) < \mathsf {CFcc}(\{a,b\}^{\le n})$. Next, we prove (ii). By Theorem 17, $\mathsf {Xc_{\infty }}(T_n) = \varOmega (2^{n/4})$ and, by Theorem 3, we have $\mathsf {Xcc}{T_n} \le 15n + 10$. Thus, $\mathsf {Xcc}{T_n} < \mathsf {Xc_{\infty }}(T_n)$, for large enough n.   $\square$

In the remainder of this subsection, we classify the relations between the X-measures under consideration according to the taxonomy from [6].

Theorem 20

Let $X \in \varGamma$ and $Y \in \{ \mathsf {REG},\mathsf {LIN}\}$. Then

• 1. $\mathsf {cc}\le _\mathsf {CFG}^3 \mathsf {c}$ and $\mathsf {cc}\le _Y^2 \mathsf {c}$, but $\tau \not \le _Y^3 \sigma$, for all $\tau ,\sigma \in \{\mathsf {c}, \mathsf {cc}\}$ with $\tau \ne \sigma$,

• 2. $\mathsf {sc}\le _X^3 \mathsf {c}$,

• 3. $\mathsf {sc}\le _Y^3 \mathsf {cc}$, but $\tau \not \le _\mathsf {CFG}^i \sigma$, for all $\tau , \sigma \in \{ \mathsf {sc},\mathsf {cc}\}$ with $\tau \ne \sigma$ and all $i \in \{1,2,3\}$,

• 4. $\mathsf {c}_\infty \le _X^3 \mathsf {c}$,

• 5. $\tau \le _X^3 \mathsf {c}$, for all $\tau \in \{\mathsf {cc}_\infty , \mathsf {sc}_\infty \}$,

• 6. $\mathsf {cc}_\infty \le _Y^3 \mathsf {cc}$, but $\tau \not \le _\mathsf {CFG}^i \sigma$, for $\tau , \sigma \in \{ \mathsf {cc},\mathsf {cc}_\infty \}$ with $\tau \ne \sigma$ and $i \in \{1,2,3\}$,

• 7. $\mathsf {cc}\le _\mathsf {CFG}^3 \mathsf {c}_\infty$, and

• 8. $\tau \le _X^3 \mathsf {c}_\infty$, for all $\tau \in \{ \mathsf {sc}, \mathsf {sc}_\infty , \mathsf {cc}_\infty \}$.

Proof

We only prove the first statement. The remaining results can be shown with similar arguments. Let $L_n = \{\, a^jb^jc^j \mid 1 \le j \le n \,\}$. In [2], it was shown that $\mathsf {CFc}(L_n) = n$. On the other hand, by Theorem 3, we have $\mathsf {CFcc}(L_n) \le 5$.

Let $Y \in \{\mathsf {REG},\mathsf {LIN}\}$ and $T_n = \{\, w\$w\#w \mid w \in \{a,b\}^n \,\}$. By Theorem 6, we have $\mathsf {Yc}(T_n) = 2^n$. On the other hand, we have $\mathsf {Ycc}(T_n) \le 15n + 10$ by Theorem 3. Hence, $\mathsf {cc}\le _Y^2 \mathsf {c}$. Finally, let L be an arbitrary finite language and $f:\mathbb {N} \rightarrow \mathbb {N}$ a function defined by $x \mapsto 2^{x-1}$. Moreover, let G be a minimal Y-grammar with $L(G) \supseteq L$. By Lemma 8, we know that $|L| \le |L(G)| \le 2^{\mathsf {Ycc}(L)-1}$. It holds that $\mathsf {Yc}(L)\le |L|$. Thus,

$$\mathsf {Yc}(L) \le |L| \le |L(G)| \le 2^{\mathsf {Ycc}(L) - 1} = f(\mathsf {Ycc}(L)).$$

Hence, $\mathsf {cc}\not \le _Y^3 \mathsf {c}$. If we set $f = id_{\mathbb {N}}$, it follows that $\mathsf {c}\not \le _Y^3 \mathsf {cc}$.    $\square$

Finally, we list some incomparability results.

Theorem 21

Let $X \in \varGamma$ and $Y \in \{ \mathsf {REG},\mathsf {LIN}\}$. Then, for $i\in \{1,2,3\}$, we have

• 1. $\tau \not \le _X^i \sigma$, for every $\tau , \sigma \in \{\mathsf {cc}_\infty , \mathsf {sc}_\infty \}$ with $\tau \ne \sigma$,

• 2. $\tau \not \le _X^i \sigma$, for every $\tau , \sigma \in \{\mathsf {cc}, \mathsf {sc}_\infty \}$ with $\tau \ne \sigma$,

• 3. $\tau \not \le _X^i \sigma$, for every $\tau , \sigma \in \{\mathsf {sc}, \mathsf {sc}_\infty \}$ with $\tau \ne \sigma$,

• 4. $\tau \not \le _X^i \sigma$, for every $\tau , \sigma \in \{\mathsf {sc}, \mathsf {cc}_\infty \}$ with $\tau \ne \sigma$, and

• 5. $\tau \not \le _Y^i \sigma$, for every $\tau , \sigma \in \{\mathsf {cc},\mathsf {c}_\infty \}$ with $\tau \ne \sigma$.

Proof

We only prove the first statement. The remaining results can be shown with similar arguments. First note that $L \subseteq \varSigma ^* \cap \varSigma ^{\le \ell }$ as well as $L \le \varSigma ^* \cap \varSigma ^{\le \ell }$, for all finite languages L over $\varSigma$ with $\ell = \max \{\, |w| \mid w \in L \,\}$. The universal language $\varSigma ^*$, for $\varSigma = \{a_1,a_2,\ldots ,a_n\}$, can be produced with the following regular productions: $S \rightarrow a_1S \mid a_2 S \mid \ldots \mid a_n S \mid \varepsilon$. Thus, $\mathsf {Xcc}_{\infty }(L) \le |\varSigma | + 1$ and $\mathsf {Xsc}_{\infty }(L) \le |\varSigma | + 1$, for all finite languages L over $\varSigma$. This, however, means that $\tau \le _X^i \sigma$ does not hold for $\tau , \sigma \in \{ \mathsf {cc}_\infty ,\mathsf {sc}_\infty \}$ and all $i \in \{1,2,3\}$.   $\square$