1 Introduction

Symmetric tensor decomposition (also known as the Waring problem) is usually studied over the complex numbers: given a multivariate form \(f \in \mathbb{C}[x_{1},\dots,x_{n}]\) of degree d the problem is to find the least integer number m such that \(f=\sum_{i=1}^{m} \ell_{i}^{d}\), where ℓ i are linear forms. The number m is known as the Waring rank or symmetric tensor rank of f. It is of significant interest to find efficient algorithms to compute a minimal representation of f [13]. The symmetric tensor decomposition problem over the complex numbers has been widely investigated [1, 4, 5, 9, 11].

A rank m is called generic for forms of degree d if forms of rank m contain a dense open subset of all forms of degree d. There is a unique generic rank for forms in \(\mathbb{C}[x_{1},\dots,x_{n}]_{d}\), which depends on the degree d and the number of variables n. The Alexander–Hirschowitz theorem gives the generic rank for all n and d [1]. The question of uniqueness, and more generally, the variety of all possible decompositions for a form f has also been studied extensively [5, 11, 12, 14].

The same question can be asked over the real numbers, which often makes more sense with respect to potential applications. Given a multivariate form \(f \in \mathbb{R}[x_{1},\dots,x_{n}]\) of degree d we now ask to find the least integer number m such that \(f=\sum_{i=1}^{m} \ell_{i}^{d}\) where \(\ell_{i}\in \mathbb{R}[x_{1},\dots,x_{n}]\) are real linear forms. The theory of real symmetric tensor decompositions is not nearly as developed. One obstacle is that there may be more than one “generic” rank. Following Comon and Ottaviani in [7] we will call any rank that occurs on an open subset of \(\mathbb{R}[x_{1},\dots,x_{n}]_{d}\) (with the Euclidean topology induced by viewing f as a vector of coefficients) a typical rank. We note that definitions of typical and generic rank are equivalent over the complex numbers: once forms of rank m contain an open subset of \(\mathbb{C}[x_{1},\dots,x_{n}]_{d}\) they will also contain a dense open subset.

The complex tensor decomposition problem in two variables has already been well understood by Sylvester and completely and algorithmically solved by Comas and Seiguer in [6]. In contrast for binary forms over \(\mathbb{R}\) the list of all typical ranks was conjectured by Comon and Ottaviani [7].

In this paper we prove this conjecture and solve the problem of finding all typical ranks for real binary forms. It has already been observed by Reznick that all ranks m with 1≤m≤d occur as real ranks of binary in \(\mathbb{R}[x,y]_{d}\) [16]. It was noted by Comon and Ottaviani that only ranks m with \(\lfloor\frac{d+2}{2}\rfloor \leq m \leq d\) may be typical ranks for forms in \(\mathbb{R}[x,y]_{d}\). They showed that all such ranks occur for d≤5 and later this was proved by Ballico for d≤7 in [2]. It was also shown in [7] that ranks \(\lfloor\frac{d+2}{2}\rfloor \) and d are typical. The case of real ranks of bivariate monomials was completely analyzed by Boij, Carlini and Geramita in [3].

The case of Waring ranks of binary forms over some field extensions of \(\mathbb{R}\) was analyzed by Reznick in [17]. Finally we note that the question of decompositions of real forms f of even degree as sums of 2dth powers of linear forms is related to the truncated moment problem in real analysis [10], [15].

2 Proofs

Let \(f \in \mathbb{R}[x,y]_{d}\) be given by \(f=\sum_{i=0}^{d} a_{i}x^{i}y^{d-i}\). We can associate to f the differential operator ∂f given by

$$\partial f=\sum_{i=0}^d a_i \frac{\partial^d}{\partial x^i\partial y^{d-i}}. $$

The apolar ideal of f, denoted by f ⊥, is the ideal of all forms in \(\mathbb{R}[x,y]\) whose differential operator annihilates f:

$$f^\perp=\bigl\{h \in \mathbb{R}[x,y] \mid \partial h(f)=0\bigr\}. $$

The following is a special bivariate version of the general Apolarity Lemma [8]. It was already known to Sylvester [16].

Lemma 2.1

(Apolarity Lemma)

Let \(f \in \mathbb{R}[x,y]_{d}\) be a bivariate form of degree d. The form f can be written as a linear combination of dth powers of linear forms,

$$f=\sum_{i=1}^rc_i(a_ix+b_iy)^d, $$

if and only if the form p=(b 1 x−a 1 y)⋯(b r x−a r y) is in the apolar ideal f ⊥.

From the Apolarity Lemma we see that \(\operatorname{rank}f=m\) if and only if m is the lowest degree for which f ⊥ contains a form with all roots real and distinct. The structure of bivariate apolar ideals is highly regular. It was already shown by Sylvester that all bivariate apolar ideals are complete (empty) intersections over \(\mathbb{C}\) and the converse also holds.

Theorem 2.2

Let \(f \in \mathbb{R}[x,y]_{d}\) then f ⊥ is a complete intersection ideal over \(\mathbb{C}\), i.e. f ⊥ is generated by two real forms g 1,g 2 such that degg 1+degg 2=d+2 and \(\mathcal{V}_{\mathbb{C}}(g_{1},g_{2})=\emptyset\). Conversely, any two such forms g 1,g 2 generate an ideal f ⊥ for some \(f\in \mathbb{R}[x,y]\) of degree degg 1+degg 2−2.

It is well known that the forms \(f \in \mathbb{R}[x,y]_{d}\) for which the generator degrees of f ⊥ are not equal to \((\frac{d+2}{2},\frac{d+2}{2} )\) if d is even or \((\frac{d+1}{2},\frac{d+3}{2} )\) if d is odd lie on a Zariski closed subset of \(\mathbb{R}[x,y]_{d}\). This can be seen for instance by considering the middle catalecticant matrix of f and observing that these generator degrees occur precisely when the middle catalecticant matrix is not of full rank [16]. Therefore when f ⊥ is generated by forms of degrees \((\frac{d+2}{2},\frac{d+2}{2} )\) if d is even or \((\frac{d+1}{2},\frac{d+3}{2} )\) if d is odd we will say that f ⊥ is generated in generic degrees.

We now observe that an essential obstruction to f being a typical form of rank m is that (f ⊥) m−1 may contain a form with all real roots, but no forms with all real distinct roots. Then perturbing f may result in a form of rank m−1. As the next lemma shows, this is the only obstruction to f being typical if f ⊥ is generated in generic degrees.

Lemma 2.3

Let \(f \in \mathbb{R}[x,y]_{d}\) such that f ⊥ is generated in generic degrees and let \(m=\operatorname{rank}f\). Then f is a typical form if and only if all forms in (f ⊥) m−1 have (at least a conjugate pair of) complex roots.

Proof

First suppose that (f ⊥) m−1 contains a form with all real roots, call this form s. For any ϵ>0 there exists a form \(r_{\epsilon}\in \mathbb{R}[x,y]_{m-1}\) in the ϵ-neighborhood of s such that s+r ϵ has distinct real zeroes. Let h ϵ =∂(s+r ϵ )(f)=∂r ϵ (f) and consider the map

$$T_{\epsilon}: \mathbb{R}[x,y]_d \rightarrow \mathbb{R}[x,y]_{d-m+1}, \quad T_{\epsilon}(p)=\partial (s+r_{\epsilon}) (p). $$

Since T ϵ is a linear map with nontrivial kernel, which for small enough ϵ is sufficiently close to just applying ∂s, we can find a form \(g_{\epsilon} \in \mathbb{R}[x,y]_{d}\), g ϵ ≠−f such that T ϵ (g ϵ )=−h ϵ and g ϵ approaches 0 as ϵ goes to 0. Then we see that ∂(s+r ϵ )(f+g ϵ )=0 and by Apolarity Lemma there exist forms of rank at most m−1 in any ϵ-neighborhood of f, which is a contradiction.

For the other direction, we note that all forms h in a sufficiently small neighborhood of f will have h ⊥ generated in generic degrees. In this neighborhood of f the ideal h ⊥ depends continuously of the coefficients of h. This can be seen, for example, by noting that h ⊥ consists of the forms in the kernels of the catalecticant matrices of h [16]. As long as h ⊥ is generated in generic degrees the dimensions of the kernels do not change and therefore there is a continuous dependence. Now perturb the coefficients of f slightly and call the resulting form h. For a small enough perturbation, given the continuous dependence of h ⊥ on the coefficients of h we can ensure that (h ⊥) m−1 has no forms with all real roots, while (h ⊥) m has such a form. Thus the rank of h is m, where h is any sufficiently small perturbation of the coefficients of f. □

We now inductively build apolar ideals f ⊥ where the form f is a typical form of rank m for all m in the desired range. Instead of constructing f directly we build generators of the ideal, which by Theorem 2.2 is an apolar ideal for some f, and then apply Lemma 2.3 to show that f is typical.

Theorem 2.4

All ranks m with \(\lfloor\frac{d+2}{2}\rfloor \leq m \leq d\) are typical for forms in \(\mathbb{R}[x,y]_{d}\).

Proof

We use induction on the degree d. The base case d=2 is just bivariate quadratic forms and the real rank corresponds to the usual rank of the matrix. Therefore there is only one typical rank, which is 2.

Inductive Step: d⇒d+1. We first note that it was already shown in [7] that rank d+1 is typical for forms in \(\mathbb{R}[x,y]_{d+1}\). Suppose that \(f \in \mathbb{R}[x,y]_{d}\) is a typical form of rank \(\lfloor \frac{d+3}{2} \rfloor \leq m \leq d\). By perturbing f we may assume that the apolar ideal f ⊥ is generated in generic degrees.

Suppose d=2k is even. Then f ⊥ is generated by forms p 1,p 2 with degp 1=degp 2=k+1.

First suppose that m=k+1. Now let \(\ell \in \mathbb{R}[x,y]_{1}\) be any linear form such that the zero of ℓ is not one of the zeroes of p 1 and consider the ideal I=〈p 1,ℓp 2〉. The forms p 1 and ℓp 2 form a complete intersection over \(\mathbb{C}\). By Theorem 2.2, I is the apolar ideal of some form \(g \in \mathbb{R}[x,y]_{d+1}\). Since we have g ⊥⊂f ⊥ by Lemma 2.3 we know that g is a typical form of rank m.

Now suppose that m>k+1. By the Apolarity Lemma there exists s∈(f ⊥) m such that s has all real distinct roots and by Lemma 2.3 we know that all forms in (f ⊥) m−1 have at least two complex roots. Since s∈(f ⊥) m we can write s in terms of the generators p 1 and p 2 of f ⊥:

$$s=p_1q_1+p_2q_2 \quad\text{for } q_1,q_2 \in \mathbb{R}[x,y]_{m-k-1}. $$

We now claim that we may choose two generators p 1 and p 2 of f ⊥ so that the multiplier q 2 has a real root distinct from the roots of p 1. If this does not hold, then we may pick a different set of generators of f ⊥: let

$$p_1'=p_1+\alpha p_2 \quad \text{with some } \alpha \in \mathbb{R}. $$

Then

$$s=p_1'q_1+p_2(q_2- \alpha q_1). $$

We can easily adjust α so that q 2−αq 1 has a real root, and we need to argue that we can also make this root distinct from the roots of \(p_{1}'=p_{1}+\alpha p_{2}\). Suppose not; then for any \((a,b) \in \mathbb{R}^{2}\) that is not a root of q 1 we may set α=−q 2(a,b)/q 1(a,b) and make (a,b) a root of q 2−αq 1. Therefore we must have \(\frac{p_{1}}{p_{2}}=-\frac{q_{2}}{q_{1}}\), which implies that s=p 1 q 1+p 2 q 2=0 and that is a contradiction. Thus we have q 2−αq 1=ℓq with \(\ell \in \mathbb{R}[x,y]_{1}\), \(q \in \mathbb{R}[x,y]_{k-m-2}\) and ℓ does not divide \(p_{1}'\). Let

$$I=\bigl\langle p_1', \ell p_2 \bigr \rangle. $$

As before, \(p_{1}'\) and ℓp 2 form a compete intersection over \(\mathbb{C}\) and by Theorem 2.2 I is the apolar ideal of some form \(g \in \mathbb{R}[x,y]_{d+1}\). Since s∈I we know that the rank of g is at most m and since I⊂f ⊥ we know that the rank of g is at least m. Therefore the rank of g is m. Further, g ⊥⊂f ⊥ has no forms of degree m−1 with all real roots and g ⊥ is generated in generic degrees. Therefore g is a typical form of rank m.

Now suppose that d=2k+1 is odd. Then f ⊥ is generated by forms p 1,p 2 with degp 1=k+1 and degp 2=k+2. We note that we only need to deal with the cases m≥k+2. By the Apolarity Lemma there exists s∈(f ⊥) m such that s has all distinct real roots and by Lemma 2.3 all forms in (f ⊥) m−1 have at least two complex roots. Since s∈(f ⊥) m we can write

$$s=p_1q_1+p_2q_2 \quad \text{for } q_1\in \mathbb{R}[x,y]_{m-k-1}, q_2 \in \mathbb{R}[x,y]_{m-k-2}. $$

The generator p 1 is uniquely determined, but p 2 is unique only modulo the ideal generated by p 1. We now claim that we may choose generators of f ⊥ so that the multiplier q 1 has a real root distinct from the roots of p 2. If this does not hold, then let

$$p_2'=p_2+\ell p_1 \quad \text{for some linear form } \ell \in \mathbb{R}[x,y]_1. $$

We have

$$s=p_1(q_1-\ell q_2)+p_2'q_2. $$

We can adjust ℓ so that q 1−ℓq 2 has a real root, and we need to argue that we may also make this root distinct from the roots of \(p_{2}'=p_{2}+\ell p_{1}\). Arguing as before we must have \(\frac{p_{2}}{p_{1}}=-\frac{q_{1}}{q_{2}}\), which implies that s=p 1 q 1+p 2 q 2=0, which is a contradiction. Let

$$I=\bigl\langle \ell p_1,p_2' \bigr\rangle. $$

Since ℓp 1 and \(p_{2}'\) form a compete intersection over \(\mathbb{C}\) by Theorem 2.2 I is the apolar ideal of some form \(g \in \mathbb{R}[x,y]_{d+1}\). Since s∈I we know that the rank of g is at most m and since I⊂f ⊥ we know that the rank of g is at least m. Therefore the rank of g is m. Further, g ⊥⊂f ⊥ has no forms of degree m−1 with all real roots and g ⊥ is generated in generic degrees. Therefore g is a typical form of rank m. □