1 Introduction

Given a sequence of polynomials \(\{p_n(x)\}_{n\in {\mathbb {N}}_0}\), one may like to know something about the nonnegativity of the coefficients \(\displaystyle {L_{k}(m,n)}\) in

$$\begin{aligned} p_n(x)p_m(x)=\sum _{k=0}^{n+m}L_{k}(m,n)p_k(x). \end{aligned}$$
(1)

Equation (1) is called the linearization formula of the polynomial sequence \(\{p_n(x)\}_{n\in {\mathbb {N}}_0}\) and \(\displaystyle {L_{k}(m,n)}\) the linearization coefficients. It is often important to know whether the linearization coefficients are positive or non-negative (see e.g. [5, 9, 13, Chap. 9 and references therein], [14, 18]). In the sequel, we use the generalized hypergeometric series defined by

$$\begin{aligned} _p F_q\left( \! \left. \begin{array}{l} \begin{array}{c} a_1,\, \ldots ,\, a_p \end{array}\\ \begin{array}{c} b_1,\, \ldots ,\, b_q \end{array}\end{array} \! \right| \displaystyle {x}\right) =\sum _{n=0}^{\infty } {(a_1)_n\cdots (a_p)_n\over (b_1)_n\cdots (b_q)_n}{x^n\over n!}, \end{aligned}$$

where \((a)_n\) denotes the Pochhammer symbol (or shifted factorial) given by

$$\begin{aligned} (a)_n=a(a+1)(a+2)\cdots (a+n-1),\, n\in {\mathbb {N}}, (a)_0=1. \end{aligned}$$

Consider the generalized Bessel polynomials defined (in 1949 by Krall and Frink [19]) for \(N\in {\mathbb {N}}\) by (see also [12, Chap. 2], [13, p. 123], [17, Section 9.13, p. 244])

$$\begin{aligned} y_n(x;\alpha )= _2 F_0\left( \! \left. \begin{array}{l} \begin{array}{c} -n,\, n+\alpha +1 \end{array}\\ \begin{array}{c} - \end{array}\end{array} \! \right| \displaystyle {-{x\over 2}}\right) ,\, n=0,1,\ldots ,N,\, \alpha <-2N-1,\, x\ge 0, \end{aligned}$$
(2)

and (see [12, p. 13], [13, p. 123], [17, Remarks, p. 246])

$$\begin{aligned} \theta _n(x;\alpha ,\beta )=x^ny_n(2(\beta x)^{-1};\alpha )=\sum _{m=0}^n {(-1)^{n-m}(-n)_{n-m}(n+\alpha +1)_{n-m}\over (n-m)!\beta ^{n-m}}x^m. \end{aligned}$$
(3)

We refer to [12, 13, 17, 19] concerning references to the literature and the history about Bessel polynomials.

In this work, we give the linearization formulae for the generalized Bessel polynomials \((y_n(x;\alpha ))_n\) and \((\theta _n(x;\alpha ,\beta ))_n\); we find recurrence relations satisfied by the linearization coefficients of the polynomial family \((\theta _n(x;0,\beta ))_n\) (from which their nonnegativity is deduced) and the polynomial family \((y_n(x;\alpha ))_n\). For \(\beta =2\) in \((\theta _n(x;0,\beta ))_n\), we recover results given by Berg and Vignat [5] and we derived an explicit formula that generalizes a result by Atia and Zeng [3]. Following the work by Atia and Zeng [3], we simplify the linearization coefficients of the polynomial family \((\theta _n(x;0,\beta ))_n\) from a double sum to a single sum.

2 Linearization formulae of the Bessel polynomials

In this section, we solve the more general linearization problem

$$\begin{aligned} p_n(ax)p_m(bx)=\sum _{k=0}^{n+m}L_{k}(m,n,a,b)p_k(x) \end{aligned}$$
(4)

for the generalized Bessel polynomials. The linearization formula (4) follows from the hypergeometric representation of the generalized Bessel polynomials and their inversion formula, that is, a formula expanding the basis \((x^n)_{n}\) into a family of polynomials \((p_n(x))_n\) with \(\deg p_n =n\)

$$\begin{aligned} x^n=\sum _{m=0}^nI_m(n)p_m(x). \end{aligned}$$

The inversion formula of the generalized Bessel polynomials \((y_n(x;\alpha ))_n\) is given by (see e.g. [7, 12, p. 73], [15, 23, Eq. (7), p. 294], [24, 27])

$$\begin{aligned} x^n = (-2)^n\sum _{m=0}^n(2m+\alpha +1){(-n)_m\Gamma (\alpha +m+1)\over m!\Gamma (n+m+\alpha +2)}y_m(x;\alpha ). \end{aligned}$$
(5)

The polynomial \(\theta _n(x;\alpha ,\beta )\) is solution of the differential equation (see [12, Eq. (29), p. 13], compare [13, p. 123])

$$\begin{aligned} xy''(x)-(2n+\alpha +\beta x)y'(x)+n\beta y(x)=0. \end{aligned}$$
(6)

Equation (6) follows immediately from the differential equation (see e.g. [12, Eq. (26), p. 12], [17, p. 245], [19, Eq. (33)])

$$\begin{aligned} x^2y''(x)+[(\alpha +2)x+2]y'(x)-n(n+\alpha +1)y(x)=0 \end{aligned}$$

satisfied by \(y_n(x;\alpha )\) and the relation \(\theta _n(x;\alpha ,\beta )=x^ny_n(2(\beta x)^{-1};\alpha )\). In [12, Eq. (5), p. 42], the so-called pseudo-generating function for \(\theta _n(x;\alpha ,\beta )\) is given as follows:

$$\begin{aligned} (1-2u)^{-1}(1-u)^{-\alpha }e^{\beta x u}=\sum _{n=0}^\infty \{\beta u(1-u)\}^n\theta _n(x;\alpha ,\beta )/n!. \end{aligned}$$
(7)

From this pseudo-generating function, we prove that

Proposition 1

The inversion formula of the Bessel polynomials \((\theta _n(x;\alpha ,\beta ))_n\) is given by

$$\begin{aligned} x^n=(n+\alpha +1)\sum _{m=0}^n{(-n)_m(-\alpha -m)_{n-m-1} \over (-1)^{m+1}\beta ^{n-m}m!}\theta _m(x;\alpha ,\beta ). \end{aligned}$$
(8)

Proof

From (7), we get

$$\begin{aligned} e^{\beta x u}=\sum _{n=0}^\infty \frac{(\beta x)^nu^n}{n!}=\sum _{n=0}^\infty \frac{\beta ^n}{n!}u^n(1-u)^{n+\alpha }(1-2u)\theta _n(x;\alpha ,\beta ). \end{aligned}$$

Since

$$\begin{aligned} u^n(1-u)^{n+\alpha }(1-2u)=u^n(1-u)^{n+\alpha +1}-u^{n+1}(1-u)^{n+\alpha } \end{aligned}$$

and

$$\begin{aligned} (1-u)^a=\sum _{m=0}^\infty \frac{(-a)_m}{m!}u^m, \end{aligned}$$

it follows that

$$\begin{aligned} u^n(1-u)^{n+\alpha }(1-2u)=\sum _{m=0}^\infty \frac{(-n-\alpha -1)_m(n+\alpha +m+1)}{m!(n+\alpha +1)}u^{n+m}. \end{aligned}$$

Therefore

$$\begin{aligned} \sum _{n=0}^\infty \frac{(\beta x)^nu^n}{n!}=\sum _{n=0}^\infty \sum _{m=0}^\infty \frac{\beta ^n(-n-\alpha -1)_m(n+\alpha +m+1)}{n!m!(n+\alpha +1)}u^{n+m}\theta _n(x;\alpha ,\beta ). \end{aligned}$$

From the relation [23, Eq. (1), p. 56]

$$\begin{aligned} \sum _{n=0}^\infty \sum _{m=0}^\infty A(m,n)=\sum _{n=0}^\infty \sum _{m=0}^nA(m,n-m), \end{aligned}$$

we deduce by equating the coefficients of \(u^n\) in the latter expression that

$$\begin{aligned} x^n=(n+\alpha +1)\sum _{m=0}^n\frac{n!(-n+m-\alpha -1)_m\beta ^{-m}}{(n-m)!m!(n-m+\alpha +1)}\theta _{n-m}(x;\alpha ,\beta ) \end{aligned}$$

from which the result follows. \(\square \)

Remark 2

For \(\alpha =0\) and \(\beta =2\), we have the Bessel polynomials (see [5, 12])

$$\begin{aligned} q_n(x)=\frac{2^nn!}{(2n)!}\theta _n(x;0,2)=\sum _{k=0}^n\frac{(-n)_k2^k}{(-2n)_kk!}x^k, \end{aligned}$$
(9)

and (8) becomes (after multiplication and division by n!)

$$\begin{aligned} x^n=(n+1)!\sum _{m=0}^n\frac{(2m)!(-n)_m(-m)_{n-m-1}}{(-1)^{m+1}2^n (m!)^2n!}q_m(x). \end{aligned}$$

Since \((-m)_{n-m-1}=0\) if \(n-m-1>m\), i.e., if \(0\le m < \frac{n-1}{2}\), and \(\frac{(-n)_m(-m)_{n-m-1} }{(-1)^{m+1} m!n! }=\frac{(-1)^{n-m}}{(n-m)!(2m+1-n)!}\), the above inversion formula coincides with the one due to Carlitz [6] (see [3, Eq. (13)], [5, Eq. (17)], [12, p. 73])

$$\begin{aligned}&x^n=\sum _{m=0}^{n}I_m(n)q_m(x),\\ \quad&\text { with } I_n(m)={\left\{ \begin{array}{ll} \frac{(n+1)!}{2^n}\frac{(-1)^{n-m}(2m)!}{(n-m)!m!(2m+1-n)!}, &{} \,\hbox {if }\quad \frac{n-1}{2}\le m\le n, \\ 0, &{} \,\hbox { if }\quad 0\le m < \frac{n-1}{2}. \end{array}\right. } \end{aligned}$$

To solve (4), we proceed in general as follows (see [1, 15]): If

$$\begin{aligned} p_n(x)=\sum _{i=0}^nA_i(n)x^i, \end{aligned}$$

then by the Cauchy product,

$$\begin{aligned} p_n(ax)p_m(bx)=\sum _{l=0}^{n+m}G_l(m,n)x^l, \end{aligned}$$

with

$$\begin{aligned} G_l(m,n)=\sum _{i=0}^lA_i(n)A_{l-i}(m)a^ib^{l-i}. \end{aligned}$$

Combining the preceding result with the inversion formula

$$\begin{aligned} x^l=\sum _{k=0}^lI_k(l)p_k(x), \end{aligned}$$

we get

$$\begin{aligned} L_k(m,n,a,b)= & {} \sum _{l=0}^{n+m-k}G_{l+k}(m,n)I_k(l+k) \\= & {} \sum _{l=0}^{n+m-k}\sum _{i=0}^{l+k}a^ib^{l+k-i}I_k(l+k)A_i(n)A_{l+k-i}(m). \end{aligned}$$

It follows then from the representations (2) and (3) of the generalized Bessel polynomials and their inversion formulae (5) and (8) that

Theorem 3

The linearization formula for the generalized Bessel polynomials

$$\begin{aligned} y_n(ax;\alpha )y_m(bx;\beta )=\sum _{k=0}^{n+m}L_k(m,n,a,b) y_k(x;\gamma ) \end{aligned}$$
(10)

has the coefficients

$$\begin{aligned}&L_k(m,n,a,b)\\&\quad = \sum _{l=0}^{n+m-k} \sum _{i=0}^{l+k}{a^ib^{l+k-i}(2k+\gamma +1)(-l-k)_k(-n,n+\alpha +1)_i (-m,m+\beta +1)_{l+k-i} \over k!i!(l+k-i)!(\gamma +k+1)_{l+k+1}}. \end{aligned}$$

The linearization formula for the Bessel polynomials

$$\begin{aligned} \theta _n(ax;\lambda ,\delta )\theta _m(bx;\mu ,\gamma )=\sum _{k=0}^{n+m}L_k(m,n,a,b)\theta _k(x;\alpha ,\beta ) \end{aligned}$$
(11)

has the coefficients

$$\begin{aligned}&L_k(m,n,a,b)\\&\quad =\sum _{l=0}^{n+m-k}\frac{(l+k+\alpha +1)(-l-k)_k(-\alpha -k)_{ l-1}}{(-1)^{n+m-l+1}\beta ^{l}k!} \\&\qquad \times \sum _{i=\max (0,l+k-m)}^{\min (n,l+k)}\frac{a^i(-n,n+\lambda +1)_{n-i}(-m,m+\mu +1)_{m+i-l-k}}{b^{i-l-k} \delta ^{n-i}\gamma ^{m+i-l-k}(n-i)!(m+i-l-k)!}. \end{aligned}$$

In the above theorem, \((a_1,a_2,\ldots ,a_k)_n=(a_1)_n(a_2)_n\cdots (a_k)_n\).

2.1 Special cases

For \(m=0\) and \(\alpha =\gamma \) in (10), we get the multiplication formula (see e.g. [8, 20, 25])

$$\begin{aligned} y_n(ax;\alpha )&=\sum _{k=0}^n\frac{(-a)^k(-n)_k(\alpha +n+1)_k}{k!(\alpha +k+1)_k}\\&\quad \times \; _2 F_1\left( \! \left. \begin{array}{l} \begin{array}{c} k-n,\, \alpha +k+n+1 \end{array}\\ \begin{array}{c} \alpha +2k+2 \end{array}\end{array} \! \right| \displaystyle {a}\right) y_k(x;\alpha ), \end{aligned}$$

and for \(m=0,\, \lambda =\alpha ,\, \delta =\beta \), we deduce from (11) the multiplication formula

$$\begin{aligned} \theta _n(ax;\alpha ,\beta )&=\sum _{k=0}^n\frac{(-1)^n(-a)^k(-n,n+\alpha +1)_{n-k}}{\beta ^{n-k}(n-k)!} \nonumber \\&\quad \times _3 F_2\left( \! \left. \begin{array}{l} \begin{array}{c} -n+k,\, -1-\alpha -k,\, \alpha +2+k \end{array}\\ \begin{array}{c} -2n+k-\alpha ,\, \alpha +k+1 \end{array}\end{array} \! \right| \displaystyle {a}\right) \theta _k(x;\alpha ,\,\beta ). \end{aligned}$$
(12)

3 Recurrence equations and nonnegativity of the linearization coefficients

In what follows, we derive a recurrence equation for the linearization coefficients \(L^{(\beta )}_k(m,n,a,1-a)\) of the linearization formula

$$\begin{aligned} \theta _n(ax;0,\beta )\theta _m((1-a)x;0,\beta )=\sum _{k=0}^{n+m}L^{(\beta )}_k(m,n,a,1-a)\theta _k(x;0,\beta ) \end{aligned}$$
(13)

in the specific case \(\alpha =0\), \(b=1-a\). Moreover, we give conditions for those coefficients to be nonnegative and we recover the result by Berg and Vignat [5] as a particular case. We also derive a mixed recurrence relation satisfied by the linearization coefficients \(L_k^{(\alpha )}(m,n,a,b)\) of the linearization formula

$$\begin{aligned} y_n(ax;\alpha )y_m(bx;\alpha )=\sum _{k=0}^{n+m}L^{(\alpha )}_k(m,n,a,b)y_k(x;\alpha ). \end{aligned}$$
(14)

Proposition 4

The polynomials \(\theta _n(x;\alpha ,\beta )\) and \(y_n(x;\alpha )\) satisfy, respectively, the structure relations (compare [13, Eq. (4.10.12)])

$$\begin{aligned} \theta '_n(x;\alpha ,\beta )&=c_n\, \theta _n(x;\alpha ,\beta )-c_n\,x\,\theta _{n-1}(x;\alpha ,\beta ),\, \text { with } c_n=\frac{n\beta }{2n+\alpha },\quad n\ge 1, \end{aligned}$$
(15)
$$\begin{aligned} x^2y'_n(x;\alpha )&=\frac{2(n+\alpha +1)}{2n+\alpha +2}y_{n+1}(x;\alpha )\nonumber \\&\quad -\left( (n+\alpha +1)x+\frac{2(n+\alpha +1)}{2n+\alpha +2}\right) y_n(x;\alpha ). \end{aligned}$$
(16)

Proof

Substitute \(\theta _n(x;\alpha ,\beta )=A_nx^n+B_{n}x^{n-1}+C_nx^{n-2}+\ldots \) and \(y_n(x;\alpha )=A'_nx^n+B'_nx^{n-1}+C'_nx^{n-2}+\cdots \), respectively, in the structure relations

$$\begin{aligned} \theta '_n(x;\alpha ,\beta )= & {} c_n\theta _n(x;\alpha ,\beta )+(d_nx+e_n)\theta _{n-1}(x;\alpha ,\beta ),\\ x^2y'_n(x;\alpha )= & {} c'_ny_{n+1}(x)+(d'_nx+e'_n)y_{n}(x;\alpha ) \end{aligned}$$

and equate the coefficients of \(x^n, x^{n-1}\) and \(x^{n-2}\) to get the coefficients \(c_n,\, d_n=-c_n,\, e_n=0, c'_n,\ d'_n,\ e'_n\). \(\square \)

Note that Relations (15) and (16) can also be obtained using the Maple procedure sumdiffrule of the hsum.mpl package (see [16]).

Now let \(m,n\ge 1\). Proceeding as Berg and Vignat [5], we differentiate (11) (for \(\lambda =\mu =\alpha \) and \(\delta =\gamma =\beta \)) to obtain

$$\begin{aligned}&a\theta '_n(ax;\alpha ,\beta )\theta _m(bx;\alpha ,\beta )+b\, \theta _n(ax;\alpha ,\beta )\theta '_m(bx;\alpha ,\beta )\\&\quad =\sum _{k=0}^{n+m}L_k(n,m,a,b)\theta '_k(x;\alpha ,\beta ). \end{aligned}$$

Using first (15) and then (11), this equation becomes

$$\begin{aligned}&\sum _{k=0}^{n+m}(ac_n+bc_m-c_k)L_k(m,n,a,b)\theta _{k}(x;\alpha ,\beta )\nonumber \\&\quad +x\sum _{k=0}^{n+m-1}\Big (a^2c_nL_k(n-1,m,a,b)\nonumber \\&\quad +b^2c_mL_k(n,m-1,a,b)-L_{k+1}(m,n,a,b)c_k \Big )\theta _{k}(x;\alpha ,\beta )=0. \end{aligned}$$
(17)

Since this relation is true for all x, therefore for \(x=0\), we have

$$\begin{aligned}&(ac_n+bc_m)L_0(m,n,a,b)\theta _{0}(0;\alpha ,\beta )\\&\quad +\sum _{k=1}^{n+m}(ac_n+bc_m-c_k)L_k(m,n,a,b)\theta _{k}(0;\alpha ,\beta )=0. \end{aligned}$$

This equation is valid if \((ac_n+bc_m)L_0(m,n,a,b)=0\), \(ac_n+bc_m-c_k=0,\, k=1,\ldots ,n+m\), that is, if \(L_0(m,n,a,b)=0\), \(\alpha =0\) and \(a+b=1\) since clearly from the definition (3), \(\beta \ne 0\). Under these conditions, we remain with

$$\begin{aligned} \frac{\beta x}{2}\sum _{k=0}^{n+m-1}&\Big (a^2L_k(n-1,m,a,1-a)+(1-a)^2L_k(n,m-1,a,1-a) \\&\quad -L_{k+1}(m,n,a,1-a) \Big )\theta _{k}(x;0,\beta )=0,\, \text { for all } x. \end{aligned}$$

Finally dividing the above equation by \(\beta x\) and equating the coefficients of \(\theta _{k}(x;0,\beta )\) yields:

Proposition 5

For \(n,m\ge 1\) and \(k=0,1,\ldots ,n+m-1\), the recurrence equation

$$\begin{aligned}&a^2L^{(\beta )}_k(m,n-1,a,1-a)+(1-a)^2L^{(\beta )}_k(m-1,n,a,1-a)\nonumber \\&\quad -L^{(\beta )}_{k+1}(m,n,a,1-a)=0, \end{aligned}$$
(18)

with \(L^{(\beta )}_0(m,n,a,1-a)=0\) is satisfied by the linearization coefficients \(L^{(\beta )}_k(n,m,a,1-a)\) of the linearization formula (13).

Remark 6

If the Bessel polynomials \(\theta _n(x;\alpha ,\beta )\) were orthogonal, this method for computing the recurrence relation of the linearization coefficients could be extended to the case \(a+b\ne 1\). In fact, Favard’s theorem [10] states that if they are orthogonal, they satisfy a three-term recurrence relation of the form

$$\begin{aligned} x\theta _n(x;\alpha ,\beta )=A_n\theta _{n+1}(x;\alpha ,\beta )+B_n\theta _n(x;\alpha ,\beta )+C_n\theta _{n-1}(x;\alpha ,\beta ), \end{aligned}$$
(19)

that we can use to substitute \(x\theta _k(x;\alpha ,\beta )\) in (17) and equate the coefficients of \(\theta _k(x;\alpha ,\beta )\) to get a mixed recurrence equation for the linearization coefficients.

If the recurrence equation (19) was valid, then to get the coefficients \(A_n, B_n, C_n\), we substitute \(\theta _n(x;\alpha ,\beta )=k_nx^n+k'_nx^{n-1}+k''_nx^{n-2}+\cdots \) in (19) and equate the coefficients of \(x^n, x^{n-1}, x^{n-2}\) to get \(A_n, B_n, C_n\). But it happens that this system has no solution meaning that such three-term recurrence relation doesn’t exist for the family \(\theta _k(x;\alpha ,\beta )\). Zeilberger’s algorithm (see e.g. [16, 21]) deals with sums of the form

$$\begin{aligned} S_n=\sum _{m=-\infty }^{\infty }A(n,m) \end{aligned}$$

and generates a homogeneous linear recurrence equation with polynomial coefficients for \(S_n\). Using Zeilberger’s algorithm implemented in Maple by the sumrecursion procedure of the hsum package [16], we find that the recurrence equation satisfied by \(\theta _n(x;\alpha ,\beta )\) is

$$\begin{aligned}&{x}^{2}n\beta \left( 2\,n+2+\alpha \right) \theta _{n-1}(x;\alpha ,\beta )\\&\quad =\left( 2\,n+\alpha \right) \left( n+ \alpha +1 \right) \beta \theta _{n+1}(x;\alpha ,\beta )\\&\qquad - \left( 2\,n+1+\alpha \right) \left( \alpha \,\beta \,x+ \left( 2\,n+2+ \alpha \right) \left( 2\,n+\alpha \right) \right) \theta _n(x;\alpha ,\beta ). \end{aligned}$$

This equation can also be obtained by substituting \(x\leftarrow \frac{2}{\beta x}\) in [17, Eq. (9.13.3), p. 245] and multiplying the resulting equation by \(x^{n+1}\).

Remark 7

With the help of a computer algebra system, extensive numerical investigations indicate that for all a and \(\beta \) in (13), the coefficients \(L^{(\beta )}_k(m,n,a,1-a)=0, 0\le k < \min (m,n), m,n\ge 1\).

Theorem 8

Let \(m,n\ge 1\). Then for \(0\le a \le 1\) and \(\beta >0, L^{(\beta )}_k(m,n,a,1-a)\ge 0, k=0,1,\ldots ,n+m\).

The proof of this theorem uses the following result.

Proposition 9

For \(k=0,1,\ldots ,n,\) the coefficients

$$\begin{aligned} D_k(n,a,\beta )=\frac{(-1)^n(-a)^k(-n,n+1)_{n-k}}{\beta ^{n-k}(n-k)!} _3 F_2\left( \! \left. \begin{array}{l} \begin{array}{c} -n+k,\, -1-k,\, 2+k \end{array}\\ \begin{array}{c} -2n+k,\, k+1 \end{array}\end{array} \! \right| \displaystyle {a}\right) , \end{aligned}$$

of the multiplication formula

$$\begin{aligned} \theta _n(ax;0,\beta )=\sum _{k=0}^n D_k(n,a,\beta ) \theta _k(x;0,\beta ) \end{aligned}$$

obtained from (12) by taking \(\alpha =0\) are nonnegative for \(0\le a\le 1\) and \(\beta >0\).

Proof

In fact for \(0\le a\le 1\) and \(\beta >0\),

$$\begin{aligned} \frac{(-1)^n(-a)^k(-n,n+1)_{n-k}}{\beta ^{n-k}(n-k)!}\ge 0,\quad k=0,1,\ldots ,n. \end{aligned}$$

It remains to prove that

$$\begin{aligned} _3 F_2\left( \! \left. \begin{array}{l} \begin{array}{c} -n+k,\, -1-k,\, 2+k \end{array}\\ \begin{array}{c} -2n+k,\, k+1 \end{array}\end{array} \! \right| \displaystyle {a}\right) \ge 0,\quad k=0,1,\ldots ,n, \, 0\le a\le 1. \end{aligned}$$

From the formula (see e.g. [22, Eq. (3), p. 388])

$$\begin{aligned} _3 F_2\left( \! \left. \begin{array}{l} \begin{array}{c} a_1,\, a_2,\, a_3 \end{array}\\ \begin{array}{c} b_1,\, b_2 \end{array}\end{array} \! \right| \displaystyle {\omega z}\right) =(1-z)^{-a_1}\sum _{j=0}^\infty \frac{(a_1)_j}{j!} _3 F_2\left( \! \left. \begin{array}{l} \begin{array}{c} -j,\, a_2,\, a_3 \end{array}\\ \begin{array}{c} b_1,\, b_2 \end{array}\end{array} \! \right| \displaystyle {\omega }\right) \left( \frac{z}{z-1} \right) ^j, \end{aligned}$$

we have for \(a_1=-k-1,\, a_2=-n+k,\, a_3=k+2,\, b_1=-2n+k,\, b_2=k+1,\ \omega =1,\, z=a\),

$$\begin{aligned}&_3 F_2\left( \! \left. \begin{array}{l} \begin{array}{c} -n+k,\, -1-k,\, 2+k \end{array}\\ \begin{array}{c} -2n+k,\, k+1 \end{array}\end{array} \! \right| \displaystyle {a}\right) \\&\quad =(1-a)^{k+1} \sum _{j=0}^{k+1}\left( {\begin{array}{c}k+1\\ j\end{array}}\right) _3 F_2\left( \! \left. \begin{array}{l} \begin{array}{c} -j,\, -n+k,\, k+2 \end{array}\\ \begin{array}{c} -2n+k,\, k+1 \end{array}\end{array} \! \right| \displaystyle {1}\right) \left( \frac{a}{1-a} \right) ^j. \end{aligned}$$

Note that the sum over j is now from 0 to \(k+1\) since \((-k-1)_j=0\) when \(j>k+1\). From the following formula (see e.g. [22, Eq. (82), p. 539])

$$\begin{aligned} _3 F_2\left( \! \left. \begin{array}{l} \begin{array}{c} -n,\, b,\, c \end{array}\\ \begin{array}{c} d,\, e \end{array}\end{array} \! \right| \displaystyle {1}\right)&=\frac{(b)_n(d+e-b-c)_n}{(d)_n(e)_n}\\&\quad \times _3 F_2\left( \! \left. \begin{array}{l} \begin{array}{c} -n,\, d-b,\, e-b \end{array}\\ \begin{array}{c} -n-b+1,\, d+e-b-c \end{array}\end{array} \! \right| \displaystyle {1}\right) , \end{aligned}$$

for \(n=j, b=k+2, c=-n+k, d=-2n+k, e=k+1\), it follows that

$$\begin{aligned}&_3 F_2\left( \! \left. \begin{array}{l} \begin{array}{c} -j,\, -n+k,\, k+2 \end{array}\\ \begin{array}{c} -2n+k,\, k+1 \end{array}\end{array} \! \right| \displaystyle {1}\right) \\&\quad =\frac{(k+2)_j(-n-1)_j}{(-2n+k)_j(k+1)_j} _3 F_2\left( \! \left. \begin{array}{l} \begin{array}{c} -1,\, -j,\, -2n-2 \end{array}\\ \begin{array}{c} -n-1,\,-j-k-1 \end{array}\end{array} \! \right| \displaystyle {1}\right) . \end{aligned}$$

This sum contains only two summands and simplifies to

$$\begin{aligned} _3 F_2\left( \! \left. \begin{array}{l} \begin{array}{c} -j,\, -n+k,\, k+2 \end{array}\\ \begin{array}{c} -2n+k,\, k+1 \end{array}\end{array} \! \right| \displaystyle {1}\right) =\frac{(k+1-j)\left( {\begin{array}{c}n+1\\ j\end{array}}\right) }{(k+1)\left( {\begin{array}{c}2n-k\\ j\end{array}}\right) }. \end{aligned}$$

We therefore deduce that

$$\begin{aligned}&_3 F_2\left( \! \left. \begin{array}{l} \begin{array}{c} -n+k,\, -1-k,\, 2+k \end{array}\\ \begin{array}{c} -2n+k,\, k+1 \end{array}\end{array} \! \right| \displaystyle {a}\right) \\&\quad =(1-a)^{k+1}\sum _{j=0}^{k+1}\left( {\begin{array}{c}k+1\\ j\end{array}}\right) \frac{(k+1-j)\left( {\begin{array}{c}n+1\\ j\end{array}}\right) }{(k+1)\left( {\begin{array}{c}2n-k\\ j\end{array}}\right) } \left( \frac{a}{1-a} \right) ^j, \end{aligned}$$

which is clearly nonnegative for \(0\le a\le 1\). \(\square \)

Proof of Theorem 8

Let \(0\le a\le 1\) and \(\beta >0\). The nonnegativity of the coefficients \(L^{(\beta )}_k(m,n,a,1-a)\) follows by induction (on m and n) from the nonnegativity of \(D_k(n,a,\beta )=L^{(\beta )}_k(0,n,a,1-a)\), \(D_k(m,1-a,\beta )=L^{(\beta )}_k(m,0,a,1-a)\) and the recurrence relation (18). \(\square \)

Proposition 10

The linearization coefficients \(L_k^{(\alpha )}(m,n,a,b)\) of the linearization formula (14) are solution of the mixed recurrence equation

$$\begin{aligned}&\frac{2\, \left( k+1 \right) \left( \alpha -k+m+n \right) }{ \left( 2\,k+2+\alpha \right) \left( \alpha +2\,k+3 \right) } L_{{k+1}}^{(\alpha )} \left( m,n,a,b\right) \\&\quad +\frac{2(n+\alpha +1)}{a(2n+\alpha +2)}L_{k}^{(\alpha )} \left( m,n+1,a,b\right) \\&\quad +\frac{2(m+\alpha +1)}{b(2m+\alpha +2)}L_{k}^{(\alpha )} \left( m+1,n,a,b\right) \\&\quad -{\frac{ 2\left( k+\alpha \right) \left( 2\,\alpha +k+m+1+n \right) }{ \left( \alpha +2\,k \right) \left( \alpha +2\,k-1 \right) }}L_{k-1}^{(\alpha )} \left( m,n,a,b\right) \\&\quad +\left( \left( \left( {\frac{ 2\left( \alpha -k+m+n+1\right) \alpha }{ \left( 2\,k+2+\alpha \right) \left( \alpha +2\,k \right) }}+{\frac{2\,(k+\alpha +1)}{2\,k+2+\alpha }} \right) b-{\frac{2\,(m+\alpha +1)}{2\,m+2+\alpha }} \right) a\right. \\&\qquad \left. -{\frac{ 2\left( n+\alpha +1 \right) b}{2\,n+2+\alpha }} \right) \frac{L_{k}^{(\alpha )} \left( m,n,a,b \right) }{ab}=0. \end{aligned}$$

Proof

Differentiate the both sides of (14) and multiply the result by \(x^2\) to use the structure relation (16). Rewrite the output with the help of (14) and use the three-term recurrence equation [17, Eq. (9.13.3)]

$$\begin{aligned} xy_n(x;\alpha )= & {} \frac{2(n+\alpha +1)}{(2n+\alpha +1)_2}y_{n+1}(x;\alpha )\\&\quad -\frac{1}{(2n+\alpha )(2n+\alpha +2)}\left( 2\alpha y_n(x;\alpha )+2ny_{n-1}(x;\alpha )\right) . \end{aligned}$$

Equating the coefficients of \(y_k(x;\alpha )\) leads to the result. \(\square \)

Remark 6 may explain why some researchers are interested by the case \(b=1-a\). The special case \(\alpha =\lambda =\mu =0\) and \(\beta =\delta =\gamma =2\) leads to the Bessel polynomials \(q_n(x)\) defined in (9) and Eq. (11) becomes (see [4])

$$\begin{aligned} q_n(ax)q_m(bx)=\sum _{k=0}^{n+m}L_k(m,n,a,b)q_k(x), \end{aligned}$$
(20)

with

$$\begin{aligned} L_k(m,n,a,b) =\frac{n!m!(2k)!}{(2n)!(2m)!k!} \sum _{l=0}^{\min (k+1,n+m-k)}\frac{(l+k+1)(-l-k)_k(-k)_{l-1}}{(-1)^{n+m-l+1}k!}\\ \times \sum _{i=\max (0,l+k-m)}^{\min (n,l+k)}\frac{a^i(-n,n+1)_{n-i}(-m,m+1)_{m+i-l-k}}{b^{i-l-k}(n-i)!(m+i-l-k)!}. \end{aligned}$$

For \(b=1-a\), that is, for \(0\le a \le 1\) and \(\beta >0\), since \(L^{(\beta )}_k(m,n,a,1-a)\ge 0\) , we deduce for \(\beta =2\) from the relation \(L_k(m,n,a,1-a) =\frac{2^{n+m}n!m!(2k)!}{(2n)!(2m)!2^kk!} L^{(2)}_k(m,n,a,1-a)\) that \(L_k(m,n,a,1-a)\ge 0\) for \(0\le a \le 1\) and \(L_k(m,n,a,1-a) = 0\) for \(k<\min (m,n)\), which are the results obtained by Berg and Vignat [5]. Equation (20) is therefore a generalization of their linearization formula for all \(a>0\) and \(b>0\) and Theorem 8 a more general result of the nonnegativity of the linearization coefficients for all \(\beta >0\).

Berg and Vignat [5] wrote that they “were unable to derive the explicit expression of the linearization coefficients \(L_k(m,n,a,1-a)\)” of the polynomial family \((q_n(x))_n\) in (20). Nevertheless, they proved using a recursion formula for \(L_k(m,n,a,1-a)\) (see [5, Lemma 3.6], [3, Eq. (5)])

$$\begin{aligned} \frac{1}{2k+1}L_{k+1}(m,n,a,1-a)=&\frac{a^2}{2n-1}L_k(m,n-1,a,1-a)\nonumber \\&\quad +\frac{(1-a)^2}{2m-1}L_k(m-1,n,a,1-a), \end{aligned}$$
(21)

for \(k=0,1,\ldots ,m+n-1\) with \(L_0(m,n,a,1-a)=0\), \(L_{n+m}(m,n,a,1-a)=a^n(1-a)^m\), that these coefficients are nonnegative for \(0\le a\le 1\) and \(L_k(m,n,a,1-a) = 0\) if \(k<\min (m,n)\). Using this nonnegativity, they deduced that the distribution of a finite convex combination of independent Student t-random variables with arbitrary odd degrees of freedom has a density which is a finite convex combination of certain Student t-densities with odd degrees of freedom. Atia and Zeng [3] were the first to derive a double sum formula for \(L_k(m,n,a,1-a)\) that they simplified to the single sum

$$\begin{aligned}&L_k(m,n,a,1-a) \nonumber \\&\quad =a^{2n+m-k}(1-a)^{-n+k}\frac{(1/2)_{n+m-k}(1/2)_k}{(1/2)_n(1/2)_m} \sum _{j=0}^{2(n+m-k)}(-1)^j\nonumber \\&\qquad \times \; \left( {\begin{array}{c}n+m+1\\ 2n+2m-2k-j\end{array}}\right) \left( {\begin{array}{c}-m+k+j\\ j\end{array}}\right) a^{-j}. \end{aligned}$$
(22)

Note that the explicit expression of \(L_k(m,n,a,b)\) in (20) generalizes the result by Atia and Zeng and in the case \(b=1-a\), it reduces to

$$\begin{aligned}&L_k(m,n,a,1-a) \nonumber \\&\quad =\frac{n!m!(2k)!}{(2n)!(2m)!k!} \sum _{l=0}^{n+m-k}\frac{(l+k+1)(-l-k)_k(-k)_{l-1}}{(-1)^{n+m-l+1}k!} \nonumber \\&\qquad \times \sum _{i=\max (0,l+k-m)}^{\min (n,l+k)}\frac{a^i(-n,n+1)_{n-i}(-m,m+1)_{m+i-l-k}}{(1-a)^{i-l-k}(n-i)!(m+i-l-k)!}. \end{aligned}$$
(23)

By direct computation, we get as in [3, p. 216]

$$\begin{aligned} L_{n+m}(m,n,a,1-a)&=a^n(1-a)^m\frac{(1/2)_{n+m}}{(1/2)_n(1/2)_m}, \end{aligned}$$
(24)
$$\begin{aligned} L_{n+m-1}(m,n,a,1-a)&=a^{n+1}(1-a)^{m-1}\frac{(1/2)(1/2)_{n+m-1}}{(1/2)_n(1/2)_m}\left( \left( {\begin{array}{c}n+m+1\\ 2\end{array}}\right) \right. \nonumber \\&\quad \left. -\left( {\begin{array}{c}n+m+1\\ 1\end{array}}\right) \left( {\begin{array}{c}n\\ 1\end{array}}\right) a^{-1}+\left( {\begin{array}{c}n+1\\ 2\end{array}}\right) a^{-2}\right) ,\end{aligned}$$
(25)
$$\begin{aligned} L_{n+m-2}(m,n,a,1-a)&=a^{n+2}(1-a)^{m-2}\frac{(1/2)_2(1/2)_{n+m-2}}{(1/2)_n(1/2)_m}\left( \left( {\begin{array}{c}n+m+1\\ 4\end{array}}\right) \right. \nonumber \\&\quad \left. -\left( {\begin{array}{c}n+m+1\\ 3\end{array}}\right) \left( {\begin{array}{c}n-1\\ 1\end{array}}\right) a^{-1}+ \left( {\begin{array}{c}n+m+1\\ 2\end{array}}\right) \left( {\begin{array}{c}n\\ 2\end{array}}\right) a^{-2}\right. \nonumber \\&\quad \left. -\left( {\begin{array}{c}n+m+1\\ 1\end{array}}\right) \left( {\begin{array}{c}n+1\\ 3\end{array}}\right) a^{-3}+ \left( {\begin{array}{c}n+2\\ 4\end{array}}\right) a^{-4}\right) . \end{aligned}$$
(26)

We make a conjecture after further computation that (23) is equivalent to (22). In fact, from the relation \( L_k(m,n,a,1-a) =\frac{2^{n+m}n!m!(2k)!}{(2n)!(2m)!2^kk!} L^{(2)}_k(m,n,a,1-a)\) and the recurrence relation (18), it follows that \(L_k(m,n,a,1-a)\) is solution of the recurrence relation (21). Since (22) and (23) are solution of the same recurrence equation (21) with the same initial conditions, it follows that they are equivalent.

One question arise at this point: Is-it also possible to reduce the linearization coefficients \(L^{(\beta )}_k(m,n,a,1-a)\) given in (13) as double sum into a single sum? In this direction, we get

$$\begin{aligned}&L^{(\beta )}_{n+m}(m,n,a,1-a)={\frac{ \left( 2\,m \right) !\, \left( 2\,n \right) !\, \left( n+m \right) !}{m!\,n!\, \left( 2\,n+2\,m \right) !}}L_{n+m}(m,n,a,1-a) , \\&L^{(\beta )}_{n+m-1}(m,n,a,1-a)={\frac{ \left( 2\,m \right) !\, \left( 2\,n \right) !\, \left( n+m-1 \right) !}{m!\,n!\, \left( 2\,n+2\,m-2 \right) !\,\beta }}L_{n+m-1}(m,n,a,1-a),\\&L^{(\beta )}_{n+m-2}(m,n,a,1-a)={\frac{ \left( 2\,m \right) !\, \left( 2\,n \right) !\, \left( n+m-2 \right) !}{m!\,n!\, \left( 2\,n+2\,m-4 \right) !\,{\beta }^{2}}}L_{n+m-2}(m,n,a,1-a), \end{aligned}$$

with \(L_{n+m}(m,n,a,1-a)\), \(L_{n+m-1}(m,n,a,1-a)\), \(L_{n+m-2}(m,n,a,1-a)\) given, respectively, by (24)–(26). It follows from (11) that

$$\begin{aligned} \theta _n(ax;0,\beta )\theta _m(bx;0,\beta )=\sum _{k=0}^{n+m}L^{(\beta )}_k(m,n,a,b)\theta _k(x;0,\beta ), \end{aligned}$$
(27)

with

$$\begin{aligned} L^{(\beta )}_k(m,n,a,b)&=\sum _{l=0}^{\min (k+1,n+m-k)}\frac{(l+k+1)(-l-k)_k(-k)_{ l-1}}{(-1)^{n+m-l+1}\beta ^{n+m-k}k!} \\&\quad \times \sum _{i=\max (0,l+k-m)}^{\min (n,l+k)}\frac{a^i(-n,n+1)_{n-i}(-m,m+1)_{m+i-l-k}}{b^{i-l-k} (n-i)!(m+i-l-k)!}. \end{aligned}$$

The latter expression and Eq. (22) lead to

$$\begin{aligned} L^{(\beta )}_{k}(m,n,a,b)=\frac{(2m)!(2n)!k!}{m!n!(2k)!\beta ^{n+m-k}}L_{k}(m,n,a,b), \end{aligned}$$
(28)

from which we deduce for \(b=1-a\)

$$\begin{aligned} L^{(\beta )}_{k}(m,n,a,1-a)=&\frac{(2m)!(2n)!k!}{m!n!(2k)!\beta ^{n+m-k}}L_{k}(m,n,a,1-a)\nonumber \\&=\frac{(2m)!(2n)!k!a^{2n+m-k}(1/2)_{n+m-k}(1/2)_k}{m!n!(2k)!\beta ^{n+m-k}(1-a)^{n-k}(1/2)_n(1/2)_m}\nonumber \\&\quad \times \sum _{j=0}^{2(n+m-k)}(-1)^j\left( {\begin{array}{c}n+m+1\\ 2n+2m-2k-j\end{array}}\right) \left( {\begin{array}{c}-m+k+j\\ j\end{array}}\right) a^{-j}, \end{aligned}$$

which is the single sum expression of \(L^{(\beta )}_{k}(m,n,a,1-a)\).

In the recent manuscript [4], Benabdallah and Atia derived the linearization formula (20) and wrote the linearization coefficients \(L_k(m,n,a,b)\) as a triple sum from which they deduce the following positivity result.

Corollary 11

(see [4]) For \(0\le k\le n+m\) and \(a+b\ne 0\), the linearization coefficients \(L_k(m,n,a,b)\) of (20) are positive if and only if \(0\le a\le a+b\le 1\).

From the above corollary and Relation (28), we deduce that

Corollary 12

For \(0\le k\le n+m\), \(a+b\ne 0\) and \(\beta >0\), the linearization coefficients \(L^{(\beta )}_{k}(m,n,a,b)\) of the linearization formula (27) are positive if and only if \(0\le a\le a+b\le 1\).