1 Introduction

A ring R is said to be prime if for any \(a,b\in R\), \(aRb= \{0\}\) implies either \(a =0\) or \( b = 0\) and is said to be semiprime if for any \(a\in R\), \(aRa=\{0\}\) implies \(a=0.\) Let Z(R) denote the center of R and U be the Utumi ring of quotients of R and \(C=Z(U)\). The symbols [xy] denote the Lie commutator \(xy-yx\) for any \(x, y \in R\). By a derivation, we mean an additive mapping \(d: R\rightarrow R\) such that \(d(xy)=d(x)y+xd(y)\) for all x, \(y \in R\).

Several authors found a number of results investigating the relationship between the behaviour of additive mappings defined on a prime (or semiprime) ring R and the structure of R. Posner [17] proved that if R is a prime ring and d a nonzero derivation on R such that \([d(r),r]\in Z(R)\), then R is commutative. Several authors have generalized the Posner’s result.

Lee and Lee in [13] proved that if \([d(f(x_1, \ldots , x_n)), f(x_1, \ldots , x_n)]_k =0\) for all \(x_1, \ldots , x_n \) in some nonzero ideal of R, then \(f(x_1, \ldots , x_n)\) is central-valued on R, except when char(\(R)=2\) and R satisfies \(s_4(x_1,x_2,x_3,x_4)\), the standard identity in four variables. Later on, De Filippis and Di Vincenzo [5] considered the situation \(\delta ([d(f(x_1,\ldots ,x_n)),f(x_1,\ldots ,x_n)])=0\) for all \(x_1,\ldots ,x_n\in R\), where d and \(\delta \) are two derivations of R. The statement of De Filippis and Di Vincenzo’s theorem is the following:

Let K be a noncommutative ring with unity, R a prime K-algebra of characteristic different from 2, d and \(\delta \) nonzero derivations of R,  and \(f(x_1,\ldots ,x_n)\) a multilinear polynomial over K. If \(\delta ([d(f(x_1,\ldots ,x_n)),f(x_1,\ldots ,x_n)])=0\) for all \(x_1,\ldots ,x_n\in R\), then \(f(x_1,\ldots ,x_n)\) is central-valued on R.

It is natural to consider the situation when derivation \(\delta \) is replaced by \(\delta ^2\), that is, \(\delta ^2([d(f(x_1,\ldots ,x_n)),f(x_1,\ldots ,x_n)])=0\) for all \(x_1,\ldots ,x_n\in R\). In the present paper, we investigate a more general case replacing \(\delta ^2\) with \(F^2\), where F is a generalized derivation of R.

On the other hand, Dhara [7] studied \([d^2(f(x_1,\ldots ,x_n)),f(x_1,\ldots ,x_n)]=0\) for all \(x_1,\ldots ,x_n\in \rho \) in prime ring R, where d is a derivation of R and \(\rho \) is a nonzero right ideal of R.

We will continue the study of analogue problems involving generalized derivations on the appropriate subsets of prime rings. An additive mapping \(F: R\rightarrow R\) associated with a derivation d on R such that \(F(xy)=F(x)y+xd(y)\) for all \(x, y \in R\), is said to be generalized derivation. For some fixed \(a, b \in U,\) an additive mapping \(F:R \rightarrow R\) defined as \(F(x)=ax+xb\) for all \(x \in R\) is an example of generalized derivation. In [2], the following result was obtained:

Let R be a prime ring of characteristic different from 2 with extended centroid C, \(f(x_1,\ldots ,x_n)\) be a multilinear polynomial over C, which is not central valued on R. If d is a derivation of R,  and F is a generalized derivation of R such that \(F([d(f(x_1,\ldots ,x_n)), f(x_1,\ldots ,x_n)])=0\) for all \(x_1,\ldots ,x_n\in R\), then either \(F=0\) or \(d=0\).

In this line of investigation, in [4], De Filippis and Di Vincenzo proved the following:

Let R be a prime algebra over a commutative ring K with unity, and \(f(x_1,\ldots , x_n)\) be a multilinear polynomial over K, not central valued on R. Suppose that d is a nonzero derivation of R,  and F is a nonzero generalized derivation of R such that \(d([F(f(r_1,\ldots ,r_n)), f(r_1,\ldots ,r_n )])=0\) for all \(r_1,\ldots , r_n\in R\). If the characteristic of R is different from 2, then one of the following holds:

  1. (1)

    there exists \(\lambda \in C\), the extended centroid of R, such that \(F(x) = \lambda x\), for all \(x\in R\);

  2. (2)

    there exists \(a\in U\), the Utumi quotient ring of R, and \(\lambda \in C = Z(U)\) such that \(F(x) = ax + xa + \lambda x\) for all \(x\in R\), and \(f(x_1,\ldots , x_n)^2\) is central-valued on R.

Furthermore, Tiwari et al. [18] investigated \(d([F^2(f(r_1,\ldots ,r_n)), f(r_1,\ldots ,r_n )])=0\) for all \(r_1,\ldots , r_n\in R\), where d is a nonzero derivation of R, and F is a generalized derivation of R.

In the present paper, we prove the following:

Main Theorem

Let R be a prime ring of characteristic different from 2 with Utumi quotient ring U and \(f(x_1,\ldots ,x_n)\) a non-central multilinear polynomial over the extended centroid C. If d is a nonzero derivation of R and F is a generalized derivation of R such that

$$\begin{aligned} F^2([d(f(x_1, \ldots , x_n)), f(x_1, \ldots , x_n)])=0 \end{aligned}$$

for all \(x_1, \ldots , x_n \in R\), then there exists \(a\in U\) with \(a^2=0\) such that \(F(x)=xa\) for all \(x\in R\) or \(F(x)=ax\) for all \(x\in R\).

Here we give an example which shows that in our result, the primeness of the ring is essential.

Example

Define \(R=\Bigg \{\left( \begin{array}{c@{\quad }c} x &{} y \\ 0 &{} 0 \\ \end{array} \right) : x,y\in \mathbb {Z} \Bigg \}\) and a multilinear polynomial \(f(r,s)=rs\). We see that R is a ring under usual operations and f(rs) is not central valued on R. Also, note that R is not a prime ring. Now we define maps \(d,F,g: R \rightarrow R\) such that \(d\left( \begin{array}{c@{\quad }c} x &{} y \\ 0 &{} 0 \\ \end{array}\right) =\left( \begin{array}{c@{\quad }c} 0 &{} y \\ 0 &{} 0 \\ \end{array}\right) \), \(F\left( \begin{array}{c@{\quad }c} x &{} y \\ 0 &{} 0 \\ \end{array}\right) =\left( \begin{array}{c@{\quad }c} x &{} 0 \\ 0 &{} 0 \\ \end{array}\right) \) and \(g\left( \begin{array}{c@{\quad }c} x &{} y \\ 0 &{} 0 \\ \end{array}\right) =\left( \begin{array}{c@{\quad }c} 0 &{} -y \\ 0 &{} 0 \\ \end{array} \right) \). Notice that d is a nonzero derivation on R and F is a generalized derivation associated to the derivation g on R. It can easily be seen that \(F^2([d(f(r,s)), f(r,s)])=0\) for all \(r,s\in R\). Thus R satisfies the hypothesis of the main theorem. However, the conclusion of the main theorem does not hold as g is a nonzero derivation of R.

2 Preliminaries

In what follows, R always denotes a prime ring and U denotes the Utumi ring of quotients of R. \(f(x_1,\ldots ,x_n)\) denotes the multilinear polynomial over C which is in the form

$$\begin{aligned} f(x_1,\ldots ,x_n)=x_1x_2\cdots x_n+\sum \limits _{\sigma \in S_n, \sigma \ne id} \alpha _\sigma x_{\sigma (1)}x_{\sigma (2)} \ldots x_{\sigma (n)}, \end{aligned}$$

for some \(\alpha _\sigma \in C\) and \(S_n\) the symmetric group of degree n.

The definition and axiomatic formulation of Utumi quotient ring U can be found in [1] and [3].

We have the following properties which we need:

  1. (1)

    \(R\subseteq U\);

  2. (2)

    U is a prime ring with identity;

  3. (3)

    The center of U is denoted by C and is called the extended centroid of R. C is a field.

Moreover, we recall some known facts.

Fact 1. Let \(\mathcal {K}\) be an algebra over a field \(\mathbb {E}.\) A generalized polynomial identity (GPI) of \(\mathcal {K}\) is a polynomial expression g in non commutative indeterminates and fixed coefficients from \(\mathcal {K}\) between the indeterminates such that g vanishes on all replacements by elements of \(\mathcal {K}\). The generalized polynomial in the context of Utumi quotient ring U is defined as follows:

Suppose that V is a set of C-independent vectors of U and \(Y=\{y_1,y_2,y_3, \ldots \}\) is a countable set, where \(y_i\) are non commuting indeterminates. Let \(C\langle Y \rangle \) be the free algebra over C in the set Y. Consider \(\mathcal {W}= U_{^{*}C} C\langle Y \rangle , \) the free product of U and \(C\langle Y \rangle \) over C. The elements of \(\mathcal {W}\) are called generalized polynomials. An element \(h \in \mathcal {W}\) of the form \(h = s_0x_1s_1x_2s_2 \ldots x_ns_n,\) where \(\{s_0, \ldots ,s_n\} \subseteq U\) and \(\{x_1, \ldots ,x_n\} \subseteq Y\) is said to be a monomial. Therefore, each \(g\in \mathcal {W}\) can be represented as a finite sum of monomials. A V-monomial is of the form \(e = v_0x_1v_1x_2v_2 \ldots x_nv_n,\) where \(\{v_0, \ldots ,v_n\} \subseteq V\) and \(\{x_1, \ldots ,x_n\} \subseteq Y.\) Thus an element \(g \in \mathcal {W}\) can be written as \(g = \sum _{i} \beta _{i}e_{i},\) where \(\beta _{i} \in C\) and \(e_{i}\) are V-monomials. An element \(g \in \mathcal {W}\) is trivial if and only if \(\beta _{i}=0\) for each i. For more details, we refer the reader to [1, 3].

Fact 2. If I is a two-sided ideal of R, then R, I and U satisfy the same generalized polynomial identities (GPIs) with coefficients in U (see [3]).

Fact 3. Every derivation d of R can be uniquely extended to a derivation of U (see Proposition 2.5.1 in [1]).

Fact 4. If I is a two-sided ideal of R, then R, I and U satisfy the same differential identities (see [14]).

Fact 5. Let d be a derivation on R. By \(f^d(x_1,\ldots ,x_n)\), \(f^{d^2}(x_1,\ldots ,x_n)\) and \(f^{d^3}(x_1,\ldots ,x_n)\), we denote the polynomials obtained from \(f(x_1,\ldots ,x_n)\) by replacing each coefficient \(\alpha _\sigma \) with \(d(\alpha _\sigma )\), \(d^2(\alpha _\sigma )\) and \(d^3(\alpha _\sigma )\), respectively. Then we have

$$\begin{aligned} d(f(x_1,\ldots ,x_n))=&f^d(x_1,\ldots ,x_n)+\sum \limits _i f(x_1,\ldots ,d(x_i),\ldots ,x_n), \\ d^2(f(x_1,\ldots ,x_n))=&f^{d^2}(x_1,\ldots ,x_n)+2\sum \limits _i f^d(x_1,\ldots ,d(x_i),\ldots ,x_n)\\&+\sum \limits _i f(x_1,\ldots ,d^2(x_i),\ldots ,x_n) \\&+\sum \limits _{i\ne j} f(x_1, \ldots , d(x_i), \ldots , d(x_j), \ldots , x_n) \end{aligned}$$

and

$$\begin{aligned} d^3(f(x_1,\ldots ,x_n))&=f^{d^3}(x_1,\ldots ,x_n) + 3\sum \limits _i f^{d^2}(x_1,\ldots ,d(x_i),\ldots ,x_n)\\&\quad +\,3\sum \limits _i f^{d}(x_1,\ldots ,d(x_i),\ldots ,d(x_j),\ldots ,x_n)\\&\quad +\, 3\sum \limits _i f^{d}(x_1,\ldots ,d^2(x_i),\ldots ,x_n)\\&\quad +\,\sum \limits _{i\ne j\ne k} f(x_1,\ldots ,d(x_i),\ldots ,d(x_j),\ldots ,d(x_k),\ldots ,x_n)\\&\quad +\,2 \sum \limits _{i\ne j} f(x_1,\ldots ,d^2(x_i),\ldots ,d(x_j),\ldots ,x_n)\\&\quad +\, 2\sum \limits _{i\ne j} f(x_1,\ldots ,d(x_i),\ldots ,d^2(x_j),\ldots ,x_n)\\&\quad +\,\sum \limits _i f(x_1,\ldots ,d^3(x_i),\ldots ,x_n). \end{aligned}$$

3 The case when F is inner

In this section, we study all the possible situation of annihilating condition of the set \(\{ [d(x),x] | x\in f(R)\}\), where d is a derivation of R. For any subset S of R, denote by \(r_R (S)\) the right annihilator of S in R, that is, \(r_R (S) = \{x \in R |Sx = 0\}\) and \(l_R (S)\) the left annihilator of S in R,  that is, \(l_R (S) = \{x \in R | x S = 0\}\). If \(r_R (S) = l_R (S)\), then \(r_R (S)\) is called an annihilator ideal of R and is written as \(\mathrm{ann}_R (S)\).

In [6], De Filippis and Di Vincenzo studied the left annihilating condition of the set \(\{ [d(x),x] | x\in f(R)\}\). More precisely, they proved that if R is a prime ring of \(\mathrm{char}(R)\ne 2\) and d is a nonzero derivation of R satisfying \(a[d(x),x]=0\) for all \(x\in f(R)\), then \(a=0\).

Now we will study a more general situation, involving left sided, right sided as well as two-sided annihilating conditions. More specifically, we study the situation \(b[d(x),x]+p[d(x),x]q+[d(x),x]c=0\) for all \(x\in f(R)\), where \(b,c,p,q\in R\).

First we consider that d is an inner derivation of R, that is, \(d(x)=[a,x]\) for all \(x\in R\). Then

$$\begin{aligned} b[d(f(r)),f(r)]+p[d(f(r)),f(r)]q+[d(f(r)),f(r)]c=0 \end{aligned}$$

gives

$$\begin{aligned}&b\big (af(r)^2 - 2f(r)af(r) + f(r)^2a) + p(af(r)^2 \\&\quad -\,2f(r)af(r)+f(r)^2a\big )q\\&\quad +\,\big (af(r)^2 - 2f(r)af(r) + f(r)^2a\big )c=0, \end{aligned}$$

that is,

$$\begin{aligned}&baf(r)^2 - 2bf(r)af(r) + bf(r)^2a + paf(r)^2q \\&\quad -\,2pf(r)af(r)q+pf(r)^2aq \\&\quad +\,af(r)^2c-2f(r)af(r)c+f(r)^2ac=0 \end{aligned}$$

for any \(r=(r_1,\ldots ,r_n) \in R^n\). We rewrite it as

$$\begin{aligned}&a_1f(r)^2 - 2a_2f(r)a_3f(r) + a_2f(r)^2a_3 + a_4f(r)^2a_5 \\&\quad -\, 2a_6f(r)a_3f(r)a_5 + a_6f(r)^2a_7\\&\quad +\, a_3f(r)^2a_8 - 2f(r)a_3f(r)a_8 + f(r)^2a_9=0 \end{aligned}$$

for any \(r=(r_1,\ldots ,r_n) \in R^n\), where \(a_1=ba, a_2=b, a_3=a, a_4=pa, a_5=q, a_6=p, a_7=aq, a_8=c, a_9=ac\). Now we study this situation in a matrix ring.

We need the following:

Lemma 3.1

[4, Lemma 1]. Let F be an infinite field and \(k \ge 2\). If \(A_1, \ldots , A_n\) are not scalar matrices in \(M_k(F)\) then there exists some invertible matrix \(P \in M_k(F)\) such that any matrices \(PA_1P^{-1}, \ldots , PA_nP^{-1}\) have all non-zero entries.

PROPOSITION 3.2

Let \(R=M_k(F)\) be the ring of all \(k \times k\) matrices over the infinite field F, \(f(x_1, \ldots , x_n)\) a non-central multilinear polynomial over F and \(a_1, a_2, \ldots , a_9 \in R\). If

$$\begin{aligned}&a_1f(r)^2 - 2a_2f(r)a_3f(r) + a_2f(r)^2a_3 + a_4f(r)^2a_5 \\&\quad -\, 2a_6f(r)a_3f(r)a_5 + a_6f(r)^2a_7\\&\quad +\, a_3f(r)^2a_8 - 2f(r)a_3f(r)a_8 + f(r)^2a_9=0 \end{aligned}$$

for all \(r=(r_1, \ldots , r_n)\in R^n\), then either \(a_3\) or \(a_5\) or \(a_6\) is central.

Proof

By the hypothesis, we have

$$\begin{aligned} \begin{aligned}&a_1f(r_1, \ldots , r_n)^2-2a_2f(r_1, \ldots , r_n)a_3f(r_1, \ldots , r_n)+a_2f(r_1, \ldots , r_n)^2a_3\\&\quad +\,a_4f(r_1, \ldots , r_n)^2a_5-2a_6f(r_1, \ldots , r_n)a_3f(r_1, \ldots , r_n)a_5 \\&\quad +\,a_6f(r_1, \ldots , r_n)^2a_7\\&\quad +\,a_3f(r_1, \ldots , r_n)^2a_8-2f(r_1, \ldots , r_n)a_3f(r_1, \ldots , r_n)a_8 \\&\quad +\,f(r_1, \ldots , r_n)^2a_9=0. \end{aligned} \end{aligned}$$

Suppose that \(a_3\notin Z(R)\), \(a_5\notin Z(R)\) and \(a_6\notin Z(R)\). Then we shall prove that this case leads to a contradiction.

Since \(a_3\notin Z(R)\), \(a_5\notin Z(R)\) and \(a_6\notin Z(R)\), by Lemma 3.1, there exists a F-automorphism \(\phi \) of \(M_k(F)\) such that \(\phi (a_3)\), \(\phi (a_5)\) and \(\phi (a_6)\) have all nonzero entries. Clearly, R satisfies the GPI,

$$\begin{aligned} \begin{aligned}&\phi (a_1)f(r_1, \ldots , r_n)^2-2\phi (a_2)f(r_1, \ldots , r_n)\phi (a_3)f(r_1, \ldots , r_n)\\&\quad +\,\phi (a_2)f(r_1, \ldots , r_n)^2\phi (a_3)+\phi (a_4)f(r_1, \ldots , r_n)^2\phi (a_5)\\&\quad -\,2\phi (a_6)f(r_1, \ldots , r_n)\phi (a_3)f(r_1, \ldots , r_n)\phi (a_5) \\&\quad +\,\phi (a_6)f(r_1, \ldots , r_n)^2\phi (a_7)\\&\quad +\,\phi (a_3)f(r_1, \ldots , r_n)^2\phi (a_8)-2f(r_1, \ldots , r_n)\phi (a_3)f(r_1, \ldots , r_n)\phi (a_8)\\&\quad +\,f(r_1, \ldots , r_n)^2\phi (a_9)=0. \end{aligned} \end{aligned}$$
(1)

As usual, by \(e_{ij}\), \(1\le i,j\le k\), we denote the matrix unit whose (ij)-entry is equal to 1 and all its other entries are equal to 0. Since \(f(x_1, \ldots , x_n)\) is non-central, by [13] (see also [15]), there exist \(s_1, \ldots , s_n \in M_k(F)\) and \(\beta \in F {\setminus } \{0\}\) satisfying \(f(s_1, \ldots , s_n)=\beta e_{\mathrm{st}}\) with \(s \ne t\). Moreover, since the set \(\{f(y_1, \ldots , y_n) : y_1, \ldots , y_n \in M_k(F)\}\) is invariant under the action of all F-automorphisms of \(M_k(F)\), for any \(i \ne j\), there exists \(u_1, \ldots , u_n \in M_k(F)\) such that \(f(u_1, \ldots , u_n)=e_{ij}\). Hence by (1) we have

$$\begin{aligned} \begin{aligned}&\phi (a_1)e_{ij}^2-2\phi (a_2)e_{ij}\phi (a_3)e_{ij}+\phi (a_2)e_{ij}^2\phi (a_3) +\phi (a_4)e_{ij}^2\phi (a_5)\\&\quad -\,2\phi (a_6)e_{ij}\phi (a_3)e_{ij}\phi (a_5)+\phi (a_6)e_{ij}^2\phi (a_7)\\&\quad +\,\phi (a_3)e_{ij}^2\phi (a_8)-2e_{ij}\phi (a_3)e_{ij}\phi (a_8)+e_{ij}^2\phi (a_9)=0. \end{aligned} \end{aligned}$$

Multiplying left side and right side by \(e_{ij}\), we obtain \(2e_{ij}\phi (a_6)e_{ij}\phi (a_3)e_{ij}\phi (a_5)e_{ij}=0\). Since char\((R)\ne 2\), we have \(\phi (a_6)_{ji}\phi (a_3)_{ji}\phi (a_5)_{ji}=0\). This is a contradiction as \(\phi (a_3)\), \(\phi (a_5)\) and \(\phi (a_6)\) have all nonzero entries. Thus we conclude that either \(a_3\) or \(a_5\) or \(a_6\) is central. \(\square \)

PROPOSITION 3.3

Let \(R=M_k(F)\) be the ring of all matrices over the field F with \(\mathrm{char} (R)\ne 2\), \(f(x_1, \ldots , x_n)\) a non-central multilinear polynomial over F and \(a_1, a_2, \ldots , a_9 \in R\). If

$$\begin{aligned}&a_1f(r)^2-2a_2f(r)a_3f(r)+a_2f(r)^2a_3+a_4f(r)^2a_5 \\&\quad -2a_6f(r)a_3f(r)a_5+a_6f(r)^2a_7\\&\quad +\,a_3f(r)^2a_8-2f(r)a_3f(r)a_8+f(r)^2a_9=0 \end{aligned}$$

for all \(r=(r_1, \ldots , r_n) \in R^n\), then either \(a_3\) or \(a_5\) or \(a_6\) is central.

Proof

If F is an infinite field, then by Proposition 3.2, we get the desired result. Next, we assume that F is finite.

Let E be an infinite field extension of the field F. Suppose that \(\bar{R} = M_{k}(E)\,\cong R\otimes _{F} E\). Note that the multilinear polynomial \(f(r_{1}, \ldots , r_{n})\) is central-valued on R if and only if it is central-valued on \(\bar{R}\). R satisfies the GPI,

$$\begin{aligned} \Psi (r_1,\ldots ,r_n)&= a_1f(r_1, \ldots , r_n)^2-2a_2f(r_1, \ldots , r_n)a_3f(r_1, \ldots , r_n)\\&\quad +\,a_2f(r_1, \ldots , r_n)^2a_3+a_4f(r_1, \ldots , r_n)^2a_5\\&\quad -\,2a_6f(r_1, \ldots , r_n)a_3f(r_1, \ldots , r_n)a_5+a_6f(r_1, \ldots , r_n)^2a_7\\&\quad +\,a_3f(r_1, \ldots , r_n)^2a_8-2f(r_1, \ldots , r_n)a_3f(r_1, \ldots , r_n)a_8\\&\quad +\,f(r_1, \ldots , r_n)^2a_9=0 \end{aligned}$$

which is multi-homogeneous of multi-degree \((2,\ldots , 2)\) in the indeterminates \(r_{1},\ldots , r_{n}\). Thus the complete linearization of \(\Psi (r_{1},\ldots , r_{n})\) is a multilinear generalized polynomial \(\Phi (r_{1},\ldots , r_{n},r_{1},\ldots , r_{n})\) in 2n indeterminates. Clearly, \(\Phi (r_{1},\ldots , r_{n},r_{1},\ldots , r_{n}) = 2^{n}\Psi (r_{1},\ldots ,r_{n})\).

Note that the multilinear polynomial \(\Phi (r_{1},\ldots , r_{n},r_{1},\ldots , r_{n})\) is a generalized polynomial identity for both R and \(\bar{R}\). Since char(\(F) \ne 2\), we obtain \(\Psi (r_{1},\ldots , r_{n})=0\) for all \(r_{1},\ldots , r_{n} \in \bar{R}\). Hence by Proposition 3.2, the proof of proposition follows. \(\square \)

Lemma 3.4

Let R be a prime ring of characteristic different from 2 with extended centroid C and \(f(x_1, \ldots , x_n)\) a non-central multilinear polynomial over C. Suppose that for some \(a_1, a_2, \ldots , a_9 \in R\),

$$\begin{aligned}&a_1f(r)^2 - 2a_2f(r)a_3f(r) + a_2f(r)^2a_3 + a_4f(r)^2a_5 \\&\quad -\, 2a_6f(r)a_3f(r)a_5 + a_6f(r)^2a_7\\&\quad +\,a_3f(r)^2a_8 - 2f(r)a_3f(r)a_8 + f(r)^2a_9=0 \end{aligned}$$

for all \(r=(r_1, \ldots , r_n)\in R^n\), then either \(a_3\) or \(a_5\) or \(a_6\) is central.

Proof

Since R satisfies the generalized polynomial identity (GPI),

$$\begin{aligned} \begin{aligned} g(x_1,\ldots ,x_n)&= a_1f(x_1,\ldots ,x_n)^2-2a_2f(x_1,\ldots ,x_n)a_3f(x_1,\ldots ,x_n)\\&+\,a_2f(x_1,\ldots ,x_n)^2a_3+a_4f(x_1,\ldots ,x_n)^2a_5\\&-\,2a_6f(x_1,\ldots ,x_n)a_3f(x_1,\ldots ,x_n)a_5+a_6f(x_1,\ldots ,x_n)^2a_7\\&+\,a_3f(x_1,\ldots ,x_n)^2a_8-2f(x_1,\ldots ,x_n)a_3f(x_1,\ldots ,x_n)a_8\\&+\,vf(x_1,\ldots ,x_n)^2a_9=0 \end{aligned} \end{aligned}$$
(2)

for all \(x_1, \ldots , x_n \in R\). Assume that \(a_3\notin C\), \(a_5 \notin C\) and \(a_6\notin C\). By Fact 2, R and U satisfy the same GPI, U satisfies \(g(x_1,\ldots ,x_n)=0.\) Suppose that \(g(x_1,\ldots ,x_n)\) is a trivial GPI for U. Let \(\mathcal {W}=U*_CC\{x_1,x_2,\ldots ,x_n\}\), the free product of U and \(C\{x_1,\ldots ,x_n\}\), the free C-algebra in noncommuting indeterminates \(x_1,x_2,\ldots , x_n\). So \(g(x_1,\ldots ,x_n)\) is a zero element in \(\mathcal {W}=U*_CC\{x_1,\ldots ,x_n\}\). In equation (2), the term \(-2a_6f(x_1,\ldots ,x_n)a_3f(x_1,\ldots ,x_n)a_5\) appears nontrivially, implying that

$$\begin{aligned} -2a_6f(x_1,\ldots ,x_n)a_3f(x_1,\ldots ,x_n)a_5=0\in \mathcal {W}. \end{aligned}$$

This implies that either \(a_3\) or \(a_5\) or \(a_6\) is central.

Now assume that \(g(x_1,\ldots ,x_n)\) is a non-trivial GPI for U. In case C is infinite, we have \(g(r_{1},\ldots , r_{n})= 0\) for all \(r_{1},\ldots , r_{n} \in U\otimes _{C} \bar{C}\), where \(\bar{C}\) is the algebraic closure of C. Moreover, both U and \(U\otimes _{C} \bar{C}\) are prime and centrally closed algebras [8]. Hence, we substitute U or \(U\otimes _{C} \bar{C}\) in place of R according to C finite or infinite respectively. Without loss of generality, we may suppose that \(C = Z(R)\) and R is a centrally closed C-algebra. Using Martindale’s theorem [16], R is then a primitive ring having nonzero Socle \(\mathrm{soc}(R)\) with C as the associated division ring. Hence by Jacobson’s theorem [10, p. 75], R is isomorphic to a dense ring of linear transformations of some vector space V over C.

First, suppose that V is finite dimensional over C, that is, \(\mathrm{dim}_CV = k\). By density of R, we have \(R\cong M_{k}(C)\). Since \(f(r_{1},\ldots , r_{n})\) is not central-valued on R, R must be noncommutative and so \(k \ge 2\). In this case, by Proposition 3.3, we get that either \(a_3\) or \(a_5\) or \(a_6\) is in C, a contradiction.

If V is infinite dimensional over C, then for any \(e^2=e\in \mathrm{soc}(R)\), we have \(eRe\cong M_t(C)\) with \(t\,{=}\,\)dim\(_CVe\). Since \(a_3\), \(a_5\) and \(a_6\) are not in C, there exist \(h_1,h_2,h_3\in \mathrm{soc}(R)\) such that \([a_3,h_1]\ne 0\), \([a_5,h_2]\ne 0\) and \([a_6,h_3]\ne 0\). By Litoff’s theorem [9], there exists idempotent \(e\in \mathrm{soc}(R)\) such that \(a_3h_1, h_1a_3, a_5h_2, h_2a_5, a_6h_3, h_3a_6, h_1, h_2,\)

\(h_3\in eRe\). Since R satisfies GPI, it follows that

$$\begin{aligned}&e\{a_1f(ex_1e,\ldots ,ex_ne)^2-2a_2f(ex_1e,\ldots ,ex_ne)a_3f(ex_1e,\ldots ,ex_ne) \\&+\,a_2f(ex_1e,\ldots ,ex_ne)^2a_3+a_4f(ex_1e,\ldots ,ex_ne)^2a_5 \\&-\,2a_6f(ex_1e,\ldots ,ex_ne)a_3f(ex_1e,\ldots ,ex_ne)a_5+a_6f(ex_1e,\ldots ,ex_ne)^2a_7 \\&+va_3f(ex_1e,\ldots ,ex_ne)^2a_8-2f(ex_1e,\ldots ,ex_ne)a_3f(ex_1e,\ldots ,ex_ne)a_8 \\&+\,f(ex_1e,\ldots ,ex_ne)^2a_9\}e=0, \end{aligned}$$

where the subring eRe satisfies

$$\begin{aligned}&ea_1ef(x_1,\ldots ,x_n)^2-2ea_2ef(x_1,\ldots ,x_n)ea_3ef(x_1,\ldots ,x_n) \\&+\,ea_2ef(x_1,\ldots ,x_n)^2ea_3e+ea_4ef(x_1,\ldots ,x_n)^2ea_5e \\&-\,2ea_6ef(x_1,\ldots ,x_n)ea_3ef(x_1,\ldots ,x_n)ea_5e+ea_6ef(x_1,\ldots ,x_n)^2ea_7e \\&+\,ea_3ef(x_1,\ldots ,x_n)^2ea_8e-2f(x_1,\ldots ,x_n)ea_3ef(x_1,\ldots ,x_n)ea_8e \\&+\,f(x_1,\ldots ,x_n)^2ea_9e=0. \end{aligned}$$

Then by the above finite dimensional case, either \(ea_3e\) or \(ea_5e\) or \(ea_6e\) is the central element of eRe. This leads to a contradiction, since \(a_3h_1=(ea_3e)h_1=h_1ea_3e=h_1a_3\), \(a_5h_2=(ea_5e)h_2=h_2(ea_5e)=h_2a_5\) and \(a_6h_3=(ea_6e)h_3=h_3(ea_6e)=h_3a_6\).

Hence we have proved that either \(a_3\) or \(a_5\) or \(a_6\) is in C. \(\square \)

Theorem 3.5

Let R be a prime ring of characteristic different from 2, \(f(x_1,\ldots ,x_n)\) a non-central multilinear polynomial over C and d a nonzero derivation of R. Suppose that for some \(b,c,p,q\in R\), \(b[d(u),u]+p[d(u),u]q+[d(u),u]c=0\) for all \(u\in f(R)\). Then one of the following holds:

  1. (1)

    b, p, \(pq+c\in C\) and \(b+pq+c=0\);

  2. (2)

    \(b+pq\), q, \(c\in C\) and \(b+pq+c=0\).

Proof

Let d be an inner derivation of R, that is, \(d(x)=[a,x]\) for all \(x\in R\). By hypothesis, R satisfies

$$\begin{aligned} b[[a,f(r)],f(r)]+p[[a,f(r)],f(r)]q+[[a,f(r)],f(r)]c=0, \end{aligned}$$
(3)

that is,

$$\begin{aligned}&baf(r)^2-2bf(r)af(r)+bf(r)^2a+paf(r)^2q \\&\quad -\,2pf(r)af(r)q+pf(r)^2aq\\&\quad +\,af(r)^2c-2f(r)af(r)c+f(r)^2ac=0 \end{aligned}$$

for all \(r=(r_1,\ldots ,r_n) \in R^n\). Since d is nonzero, \(a\notin C\). In this case, by Lemma 3.4, we have either \(p\in C\) or \(q\in C\).

Case i. Let \(p\in C\). Then by hypothesis, R satisfies

$$\begin{aligned} b[[a,f(r)],f(r)]+[[a,f(r)],f(r)](pq+c)=0. \end{aligned}$$

By Lemma 3.3 in [2], \(b, pq+c\) and \((b+pq+c)a\) are in C. Since \(a\notin C\), we conclude that \(b+pq+c=0\). This is our conclusion (1).

Case ii. Let \(q\in C\). By hypothesis, R satisfies

$$\begin{aligned} (b+pq)[[a,f(r)],f(r)]+[[a,f(r)],f(r)]c=0. \end{aligned}$$

By Lemma 3.3 in [2], \(b+pq, c\) and \((b+pq+c)a\) are in C. Since \(a\notin C\), we conclude that \(b+pq+c=0\). This is our conclusion (2).

Next, suppose that d is an outer derivation of R. By using Fact 5 and Kharchenko’s theorem [11], we can replace \(d(x_{i})\) with \(y_{i}\) and then R satisfies

$$\begin{aligned}&b[f^d(x_1,\ldots ,x_n)+\sum \limits _i f(x_1,\ldots ,y_i,\ldots ,x_n), f(x_1,\ldots ,x_n)]\\&\qquad +\,p[f^d(x_1,\ldots ,x_n)+\sum \limits _i f(x_1,\ldots ,y_i,\ldots ,x_n), f(x_1,\ldots ,x_n)]q\\&\qquad +\,[f^d(x_1,\ldots ,x_n)+\sum \limits _i f(x_1,\ldots ,y_i,\ldots ,x_n), f(x_1,\ldots ,x_n)]c=0. \end{aligned}$$

In particular, R satisfies blended component

$$\begin{aligned} \begin{aligned}&b[\underset{i}{\Sigma } f(x_1,\ldots ,y_i,\ldots , x_n), f(x_1,\ldots ,x_n)] \\&\quad +p[\underset{i}{\Sigma } f(x_1,\ldots ,y_i,\ldots , x_n), f(x_1,\ldots ,x_n)]q\\&\quad +[\underset{i}{\Sigma } f(x_1,\ldots ,y_i,\ldots , x_n), f(x_1,\ldots ,x_n)]c=0. \end{aligned} \end{aligned}$$
(4)

Since R is noncommutative, we choose \(a'\in R\) such that \(a'\notin C\). Replacing \([a', x_i]\) in place of \(y_i\) in equation (4), we get

$$\begin{aligned} b[[a',f(r)],f(r)]+p[[a',f(r)],f(r)]q+[[a',f(r)],f(r)]c=0 \end{aligned}$$

for all \(r=(r_1,\ldots ,r_n) \in R^n\), which is the same as equation (3). Then by the same argument as above, we have our conclusions. \(\square \)

In particular, for right-sided annihilator condition, we have the following.

COROLLARY 3.6

Let R be a prime ring of characteristic different from 2, \(f(x_1,\ldots ,x_n)\) a non-central multilinear polynomial over C and d a nonzero derivation of R. Suppose that for some \(a\in R\), \([d(u),u]a=0\) for all \(u\in f(R)\). Then \(a=0\).

In particular, for two-sided annihilator condition, we have the following.

COROLLARY 3.7

Let R be a prime ring of characteristic different from 2, \(f(x_1,\ldots ,x_n)\) a non-central multilinear polynomial over C and d a nonzero derivation of R. Suppose that for some \(a,b\in R\), \(a[d(u),u]b=0\) for all \(u\in f(R)\). Then either \(a=0\) or \(b=0\).

Putting \(p=0\) and \(q=0\) in Theorem 3.5, we have the inner part of Theorem 5.3 of [2]. More precisely, we obtain the following.

COROLLARY 3.8

Let R be a prime ring of characteristic different from 2, \(f(x_1,\ldots ,x_n)\) a non-central multilinear polynomial over C and d a nonzero derivation of R. Suppose that for some \(b,c\in R\), \(b[d(u),u]+[d(u),u]c=0\) for all \(u\in f(R)\). Then \(b=-c\in C\).

Replacing b by \(s^{2}\), c by \(t^{2}\), \(p=2s\) and \(q=t\) in Theorem 3.5, we obtain the following.

COROLLARY 3.9

Let R be a prime ring of characteristic different from 2 and \(f(x_1,\ldots ,x_n)\) a non-central multilinear polynomial over C. If d is a nonzero derivation of R, and F is an inner generalized derivation of R such that

$$\begin{aligned} F^2([d(f(x_1,\ldots ,x_n)), f(x_1,\ldots ,x_n)])=0 \end{aligned}$$

for all \(x_1,\ldots ,x_n\in R\), then there exists \(a\in U\) such that \(F(x)=xa\) for all \(x\in R\) or \(F(x)=ax\) for all \(x\in R\), with \(a^2=0\).

In the next section, we will extend Corollary 3.9 to the arbitrary generalized derivation. Now we are ready to prove the main theorem.

4 The proof of the main theorem

Lee [12] proved that every generalized derivation can be uniquely extended to a generalized derivation of U, and thus all generalized derivations of R will be implicitly assumed to be defined on the whole U. In particular, Lee proved that every generalized derivation g on a dense right ideal of R can be uniquely extended to U and has the form \(g(x) = ax + d(x)\) for some \(a\in U\) and a derivation d of R.

Theorem 4.1

Suppose that R is a prime ring of characteristic different from 2 and \(f(x_1,\ldots ,x_n)\) is a non-central multilinear polynomial over C. If d is a nonzero derivation of R, and F is a generalized derivation of R such that

$$\begin{aligned} F^2([d(f(x_1,\ldots ,x_n)), f(x_1,\ldots ,x_n)])=0 \end{aligned}$$

for all \(x_1,\ldots ,x_n\in R\), then there exists \(a\in U\) such that \(F(x)=xa\) for all \(x\in R\) or \(F(x)=ax\) for all \(x\in R\), with \(a^2=0\).

Proof

In light of [12, Theorem 3], we may assume that there exist \(b \in U\) and derivation \(\delta \) of U such that \(F(x) = bx + \delta (x)\) and so, \(F^2(x)=b^2x+2b\delta (x)+\delta (b)x+\delta ^2(x)\). Since R and U satisfy the same generalized polynomial identities (see Fact 2) as well as the same differential identities (see Fact 4), without loss of generality, we have

$$\begin{aligned} F^2[d(f(r_1,\ldots ,r_n)), f(r_1,\ldots ,r_n)]=0 \end{aligned}$$

for all \(r_1,\ldots ,r_n \in U\). If F is an inner generalized derivation of R, then assume that \(F(x)=bx+xc\) for all \(x\in R\), with some \(b,c\in U\). In this case, by the hypothesis

$$\begin{aligned}b^2[d(r), r]+2b[d(r), r]c+[d(r), r]c^2=0\end{aligned}$$

for all \(r\in f(R)\). Then by Theorem 3.5, one of the following holds:

  1. (i)

    \(b^2,b, 2bc+c^2\in C\) and \(b^2+2bc+c^2=0\), that is \((b+c)^2=0\). In this case, \(F(x)=x(b+c)\) for all \(x\in R\) with \((b+c)^2=0\).

  2. (ii)

    \(b^2+2bc, c, c^2\in C\) and \(b^2+2bc+c^2=0\), that is, \((b+c)^2=0\). In this case, \(F(x)=(b+c)x\) for all \(x\in R\) with \((b+c)^2=0\).

Now, we assume that F is outer. By the hypothesis, U satisfies

$$\begin{aligned} b^2[d(r),r]+2b\delta ([d(r),r])+\delta (b)[d(r),r]+\delta ^2([d(r),r])=0 \end{aligned}$$
(5)

for all \(r\in f(R)\).

Case I. Let d and \(\delta \) be C-dependent modulo inner derivations of U, that is, \(\alpha d+\beta \delta =ad_q\), where \(\alpha ,\beta \in C\), \(q\in U\) and \(ad_q(x)=[q,x]\) for all \(x\in U\). If \(\alpha =0\), then \(\delta \) must be inner and so F is inner, a contradiction. Hence \(\alpha \ne 0\), and hence \(d=\lambda \delta +ad_p\), where \(\lambda =-\alpha ^{-1}\beta \) and \(p=\alpha ^{-1}q\).

Then by the hypothesis, it follows that

$$\begin{aligned} \begin{aligned}&b^2[\lambda \delta (r)+[p,r], r]+2b\delta ([\lambda \delta (r)+[p,r],r]) \\&\quad + \delta (b)[\lambda \delta (r) + [p,r],r] \\&\quad +\delta ^2([\lambda \delta (r)+[p,r],r])=0 \end{aligned} \end{aligned}$$
(6)

for all \(r\in f(R)\).

Using Fact 5, substitute the values of \(\delta (f(r_1,\ldots ,r_n))\), \(\delta ^2(f(r_1,\ldots ,r_n))\) and \(\delta ^3(f(r_1,\ldots ,r_n))\) in equation (6). Then by Kharchenko’s theorem [11], we can replace \(\delta (r_i)\) with \(y_i\), \(\delta ^2(r_i)\) with \(w_i\) and \(\delta ^3(r_i)\) with \(z_i\) in equation (6) and then U satisfies the blended component

$$\begin{aligned}{}[\lambda \underset{i}{\Sigma } f(r_1,\ldots ,z_i,\ldots ,r_n), f(r_1,\ldots ,r_n)]=0. \end{aligned}$$

We choose \(q\in U\) such that \(q\notin C\) and replace \(z_i\) by \([q,r_i]\). Then U satisfies

$$\begin{aligned}{}[\lambda q, f(r_1,\ldots ,r_n)]_2=0. \end{aligned}$$

By [13, Theorem], \(\lambda q\in C\). Since \(q\notin C\), \(\lambda =0\). Hence by equation (6),

$$\begin{aligned} b^2[[p,r], r]+2b\delta ([[p,r],r])+\delta (b)[[p,r],r]+\delta ^2([[p,r],r])=0 \end{aligned}$$
(7)

for all \(r\in f(R)\).

Putting the values of \(\delta (f(r_1,\ldots ,r_n))\) and \(\delta ^2(f(r_1,\ldots ,r_n))\) in equation (7), then again by Kharchenko’s theorem [11], we can replace \(\delta (r_i)\) with \(y_i\) and \(\delta ^2(r_i)\) with \(w_i\) in (7), and then U satisfies the blended component

$$\begin{aligned}&{[[}p, \underset{i}{\Sigma } f(r_1,\ldots ,w_i,\ldots ,r_n)], f(r_1,\ldots ,r_n)]\\&+[[p, f(r_1,\ldots ,r_n)], \underset{i}{\Sigma } f(r_1,\ldots ,w_i,\ldots ,r_n)]=0. \end{aligned}$$

By taking \(w_1=r_1\) and \(w_2=\cdots =w_n=0\), U satisfies

$$\begin{aligned} 2[[p, f(r_1,\ldots ,r_n)], f(r_1,\ldots ,r_n)]=0. \end{aligned}$$

Since char(\(R)\ne 2\), by [13, Theorem] \(p\in C\). This gives that \(d=0\), a contradiction.

Case II. Let d and \(\delta \) be C-independent modulo inner derivations of U. Then by applying Fact 5 and Kharchenko’s theorem [11] to equation (5), we can replace \(d(r_i)\) with \(y_i\), \(\delta (r_i)\) with \(z_i\), \(\delta d(r_i)\) with \(s_i\), \(\delta ^2(r_i)\) with \(t_i\) and \(\delta ^2 d(r_i)\) with \(u_i\). Then U satisfies the blended component

$$\begin{aligned}{}[\underset{i}{\Sigma } f(r_1,\ldots ,u_i,\ldots ,r_n), f(r_1,\ldots ,r_n)]=0. \end{aligned}$$

In particular, replacing \(u_i\) with \([q, r_i]\) for some \(q\notin C\), U satisfies

$$\begin{aligned}{}[q, f(r_1,\ldots ,r_n)]_2=0. \end{aligned}$$

Again by [13, Theorem], \(q\in C\), a contradiction. \(\square \)

COROLLARY 4.2

Let R be a prime ring of characteristic different from 2 with extended centroid C and \(f(x_1,\ldots ,x_n)\) a multilinear polynomial over C. If d and \(\delta \) are two nonzero derivations of R such that

$$\begin{aligned} \delta ^2([d(f(x_1,\ldots ,x_n)), f(x_1,\ldots ,x_n)])=0\end{aligned}$$

for all \(x_1,\ldots ,x_n\in R\), then \(f(x_1,\ldots ,x_n)\) is central-valued on R.