On finite groups in which the twisted conjugacy classes of the unit element are subgroups

Abstract

We consider groups G such that the set $[G,\varphi ]=\{g^{-1}g^{\varphi }|g\in G\}$ is a subgroup for every automorphism $\varphi$ of G, and we prove that there exists such a group G that is finite and nilpotent of class n for every $n\in \mathbb N$. Then there exists an infinite not nilpotent group with the above property and the Conjecture 18.14 of Khukhro and Mazurov (The Kourovka Notebook No. 20, 2022) is false.

1 Introduction

Let G be a group and $\varphi$ be an endomorphism of G; we say that elements $x,y\in G$ are $\varphi$-conjugate if there exists an element $z\in G$ such that $y=z^{-1}xz^{\varphi }$.

It is easy to check that the relation of $\varphi$-conjugation is an equivalence relation in G. In particular, it is the usual conjugation if $\varphi =\textrm{id}_G$ and it is the total equivalence relation if $\varphi =0_G$ is the zero endomorphism.

Equivalence classes are called twisted conjugacy classes or $\varphi$-conjugacy classes and their number $R(\varphi )$ is called the Reidemeister number of the endomorphism $\varphi$.

The $\varphi$-conjugacy class $[1]_{\varphi }=\{x^{-1}x^{\varphi }|x\in G\}$ of the unit element of the group G is a subset whose cardinality is equal to the index $|G:C_G(\varphi )|$ of the centralizer $C_G(\varphi )=\{x\in G|x^{\varphi }=x\}$ of $\varphi$ in G; in what follows, we will put $[1]_{\varphi }=:[G,\varphi ]$ and we will write $[x,\varphi ]:=x^{-1}x^{\varphi }$.

If $\varphi =\textrm{id}_G$, then $[G,\varphi ]=\{1\}$ and if $\varphi =0_G$, then $[G,\varphi ]=G$ so, in these cases, $[G,\varphi ]$ is a subgroup of G. However, in the general case, $[G,\varphi ]$ is not a subgroup, it is not if we consider an automorphism $\varphi \in \hbox {Aut}(G)$ and not even if this automorphism $\varphi \in \hbox {Inn}(G)$ is inner. For instance, if $G=S_3$ is the symmetric group of degree 3 and $\varphi =\bar{g}$ is the inner automorphism induced by $g=(123)$, then $[1]_{\bar{g}}=\{1, (132)\}\not \le G.$

Notice that if $\varphi \in \hbox {Aut}_C(G)$ is a central automorphism of a group G, that is, if $g^{-1}g^{\varphi }\in Z(G)$ for every $g\in G$, then $[G,\varphi ]\le G$. So, in particular, if the group G is nilpotent of class $\le 2$, then $[G,\varphi ]$ is a subgroup for every $\varphi \in \hbox {Inn}(G)$.

As it is easy to verify, if a group G is abelian, then not only $H=[G,\varphi ]\le G$ for every $\varphi \in \hbox {End}(G)$, but for every element $x\in G$, its $\varphi$-conjugacy class is equal to the coset $[x]_{\varphi }=xH$, that is, $\varphi$-conjugation is a congruence in G.

It is possible to prove that this property characterizes abelian groups. In fact, if $\varphi$-conjugation is a congruence for every $\varphi \in \hbox {End}(G)$, then in particular, conjugation is a congruence. This implies that every conjugacy class has order 1, that is, $g^x=g$ for every $g,x\in G$, hence G is abelian. Actually for a group to be abelian, it is enough that there exists an inner automorphism $\bar{g}\in \hbox {Inn}(G)$ such that $\bar{g}$-conjugation is a congruence in G.

Proposition 1.1

A group G is abelian if and only if there exists $g\in G$ such that $\bar{g}$-conjugation is a congruence in G.

Proof

We only have to show that if $\bar{g}$-conjugation is a congruence for an element $g\in G$, then the group is abelian.

If $\bar{g}$ is a congruence, then $|[x]_{\bar{g}}|=|[1]_{\bar{g}}|=|\{h^{-1}h^g|h\in G\}|$ for every $x\in G$. In particular, $|\{h^{-1}h^g=[h,g]|h\in G\}|=|[g]_{\bar{g}}|$ so $|\{[h,g]|h\in G\}|=1$, that is, $\{[h,g]|h\in G\}=\{1\}$ since $[g]_{\bar{g}}=\{g\}$. This means that $g\in Z(G)$ hence $\bar{g}=\textrm{id}_G$, conjugation is a congruence and so G is abelian. $\square$

In the paper [1], the authors prove that every finite group G in which the $\varphi$-conjugacy class of the unit element is a subgroup for every inner automorphism $\varphi \in \hbox { Inn}(G)$ is nilpotent. Anyway, it is not possible to bound the nilpotency class of such groups; in fact, in [3], the authors construct, for any integer $n>2$ and for any prime $p>2$, a finite p-group G nilpotent of class $\ge n$ with this property. However, they notice that there exist automorphisms $\phi \in \hbox {Aut}(G){\setminus } \hbox {Inn}(G)$ such that $[G,\phi ]$ is not a subgroup of G.

It is therefore natural to ask if it is possible to bound the nilpotency class of a finite group G such that the $\varphi$-conjugacy class of the unit element is a subgroup for every $\varphi \in \hbox {Aut}(G)$. In this regard, in [1], the conjecture is made that such groups could be abelian (cfr. also [5, 18.14]). This conjecture is certainly false, in fact, there exist finite non-abelian p-groups in which every automorphism is central (see for instance [2, 7]). Of course, such groups are nilpotent of class 2 but we will prove the existence of nilpotent groups of every class n with this property. So also the answer to the previous question is negative.

Our main result is the following

Theorem 1.2

(Main Theorem). For every integer $n\in \mathbb N$ and for every odd prime p, there exists a finite p-group G, of class n, in which the $\varphi$-conjugacy class of the unit element is a subgroup for every $\varphi \in \textrm{Aut}(G)$.

As we will see, these groups are abelian-by-cyclic, and this result will give a negative answer also to Problem 3 of [3].

Moreover, from Theorem 1.2, it follows the existence of an infinite non-nilpotent group G such that $[G,\varphi ]\le G$ for every $\varphi \in \hbox {Aut}(G)$ and so Conjecture 18.14 of [5] is false.

2 The proof of the main theorem

Let n be an integer $n\ge 2$, p be an odd prime, and $G=A\rtimes \langle x\rangle$, where $A=\langle a\rangle \times \langle b\rangle \simeq \mathbb Z_{p^n}\times \mathbb Z_{p^n}$, $\langle x\rangle \simeq \mathbb Z_{p^{n-1}}$, and $c^x=c^{1+p}$ for every $c\in A$.

It is easy to verify that $G'=\langle a^p\rangle \times \langle b^p\rangle$ and that G is nilpotent of class n. Moreover, for every $c\in A$, we have that $\langle c\rangle$ is normal in G; in particular, for every $g_1,g_2\in G$, we have that $\langle [g_1,g_2]\rangle$ is normal in G, then $\langle g_1,g_2\rangle '=\langle [g_1,g_2]\rangle$ and this implies that $\langle g_1,g_2\rangle$ is a regular p-group since its derived subgroup is cyclic ([4, III 10.2 (c)]). Therefore G is a regular p-group.

Let $\varphi \in \hbox {Aut}(G)$ be an automorphism of G, we will prove that $[1]_{\varphi }=\{g^{-1}g^{\varphi }|g\in G\}=[G,\varphi ]$ is a subgroup.

First of all we have that

$$\begin{aligned} \qquad \qquad \qquad \qquad \qquad \qquad [c,\varphi ]\in A\>\>\>\forall c\in A. \qquad \qquad \qquad \qquad \qquad \qquad (*) \end{aligned}$$

In fact, if there exists $c\in A$ such that $[c,\varphi ]\not \in A$, then either $[a,\varphi ]\not \in A$ or $[b,\varphi ]\not \in A$. Without loss of generality. we may assume that $[a,\varphi ]\not \in A$, that is, $[a,\varphi ]=yx^{\alpha }$ with $y\in A$ and $\alpha \in \mathbb Z$ such that $p^{n-1}$ does not divide $\alpha$.

Observe that $a^{\varphi }=ayx^{\alpha }$, $(a^{\varphi })^{p^{n-1}}=(ay)^{p^{n-1}} (x^{\alpha })^{p^{n-1}}z^{p^{n-1}}$ with $z\in G'$ since G is regular, and then $(a^{\varphi })^{p^{n-1}}=(ay)^{p^{n-1}}$ because $G'$ has exponent $p^{n-1}$ and, in particular, ay has order $p^n$.

Let $[b,\varphi ]=tx^{\beta }$ with $t\in A$ and $\beta \in \mathbb Z$, then $b^{\varphi }=btx^{\beta }$ and we have that also bt has order $p^{n}$ since $(b^{\varphi })^{p^{n-1}}=(bt)^{p^{n-1}}$.

Now $1=[a,b]=[a^{\varphi }, b^{\varphi }]=[ayx^{\alpha }, btx^{\beta }]=[ay, x^{\beta }]^{x^{\alpha }}[x^{\alpha }, bt]^{x^{\beta }}$ and this implies that $[ay, x^{\beta }]^{x^{\alpha }}=[bt, x^{\alpha }]^{x^{\beta }}.$ Therefore there exist integers $\gamma , \delta \in \mathbb Z$ such that $(ay)^{\gamma }=(bt)^{\delta }$ because $[ay, x^{\beta }]^{x^{\alpha }}\in \langle ay\rangle$ and $[bt, x^{\alpha }]^{x^{\beta }}\in \langle bt\rangle$.

From $\langle a\rangle \cap \langle b\rangle =\{1\}$, it follows that $\langle a^{\varphi }\rangle \cap \langle b^{\varphi }\rangle =\{1\}$, then

$$\begin{aligned} \langle (a^{\varphi })^{p^{n-1}}\rangle \cap \langle (b^{\varphi })^{p^{n-1}}\rangle =\langle (ay)^{p^{n-1}}\rangle \cap \langle (bt)^{p^{n-1}}\rangle =\{1\}. \end{aligned}$$

This implies that $\langle ay\rangle \cap \langle bt\rangle =\{1\}$ because the intersection between the unique subgroups of order p of the cyclic p-groups $\langle ay\rangle$ and $\langle bt\rangle$ is the trivial subgroup. Hence $p^n$ divides both $\gamma$ and $\delta$, so $x^{\beta }\in C_G(ay)$ and $x^{\alpha }\in C_G(bt)$. In particular, $(bt)^{x^{\alpha }}=(bt)^{(1+p)^{\alpha }}=bt$, then $(1+p)^{\alpha }\equiv 1\, (\text{ mod } p^n)$ and, since the multiplicative order of $(p+1)$ in $\mathbb Z_{p^n}$ is $p^{n-1}$, we have that $p^{n-1}$ divides $\alpha$, that is a contradiction. From $(*)$, it follows that $c^{\varphi }\in A$ for every $c\in A$, and in particular, $a^\varphi \in A$.

Now we prove that $x^{\varphi }=sx$ with $s\in A$ and so $[x,\varphi ]=x^{-1}x^{\varphi }=s^x=s^{1+p}\in A$. Let $x^{\varphi }=sx^{\lambda }$ with $\lambda \in \mathbb Z$ and $s\in A$; from $a^x=a^{1+p}$, it follows that $(a^{\varphi })^{x^{\varphi }}=(a^{\varphi })^{(1+p)}$, that is, $(a^{\varphi })^{x^{\lambda }}=(a^{\varphi })^{1+p}$ and $(a^{\varphi })^{(1+p)^{\lambda }}= (a^{\varphi })^{1+p}$. Therefore $(1+p)^{\lambda }\equiv (1+p)\, (\text{ mod } p^n)$, $\lambda \equiv 1\, (\text{ mod } p^n)$ and so $x^{\varphi }=sx$.

Since $[x,\varphi ] \in A$, we have that $[x^{\alpha },\varphi ]=[x,\varphi ]^{\beta }$ with $\beta \in \mathbb Z$, that is, $[x^{\alpha },\varphi ]\in \langle [x,\varphi ]\rangle$ for every $\alpha \in \mathbb Z$ and so $V=\{[x^{\alpha },\varphi ]|\alpha \in \mathbb Z\}\subseteq \langle [x,\varphi ]\rangle$.

Put $[x,\varphi ]=c$, that is, $x^{\varphi }=xc$. If $|V|=|\langle x\rangle :C_{\langle x\rangle }(\varphi )|=p^k$, then $(x^{\varphi })^{p^{k}}=(x^{p^{k}})^{\varphi }=x^{p^k}$, that is, $(xc)^{p^{k}}=x^{p^{k}}$. Hence $x^{p^{k}}=x^{p^{k}}c^{p^{k}}z^{p^{k}}$ with $z\in (\langle x,c\rangle )'=\langle c^p\rangle$, that is, $x^{p^{k}}=x^{p^{k}}c^{p^{k}}(c^{lp})^{p^{k}}=x^{p^{k}}c^{(1+lp)p^{k}}$ which implies $c^{(1+lp)p^{k}}=1$ and so the order o(c) divides $p^k$. Then $p^k=|V|\le |\langle c\rangle |\le p^k$ and $|V|=|\langle c\rangle |$. So we have that $(**)\>\>V=\langle [x,\varphi ]\rangle .$

Let $B=\{[c,\varphi ]|c\in A\}$. From $(*)$, we have that $B\subseteq A$ and we prove that $B\le A$. In fact, for every $c,d\in A$, we get $[cd,\varphi ]=[c,\varphi ]^d[d,\varphi ]=[c,\varphi ][d,\varphi ]$ and $[c^{-1},\varphi ]=[c,\varphi ]^{-1}$. Moreover, $\langle [x,\varphi ]\rangle \le A$ since $[x,\varphi ]\in A$. Then $B\langle [x,\varphi ]\rangle \le A,$ and we show that $B\langle [x,\varphi ]\rangle =[G,\varphi ]$ so in particular $[G,\varphi ]\le G$.

For every $g\in G$, there exist $f\in A$ and $\alpha \in \mathbb Z$ such that $g=fx^{\alpha }$; then $[g,\varphi ]=[fx^{\alpha }, \varphi ]=[f,\varphi ]^{x^{\alpha }}[x^{\alpha }, \varphi ]=[f,\varphi ]^{\xi }[x,\varphi ]^{\eta }$ with $\xi ,\eta \in \mathbb Z$ and so $[g,\varphi ]\in B\langle [x,\varphi ]\rangle$.

Conversely, if we consider $[d,\varphi ][x,\varphi ]^{\eta }$ with $d\in A$, then we have that there exists $\xi \in \mathbb Z$ such that $[d,\varphi ][x,\varphi ]^{\eta }= [d,\varphi ][x^{\xi },\varphi ]$ since $(**)$ implies that there exists $\xi \in \mathbb Z$ such that $[x,\varphi ]^{\eta }=[x^{\xi },\varphi ]$. Let $\beta \in \mathbb Z$ be such that $(1+p)^{\xi }\beta \equiv 1\, (\text{ mod } p^n)$, then

$$\begin{aligned}{}[d^{\beta }x^{\xi },\varphi ]=[d^{\beta },\varphi ]^{x^{\xi }}[x^{\xi },\varphi ]=[d,\varphi ]^{\beta x^{\xi }}[x,\varphi ]^{\eta }=[d, \varphi ]^{(1+p)^{\xi }\beta }[x,\varphi ]^{\eta }=[d,\varphi ][x,\varphi ]^{\eta },\end{aligned}$$

that is, $[d,\varphi ][x,\varphi ]^{\eta }\in [G,\varphi ]$ and this completes the proof.

3 Further remarks and open questions

From Theorem 1.2, it follows that for every $n\in \mathbb N$ and for every odd prime p, there exists a finite p-group P of class n such that $[P,\varphi ]\le P$ for every $\varphi \in \hbox {Aut}(P)$, let us denote such a finite p-group by G(n, p). It is possible to construct an infinite non-nilpotent group with the same property.

Corollary 3.1

There exists an infinite non-nilpotent group G such that $[G,\varphi ]\le G$ for every $\varphi \in \textrm{Aut}(G)$.

Proof

For every $n\in \mathbb N$, fix a prime $p_n$ such that $p_n\not =p_m$ for every $m<n$ and put $P_n:=G(n,p_n)$. The restricted direct product ([8, p. 20]) $G:=\textrm{Dir}_{n\in \mathbb N}P_n$ is non-nilpotent and $[G,\varphi ]\le G$ for every $\varphi \in \hbox {Aut}(G)$.

Note that, for every $\varphi \in G$ and for every $n\in \mathbb N$, we have that the restriction $\varphi _{| P_n}=:\varphi _{n}\in \hbox { Aut}(P_n)$, so $[P_n,\varphi _n]$ is a subgroup of $P_n$. We will show that $[G,\varphi ]=\langle [P_n,\varphi _n]|n\in \mathbb N\rangle$ so, in particular, it is a subgroup.

Let $g\in G$, then $g=g_{n_1}\cdots g_{n_t}$ with $t, n_1,\ldots ,n_t\in \mathbb N$, and $g_{n_i} \in P_{n_i}$ for every $i\in \{1,\ldots , t\}$. So $[g,\varphi ]= [g_{n_1}\cdots g_{n_t},\varphi ]= g_{n_t}^{-1}\cdots g_{n_1}^{-1} (g_{n_1}\cdots g_{n_t})^{\varphi }= g_{n_1}^{-1}g_{n_1}^{\varphi _{n_1}}\cdots g_{n_t}^{-1}g_{n_t}^{\varphi _{n_t}}=[g_{n_1},\varphi _{n_1}]\cdots [g_{n_t},\varphi _{n_t}]$ and $[G,\varphi ]\subseteq \langle [P_n,\varphi _n]|n\in \mathbb N\rangle$.

Conversely, if $y\in \langle [P_n,\varphi _n]|n\in \mathbb N\rangle$, then $y=y_{n_1}\cdots y_{n_k}$ for some $k, n_1,\ldots ,n_k\in \mathbb N$, $y_{n_i}=[z_{n_i}, \varphi _{n_i}]\in [P_{n_i},\varphi _{n_i}]$ and, as before, it is easy to see that $y=[z_{n_1}\cdots z_{n_t}, \varphi ]\in [G,\varphi ].$ $\square$

If $n=2$, Theorem 1.2 gives an example of an abelian-by-(cyclic of order p) p-group that is a conterexample to Problem 3 of [3] for every odd prime p.

Regarding the case where $p=2$, in [2], there is an example of a 2-group of class 2 such that $[G,\varphi ]\le G$ for every $\varphi \in \hbox {Aut}(G)$, is it possible to have such a 2-group of class $>2$?

Remark 3.2

Notice that if we consider the 2-group $G=A\rtimes \langle x\rangle$, with $A=\langle a\rangle \times \langle b\rangle \simeq \mathbb Z_{2^n}\times \mathbb Z_{2^n}$, $\langle x\rangle \simeq \mathbb Z_{2^{n-1}}$, $n>1$, and $c^x=c^3$ for every $c\in A$, then there exists $\varphi \in \hbox {Aut}(G)$ such that $[G,\varphi ]\not \le G$.

In order to define such an automorphism, first of all, we prove, by induction, that for every $c\in A$ and for every $t\ge 1$, we have that

$$\begin{aligned} (xc)^{2^t}=x^{2^t}c^{2^{t+1}h}\>\>\text {for some}\>\> h\in \mathbb N. \end{aligned}$$

For if $t=1$, then

$$\begin{aligned} (xc)^2=xcxc=x^2c^xc=x^2c^4. \end{aligned}$$

Suppose that, for some $t\in \mathbb N$, we have

$$\begin{aligned} (xc)^{2^t}=x^{2^t}c^{2^{t+1}h}, \end{aligned}$$

then $(xc)^{2^{t+1}}=((xc)^{2^{t}})^2=x^{2^{t+1}}(c^{2^{t+1}h})^{x^{2^t}}(c^{2^{t+1}h})=x^{2^{t+1}}(c^{2^{t+1}h})^{3^{2^t}}c^{2^{t+1}h}=x^{2^{t+1}}c^{2^{t+1}h(3^{2^t}+1)}= x^{2^{t+1}}c^{2^{t+2}k}$ since $3^{2^t}+1$ is even.

Therefore, if $c\in A$ has order $2^n$, then xc has order $2^{n-1}$ and we may consider $\varphi \in \text {Aut}(G)$ such that $y^{\varphi }=y$ for every $y\in A$ and $x^{\varphi }=xc$. We have that $[x,\varphi ]\in A$ and, by induction, $[x^{\alpha },\varphi ]=[x,\varphi ]^{x^{\alpha -1}}[x^{\alpha -1},\varphi ]\in A$ for every $\alpha \in \mathbb N$. Therefore $[G,\varphi ]=[\langle x\rangle , \varphi ]$ because for every $g=x^{\alpha }y\in G$ with $\alpha \in \mathbb N$ and $y\in A$, we have that $[g,\varphi ]=[x^{\alpha }y, \varphi ]=[x^{\alpha }, \varphi ]\in [\langle x\rangle , \varphi ]$. This implies that $|[G,\varphi ]|=|[\langle x\rangle , \varphi ]|=|\langle x\rangle :C_{\langle x\rangle }(\varphi )|\le 2^{n-1}$. Now $x^{-1}x^{\varphi }=c\in [G,\varphi ]$, so $[G,\varphi ]\not \le G$ since c has order $2^{n}$.

Remark 3.3

Also every dihedral 2-group $G=D_{2^n}=A\rtimes \langle x\rangle$ with $A=\langle a\rangle \simeq \mathbb Z_{2^{n-1}}$ and $\langle x\rangle \simeq \mathbb Z_2$ $(n>2)$ has an automorphism $\varphi \in \text {Aut}(G)$ such that $[G,\varphi ]\not \le G$. In fact, if we consider the automorphism $\varphi$ defined by $a^\varphi =a$ and $x^{\varphi }=xa^{-1}$, then we have that $[G,\varphi ]=[\langle x\rangle , \varphi ]$ so $|[G,\varphi ]| \le 2.$ But $a^{-1}=x^{-1}x^{\varphi }\in [G,\varphi ]$, then it is not a subgroup because $a^{-1}$ has order $2^{n-1}>2$.

Notice that, in particular, if $n=3$, that is, $G=D_8$, then $\varphi _{|Z(G)}=\textrm{id}_{Z(G)}$ because $[a,x]^{\varphi }=[a,xa^{-1}]=[a,x]$.

Also the quaternion group $Q_8=\{1,-1,i,j,k,-i,-j,-k\}$ has an automorphism $\varphi$ such that $[Q_8,\varphi ]\not \le Q_8$ and $\varphi _{|Z(Q_8)}= \textrm{id}_{{Z(Q_8)}}$. Actually, if we consider the automorphism $\varphi$, defined by $i^{\varphi }=i$ and $j^{\varphi }=k$, then we have $[Q_8,\varphi ]=\{1,-i\}$.

Indeed, it is possible to prove the following result:

Proposition 3.4

For every prime p, if G is an extraspecial p-group, then there exists $\varphi \in \textrm{Aut}(G)$ such that $[G,\varphi ]\not \le G$.

In order to show this proposition, first of all, we recall that a group G is the central product of two subgroups H and K if $G=HK$, $[H,K]=1$, and $H\cap K=Z(G)$. Then we prove the following two lemmas.

Lemma 3.5

Suppose that a group G is the central product of two subgroups H and K; if there exists an automorphism $\phi \in \textrm{Aut}(H)$ such that $z^{\phi }=z$ for every $z\in Z(G)$ and $[H,\phi ]\not \le H$, then there also exists $\varphi \in \textrm{Aut}(G)$ such that $[G,\varphi ]\not \le G$.

Proof

The group G is the central product of H and K, hence for every $g\in G$, we have that $g=hk$ with $h\in H$ and $k\in K$.

Let $\phi \in \text {Aut}(H)$; if the restriction $\phi _{|Z(G)}=\text {id}_{Z(G)}$, then $g^{\varphi }:=(hk)^{\varphi }=h^{\phi }k$ defines a map $\varphi :G\rightarrow G$. In fact, $hk=h_1k_1$, with $h,h_1\in H$ and $k,k_1\in K$, if and only if $h_1^{-1}h=k_1k^{-1}\in Z(G)$; so $(h_1^{-1})^{\phi }h^{\phi }=(h_1^{-1}h)^{\phi }=(k_1k^{-1})^{\phi }=k_1k^{-1}$, that is, $h^{\phi }k=h_1^{\phi }k_1$.

It is easy to check that $\varphi \in \textrm{Aut}(G)$, moreover $[G,\varphi ]=\{g^{-1}g^{\varphi }|g\in G\}=\{k^{-1}h^{-1}h^{\phi }k|h\in H, k\in K\}=\{h^{-1}h^{\phi }|h\in H\}=[H,\phi ].$ $\square$

Lemma 3.6

Let p be a prime, and G be a group of order $p^3$. If G is non-abelian, then there exists $\varphi \in \text {Aut}(G)$ such that $z^{\varphi }=z$ for every $z\in Z(G)$ and $[G,\varphi ]\not \le G$.

Proof

If $p=2$, then either $G\simeq D_8$ or $G\simeq Q_8$ and the claim is true, as we have seen before. So we may suppose that p is odd and in this case we have that either $G=\langle x,y,z|x^p=y^p=z^p=1, [x,z]=[y,z]=1,[x,y]=z\rangle$, that is, $G=H\rtimes \langle y\rangle$ with $H=\langle x,z\rangle \simeq \mathbb Z_p\times \mathbb Z_p$, $\langle y\rangle \simeq \mathbb Z_p$, and $x^y=xz$, $z^y=z$, or $G=\langle x,y|x^{p^2}=y^p=1, x^y=x^{1+p}\rangle$, that is, $G=\langle x\rangle \rtimes \langle y\rangle$ with $x^{p^2}=1$, $y^p=1$, and $x^y=x^{1+p}$. Observe that in both cases, G is nilpotent of class 2, then for every $n\in \mathbb N$ such that $n\ge 2$, we have $(yx)^n=y^nx^n[x,y]^{n(n-1)\over 2}$, that is, $(yx)^n=y^nx^nz^{n(n-1)\over 2}$ in the first case, $(yx)^n=y^nx^nx^{p{n(n-1)\over 2}}=y^nx^{n+{p{n(n-1)\over 2}}}$ in the second case (see [8, 5.3.5 p. 137]).

In the first case, we may consider $\varphi \in \text {Aut}(G)$ defined by $x^{\varphi }=x$ and $y^{\varphi }=yx$ and we have that $z^{\varphi }=[x,y]^{\varphi }=[x,yx]=[x,y]=z$, that is, $\varphi _{|Z(G)}=\textrm{id}_{Z(G)}$. Moreover, if we consider $g=y^tx^nz^m\in G$ with $0\le t,n,m\le p-1$, then we have that $g^{\varphi }=(yx)^tx^nz^m=y^tx^tz^{{t(t-1)\over 2}}x^nz^m= y^tx^{t+n}z^{{t(t-1)\over 2}+m}$ and so $g^{-1}g^{\varphi }=(y^{-t}x^{-n}z^{nt-m})(y^tx^{t+n}z^{{t(t-1)\over 2}+m})=(x^{-n})^{y^t}x^{t+n}z^{nt+{t(t-1)\over 2}}=(xz^t)^{-n}x^{t+n}z^{nt+{t(t-1)\over 2}}= x^tz^{{t(t-1)\over 2}}$ with $0\le t\le p-1$. Then $[G,\varphi ]=\{x^tz^{{t(t-1)\over 2}}|0\le t\le p-1\}$ and this set is not a subgroup of G.

In the second case, we may consider the automorphism $\phi \in \text {Aut}(G)$ defined by $y^{\phi }=y$ and $x^{\phi }=yx$; since $[x,y]^{\phi }=[yx,y]=[x,y]$, we have that $\phi _{|Z(G)}=\textrm{id}_{Z(G)}$. Moreover, if we consider $g=y^nx^m\in G$ with $0\le n\le p-1$ and $0\le m\le p^2-1$, then we have that $g^{\phi }=y^n(yx)^m=y^{n+m}x^{m(1+p{(m-1)\over 2})}$. Therefore $g^{-1}g^{\phi }=y^{p-n}x^{p^2-m(1+p)^{p-n}}y^{n+m}x^{m(1+p{(m-1)\over 2})}=y^mx^{m(1-(1+p)^m+p{(m-1)\over 2})}$ and the set of these elements, with $0\le m\le p^2-1$, is not a subgroup of G. $\square$

For every prime p, an extra-special p-group is the iterated central product of non-abelian groups of order $p^3$ (see for instance Lemma 2.2.9 of [6]), then, from the two previous lemmas, Proposition 3.4 follows.