On the $\sigma$-Nilpotent Norm and the $\sigma$-Nilpotent Hypernorm of a Finite Group

Abstract

A subgroup $\mathcal {N}_{\sigma }(G)$ of a finite group G is the $\sigma$-nilpotent norm of G if $\mathcal {N}_{\sigma }(G)$ is the intersection of the normalizers of the $\sigma$-nilpotent residuals of all subgroups of G. Let $\mathcal {N}_{\sigma }^{0}(G) = 1$ and define $\mathcal {N}_{\sigma }^{i+1}(G)/\mathcal {N}_{\sigma }^{i}(G) = \mathcal {N}_{\sigma }(G/\mathcal {N}_{\sigma }^{i}(G))$ for $i= 0, 1, 2, ...$. By $\mathcal {N}_{\sigma }^{\infty }(G)$ denote the terminal term of the ascending series and say that $\mathcal {N}_{\sigma }^{\infty }(G)$ is the $\sigma$-nilpotent hypernorm of G. We study the influence of the $\sigma$-nilpotent norm and $\sigma$-nilpotent hypernorm of G on the structure of a finite group G. In particular, we proved that $G = \mathcal {N}_{\sigma }^{\infty }(G)$ if and only if $G^{\mathfrak {N}_{\sigma }}$ is nilpotent, where $G^{\mathfrak {N}_{\sigma }}$ is the $\sigma$-nilpotent residual of G, that is, the intersection of all normal subgroups N of G with $\sigma$-nilpotent quotient G/N.

1 Introduction

Throughout this paper, all groups are finite and G always denotes a finite group. The notation and terminology used in this paper are standard as in [8, 12].

Baer in [1] considered the intersection of normalizers of all subgroups of G, which is called the norm of G, and denoted by $\mathcal {N}(G)$. Much investigation has focused on using the concepts of the norm to determine the structure of groups (see, for example, [1,2,3,4, 22]).

Recall that a class of groups $\mathfrak {F}$ is called a formation if $\mathfrak {F}$ is closed under taking homomorphic images and subdirect products. A formation $\mathfrak {F}$ is said to be saturated if $G \in \mathfrak {F}$ whenever $G/\Phi (G) \in \mathfrak {F}$. If $\mathfrak {F}$ is a non-empty formation, then the $\mathfrak {F}$-residual of G, denoted by $G^{\mathfrak {F}}$, is the smallest normal subgroup N of G with $G/N \in \mathfrak {F}$. Li and Shen in [21] considered D(G), where D(G) is the intersection of the normalizers of derived subgroups of all subgroups of G. Shen et al. in [24] introduced S(G), which is the intersection of the normalizers of nilpotent residuals of all subgroups of G. Recall that a group G is called p-decomposable if there exists a subgroup H of G such that $G = P \times H$ for the Sylow p-subgroup P of G. In 2020, Fu, Shen and Yan [9] introduced $\mathcal {N}^{\mathcal {D}_p}(G)$, which is the intersection of the normalizers of p-decomposable residuals of all subgroups of G.

In recent years, a new theory of $\sigma$-groups has been established by Skiba and Guo (See [13, 14, 25,26,27]).

In fact, following Shemetkov [23], $\sigma =\{\sigma _i |i\in I\}$ is some partition of all primes $\mathbb {P}$, that is, $\mathbb {P}=\bigcup _{i\in I}\sigma _i$ and $\sigma _i\cap \sigma _j=\emptyset$ for all $i\ne j$. We write $\sigma (G)=\{\sigma _i|\sigma _i\cap \pi (G)\ne \emptyset \}$.

Following [25, 26], G is said to be: $\sigma$-primary if $|\sigma (G)|\le 1$; $\sigma$-soluble if every chief factor of G is $\sigma$-primary; $\sigma$-nilpotent if $G = G_1 \times \cdots \times G_n$ for some $\sigma$-primary groups $G_1, \cdots , G_n$. Clearly, a $\sigma$-nilpotent group is $\sigma$-soluble. We use $\mathfrak {S}_{\sigma }$ and $\mathfrak {N}_{\sigma }$ to denote the class of all $\sigma$-soluble groups and the class of all $\sigma$-nilpotent groups, respectively. If $G \not \in {\mathfrak {N}}_{\sigma }$ but every proper subgroup of G belongs to ${\mathfrak {N}}_{\sigma }$, then G is called an ${\mathfrak {N}}_{\sigma }$-critical or a minimal non-$\sigma$-nilpotent group.

Remark 1.1

When $\sigma ={\sigma }^1=\{\{2\},\{3\},\cdots \}$ (we use here the notation in [26]), then $\sigma$-soluble groups and $\sigma$-nilpotent groups are just soluble groups and nilpotent groups respectively, and an ${\mathfrak {N}}_{{\sigma }^{1}}$-critical group G (that is, G is not nilpotent but every proper subgroup of G is nilpotent) is a

Schmidt group. Let $\sigma =\{\{p\},p'\}$, then $\sigma$-soluble groups and $\sigma$-nilpotent groups are just p-soluble groups and p-decomposable groups, respectively. For the set $\pi =\{p_{1}, \ldots , p_{n}\}$ of primes, we deal with the partition $\sigma =\sigma ^{1\pi }=\{\{p_{1}\}, \ldots , \{p_{n}\}, \pi ' \}$ of $\mathbb {P}$ [26]. Then G is: $\sigma ^{1\pi }$-soluble if and only if G is $\pi$-soluble; $\sigma ^{1\pi }$-nilpotent if and only if G is $\pi$-special [7], that is, $G=O_{p_{1}}(G)\times \cdots \times O_{p_{n}}(G)\times O_{\pi '}(G)$.

This new theory of $\sigma$-groups is actually the development and popularization of the famous Sylow theorem, the Hall theorem of the soluble groups and the Chunihin theorem of $\pi$-soluble groups. A series of studies have been caused (See, for example, [13, 15, 18, 25, 26, 30,31,32]).

In 2021, Hu et al. [18] introduced the notion of $\sigma$-nilpotent norm as follows:

Definition 1.2

([18, Definition 1.2]) A subgroup $\mathcal {N}_{\sigma }(G)$ of G is called the $\sigma$-nilpotent norm of G if $\mathcal {N}_{\sigma }(G)$ is the intersection of the normalizers of the $\sigma$-nilpotent residuals of all subgroups of G, that is,

$$\begin{aligned} \mathcal {N}_{\sigma }(G) = \bigcap _{H \le G} N_G(H^{\mathfrak {N}_{\sigma }}). \end{aligned}$$

Now let $\mathcal {N}_{\sigma }^{0}(G) = 1$ and define $\mathcal {N}_{\sigma }^{i+1}(G)/\mathcal {N}_{\sigma }^{i}(G) = \mathcal {N}_{\sigma }(G/\mathcal {N}_{\sigma }^{i}(G))$ for $i= 0, 1, 2, \ldots$. Then, there exists a series of normal subgroups of G:

$$\begin{aligned} 1 = \mathcal {N}_{\sigma }^{0}(G) \le \mathcal {N}_{\sigma }^{1}(G) \le \mathcal {N}_{\sigma }^{2}(G) \le \cdots \le \mathcal {N}_{\sigma }^{n}(G) = \mathcal {N}_{\sigma }^{n+1}(G) = \cdots . \end{aligned}$$

Denote by $\mathcal {N}_{\sigma }^{\infty }(G)$ the terminal term of this ascending series and say that $\mathcal {N}_{\sigma }^{\infty }(G)$ is the $\sigma$-nilpotent hypernorm of G.

Remark 1.3

The $\sigma$-nilpotent norm $\mathcal {N}_{\sigma }(G)$ of G covers many possible definitions. For example, in one case when $\sigma = {\sigma }^1 =\{\{2\},\{3\},\ldots \}$, the $\sigma$-nilpotent norm $\mathcal {N}_{\sigma }(G)$ is S(G), that is, the intersection of the normalizers of nilpotent residuals of all subgroups in G.

In the case when $\sigma =\{\{p\},p'\}$, the $\sigma$-nilpotent norm $\mathcal {N}_{\sigma }(G)$ is $\mathcal {N}^{\mathcal {D}_p}(G)$, that is, the intersection of the normalizers of p-decomposable residuals of all subgroups of G.

In the other case when $\sigma =\sigma ^{1\pi }=\{\{p_{1}\}, \ldots , \{p_{n}\}, \pi ' \}$, the $\sigma$-nilpotent norm is the $\pi$-special norm, that is, the intersection of the normalizers of $\pi$-special residuals of all subgroups of G.

In [18], the authors studied the relationship of the $\sigma$-nilpotent length with the $\sigma$-nilpotent norm of G, and get some important results. In this paper, we continue to study the influence of the $\sigma$-nilpotent norm and $\sigma$-nilpotent hypernorm of G on the structure of G.

For any $\sigma$-nilpotent group G, it is easy to see that $\mathcal {N}_{\sigma }(G) = \mathcal {N}_{\sigma }^{\infty }(G) = G.$ If $\sigma = \{ \{2\}, \{2\}'\}$ and $G = S_4$, the symmetry group of degree four. Then $\mathcal {N}_{\sigma }(G) = \mathcal {N}_{\sigma }^{\infty }(G) = 1.$

Motivated by the above observations, the following question naturally arise:

Question 1.4

What is the structure of G under the condition that $\mathcal {N}_{\sigma }^{\infty }(G) = G?$

We use $\mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$ to denote the class of all finite group G with $G^{\mathfrak {N}_{\sigma }}$ nilpotent.

In this paper, we give the affirmative answer to this above problem and get the following theorem:

Theorem 1.5

Let G be a finite group. Then, the following statements are equivalent:

• (i) $G \in \mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }};$

• (ii) $G/\mathcal {N}_{\sigma }^{\infty }(G) \in \mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }};$

• (iii) $G = \mathcal {N}_{\sigma }^{\infty }(G);$

• (iv) $\mathcal {N}_{\sigma }(G/N) > 1$ for any proper normal subgroup N of G.

For a formation $\mathfrak {X}$, G is called an $\mathfrak {X}$-group if $G \in \mathfrak {X}$.

Remark 1.6

It follows from the Theorem 1.5 that G is an $\mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$-group if and only if $\mathcal {N}_{\sigma }^{\infty }(G) = G.$

In [24], Shen, Shi and Qian introduced the definition of S-groups:

Definition 1.7

A group G is called an S-group if $G = S(G)$, that is, the nilpotent residuals of all subgroups of G are normal.

The authors in [24] discussed some characters and the structure of S-groups. Recently, the S-groups are also studied by Guo and Gong, see [10, 16].

Here, we generalize the definition of S-groups and give the following definition:

Definition 1.8

A group G is called an $N_{\sigma }$-group if $G = \mathcal {N}_{\sigma }(G)$, that is, the $\sigma$-nilpotent residuals of all subgroups of G are normal.

In this paper, we would also study the properties and structure of $N_{\sigma }$-groups.

The paper is organized as follows. In Sect. 2, we prove some basic properties of the subgroups $\mathcal {N}_{\sigma }(G)$ and $\mathcal {N}_{\sigma }^{\infty }(G).$ In Sect.. 3, we give the structure of G under the condition that $\mathcal {N}_{\sigma }^{\infty }(G) = G$ and prove Theorem 1.5. In Sect. 4, we study the properties and structure of $N_{\sigma }$-groups.

2 Preliminaries

In this section, we give some basic properties of the subgroups $\mathcal {N}_{\sigma }(G)$ and $\mathcal {N}_{\sigma }^{\infty }(G).$

Lemma 2.1

(See [18, Proposition 2.5] and [6, Lemma 2.4])

• (1) If $M \le G$, then $M \cap \mathcal {N}_{\sigma }(G) \le \mathcal {N}_{\sigma }(M)$ and so $M \cap \mathcal {N}_{\sigma }^{\infty }(G) \le \mathcal {N}_{\sigma }^{\infty }(M).$

• (2) If $N \unlhd G$, then $\mathcal {N}_{\sigma }(G)N/N \le \mathcal {N}_{\sigma }(G/N)$ and so $\mathcal {N}_{\sigma }^{\infty }(G)N/N \le \mathcal {N}_{\sigma }^{\infty }(G/N)$.

• (3) If $N \unlhd G$ and $N \le \mathcal {N}_{\sigma }^{\infty }(G)$, then $\mathcal {N}_{\sigma }^{\infty }(G/N) = \mathcal {N}_{\sigma }^{\infty }(G)/N.$

Lemma 2.2

• (1) The class ${\mathfrak {N}}_{\sigma }$ of all $\sigma$-nilpotent groups is closed under taking direct products, homomorphic images and subgroups. Moreover, if H is a normal subgroup of G and $H/H\cap \Phi (G)$ is $\sigma$-nilpotent, then H is $\sigma$-nilpotent. [25, Lemma 2.5]

• (2) The class $\mathfrak {S}_\sigma$ of all $\sigma$-soluble groups is closed under taking direct products, homomorphic images and subgroups. Moreover, any extension of the $\sigma$-soluble group by a $\sigma$-soluble group is a $\sigma$-soluble group as well. [28, Lemma 2.1]

Remark 2.3

From the above Lemma 2.2(1), we know that ${\mathfrak {N}}_{\sigma }$ is a subgroup closed saturated formation.

Recall that $\Phi (G)$ is the intersection of all maximal subgroups of G and F(G) is the Fitting subgroup of G, that is the maximal normal nilpotent subgroup of G.

Lemma 2.4

(See [23, Ch. V, Theorem 26.1]) If G is a Schmidt group, then $G=P\rtimes Q$, where $P=G^{\mathfrak {N}}=G'$ is a Sylow p-subgroup of G and $Q=\langle x \rangle$ is a cyclic Sylow q-subgroup of G with $\langle x^{q} \rangle \le Z(G)\cap \Phi (G)$. Hence $Q^{G}=G$.

Lemma 2.5

(See [5, Theorem 1.2]) If G is an ${\mathfrak {N}}_{\sigma }$-critical group, then G is a Schmidt group.

Proposition 2.6

If $G = \mathcal {N}_{\sigma }(G)$, then $G^{\mathfrak {N}_{\sigma }}$ is nilpotent.

Proof

Suppose it is false and let G be a counterexample of minimal order. Then:

• (1) For every proper subgroup H of G, $H^{\mathfrak {N}_{\sigma }}$ is nilpotent and so $H^{\mathfrak {N}_{\sigma }} \le F(G)$. For every proper subgroup H of G, $H = H \cap \mathcal {N}_{\sigma }(G) \le \mathcal {N}_{\sigma }(H)$ by the hypothesis and Lemma 2.1(1) and so $H = \mathcal {N}_{\sigma }(H)$. Hence, H satisfies the hypothesis of the proposition. The choice of G implies that $H^{\mathfrak {N}_{\sigma }}$ is nilpotent. From the definition of $\mathcal {N}_{\sigma }(G)$, $H^{\mathfrak {N}_{\sigma }}$ is normal in $G = \mathcal {N}_{\sigma }(G)$. Hence $H^{\mathfrak {N}_{\sigma }} \le F(G)$. Therefore Claim (1) holds.

• (2) $\Phi (G) = 1$ If $\Phi (G) \ne 1$, then $G/\Phi (G) = \mathcal {N}_{\sigma }(G)\Phi (G)/\Phi (G) \le \mathcal {N}_{\sigma }(G/\Phi (G))$ by Lemma 2.1(2). So $G/\Phi (G)$ satisfies the hypothesis of the proposition. The choice of G implies that $(G/\Phi (G))^{\mathfrak {N}_{\sigma }} = G^{\mathfrak {N}_{\sigma }}\Phi (G)/\Phi (G) \cong G^{\mathfrak {N}_{\sigma }}/G^{\mathfrak {N}_{\sigma }} \cap \Phi (G)$ is nilpotent. Then $G^{\mathfrak {N}_{\sigma }}$ is nilpotent, a contradiction. Therefore Claim (2) holds.

• (3) $F(G) > 1$. Assume that $F(G) = 1$. Then $H^{\mathfrak {N}_{\sigma }} = 1$ for every proper subgroup H of G by Claim (1). This shows that every proper subgroup of G is $\sigma$-nilpotent. Therefore G is either a $\sigma$-nilpotent group or an ${\mathfrak {N}}_{\sigma }$-critical group. If G is a $\sigma$-nilpotent group, then $G^{\mathfrak {N}_{\sigma }} = 1$, a contradiction. Assume that G is an ${\mathfrak {N}}_{\sigma }$-critical group. Then by Lemma 2.5, G is a Schmidt group. It follows from Lemma 2.4 that $G^{\mathfrak {N}} = G^{\mathfrak {N}_{\sigma }} \le G' < G$. Hence $G^{\mathfrak {N}_{\sigma }}$ is nilpotent. This contradiction shows that Claim (3) holds.

• (4) The finial contradiction.

By Claim (3), there exists a minimal normal subgroup N of G such that $N \le F(G)$. Then N is an elementary abelian group. By Claim (2), there exists a maximal subgroup M of G such that $G = NM$ and $N \cap M =1$. Since $G/NM^{\mathfrak {N}_{\sigma }} = MNM^{\mathfrak {N}_{\sigma }}/NM^{\mathfrak {N}_{\sigma }} \simeq M/M^{\mathfrak {N}_{\sigma }} \in \mathfrak {N}_{\sigma },$ $G^{\mathfrak {N}_{\sigma }} \le NM^{\mathfrak {N}_{\sigma }}$. By Claim (1), $M^{\mathfrak {N}_{\sigma }}\le F(G)$. Hence $G^{\mathfrak {N}_{\sigma }}\le F(G)$. The final contradiction completes the proof. $\square$

Proposition 2.7

For any group G, the subgroup $\mathcal {N}_{\sigma }^{\infty }(G)$ of G is $\sigma$-soluble.

Proof

In view of Lemma 2.2(2), we only need to show that $\mathcal {N}_{\sigma }(G)$ is $\sigma$-soluble. Write $X = \mathcal {N}_{\sigma }(G)$. Let M be a maximal subgroup of X. If $M^{\mathfrak {N}_{\sigma }} > 1$, then $M^{\mathfrak {N}_{\sigma }}$ is normal in X. By Lemmas 2.1(1) and 2.1(2) and induction, $X/M^{\mathfrak {N}_{\sigma }}$ and $M^{\mathfrak {N}_{\sigma }}$ are $\sigma$-soluble. Therefore X is $\sigma$-soluble by Lemma 2.2(2). If $M^{\mathfrak {N}_{\sigma }} = 1$ for all maximal subgroups of X, then X is either a $\sigma$-nilpotent group or an ${\mathfrak {N}}_{\sigma }$-critical group. Therefore by Lemmas 2.4 and 2.5, X is $\sigma$-soluble. The proof is completed. $\square$

Theorem 2.8

Suppose that G has no $\sigma$-primary Schmit subgroups. Then $G^{\mathfrak {N}_{\sigma }} \cap \mathcal {N}_{\sigma }(G) = 1$ if and only if $Z(G^{\mathfrak {N}_{\sigma }}) = 1$.

Proof

Assume that $G^{\mathfrak {N}_{\sigma }} \cap \mathcal {N}_{\sigma }(G) = 1$. By [9, Lemma 2.1] and Lemma 2.2, we have $C_G(G^{\mathfrak {N}_{\sigma }}) \le \mathcal {N}_{\sigma }(G)$. It follows that $Z(G^{\mathfrak {N}_{\sigma }}) = C_G(G^{\mathfrak {N}_{\sigma }}) \cap G^{\mathfrak {N}_{\sigma }} \le \mathcal {N}_{\sigma }(G) \cap G^{\mathfrak {N}_{\sigma }} = 1.$ Hence, the necessity holds.

Next we prove the sufficiency, that is, if $Z(G^{\mathfrak {N}_{\sigma }}) = 1$, then $G^{\mathfrak {N}_{\sigma }} \cap \mathcal {N}_{\sigma }(G) = 1$.

Suppose that $Z(G^{\mathfrak {N}_{\sigma }}) = 1$ and $G^{\mathfrak {N}_{\sigma }} \cap \mathcal {N}_{\sigma }(G) > 1$. Hence there exists a minimal normal subgroup N of G such that $N \le G^{\mathfrak {N}_{\sigma }} \cap \mathcal {N}_{\sigma }(G)$. Since $\mathcal {N}_{\sigma }(G)$ is $\sigma$-soluble by Proposition 2.7, N is a $\sigma _i$-group for some i. Set $C = C_G(N)$.

We show that the factor G/C is a $\sigma$-nilpotent group. Suppose it is false. Then, there exists a minimal non-$\sigma$-nilpotent subgroup K/C of G/C, that is, K/C is not $\sigma$-nilpotent but all of whose proper subgroups are $\sigma$-nilpotent. Choose a subgroup L of K such that $K = CL$ and $C \cap L \le \Phi (L)$. Then by Remark 2.3, $L/\Phi (L)$ is a minimal non-$\sigma$-nilpotent subgroup. By Lemmas 2.4 and 2.5, $L/\Phi (L) = (Q/\Phi (L))(P/\Phi (L))$, where $Q/\Phi (L)$ is a normal Sylow q-subgroup of $L/\Phi (L)$ and $P/\Phi (L)$ is a cyclic Sylow p-subgroup of $L/\Phi (L)$. Since $L/\Phi (L)$ is not $\sigma$-nilpotent, clearly p and q belong to different partitions, that is, $p \in \sigma _{i}$ and $q \in \sigma _{j}$, where $\sigma _{i} \cap \sigma _{j} = \emptyset$. By Lemma 2.4, we also know that $(L/\Phi (L))^{\mathfrak {N}_{\sigma }} = L^{\mathfrak {N}_{\sigma }}\Phi (L)/\Phi (L)$ is a q-subgroup, so $L^{\mathfrak {N}_{\sigma }}$ is nilpotent. Take the Sylow q-subgroup $Q_0$ of $L^{\mathfrak {N}_{\sigma }}$. Then $Q_0$ is characterize in $L^{\mathfrak {N}_{\sigma }}$. By definition of $\mathcal {N}_{\sigma }(G)$, $\mathcal {N}_{\sigma }(G)$ normalizers $L^{\mathfrak {N}_{\sigma }}$. Hence $\mathcal {N}_{\sigma }(G)$ normalizers $Q_0$. If $N \cap Q_0 =1$, then $Q_0 \le C \cap L \le \Phi (L)$. It follows that $(L/\Phi (L))^{\mathfrak {N}_{\sigma }} = 1$. Hence $L/\Phi (L)$ is $\sigma$-nilpotent, a contradiction. Assume $N \cap Q_0 \ne 1$. It follows from N is a minimal normal subgroup of G and $N \cap Q_0$ is a normal q-subgroup of N that N is an elementary abelian q-subgroup of G. Now we show that G/C is p-nilpotent for every prime $p \ne q$. Assume that it is false, then there exists a minimal non-p-nilpotent subgroup H/C of G/C. Choose a subgroup M of H such that $H = CM$ and $C \cap M \le \Phi (M)$. Then $M/\Phi (M)$ is a minimal-non-p-nilpotent subgroup. In the light of a theorem of Ito [19, Chapert IV, Theorem 5.4], $M/\Phi (M) = (P^{*}/\Phi (M))(R/\Phi (M))$ is a Schmit group of order $p^{m}r^{n}$, where $P^{*}/\Phi (M)$ is a normal Sylow p-subgroup of $M/\Phi (M)$ and $R/\Phi (M)$ is a cyclic Sylow r-subgroup of $M/\Phi (M)$, $r \ne p$ is a prime, $m, n \ge 1$. It is clear that p and r belong to different partitions. In fact, if $p, r \in \sigma _{j}$ for some j. Then $M/\Phi (M)$ is a $\sigma$-primary Schmit group. Since $M/\Phi (M)$ is a $\sigma _{j}$-group, M is a $\sigma _{j}$-group. Clearly M is not nilpotent, so there exists a Schmit subgroup S of M. Hence, S is a $\sigma$-primary Schmit group of G, which contradicts the hypothesis. Therefore, p and r belong to different partitions. Then $(M/\Phi (M))^{\mathfrak {N}_{\sigma }} = (M/\Phi (M))^{\mathfrak {N}} = P^{*}/\Phi (M)$. Now as $M^{\mathfrak {N}_{\sigma }}\Phi (M)/\Phi (M) = (M/\Phi (M))^{\mathfrak {N}_{\sigma } }= (M/\Phi (M))^{\mathfrak {N}} = M^{\mathfrak {N}}\Phi (M)/\Phi (M)$ is a p-subgroup, $M^{\mathfrak {N}_{\sigma }}$ is nilpotent. Take the Sylow p-subgroup $P_0$ of $M^{\mathfrak {N}_{\sigma }}$. Then $P_0$ is characterize in $M^{\mathfrak {N}_{\sigma }}$. By definition of $\mathcal {N}_{\sigma }(G)$, N normalizes $M^{\mathfrak {N}_{\sigma }}$, and so N normalizes $P_0$. Since $N \cap P_0 =1$, we have that $P_0 \le C \cap M \le \Phi (M)$. It follows that $(M/\Phi (M))^{\mathfrak {N}} = 1$. Hence $M/\Phi (M)$ is nilpotent, a contradiction. Therefore G/C is p-nilpotent for every prime $p \ne q$. We now show that G/C is q-closed. Let H(p)/C be the normal Hall $p'$-subgroup of G/C for every prime p with $p \ne q$. Write $T/C = \bigcap _{p \ne q}H(p)/C$. It is clear that T/C is a normal Sylow q-subgroup of G/C. Therefore G/C is q-closed. It follows that G/T is a $q'$-group. Then by [23, Chapter III, Lemma 11.2], there exists a $q'$-subgroup W of G such that $G = TW$. Since $[N, W^{\mathfrak {N}_{\sigma }}] \subseteq N\cap W^{\mathfrak {N}_{\sigma }}= 1$, $W^{\mathfrak {N}_{\sigma }} \le C$. It follows that G/T is $\sigma$-nilpotent. Thus $G^{\mathfrak {N}_{\sigma }} \le T$. Take $Q_1$ is a Sylow q-subgroup of T containing N. Then $N \cap Z(Q_1) \ne 1$. Hence, there exists an element x such that $1 \ne x \in N \cap Z(Q_1)$. Now $C \le C_{G}(x)$ and $Q_1 \le C_{G}(x)$. Since $T = CQ_1$, we have that $T \le C_{G}(x)$. Hence $G^{\mathfrak {N}_{\sigma }} \le T \le C_{G}(x)$ and $x \in C_{G}(G^{\mathfrak {N}_{\sigma }}) \cap N \le C_{G}(G^{\mathfrak {N}_{\sigma }}) \cap G^{\mathfrak {N}_{\sigma }} = Z(G^{\mathfrak {N}_{\sigma }})$, contrary to $Z(G^{\mathfrak {N}_{\sigma }}) = 1$. This contradiction shows that G/C is a $\sigma$-nilpotent group.

Now G/C is $\sigma$-nilpotent, that is, $(G/C)^{\mathfrak {N}_{\sigma }} = 1$. Hence, $G^{\mathfrak {N}_{\sigma }} \le C = C_{G}(N)$. Consequently $N \le C_G(G^{\mathfrak {N}_{\sigma }}) \cap G^{\mathfrak {N}_{\sigma }} = Z(G^{\mathfrak {N}_{\sigma }})$, contrary to $Z(G^{\mathfrak {N}_{\sigma }}) = 1$. The proof is completed. $\square$

Remark 2.9

We do not know whether the result of Theorem 2.8 is still true if we remove the hypothesis that G has no $\sigma$-primary Schmit subgroups. Note also that when $\sigma = {\sigma }^1 =\{\{2\},\{3\},\cdots \}$, then clearly G has no $\sigma$-primary Schmit subgroups and so the hypothesis of Theorem 2.8 natural holds. Hence our Theorem 2.8 covers the Theorem 2.5 in [24].

3 The Proof of Theorem 1.5

Let $\mathfrak {M}$ and $\mathfrak {H}$ be non-empty formations, then the Gaschütz product $\mathfrak {M}\circ \mathfrak {H}$ of $\mathfrak {M}$ and $\mathfrak {H}$ is defined as follows: $G\in \mathfrak {M}\circ \mathfrak {H}$ if and only if $G^{\mathfrak {H}}\in \mathfrak {M}$. In particular, $\mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }} = \mathfrak {N}\circ \mathfrak {N}_{\sigma }$ is the class of finite groups G with $G^{\mathfrak {N}_{\sigma }}$ nilpotent.

In order to prove Theorem 1.5, we first need to prove the following two propositions.

Proposition 3.1

• (1) $\mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$ is a formation;

• (2) $\mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$ is subgroup closed, that is , if $G \in \mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$ and $H \le G$, then $H \in \mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$.

• (3) $G \in \mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$ if and only if $G/\Phi (G) \in \mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$.

Proof

• (1) It follows that [8, IV, Theorem 1.8(a)].

• (2) It is well known that the class $\mathfrak {N}$ of all nilpotent groups is subgroup closed. By Lemma 2.2, we also know that $\mathfrak {N}_{\sigma }$ is subgroup closed. Now we only need to prove the more general result: If two formations $\mathfrak {M}$ and $\mathfrak {H}$ are subgroup closed, then $\mathfrak {M}\circ \mathfrak {H}$ is subgroup closed. In fact, if $G\in \mathfrak {M}\circ \mathfrak {H}$ and $H\le G$, then $G^{\mathfrak {H}}\in \mathfrak {M}$. Since $\mathfrak {H}$ is subgroup closed and $G/G^{\mathfrak {H}}\in \mathfrak {H}$, we have that $HG^{\mathfrak {H}}/G^{\mathfrak {H}}\in \mathfrak {H}$. By the isomorphism $HG^{\mathfrak {H}}/G^{\mathfrak {H}}\simeq H/H\cap G^{\mathfrak {H}}$, we see that $H/H\cap G^{\mathfrak {H}}\in \mathfrak {H}$, and so $H^{\mathfrak {H}}\le G^{\mathfrak {H}}\in \mathfrak {M}$. But as $\mathfrak {M}$ is subgroup closed, we obtain that $H^{\mathfrak {H}}\in \mathfrak {M}$. Hence $H\in \mathfrak {M}\circ \mathfrak {H}$.

• (3) It follows from [11, Theorems 3.1.11 and 3.1.20].

$\square$

Remark 3.2

The Proposition 3.1 shows that $\mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$ is a subgroup closed saturated formation.

Proposition 3.3

The following statements are equivalent:

• (1) $G \in \mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$

• (2) $G/\mathcal {N}_{\sigma }(G) \in \mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$

Proof

$(1) \Longrightarrow (2)$:

It follows from Proposition 3.1(1).

$(2) \Longrightarrow (1)$:

Assume that this is false and let G be a counterexample of minimal order. We proceed via the following steps.

• (1) For any proper subgroup L of G, $L^{\mathfrak {N}_{\sigma }}$ is nilpotent. Since $G/\mathcal {N}_{\sigma }(G) \in \mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$, we have that $L\mathcal {N}_{\sigma }(G)/\mathcal {N}_{\sigma }(G) \in \mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$ by Proposition 3.1(2). It follows that $L\mathcal {N}_{\sigma }(G)/\mathcal {N}_{\sigma }(G) \cong L/(\mathcal {N}_{\sigma }(G) \cap L)$ that $L/(\mathcal {N}_{\sigma }(G) \cap L) \in \mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}.$ By Lemma 2.1(1), $\mathcal {N}_{\sigma }(G) \cap L \le \mathcal {N}_{\sigma }(L).$ So $L/\mathcal {N}_{\sigma }(L) \in \mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$. Hence $L \in \mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$ by the minimal choice of G.

• (2) For any nontrivial normal subgroup K of G, $(G/K)^{\mathfrak {N}_{\sigma }}$ is nilpotent. Since $G/\mathcal {N}_{\sigma }(G)K \cong (G/\mathcal {N}_{\sigma }(G))/(\mathcal {N}_{\sigma }(G)K/\mathcal {N}_{\sigma }(G))$ and $G/\mathcal {N}_{\sigma }(G) \in \mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$, we have that $G/\mathcal {N}_{\sigma }(G)K \in \mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$. By Lemma 2.1(2), $\mathcal {N}_{\sigma }(G)K/K \le \mathcal {N}_{\sigma }(G/K).$ This shows that G/K satisfies the hypothesis of the proposition. Therefore $(G/K)^{\mathfrak {N}_{\sigma }}$ is nilpotent by the choice of G.

• (3) $\Phi (G) = 1$. If $\Phi (G) \ne 1$, then $(G/\Phi (G))^{\mathfrak {N}_{\sigma }} = G^{\mathfrak {N}_{\sigma }}\Phi (G)/\Phi (G)$ is nilpotent by Claim (2). So $G^{\mathfrak {N}_{\sigma }}$ is nilpotent, that is, $G \in \mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$, a contradiction. Therefore $\Phi (G) = 1$.

• (4) The minimal normal subgroup of G is unique. Assume N is a minimal normal subgroup of G. By Claim (2), $(G/N)^{\mathfrak {N}_{\sigma }} = G^{\mathfrak {N}_{\sigma }}N/N$ is nilpotent. If $N \nsubseteq G^{\mathfrak {N}_{\sigma }}$ , then $N \cap G^{\mathfrak {N}_{\sigma }} = 1$. From the isomorphism $G^{\mathfrak {N}_{\sigma }}N/N \simeq G^{\mathfrak {N}_{\sigma }}$, we have $G^{\mathfrak {N}_{\sigma }}$ is nilpotent, a contradiction. Hence, any minimal normal subgroup of G is contained in $G^{\mathfrak {N}_{\sigma }}$. If there exist two different minimal normal subgroups of G, says $N_1, N_2$. Then $N_1 \cap N_2 = 1$ and $G^{\mathfrak {N}_{\sigma }}/N_1, G^{\mathfrak {N}_{\sigma }}/N_2$ are both nilpotent. So we have $G^{\mathfrak {N}_{\sigma }}$ is nilpotent, a contradiction. Thus the minimal normal subgroup of G is unique.

• (5) The finial contradiction.

Let N be the unique minimal normal subgroup of G. Clearly, $\mathcal {N}_{\sigma }(G) > 1$. Then, $N \le \mathcal {N}_{\sigma }(G)$. If N is solvable, then N is an elementary abelian group. Since $\Phi (G) = 1$ by Claim (3), there exists a maximal subgroup M of G such that $G = NM$ and $N \cap M = 1$. Then $G^{\mathfrak {N}_{\sigma }} \le NM^{\mathfrak {N}_{\sigma }}$. By Claim (1), $M^{\mathfrak {N}_{\sigma }}$ is nilpotent. Now as $N \le \mathcal {N}_{\sigma }(G)$ and $\mathcal {N}_{\sigma }(G)$ normalizes $M^{\mathfrak {N}_{\sigma }}$, we have that $M^{\mathfrak {N}_{\sigma }}$ is normal in G. The minimality of N implies that $N \cap M^{\mathfrak {N}_{\sigma }} = 1$ and so $NM^{\mathfrak {N}_{\sigma }} = N \times M^{\mathfrak {N}_{\sigma }}$. Since N is an elementary abelian group and $M^{\mathfrak {N}_{\sigma }}$ is nilpotent, we conclude that $G^{\mathfrak {N}_{\sigma }}$ is nilpotent. The contradiction shows that N is not soluble, and so N is a direct product of copies of a nonabelian simple group. Since N is the unique minimal normal subgroup of G, we have $C_G(N) = 1$. Let H be any proper subgroup of G. By Claim (1), $H^{\mathfrak {N}_{\sigma }}$ is nilpotent. Therefore $H^{\mathfrak {N}_{\sigma }} \cap N = 1$. Since $N \le \mathcal {N}_{\sigma }(G)$ and $\mathcal {N}_{\sigma }(G)$ normalizes $H^{\mathfrak {N}_{\sigma }}$, N normalizes $H^{\mathfrak {N}_{\sigma }}$. Hence $H^{\mathfrak {N}_{\sigma }} \le C_G(N) = 1$ and so $H^{\mathfrak {N}_{\sigma }} = 1$ for any proper subgroup H of G. It follows that G is either a $\sigma$-nilpotent group or an ${\mathfrak {N}}_{\sigma }$-critical group. If G is a $\sigma$-nilpotent group, then $G^{\mathfrak {N}_{\sigma }} = 1$, and so $G \in \mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$, a contradiction. If G is an ${\mathfrak {N}}_{\sigma }$-critical group, then $G^{\mathfrak {N}_{\sigma }} \le G^{\mathfrak {N}}$ is nilpotent by Lemmas 2.5 and 2.4, a contradiction also. The proposition is proved. $\square$

The Proof of Theorem 1.5.

Proof

$(i) \Longrightarrow (ii)$: It follows from Proposition 3.1(1).

$(ii) \Longrightarrow (iii)$: Firstly, we prove a fact that if $X > 1$ is an $\mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$-group, then $\mathcal {N}_{\sigma }(X) > 1$. In fact, if $X \in \mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$, then $X^{\mathfrak {N}_{\sigma }}$ is nilpotent, and so $C_X(X^{\mathfrak {N}_{\sigma }}) > 1$. But $C_X(X^{\mathfrak {N}_{\sigma }}) \le \mathcal {N}_{\sigma }(X)$ by Remark 2.3 and [9, Lemma 2.1], so we have $\mathcal {N}_{\sigma }(X) > 1$. Using the fact and the noting that $\mathcal {N}_{\sigma }(G/\mathcal {N}_{\sigma }^{\infty }(G)) = 1$, we get $G = \mathcal {N}_{\sigma }^{\infty }(G)$.

$(iii) \Longrightarrow (i)$: By Lemma 2.1(3), $\mathcal {N}_{\sigma }^{\infty }(G/\mathcal {N}_{\sigma }(G)) = \mathcal {N}_{\sigma }^{\infty }(G)/\mathcal {N}_{\sigma }(G)$. Then by induction, $G/\mathcal {N}_{\sigma }(G) \in \mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$. It follows from Proposition 3.3 that $G \in \mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$.

$(i) \Longrightarrow (iv)$: Using the fact which is proved in $(ii) \Longrightarrow (iii)$, it is obvious.

$(iv) \Longrightarrow (iii)$: By definition, $\mathcal {N}_{\sigma }^{i+1}(G)/\mathcal {N}_{\sigma }^{i}(G) = \mathcal {N}_{\sigma }(G/\mathcal {N}_{\sigma }^{i}(G))$ for $i = 0, 1, 2, \ldots$. By the hypothesis, $\mathcal {N}_{\sigma }(G/\mathcal {N}_{\sigma }^{i}(G)) > 1$ if $\mathcal {N}_{\sigma }^{i}(G) < G$, so $\mathcal {N}_{\sigma }^{i+1}(G) > \mathcal {N}_{\sigma }^{i}(G)$ for any $\mathcal {N}_{\sigma }^{i}(G) < G$. Hence, the terminal term $\mathcal {N}_{\sigma }^{\infty }(G)$ of the ascending series must be G. $\square$

Corollary 3.4

If H is an $\mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$-subgroup of G, then $\mathcal {N}_{\sigma }^{\infty }(G)H$ is an $\mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$-group. Consequently, $\mathcal {N}_{\sigma }^{\infty }(G)$ is contained in every maximal $\mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$-subgroup of G.

Proof

Let $M = \mathcal {N}_{\sigma }^{\infty }(G)H$. By Lemma 2.1(1), we have $\mathcal {N}_{\sigma }^{\infty }(G) \le \mathcal {N}_{\sigma }^{\infty }(M)$. Hence $M/\mathcal {N}_{\sigma }^{\infty }(M) \le \mathcal {N}_{\sigma }^{\infty }(M)H/\mathcal {N}_{\sigma }^{\infty }(M) \cong H/(H \cap \mathcal {N}_{\sigma }^{\infty }(M))$. Since $H \in \mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$, $M/\mathcal {N}_{\sigma }^{\infty }(M) \in \mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$ by Proposition 3.1. It follows from Theorem 1.5 that $M \in \mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$. In particular, we take H is any maximal $\mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$-subgroup of G. Then $M = \mathcal {N}_{\sigma }^{\infty }(G)H = H$, and so $\mathcal {N}_{\sigma }^{\infty }(G) \le H$. Therefore $\mathcal {N}_{\sigma }^{\infty }(G)$ is contained in every maximal $\mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}$-subgroup of G. $\square$

Following [12], we use $Int_{\mathfrak {X}}(G)$ to denotes the intersection of all $\mathfrak {X}$-maximal subgroups of G, where $\mathfrak {X}$ is a class of finite groups.

Remark 3.5

The above Corollary 3.4 shows that $\mathcal {N}_{\sigma }^{\infty }(G) \le Int_{\mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}}(G).$ But we do not know whether $Int_{\mathcal {F}_{\mathfrak {N}\mathfrak {N}_{\sigma }}}(G) \le \mathcal {N}_{\sigma }^{\infty }(G)$ is true.

4 The Properties and Structures of $N_{\sigma }$-groups

In this section, we discuss the class of $N_{\sigma }$-groups.

Recall that a group G is an $N_{\sigma }$-group if $G = \mathcal {N}_{\sigma }(G)$, that is, the $\sigma$-nilpotent residual of every subgroup of G is normal in G; a group G is an S-group if $G = S(G)$, that is, the nilpotent residual of every subgroup of G is normal in G. Firstly we discuss the relationship between $N_{\sigma }$-groups and S-groups.

The following example shows that $N_{\sigma }$-groups and S-groups are different.

Example 4.1

Let $G=A_5$ be the alternating group with degree 5 and $\sigma = \{ \sigma _1, \sigma _2 \}$, where $\sigma _1 = \{ 2, 3,5\}$ and $\sigma _2 = \{2,3,5\}'$. Clearly, G is a $\sigma$-nilpotent, so every subgroup of G is $\sigma$-nilpotent by Lemma 2.2(1). Hence G is an $N_{\sigma }$-group. However, G is not an S-group. In fact, $A_4{^{\mathfrak {N}}} \ne 1$ is not normal in G.

Now we give the following proposition to show $N_{\sigma }$-groups and S-groups are same under certain conditions.

Recall (see [25, 26, 29]) that a set $\mathcal {H}$ of subgroups of G is said to be a complete Hall $\sigma$-set of G if every non-identity member of $\mathcal {H}$ is a Hall $\sigma _i$-subgroup of G for some $\sigma _i$ and $\mathcal {H}$ contains exactly one Hall $\sigma _i$-subgroup for every $\sigma _i\in \sigma (G)$. G is said to be: $\sigma$-full if G possesses a complete Hall $\sigma$-set; a $\sigma$-full group of Sylow type if every subgroup of G is a $D_{\sigma _i}$-group for all $\sigma _i\in \sigma (G)$.

Proposition 4.2

Let G be a $\sigma$-full group of Sylow type and $\mathcal {H} = \{W_1, \ldots , W_t\}$ be a complete Hall $\sigma$-set of G such that $W_i$ is a nilpotent $\sigma _i$-subgroup for all $i = 1, \ldots , t$. Then G is an $N_{\sigma }$-group if and only if G is an S-group.

Proof

Let H be any subgroup of G. It is clear that $H^{\mathfrak {N}_{\sigma }} \le H^{\mathfrak {N}}$. Now we show $H^{\mathfrak {N}} \le H^{\mathfrak {N}_{\sigma }}$. Since $H/H^{\mathfrak {N}_{\sigma }} \in \mathfrak {N}_{\sigma }$, we have that $H/H^{\mathfrak {N}_{\sigma }} = H_1/H^{\mathfrak {N}_{\sigma }} \times H_2/H^{\mathfrak {N}_{\sigma }} \times \ldots H_t/H^{\mathfrak {N}_{\sigma }},$ where $H_i/H^{\mathfrak {N}_{\sigma }}$ is a Hall $\sigma _i$-subgroup of $H/H^{\mathfrak {N}_{\sigma }}$ for all $i = 1, 2, \ldots , t.$ By the hypothesis, we see that $H_i/H^{\mathfrak {N}_{\sigma }}$ is nilpotent. Hence $H/H^{\mathfrak {N}_{\sigma }}$ is nilpotent, that is, $H/H^{\mathfrak {N}_{\sigma }} \in \mathfrak {N}$. Therefore $H^{\mathfrak {N}} \le H^{\mathfrak {N}_{\sigma }}$. It follows that $H^{\mathfrak {N}} = H^{\mathfrak {N}_{\sigma }}$. If G is an $N_{\sigma }$-group, then $H^{\mathfrak {N}} = H^{\mathfrak {N}_{\sigma }}$ is normal in G for every subgroup H of G. Hence G is an S-group. If G is an S-group, then $H^{\mathfrak {N}_{\sigma }} = H^{\mathfrak {N}}$ is normal in G for every subgroup H of G. It follows that G is an $N_{\sigma }$-group. Therefore, the proof is completed. $\square$

Next we give some characters of $N_{\sigma }$-groups and some judging theorems for a group to be an $N_{\sigma }$-group.

Note that for any normal subgroup N of G, $(G/N)^{\mathfrak {N}_{\sigma }} = G^{\mathfrak {N}_{\sigma }}N/N$ (see [11, Lemma 2.1.3]). Hence the following facts are clear from Definition 1.8, Lemma 2.2(1), Lemmas 2.4 and 2.5:

Proposition 4.3

• (i) The subgroups of an $N_{\sigma }$-group are $N_{\sigma }$-groups;

• (ii) The quotient groups of an $N_{\sigma }$-group are $N_{\sigma }$-groups;

• (iii) If G is a $\sigma$-nilpotent group or a minimal non-$\sigma$-nilpotent group, then G is an $N_{\sigma }$-group.

Lemma 4.4

G is a $\sigma$-nilpotent group if and only if there is a normal nilpotent subgroup N of G such that $G/N'$ is a $\sigma$-nilpotent group.

Proof

The necessity follows from Lemma 2.2(1). We now show the sufficiency. Since N is nilpotent, then $N' \le \Phi (G)$. Then as $G/N'$ is a $\sigma$-nilpotent group, we have that $G/\Phi (G)$ is $\sigma$-nilpotent by Lemma 2.2(1). Hence G is a $\sigma$-nilpotent group by Remark 2.3. The lemma is proved. $\square$

Remark 4.5

Through the proof process of the above Lemma 4.4, we can see that the Lemma 4.4 is also valid when the class of $\sigma$-nilpotent groups is replaced by any saturated formation. Hence Lemma 4.4 covers a famous result of Hall (see [17, page 2]).

From Lemmas 4.4,  2.2 and Proposition 2.6, we directly obtain the following:

Theorem 4.6

G is a $\sigma$-nilpotent group if and only if G is an $N_{\sigma }$-group and $G/(G^{{\mathfrak {N}}_{\sigma }})'$ is a $\sigma$-nilpotent group.

Following Definition 3.10 in [20], let $G_{n+1}=[G_n,G]$, if a group G satisfies $G_{c+1}=1$, then we say that G is nilpotent of class $\le c$; the least such number c is called the nilpotency class of G.

Theorem 4.7

If $G/(G^{{\mathfrak {N}}_{\sigma }})'$ is an ${\mathfrak {N}}_{\sigma }$-critical group and $G^{{\mathfrak {N}}_{\sigma }}$ is of nilpotency class 2, then G is an $N_{\sigma }$-group.

Proof

Assume that the theorem is false and let G be a counterexample of minimal order. Then there exists $M < G$ such that $M^{{\mathfrak {N}}_{\sigma }}$ is not normal in G. Consider $N_G(M^{{\mathfrak {N}}_{\sigma }})$. As $G^{{\mathfrak {N}}_{\sigma }}$ is nilpotent of class 2, $(G^{{\mathfrak {N}}_{\sigma }})' \le Z(G^{{\mathfrak {N}}_{\sigma }})$. Since $M/M \cap G^{{\mathfrak {N}}_{\sigma }} \simeq MG^{{\mathfrak {N}}_{\sigma }}/G^{{\mathfrak {N}}_{\sigma }} \le G/G^{{\mathfrak {N}}_{\sigma }} \in {\mathfrak {N}}_{\sigma }$ and ${\mathfrak {N}}_{\sigma }$ is subgroup closed by Lemma 2.2(1), $M^{{\mathfrak {N}}_{\sigma }} \le G^{{\mathfrak {N}}_{\sigma }}$. Then $G > N_G(M^{{\mathfrak {N}}_{\sigma }}) \ge M(G^{{\mathfrak {N}}_{\sigma }})'.$ Since $G/(G^{{\mathfrak {N}}_{\sigma }})'$ is an ${\mathfrak {N}}_{\sigma }$-critical group, $M(G^{{\mathfrak {N}}_{\sigma }})'/(G^{{\mathfrak {N}}_{\sigma }})'$ is $\sigma$-nilpotent. It follows from $M(G^{{\mathfrak {N}}_{\sigma }})'/(G^{{\mathfrak {N}}_{\sigma }})' \simeq M/M \cap (G^{{\mathfrak {N}}_{\sigma }})' \in {\mathfrak {N}}_{\sigma }$ that $M^{{\mathfrak {N}}_{\sigma }} \le M \cap (G^{{\mathfrak {N}}_{\sigma }})' \le Z(G^{{\mathfrak {N}}_{\sigma }}).$ Hence $G^{{\mathfrak {N}}_{\sigma }} \le N_G(M^{{\mathfrak {N}}_{\sigma }}),$ and so $MG^{{\mathfrak {N}}_{\sigma }} \le N_G(M^{{\mathfrak {N}}_{\sigma }}) < G.$ Then $MG^{{\mathfrak {N}}_{\sigma }}/(G^{{\mathfrak {N}}_{\sigma }})'$ is $\sigma$-nilpotent as $G/(G^{{\mathfrak {N}}_{\sigma }})'$ is an ${\mathfrak {N}}_{\sigma }$-critical group. Hence by Lemma 4.4, $MG^{{\mathfrak {N}}_{\sigma }}$ is $\sigma$-nilpotent. It follows from Lemma 2.2(1) that M is $\sigma$-nilpotent and so $M^{{\mathfrak {N}}_{\sigma }} = 1$ is normal in G. The contradiction completes the proof. $\square$

By using the same method of the Proof of Theorem 4.7, we can also get the following general result.

Theorem 4.8

If $G/(G^{{\mathfrak {N}}_{\sigma }})_n$ is an ${\mathfrak {N}}_{\sigma }$-critical group and $G^{{\mathfrak {N}}_{\sigma }}$ is of nilpotency class n, where n is an integer and $n \ge 2$, then G is an $N_{\sigma }$-group.

Proof

Assume that the theorem is false and let G be a counterexample of minimal order. Then there is $M < G$ such that $M^{{\mathfrak {N}}_{\sigma }}$ is not normal in G. Consider $N_G(M^{{\mathfrak {N}}_{\sigma }})$. If $G^{{\mathfrak {N}}_{\sigma }}$ is nilpotent of class n, then $(G^{{\mathfrak {N}}_{\sigma }})_n \le Z(G^{{\mathfrak {N}}_{\sigma }})$. Clearly $M^{{\mathfrak {N}}_{\sigma }} \le G^{{\mathfrak {N}}_{\sigma }}$, so $G > N_G(M^{{\mathfrak {N}}_{\sigma }}) \ge M(G^{{\mathfrak {N}}_{\sigma }})_n.$ Since $G/(G^{{\mathfrak {N}}_{\sigma }})_n$ is an ${\mathfrak {N}}_{\sigma }$-critical group, $M(G^{{\mathfrak {N}}_{\sigma }})_n/(G^{{\mathfrak {N}}_{\sigma }})_n$ is $\sigma$-nilpotent. It follows from $M(G^{{\mathfrak {N}}_{\sigma }})_n/(G^{{\mathfrak {N}}_{\sigma }})_n \simeq M/M \cap (G^{{\mathfrak {N}}_{\sigma }})_n \in {\mathfrak {N}}_{\sigma }$ that $M^{{\mathfrak {N}}_{\sigma }} \le M \cap (G^{{\mathfrak {N}}_{\sigma }})_n \le Z(G^{{\mathfrak {N}}_{\sigma }}).$ Hence $G^{{\mathfrak {N}}_{\sigma }} \le N_G(M^{{\mathfrak {N}}_{\sigma }}),$ and so $MG^{{\mathfrak {N}}_{\sigma }} \le N_G(M^{{\mathfrak {N}}_{\sigma }}) < G.$ Therefore $MG^{{\mathfrak {N}}_{\sigma }}/(G^{{\mathfrak {N}}_{\sigma }})_n$ is $\sigma$-nilpotent as $G/(G^{{\mathfrak {N}}_{\sigma }})_n$ is an ${\mathfrak {N}}_{\sigma }$-critical group. Since $(G^{{\mathfrak {N}}_{\sigma }})_n \le (G^{{\mathfrak {N}}_{\sigma }})'$, $MG^{{\mathfrak {N}}_{\sigma }}/(G^{{\mathfrak {N}}_{\sigma }})'$ is $\sigma$-nilpotent by Lemma2.2. Hence by Lemma 4.4, $MG^{{\mathfrak {N}}_{\sigma }}$ is $\sigma$-nilpotent. Consequently M is $\sigma$-nilpotent, and so $M^{{\mathfrak {N}}_{\sigma }} = 1$ is normal in G. The contradiction completes the proof. $\square$

Remark 4.9

It is clear that Theorem 4.5 and 4.6 in [24] are corollaries of our Theorem 4.8.