1 Introduction

With the advancement of IoT-based services and applications, the academicians and researchers of 3GPP have recommended 5G communication technology of the cellular network from the recent past [1,2,3]. The 5G technology suggests advanced aspects related to LTE-A network as non-3GPP inter-working, the formative arrangement of User Plane (UP) operations which are described as logical networks (user and control plane operations) with different potentials [4]. Further, User Equipment (UE) may broadcast Non-Access Stratum (NAS) information to the core network of the 5G for session and mobility administration, that hasn’t been attained in preceding cellular network technologies [5, 6]. Moreover, these attributes identifies various aspects in the security framework of the 5G handover network. There are different handover services and applications as a vehicular management system, e-health care, and multimedia services, etc. because of the portability of numerous IoT devices/equipment in the 5G network [7,8,9,10].

Although, a key structure of 5G handover suffers from authentication complexities and various security susceptibilities [11]. In the handover key structure, an attacker can breach the secret session keys from genuine base-stations. Nonetheless, the partition of secret keys among base-stations avoids these issues at the time of handover. However, this approach neglects the negotiated key in one particular gNB from the other one. The source Next Generation (5G) Base-Station Node (\(gNB_{s}\)) broadcasts session key to the target Next Generation (5G) Base-Station Node (\(gNB_{t}\)). The \(gNB_{s}\) obtains a fresh session key by adopting a one-way operation and obtains key backward secrecy (KBS). The KBS restrains gNB’s from generating the preceding keys from the established key. Contrarily, the gNB’s might learn the entire keys used in earlier sessions of handover. Correspondingly, the KFS (forward secrecy) is preserved to provide that the communicating participants place various specifications in obtaining the new key for subsequent gNB. Moreover, the current gNB doesn’t form subsequent keys. The structure of the 5G handover key fails to establish KFS if an attacker negotiates an honest base-station. In this situation, \(gNB_{t}\) doesn’t provide fresh session keys because of de-synchronization. Hence, it demonstrates the security deficiencies in the handover key structure, and an attacker may negotiate prior keys between gNB and UE. The potential attacks may be sustained before the aforesaid modifications of the current key as the key specifications are obtained from preceding keys [12]. Furthermore, inter-gNB handover scheme in 5G networks degrades the transmission overhead because of numerous rounds of information transmission among the communicating participants. Hence, it is recommended to introduce a cost-efficient and attack resilient inter-gNB handover protocol in the 5G network.

1.1 Fundamental Security Properties of Handover Protocol

The security properties of the 5G handover are required to establish mutual authentication and shared secret key compliance between the communicating participants to satisfy the integrity for subsequent handover. The proposed 5G inter-gNB handover protocol must conclude the following properties.

  • The protocol should maintain the privacy of the communicating participants during the authentication process. Only the home network can obtain the permanent identity of mobile devices.

  • The protocol should maintain forward/backward secrecy with key re-freshness in each new handover authentication connection even if an attacker knows the private keys.

  • The protocol must establish robust secrecy during the authentication to reduce the possible attacks in the 5G network.

  • It is known that the UE is a low power resource device and the network channel has controlled frequency. Therefore, the protocol must be designed in a form that mandates the reduced overhead.

To achieve the necessary security properties during the handover process, 3GPP has introduced the handover mechanism [11]. However, the protocol incurs security vulnerabilities such as 1) several messages correspondence are needed to communicate with the AMF (serving network). Therefore, the 5G network reduces the transmission efficiency. 2) The 5G handover key derivation structure proposed by 3GPP brings out various gNB keys based on the horizontal/vertical key approach. Hence, the researchers have proposed various handover protocols in 5G communication networks [13,14,15,16,17]. Unfortunately, authentication complexity, high communication, and computation overhead are observed in these protocols. In addition, these protocols are susceptible to several security attacks. Hence, these handover protocols are not much suitable for efficient handover authentication in the 5G communication network.

To overcome these issues, we introduce Secrecy and Efficiency Aware Inter-gNB (SEAI) handover AKA protocol in 5G network. The proposed protocol avoids the problem of key escrow without involving any third party in establishing the secret keys. Also, the UE/gNB shows a secret correspondence of their identity by collision avoidance hash function and chooses secret keys in the handover initialization stage. The protocol doesn’t execute the time-consuming exponentiation operations and shows less overhead. Moreover, the protocol doesn’t transmit the secret keys over the public channel to preserve the handover key authentication.

1.2 Core Technical Improvements

To overcome the above-raised issues, we propose the Secrecy and Efficiency Aware Inter-gNB (SEAI) handover AKA protocol in 5G communication network. The main improvements of the protocol compared to previous handover schemes are:

  1. 1.

    We investigate the current 5G handover key structure and analyze its security deficiencies such as bogus base-station attack and synchronization failure.

  2. 2.

    We introduce the SEAI handover AKA protocol to overcome the security deficiencies from the current handover protocols of the 5G communication network. In the proposed protocol, \(gNB_{t}\) and UE establish mutual authentication at the time of handover execution without broadcasting the secret keys in the air. Moreover, the protocol mandates the KFS and KBS.

  3. 3.

    The confidentiality, integrity, and session key secrecy in the SEAI handover AKA protocol are proven secure by adopting ROM. Also, the Automated Validation of Internet Security Protocols and Applications (AVISPA) tool presents correctness and verification of the protocol. Moreover, the attack and security analysis are provided for numerous security specifications. The analysis represents that the protocol averts the potential attacks.

  4. 4.

    The performance estimation of current and proposed handover protocols is calculated on the basis of communication, computation, and transmission overhead. The estimation results represent that the SEAI handover AKA protocol is efficient and secure compared to the previously proposed handover schemes.

  5. 5.

    The handover delay & key size is computed for the proposed and existing handover protocols based on hope count, number of users. Also, we analyze the protocols based upon the energy consumption during the handover authentication process.

The rest of the article is formed as follows: Sect. 2 illustrates the network model of 5G handover, key hierarchy, handover structure, and the existing handover methodologies. The security susceptibilities of the 5G handover protocol are discussed in Sect. 3. Section 4 discusses the proposed SEAI handover AKA protocol in the 5G network. The formal security proof using ROM, correctness, and informal analysis of the protocol are presented in Sect. 5. Section 6 demonstrates the performance estimation of 5G handover AKA protocols. Lastly, Sect. 7 concludes the article.

2 Overview and Existing Methodologies

The 5G network derives a fundamental security architecture of the LTE-A network. 3GPP has done some security design contributions in the 5G network after the performance and practical operations. Although, a novel handover authentication framework is required to mandate these modifications for the 5G network. In this section, we demonstrate the overview of the 5G handover framework, handover key structure, and key hierarchy. To obtain mutual authentication and overcome the bandwidth consumption from the 5G network, researchers and academicians have introduced numerous handover methodologies. We illustrate these protocols based on their security features and issues in this section also.

2.1 Network Model of 5G Communication Network in Handover

The communication in 5G network framework is established by the following participants as Access and Mobility Management Function (AMF)/Security Anchor Function (SEAF), Authentication Credential Repository and Processing Function (ARPF), Session Management Function (SMF), Policy Control Function (PCF), and Authentication Server Function (AUSF) as shown in Fig. 1 [18,19,20]. In this framework, UE establishes the connection with various gNBs and AMF maintains secure communication using \(Key_{AMF}\). Further, UE verifies the AUSF while subscription information is kept by the ARPF. For the authentication with UE, the ARPF stores the secure symmetric key \(S_{key}\). Also, ARPF computes the authentication vectors (AVs) by executing the cryptographic operations with the security parameters. The Security Policy Control Function (SPCF) consists of security to the SMF and AMF. The security credentials has the key length, integrity and confidentiality algorithm, and AUSF information. The Non-access Stratum(NAS) and AS layers maintain their communication traffic to establish gNB security [21]. Whenever UE communicates in the 5G network, the AS layer establishes the secrecy between the UE, NAS layer, and gNB. In addition, the N3-UP (path of user plane signaling) and N2-CP (path of control plane signaling) are established between UE & User Plane Function (UPF) and UE & AMF respectively [22]. These new updates are the autonomous paths for user/control planes and key algorithms (integrity and encryption).

Fig. 1
figure 1

A handover framework of 5G communication network

2.2 Key Hierarchy

The 5G network key hierarchy is designed for the efficient structure of numerous keys among the participating entities in the communication [11]. The first transition key \(Key_{AUSF}\) is computed by the ARPF to maintain secret communication between UE and ARPF. From this key, another transition key \(Key_{SEAF}\) is computed between UE and AUSF to determine \(Key_{AMF}\). In addition, the key \(Key_{gNB}\) is retrieved at AMF and send to the gNB. The UE establishes authentication compliance with AMF in support of AUSF/ARPF. The AMF and UE compute the \(Key_{AMF}\) using \(Key_{SEAF}/ Key_{AUSF}\) after obtaining the mutual authentication. The \(Key_{AMF}\) is valid for the certain period computed for the successive AKA process and generates four sub-keys from it. The two sub-keys \(Key_{NASenc}\) and \(Key_{NASint}\) are computed for encryption verification and integrity respectively. UE and AMF derives the third sub-key Non-3GPP access Inter-working Function (\(Key_{N3IWF}\)) from \(Key_{AMF}\) for non-3GPP access. Moreover, UE and gNB generate the fourth sub-key \(Key_{gNB}\) that computes another four keys. Firstly, two keys \(Key_{RRCenc}\) and \(Key_{RRCint}\) are required to authenticate the Radio Resource Control (RRC) signaling encryption and its integrity respectively. In addition, the keys \(Key_{UPenc}\) and \(Key_{UPint}\) are required to verify the UP data traffic encryption and integrity respectively. Also, \(Key_{gNB}\) is renewed during handover whenever the UE enters into the coverage area of another gNB.

2.3 Handover Structure

In this section, we will demonstrate the Xn-based (inter-gNB) 5G handover structure. In the inter-gNB handover, AMF and UE obtain the authentication process to fulfill the security properties. For secure communication during handover, \(gNB_{s}\) generates the \({Key_{NG-RAN}}^{'}\) (preceding \(Key_{gNB}\)) for \(gNB_{t}\). Also, \(Key_{gNB}\) is concatenated at handover key chaining before the subsequent AKA process [11]. By using the one-way hash, \(gNB_{s}\) generates the next \(Key_{gNB}\) from the present gNB and applies the current key from AMF. Then, AMF transmits these information to \(gNB_{t}\) after accomplishing the inter-gNB handover and apply it for subsequent handover. NH Chaining Counter (NCC) and Next Hop (NH) are the key parameters in handover key chaining. AMF setups the next NH parameters generated from \(Key_{AMF}\) for respective handover repeatedly. The communication mechanism of 5G inter-gNB handover is shown in Fig. 2 [11]. It is analyzed that the \(gNB_{s}\) obtains the specific key parameters \(\lbrace NH_{NCC}, NCC \rbrace \) from the preceding handover. The counter of NH key update is \(NH_{NCC}\). The \(gNB_{s}\) computes \({Key_{NG-RAN}}^{'}\) from NH key and \(Key_{gNB}\) by performing horizontal and vertical key operations respectively for \(gNB_{t}\). The horizontal and vertical key operations are \({Key_{NG-RAN}}^{'}= KDF(\eta ||NH_{NCC})\) and \({Key_{NG-RAN}}^{'}= KDF(\eta ||Key_{gNB})\) respectively, where \(\eta = ARFC-DL||PCIA\), \({NH_{NCC}}^{'} = KDF(Key_{gNB}||Key_{AMF})\) (original value of NH), \(NH_{NCC}= KDF(NH_{NCC-1}||Key_{AMF})\), \(NH_{NCC-1}\) (preceding value of NH), absolute radio frequency channel-down link(ARFC-DL), and physical cell identity allocation (PCIA). In the horizontal handover, \(gNB_{s}\) doesn’t achieve the specific NH key, and \(\lbrace NH_{NCC}, NCC \rbrace \) are appeared before the completion of inter-gNB 5G handover. On the other hand in vertical handover, \(gNB_{s}\) has specific NH key derived in 5G inter-gNB handover, and AMF and UE could fetch the NH only.

Fig. 2
figure 2

Inter-gNB 5G handover mechanism

The \(gNB_{s}\) transmits \(\lbrace NCC, {Key_{NG-RAN}}^{'} \rbrace \) to \(gNB_{t}\) in inter-gNB handover. It is analyzed that the \(gNB_{s}\) executes the vertical operation and future keys between \(gNB_{t}\) and UE. In this handover, the AMF and \(gNB_{t}\) transmit their handover request/response to UE. Later, UE verifies the acknowledged NCC from the equipped NCC. If it authenticates, UE performs vertical operation from the current \(Key_{gNB}\) to generate \({Key_{NG-RAN}}^{'}\). Or, UE tries to integrate the NCC by generating NH key regularly, until it authenticates and executes the horizontal key operation to derive \({Key_{NG-RAN}}^{'}\). Moreover, the \(gNB_{t}\) transmits the path change request to the AMF in inter-gNB 5G handover after the handover accomplishment with UE. Then, AMF increases NCC value by one and derives the specific NH key. Also, AMF transmits the \(\lbrace NH_{NCC+1}, NCC+1 \rbrace \) to \(gNB_{t}\) for further handover.

2.4 Existing Methodologies

Cao et al. [13] discussed the privacy-preserving handover authentication protocol for 5G HetNets using the Software Defined Network (SDN). The protocol obtains the mutual authentication and key agreement between base-stations and mobile devices without any other entities. Also, the protocol overcomes the system authentication complexity and minimizes bandwidth consumption. However, similar to the 3GPP-5G handover AKA protocol, the protocol fails to avoid the de-synchronization of communicating entities that lead to DoS attack because of sequence number (SQN) mismatch. In the protocol, it is considered that the SQN is maintained between base-station and UE. In one registration, the value of SQN is used for entire the n connections and increases the value by one at UE/base-station. An adversary may attempt a bogus registration attempt by using previous messages and SQN value become inconsistent. If the genuine UE attempts to create the connection with the target base-station, the session keys and message authentication code are not matched. Therefore, the genuine UE will be unauthorized to access the network during handover. To avoid the above issues, Sharma et al. [14] proposed the handover authentication protocol that maintains the privacy-preservation and key secrecy. Also, the protocol avoids all the security susceptibilities and withstands security attacks. However, numerous message correspondence with the base-station and terminal (UE) carries handover breach and increases the overhead because the serving network is very far from base-station. Hence, the protocol incurs authentication complexity. Also, the source base-station computes numerous keys for target base-stations that enhance the probability of dodging the secret keys.

Zhang et al. [15] introduced the Elliptic Curve Cryptography (ECC)-based handover authentication protocol by using chameleon hash function key pairs to avoid the authentication complexity. However, the protocol obtains all the security characteristics but suffer from identity privacy preservation and MitM attack. Also, the protocol exhibits a huge network and transmission overhead due to the additional use of point multiplication key operations. Han et al. [16] designed the efficient handover AKA to enhance security properties and maintain mutual authentication. Also, the protocol incurs less overhead and establishes the key secrecy. However, the protocol suffers from DoS attack similar to Cao’s protocol. Due to the use of Extensible Authentication Protocol (EAP)-AKA [23], the proposed protocol suffers from identity privacy preservation and security vulnerabilities such as redirection and MitM attack. Recently, Kumar et al. [17] designed the ECC-based handover authentication protocol for 5G-wireless LAN networks. The protocol obtains mutual authentication and most of the security properties such as key forward/backward secrecy, anonymity. However, the protocol fails to preserve the identity of the communicating participants and suffers from redirection, MitM attack. In addition, the protocol incurs huge communication and computational overhead due to the additional use of point multiplication functions during the handover authentication process.

From the existing handover methodologies, it is noticed that these protocols are susceptible to various known attacks and exhibit huge network overhead. Also, the protocols fail to provide the key secrecy and suffer from authentication complexity. Therefore, the above-discussed protocols are not well suited for efficient handover development in the 5G communication network. To avoid these problems, we introduce the SEAI handover AKA protocol in the 5G network to obtain necessary security requirements. The SEAI protocol is free from the problem of key escrow as there is no entanglement of any third party in establishing the secret keys. Also, the communicating participants send their identity securely in the handover process and don’t transmit the secret keys in the public channel during the handover agreement. The protocol operates the key operations using the point multiplication functions and enhances its efficiency compared to the existing protocols. Moreover, the protocol avoids potential attacks and provides all the security properties.

3 Security Weaknesses in 5G Handover Mechanism

This section illustrates the security susceptibilities in the 5G handover mechanism proposed by the 3GPP and other various researchers. These security problems represent various adversities in the steady communication of the 5G handover network. Let consider, an attacker \({\mathcal {ATT}}\) impersonates the genuine base-station (gNB) and implants the forged base-station \(gNB_{{\mathcal {ATT}}}\) in the communication network. \({\mathcal {ATT}}\) may approach its stored parameters by massive attacks as gNB is implanted very far to the AMF.

3.1 De-synchronization Attack

\({{\mathcal {ATT}}}\) can install the \(gNB_{{\mathcal {ATT}}}\) that performs the Denial-of-Service (Dos) and leads to de-synchronization during the 5G handover. The prime target of \(gNB_{{\mathcal {ATT}}}\) is to build the bogus information of NCC and dodge the imminent keys. The \({{\mathcal {ATT}}}\) can impose to \(gNB_{t}\) to disturb the key forward secrecy by performing horizontal key operations. The value of NCC can be compromised by manipulating the information between \(gNB_{s}\) and \(gNB_{t}\) in the 5G handover mechanism. The \(gNB_{{\mathcal {ATT}}}\) chooses a large prime number to impersonate the NCC and transmits to \(gNB_{t}\) during second handover response as shown in Fig. 2.

\({{\mathcal {ATT}}}\) sends the original and false NCC to UE for maintaining the synchronization. The NCC value in path shifting information is negligible than that obtained by \(gNB_{{\mathcal {ATT}}}\). In addition, the \(gNB_{t}\) and UE generate future handover keys on the basis of present \(Key_{gNB}\) in place of \(NH_{NCC+1}\). Therefore, \(gNB_{{\mathcal {ATT}}}\) may not obtain the following \(Key_{gNB}\) because of forward secrecy failure. The gNB acquires the following key of \({Key_{NG-RAN}}^{'}\) from \(Key_{gNB}\) because \({{\mathcal {ATT}}}\) can know ARFC-DL and PCIA. Moreover, \({{\mathcal {ATT}}}\) impersonates the UE by sending the original value of NCC and executes de-synchronization. \({{\mathcal {ATT}}}\) can damage the NCC by disguising the information AMF to \(gNB_{t}\). The \(gNB_{t}\) fails to accommodate to the fresh value of NCC because bogus information has a lesser value of NCC compared to the initial one. To overcome the above security concerns, the Internet Protocol Security scheme is applied in path shifting and its confirmation message. Although, numerous links of IPSec with gNBs are prescribed to establish in these transmitted messages with AMF. \({{\mathcal {ATT}}}\) may deploy the de-synchronization by information flooding/drop to block the \(gNB_{t}\) from recovering the NCC. Accordingly, the \(gNB_{t}\) may not modify the NCC and synchronization of the keys is not established. \({{\mathcal {ATT}}}\) may know the secret handover information from the communicating parties from \(gNB_{{\mathcal {ATT}}}\) and degrades the network efficiency.

3.2 Verification Failure

The 5G inter-gNB handover mechanism needs various request/response message communication rounds with the AMF and \(gNB_{s}/gNB_{t}\) that suffers from handover explosion. Also, it increases the overhead because the AMF is installed far from gNB. Hence, the 5G handover network suffers from authentication complexity/verification failure. The \(gNB_{s}\) generates legitimate keys for numerous \(gNB_{t}\) from the current one by using required specifications in the 5G handover mechanism. For explanation, \(gNB_{s}\) may obtain the \({Key_{NG-RAN}}^{''}\) between the UE and \(gNB_{t}\) from \({Key_{NG-RAN}}^{'}\). Once the \(gNB_{s}\) is attacked, the \({{\mathcal {ATT}}}\) knows all the subsequent keys. Therefore, the key backward secrecy is not obtained in the current 5G handover communication.

4 Proposed SEAI Handover AKA Protocol

In this section, we discuss the SEAI handover AKA protocol to avoid the security deficiencies from the previously proposed handover protocols. The proposed protocol has three stages: a) establishment stage; b)handover initialization stage and c) handover authentication stage. The methodology of Elliptic Curve Cryptography (ECC) is illustrated in the establishment stage. UE is authenticated at AMF and \(gNB_{s}\) defines the handover request/response information to UE for preceding communication in the initial authentication stage. Moreover, the \(gNB_{t}\) and UE executes the handover authentication stage when UE arrives in the area of \(gNB_{t}\). The used notations and their meaning in the proposed protocol are reported in Table 1.

Table 1 Used notations and their meaning in the proposed SEAI protocol

4.1 Establishment Stage

In order to achieve the authentication between \(gNB_{t}\) and UE in the SEAI handover AKA protocol, we are applying ECC [24]. Let \(\lambda \) be a security parameter, a prime number w and an elliptic curve \(E(F_{w})\) over \(F_{w}\) with w elements. Here, two elements a, b are designated in E over \({F}_{w}\) of an equation \({b}^2 + x_{1}ab + x_{3}b\)= \({a}^3 + x_{2}a^2 + x_{4}a + x_{5}\), where \(x_{1}, x_{2}, x_{3}, x_{4}, x_{5} \in {F}_w\). Suppose, q is a prime order in \(E(F_{w})\) with point P, where \(q|\#E(F_{w})\). Moreover, finite field of integers modulo prime q is the \({{Z}_q}\) and \({{Z}_q}^*\) is multiplicative sub-group of \({{Z}_q}\). Also, the cyclic group C has the generator P. The ARPF initializes the SEAI handover AKA stage as following.

  1. 1.

    The ARPF selects the secure one way collision resistant hash functions:

    • \(H_{1}: \lbrace {0,1}\rbrace ^* \times C \longrightarrow {{Z}_q}^*\)

    • \(H_{2}: \lbrace {0,1}\rbrace ^* \times {{Z}_q}^* \longrightarrow {{Z}_q}^*\)

    • \(H_{3}: \lbrace {0,1}\rbrace ^* \times C^2 \times \lbrace {0,1}\rbrace ^* \longrightarrow {{Z}_q}^*\)

    • \(H_{4}: \lbrace {0,1}\rbrace ^* \times C^2 \times \lbrace {0,1}\rbrace ^* \times {{Z}_q}^* \times C^2 \times \lbrace {0,1}\rbrace ^* \times C \longrightarrow \lbrace {0,1}\rbrace ^\lambda \)

    • \(H_{5}: C \times \lbrace {0,1}\rbrace ^\lambda \times \lbrace {0,1}\rbrace ^\lambda \times \lbrace {0,1}\rbrace ^* \times {{Z}_q}^* \times C^2 \times \lbrace {0,1}\rbrace ^* \longrightarrow \lbrace {0,1}\rbrace ^\lambda \)

  2. 2.

    Furthermore, ARPF distributes/publishes these system specifications/public parameters \(PK= \lbrace KDF,P,C,w,q,H_{1},H_{2},H_{3},H_{4},H_{5} \rbrace \) to all the entities that establish the communication in initial and handover authentication stage.

As the protocol believes in the elliptic curve discrete logarithmic problem (ECDLP) assumption [25, 26]. It is admitted that the ECDLP computation is not feasible in polynomial-time and the key of ECC (size: 256 bits) obtains the same secrecy as RSA (size: 3072 bits).

  1. 1.

    Note-(a): Let, C be a group of q prime order and point P. \(xP \in C\) is an element, where \(x \in {{Z}_q}^*\). It is computationally difficult to derive x from xP and P.

  2. 2.

    Note-(b): Let, C be a group of q prime order and point P. \(xP,yP,P \in C\) are the elements where \(x,y \in {{Z}_q}^*\). It is computationally difficult to derive the xyP by using any polynomial time algorithm.

4.2 Handover Initialization Stage

In this stage, UE is verified at AUSF and AMF followed by ARPF [4]. During the verification process, some handover specifications are confined to message authentication requests/responses of the original 5G-AKA protocol. These specifications in 5G-AKA don’t mitigate the efficiency of the network. In the SEAI handover AKA protocol, the AMF sends the secret keys to \(gNB_{s}\) and then, \(gNB_{s}\) broadcasts the information to UE for subsequent handover after accomplishing the UE’s verification. The descriptive explanation of the handover initialization is exhibited in Fig. 3 and step-wise discussion is as follows:

Fig. 3
figure 3

Handover initialization stage

  • Step-1: \(m_{UE} \in {{Z}_{q}}^*\) is private key chosen by the UE and computes \(M_{UE}= m_{UE}.P\). Then, UE sends the message \(SUCI,M_{UE},Msg_{UE},M_{AMF}\) to AMF and initiate the authentication mechanism with ARPF. The Subscription Permanent Identifier (SUPI) is never broadcasted in the communication channel and Subscription Concealed Identifier (SUCI) is the privacy-preserving identifier containing the concealed SUPI. Only ARPF uses the Subscriber Identity De-concealing Function (SIDF) and decrypts the SUCI to achieve the original SUPI.

  • Step-2: AMF authenticates the message from the UE and verifies \(Msg_{UE}\). After this, it chooses \(m_{AMF} \in {{Z}_{q}}^*\) (private key) and derives public key \(M_{AMF} = m_{AMF}.P\). Finally, AMF sends the \(SUCI,M_{ARPF},M_{AMF},M_{UE}, Msg_{AMF},SEAF_{ID} \) to the ARPF.

  • Step-3: \(Mgs_{AMF}\) is verified at the ARPF and authentication of UE is accomplished. Then, ARPF authenticates \(SEAF_{ID}\) and checks the \(SEAF_{ID}\) of UE. The \(SEAF_{ID}\) is verified if they are same, otherwise; ARPF rejects an authentication request. Moreover, the ARPF choses \(m_{ARPF} \in {{Z}_{q}}^*\) and derives \(M_{ARPF} = m_{ARPF}.P\). It generates the \(IKey,CKey,Key_{AUSF},AUTN_{ARPF},XRES^{'}\) and transmits the \(M_{ARPF}, AUTN_{ARPF} \) to the AUSF.

  • Step-4: AUSF keeps \(XRES^{'}\) and generates the \(Key_{SEAF},AUTN_{AUSF},XRESV^{'}\). Then, it transmits the \(M_{AUSF},AUTN_{AUSF}\) to the AMF.

  • Step-5: AMF sends the \(M_{ARPF},M_{AUSF},ngKSI, XRESV^{'}\) to the UE. Then, UE generates the \(XRES^{'}, XRESV^{'}, Key_{AMF},Key_{AUSF},Key_{SEAF}\). It compares these derived values with the obtained ones. UE verifies and confirms the authenticity of AUSF and ARPF, if these value matches. Moreover, UE computes \(RES^{'}\) and sends to AMF.

  • Step-6: AMF obtains \(RESV^{'}\) and compares with \(XRESV^{'}\). If it verifies, AMF confirms the UE’s verification and generates \(Key_{AMF}\). Further, AMF transmits \(RES^{'}\) to AUSF and \(Key_{{gNB_{s}}}^{UE}, SUCI \) to the \(gNB_{s}\).

  • Step-7: The AUSF achieves the \(RES^{'}\) and compares with \(XRES^{'}\). If they match successfully, authentication of the UE is accomplished at AUSF. Moreover, \(gNB_{s}\) retrieves the \(RHI_{UE}\) from \(Key_{{gNB_{s}}}^{UE}\) and sends to UE for subsequent handover. Here, rspec is the related specifications of \(gNB_{s}\) as \(ID_{gnb_{s}},ECI,frequency, PCI\). Then, UE retrieves \(Key_{{gNB_{s}}}^{UE}\) and securely stores \(RHI_{UE}\).

4.3 Authentication Stage of Handover

When UE moves into the range of \(gNB_{t}\), the \(gNB_{t}\) and UE initiate mutual authentication and key agreement mechanism. Here, UE uses the \(RHI_{UE}\) which is retrieved in the handover initialization stage. The inter-gNB handover follows the traditional handover authentication mechanism. Figure 4 represents the flow of the authentication messages in the SEAI handover AKA mechanism. The illustration of the handover authentication steps is shown below.

  • Step-1: When UE is in the area of \(gNB_{t}\), it obtains public parameters of associated gNBs and another specifications such as cell ID (ECI), PLMN-ID, location area identity (LAI), PCI of \(gNB_{t}\). After this, UE chooses a random nonce \(n_{UE} \in {{Z}_q}^*\) and generates \(N_{UE}=n_{UE}.P\). Then, UE retrieves \(MAC_{UE}\) and sends the \(N_{UE}||RHI_{UE}||MAC_{UE}||inau_{UE}\) to \(gNB_{t}\); where, the \(inau_{UE}\) has the related specifications as \(ECI, PLMN_{ID}, PCI\) of \(gNB_{t}\) and targeted LAI.

  • Step-2: Now, \(gNB_{t}\) retrieves the \(Key_{{gNB_{s}}}^{UE}\) by applying \(RHI_{UE}\). It also confirms the authenticity of \(RHI_{UE}\) from \(T_{exp}\). If it is not verified, \(gNB_{t}\) rejects the handover query. After this, \(gNB_{t}\) computes and checks the \(MAC_{UE}\) by using \(Key_{{gNB_{s}}}^{UE}\). If it verifies, \(gNB_{t}\) accepts the acknowledged \(MAC_{UE}\) that is transferred from genuine UE. Or, authentication is rejected.

  • Step-3: After this, \(gNB_{t}\) chooses a random nonce \({n_{gNB}}_{t} \in {{Z}_q}^*\) and retrieves \({n_{gNB}}_{t}.P={N_{gNB}}_{t}\). Moreover, it generates the \({MAC_{gNB}}_{t}\) for UE and session key \(Key_{{gNB_{t}}}^{UE}\). Also, it sends the handover message \({MAC_{gNB}}_{t}||{N_{gNB}}_{t}||{ID_{gNB}}_{t}||{inau_{gNB}}_{t}\) to the UE. The \({inau_{gNB}}_{t}\) has the specifications as \(ID_{AMF}\), ECI, \(PLMN_{ID}\), and PCI.

  • Step-4: Now, UE calculates the \(Key_{{gNB_{t}}}^{UE}\) and checks the \({MAC_{gNB}}_{t}\). If it is incorrect, UE transmits the authentication failure response to \(gNB_{t}\). On the other hand, UE accepts the \(gNB_{t}\) and transmits successful handover acknowledgement (\(MAC_{cfm}\)) to \(gNB_{t}\) with the \(Key_{{gNB_{t}}}^{UE}\). Then, \(gNB_{t}\) approves the handover confirmation with the UE.

Fig. 4
figure 4

Handover authentication stage

5 Security Analysis

This section discusses that the proposed protocol fulfills the security requirements in the ROM. The used assumptions and security model are shown in this proof. The correctness of the protocol is obtained from the AVISPA tool. Also, the informal analysis of protocol is discussed for various security attacks.

5.1 Security Model

For the resistance of identified attacks in the SEAI protocol, we are using a provable security mechanism. We are showing the security proof based on the modeling introduced by [27].

5.1.1 Participants

The protocol \(\Pi \) executes with numerous number of associated participants in 5G network where the participant could be a client \(W\in \omega \) or server \(N\in \eta \). The set \(\eta \) is considered that only a single server is involved at one time. Every participants could have numerous instances (oracles) in distinct executions of \(\Pi \). We indicate the \(i_{th}\) instance of W and N in sessions as \(\Pi ^{i}_{W}\) and \(\Pi ^{i}_{N}\) respectively. Each instance \(\Pi ^{i}_{W}\)/\(\Pi ^{j}_{N}\) has its session identity \(sid^{i}_{W}\)/\(sid^{j}_{N}\) (set of identities that shows the message flow sending/receiving in this instance), partner identity \(pid^{i}_{W}\)/\(pid^{j}_{N}\) (set of identities which are executed in this instance), and session key as \(sk^{i}_{W}\)/\(sk^{j}_{N}\). The instances \(\Pi ^{i}_{W}\), \(\Pi ^{i}_{N}\) can be accepted if it maintains the \(sid^{i}_{W}\)/\(sid^{j}_{N}\), \(sk^{i}_{W}\)/\(sk^{j}_{N}\), and \(pid^{i}_{W}\)/\(pid^{j}_{N}\). \(\Pi ^{i}_{W_{1}}\)/\(\Pi ^{j}_{W_{2}}\) are acknowledged as a partner if (i) both are successfully accepted; (ii) \(sid^{i}_{W_{1}}=sid^{j}_{W_{2}}\); (iii) \(sk^{i}_{W_{1}}=sk^{j}_{W_{2}}\); (iv) \(pid^{i}_{W_{1}}=pid^{j}_{W_{2}}\).

5.1.2 Attacker Model

It is considered that the attacker \({\mathcal {ATT}}\) completely controls the network, which initiates the communication sessions among the participants [28]. The \({\mathcal {ATT}}\) can execute the following queries as:

Execute(\(\Pi ^{i}_{W_{1}}, \Pi ^{j}_{W_{2}}, \Pi ^{k}_{N} \)): The query forms passive attacks where an adversary dodges the legitimate operations among the instances of client \(\Pi ^{i}_{W_{1}}, \Pi ^{j}_{W_{2}}, \Pi ^{k}_{N}\). The result of the query is the exchange of messages at the time of the genuine operation of \(\Pi \).

Send_Client(\(\Pi ^{i}_{W}, m \)): The attacker may use this query to trace the message and update it or forward to the client \(\Pi ^{i}_{W}\). The result of the query is the information that the client \(\Pi ^{i}_{W}\) might compute upon acceptance of message m. Moreover, an attacker is granted to start the protocol by appealing to Send_Client(\(\Pi ^{i}_{W_{1}}, (W_{1}, Start) \)).

Send_Server(\(\Pi ^{i}_{N}, m \)): The query builds active attacks counter to server. The result of the query is the information that the server \(\Pi ^{i}_{N}\) might compute upon acceptance of message m.

Reveal(\(\Pi ^{i}_{W}\)): The query builds identified session key attack. An attacker executes the query to achieve the secret keys of instance \(\Pi ^{i}_{W}\).

Corrupt(W): The query sends the long-term secret/private keys to an attacker for participant W.

Test(\(\Pi ^{i}_{W}\)): An attacker can build this type of query only one time to a fresh instance. On the response of the query, random number \(e\in {0,1}\) is chosen. If \(e=1\), session key obtained by \(\Pi ^{i}_{W}\) is send. Or, return the consistently chosen random number.

5.1.3 Fresh Instances

An instance \(\Pi ^{i}_{W}\) is fresh if following condition satisfies: (i) \(\Pi ^{i}_{W}\) is accepted; (ii) \(\Pi ^{i}_{W}\) or its corresponding partner hasn’t run the Reveal query after acceptance; (iii) client’s corresponding partner with \(\Pi ^{i}_{W}\), hasn’t run the Corrupt query.

5.1.4 Protocol Security

The security of proposed protocol \(\Pi \) is formed by game \(Game^{protocol}(\Pi ,{\mathcal {ATT}})\). As running this game, \({\mathcal {ATT}}\) can execute several queries to \(\Pi ^{i}_{W}\) and \(\Pi ^{j}_{N}\). If \({\mathcal {ATT}}\) asks a Test(\(\Pi ^{i}_{W}\)) query, and \(\Pi ^{i}_{W}\) is fresh and accepted, \({\mathcal {ATT}}\) generates the \(e^{'}\). The objective of \({\mathcal {ATT}}\) is know e correctly in test query. The advantage of \({\mathcal {ATT}}\) can be written as:

$$\begin{aligned} Adv^{protocol}_{\Pi }({\mathcal {ATT}}) = |2Pr[e=e^{'}]-1 | \end{aligned}$$
(1)

The protocol \(\Pi \) is secure if \(Adv^{protocol}_{\Pi }({\mathcal {ATT}})\) is negligibly higher than \(O(q_{se})\), where \(q_{se}\) is the number of Send queries.

5.1.5 Assumption

The CDH assumption can be stated by two experiments, \(Exp1^{CDH-real}_{q}(\Phi )\) and \(Exp2^{CDH-rand}_{q}(\Phi )\). Adversary \(\Phi \) is obtained with xPyPxyP in the \(Exp1^{CDH-real}_{q}(\Phi )\); and xPyPzP in the \(Exp2^{CDH-rand}_{q}(\Phi )\), where \(x,y,z \in {{Z}_{q}}^*\). The advantage of \(\Phi \) in breaching the CDH assumption, \(Adv^{CDH}_{q}(\Phi ) = max\lbrace |Pr(Exp1^{CDH-real}_{q}(\Phi )=1)-Pr(Exp1^{CDH-rand}_{q}(\Phi )=1)|\rbrace \)

5.2 Security Proof

Theorem: Let proposed protocol \(\Pi \) runs the \(q_{se}\) number of Send queries, \(q_{ex}\) number of Execute queries, and \(q_{hash}\) number of hash queries. Then CDH assumption holds the following

\(Adv^{protocol}_{\Pi }({\mathcal {ATT}}) \le \dfrac{(q_{se} + q_{ex}^{2})}{q} + \dfrac{{q_{hash}}}{2^{l}} + 2q_{ex}Adv^{CDH}_{q}(\Phi ) + 4maxima\lbrace \dfrac{q_{se}+q_{ex}}{2^{l}},\dfrac{q_{hash}}{l} \rbrace \).

Proof: The proof has a combination of games, initiating from real attack \(G_{1}\) and finishing at game \(G_{5}\) where an attacker has no power. In each game, we set \(Succ_{i}\) as event that \({\mathcal {ATT}}\) knows e correctly in test query.

Game \(G_{1}\): This is the real attack by \({\mathcal {ATT}}\) in protocol. In this game, the entire instances of participants are formed as real run/execution in ROM. As per the definition of \(Succ_{i}\), we have

$$\begin{aligned} Adv^{protocol}_{\Pi }({\mathcal {ATT}}) = |2Pr[Succ_{1}]-\dfrac{1}{2}| \end{aligned}$$
(2)

Game \(G_{2}\): This is very similar game to Game \(G_{1}\) except the simulation of hash oracles h by constructing hash records \(h_{rec}\) with input/output entries. By executing h inp query, the output result is generated from the \(h_{rec}\), otherwise randomly select the \(Output\in \lbrace 0,1\rbrace ^{l}\) and transmit to the \({\mathcal {ATT}}\) with storing new entry of input/output in \(h_{rec}\). Moreover, we simulate the oracles of the entire queries. As per the knowledge of \({\mathcal {ATT}}\), the game \(G_{2}\) is indistinguishable from real attack (game \(G_{1}\)). Therefore,

$$\begin{aligned} Pr[Succ_{2}] = Pr[Succ_{1}] \end{aligned}$$
(3)

Game \(G_{3}\): Here, we simulate the entire instances of game \(G_{2}\), except we omit the game by which collisions may appear on transcripts as \((Msg_{UE},Msg_{AMF})\), \((MAC_{UE}, MAC_{gNB_{t}})\), and hash values in the protocol. As per the definition of birthday paradox, in the result of h instances, the probability of collisions is \(\dfrac{{q_{hash}}}{2^{l+1}}\). Also, collisions probability in the transcripts is no more than \(\dfrac{(q_{se} + q_{ex}^{2})}{2q}\). Therefore,

$$\begin{aligned} |Pr[Succ_{3}] - Pr[Succ_{2}]| \le \dfrac{(q_{se} + q_{ex}^{2})}{2q} + \dfrac{{q_{hash}}}{2^{l+1}} \end{aligned}$$
(4)

Game \(G_{4}\): Here, we change queries to the \(Send\_Client\) instances. Also, select a random session initiated by legitimate clients UE and \(gNB_{t}\) for partner oracles \(\Pi ^{i}_{UE}\) and \(\Pi ^{j}_{gNB_{t}}\).

  • Send_Client(\(\Pi ^{i}_{UE}, (gNB_{t}, Start) \)) is requested and send output \(SUCI, MAC_{UE}, RHI_{UE}\) to the \({\mathcal {ATT}}\).

  • Send_Client(\(\Pi ^{i}_{UE}, (AUTN_{AUSF}, XRESV^{'}) \)) is requested, randomly select \(x \in {{Z}_q}^*\) and generates \(N_{UE}=x.P\). Then, UE computes \(MAC_{UE}=H_{1}(SUPI||x||RHI_{UE}||inau_{UE}||K^{UE}_{gNB_{s}})\) and \(RHI_{UE}=E\lbrace SUCI||ID_{gNB_{s}}||K^{UE}_{gNB_{s}}||T_{exp} \rbrace \) as real protocol. Then, send the output as \(N_{UE}||RHI_{UE}||MAC_{UE}||SUCI||inau_{UE}\) to \({\mathcal {ATT}}\).

  • Send_Client(\(\Pi ^{j}_{gNB_{t}}, (N_{UE}||RHI_{UE}||MAC_{UE}|| SUCI||inau_{UE}) \)) is requested and randomly select \(y \in {{Z}_q}^*\) and generates \(y.P={N_{gNB}}_{t}\). Also. computes \({MAC_{gNB}}_{t}=H_{2}({ID_{gNB}}_{t}||y||{inau_{gNB}}_{t})\) and \(Key_{{gNB_{t}}}^{UE}=Key_{gNB_{t}}=KDF(Key_{{gNB_{s}}}^{UE}||{ID_{gNB}}_{t}||{inau_{gNB}}_{t}||N_{UE}.y)=xyP\). Then, it sends the output \({MAC_{gNB}}_{t}||{N_{gNB}}_{t}||{ID_{gNB}}_{t}||{inau_{gNB}}_{t}\) to \({\mathcal {ATT}}\).

  • Send_Client(\(\Pi ^{i}_{UE}, ({MAC_{gNB}}_{t}||{N_{gNB}}_{t}||{ID_{gNB}}_{t}|| {inau_{gNB}}_{t}) \)) is requested, compute\( Key_{UE}=xyP\), \(MAC_{cfm}=H_{3}(Key_{{gNB_{t}}}^{UE}||SUPI||{ID_{gNB}}_{t}||x.{N_{gNB}}_{t})\) and session key \(Key_{{gNB_{t}}}^{UE}\) in real protocol. Then it send \(MAC_{cfm}\) to \({\mathcal {ATT}}\).

Hence, it is observed that the game is indistinguishable from game \(G_{3}\). So,

$$\begin{aligned} Pr[Succ_{4}] = Pr[Succ_{3}] \end{aligned}$$
(5)

Game \(G_{5}\): Here, we update the simulation queries of \(Send\_Client\) instances for randomly chosen session in \(G_{3}\). In this game, we choose another way for computing the value of \(Key_{{gNB_{t}}}, Key_{UE}\) so it will be autonomous for handover acknowledgment value and keys. When Send_Client(\(\Pi ^{j}_{gNB_{t}}, (N_{UE}||RHI_{UE}||MAC_{UE}||SUCI||inau_{UE}) \)) and Send_Client(\(\Pi ^{i}_{UE}, ({MAC_{gNB}}_{t}||{N_{gNB}}_{t}||{ID_{gNB}}_{t}||{inau_{gNB}}_{t}) \)) are requested \(Key_{{gNB_{t}}}= Key_{UE}=T_{z}(\psi )\) (for UE and \(gNB_{t}\)), where \(z \in {{Z}_q}^*\). The difference between game \(G_{5}\) and \(G_{4}\) is:

$$\begin{aligned} |Pr[Succ_{5}]- Pr[Succ_{4}]| \le q_{ex}Adv^{CDH}_{\psi ,q}(\Phi ) \end{aligned}$$
(6)

By considering a successful attacker \({\mathcal {ATT}}\) to analyze \(G_{5}\) and \(G_{4}\), we make the CDH fixer \(\Phi \). The difference between \(G_{5}\) and \(G_{4}\) is the way of calculation of \(Key_{{gNB_{t}}}, Key_{UE}\) for chosen session. Firstly, \(\Phi \) obtains the CDH value (xPyPZ). As \(G_{5}\) and \(G_{4}\), the fixer \(\Phi \) chooses a verifying session for \(\Pi ^{i}_{UE}\) and \(\Pi ^{j}_{gNB_{t}}\) initiated legitimate clients UE and \(gNB_{t}\) respectively. When Send_Client(\(\Pi ^{i}_{UE}, (AUTN_{AUSF}, XRESV^{'}) \)) is requested, the \(\Phi \) sets \(N_{UE}=x.P\). In addition, when, Send_Client(\(\Pi ^{j}_{gNB_{t}}, (N_{UE}||RHI_{UE}||MAC_{UE}||SUCI||inau_{UE}) \)) and Send_Client(\(\Pi ^{i}_{UE}, ({MAC_{gNB}}_{t}||{N_{gNB}}_{t}||{ID_{gNB}}_{t}||{inau_{gNB}}_{t}) \)) are requested, \(\Phi \) sets \(y.P={N_{gNB}}_{t}\) and \(Key_{{gNB_{t}}}^{UE}=Z\).

The analyzer \({\mathcal {ATT}}\) selects a random session for the test queries (Test(\(\Pi ^{i}_{UE}\)), Test(\(\Pi ^{j}_{gNB_{t}}\))), then the probability is \(\dfrac{1}{q_{ex}}\). Hence, the \(\Phi \) simulates all instances query without having information of xy. From this, analyzer \({\mathcal {ATT}}\) may generate \(N_{UE}=x.P, y.P={N_{gNB}}_{t}\) but not the correct \(Key_{{gNB_{t}}}, Key_{UE}\). In case, \(Z=xyP\), this setting for the analyzer is similar to \(G_{4}\). In case, \(Z=zP\), this setting for the analyzer is similar to \(G_{5}\).

Lastly, if analyzer \({\mathcal {ATT}}\) interacts with \(G_{4}\), the fixer \(\Phi \) decides that \(Z=xyP\). And, if \({\mathcal {ATT}}\) interacts with \(G_{5}\), the fixer \(\Phi \) decides that \(Z \ne xyP\). Hence, eq. (6) holds. In this game, the keys \(Key_{{gNB_{t}}}, Key_{UE}\) are independent and random with secret keys. Therefore, three possibilities can be arises where an attacker analyzes the random and secret session keys as:

Case-1: Attacker queries \((zP,SUCI,{ID_{gNB}}_{t})\) to h. Then, this event obtains in \(\dfrac{2q_{hash}}{{l}}\).

Case-2: Attacker requests Send_Client query excepting Send_Client(\(\Pi ^{j}_{gNB_{t}}, m \)) and impersonates UE to \(gNB_{t}\). If an attacker, tries to impersonate UE in random session by generating \(MAC_{UE}\) and got success, it will make the discrepancy but the probability is less than to \(\dfrac{1}{{2^{l}}}\). As there are maximum \({2(q_{se}+q_{ex})}\) sessions, then the total probability that this event is obtained will be less than to \(\dfrac{2(q_{se}+q_{ex})}{{2^{l}}}\).

Case-3: Attacker requests Send_Client query excepting Send_Client(\(\Pi ^{i}_{UE}, m \)) and masquerades the \(gNB_{t}\) to UE. Similar to Case-2:, the probability of this event is obtained less than to \(\dfrac{2(q_{se}+q_{ex})}{{2^{l}}}\). Therefore, from above three cases;

$$\begin{aligned} |Pr[Succ_{5}]| = \dfrac{1}{2} + 2maxima \lbrace \dfrac{q_{se}+q_{ex}}{{2^{l}}},\dfrac{q_{hash}}{l} \rbrace \end{aligned}$$
(7)

By combining the eq. from (1) to (7), the results are:

$$\begin{aligned} \begin{aligned} Adv^{protocol}_{\Pi }({\mathcal {ATT}})&= 2Pr[Succ_{1}]-\dfrac{1}{2}|\\&\le (|Pr[Succ_{2}]- Pr[Succ_{3}]| + \\&\quad |Pr[Succ_{4}] - Pr[Succ_{5}]| + \\&\quad 2maxima \lbrace \dfrac{q_{se}+q_{ex}}{{2^{l}}},\dfrac{q_{hash}}{l} \rbrace ) \\&\le \dfrac{(q_{se} + q_{ex}^{2})}{q} + \dfrac{{q_{hash}}}{2^{l}} \\&\quad + 2q_{ex}Adv^{CDH}_{q}(\Phi ) + \\&\quad 4maxima \lbrace \dfrac{q_{se}+q_{ex}}{{2^{l}}},\dfrac{q_{hash}}{l} \rbrace \rbrace \\ \end{aligned} \end{aligned}$$

5.3 Correctness of the Protocol

The proposed SEAI-AKA handover protocol is simulated using the AVISPA tool to prove its correctness. The protocol is programmed coded in classic High-Level Protocol Specification Language (HLPSL) to define its characteristics [29]. AVISPA tool simulates the protocol in numerous backends as On-the-Fly Model Checker (OFMC) and SAT-based Model-Checker (SATMC). There are two participants titled gNB and UE in the protocol. We have programmed the fundamental role of these participants in HLPSL and simulated the mechanism by adopting the AVISPA tool. The HLPSL program of the communicating participants is demonstrated in the Appendix-A. Also, the objectives of the protocol are described in Fig. 5.

Fig. 5
figure 5

Objectives of the SEAI handover protocol

The simulation of the protocol is implemented by applying the OFMC backend with a restricted number of terms. Essentially, the OFMC simulates handover protocol, and then attacker fetches the information from preceding executions. Therefore, OFMC obtains the session complexity and avoids replay attack without executing different sessions between communicating participants. Also, OFMC checks whether the genuine participants can execute the protocol by seeking the passive attacker and broadcasts the instructions of a few sessions to the attacker between genuine participants [30]. The test outputs show that the protocol dodges replay attack. The output of OFMC back-end model is represented in Fig. 6. The keyword SAFE in result proves the correctness of the protocol. Moreover, the protocol averts from the MitM attack by adopting the tests of OFMC back-end. Therefore, the SEAI handover AKA protocol gains the essential security characteristics and dodges the known attacks from the 5G network.

Fig. 6
figure 6

Output of OFMC back-end

5.4 Informal Analysis

In this section, we discuss various malicious attacks to show that the SEAI handover protocol is not vulnerable to the probable attacks.

  • KFS/KBS: To preserve the KFS/KBS, the secret keys must not be acknowledged in the preceding and successive sessions even if it is compromised. In the protocol, UE achieves the \(RHI_{UE}\) and \(Key_{{gNB_{s}}}^{UE}\) from \(gNB_{s}\) and AMF respectively in a secure communication even if \({\mathcal {ATT}}\) generates the required public keys. Moreover, \({\mathcal {ATT}}\) aims to achieve \(MAC_{UE}/{MAC_{gNB}}_{t} \) for self-verification at any participant. However, \({\mathcal {ATT}}\) can’t obtain these authentication values as \(n_{UE}\) and \({n_{gNB}}_{t}\) are random values at unique communication of handover. \({\mathcal {ATT}}\) needs the information of private keys to generate the preceding and following session keys of \(Key_{{gNB_{t}}}^{UE}\). However, it fails to obtain these values as ECDLP is computationally hard. Also, the protocol doesn’t follow the key chain framework and interaction with \(gNB_{s}\). Therefore, \({\mathcal {ATT}}\) will never have the information of earlier/subsequent private keys.

  • Key Escrow Problem: The UE or \(gNB_{t}\) select the secret keys in each handover authentication. To compute these secret keys, there is no association of the third party such as a key generation center (KGC)/private key generator (PKG). Therefore, the protocol avoids the key escrow problem.

  • DoS Attack: The \({\mathcal {ATT}}\) may transmit a large number of false handover requests to UE or \(gNB_{t}\) in the authentication stage to drain its network bandwidth. In the protocol, \(gNB_{t}\) obtains the \(Key_{{gNB_{t}}}^{UE}, {MAC_{gNB}}_{t}\), and transfers the sequence message \(S_{2}\) to the UE (as presented in Fig. 4). UE generates \(Key_{{gNB_{t}}}^{UE}\) and authenticates \({MAC_{gNB}}_{t}\). After this, it sends the \(MAC_{cfm}\) to \(gNB_{t}\). If the authentication is not successful, an authentication reject information is send to UE. As per the ECDLP infeasibility assumption, it is impractical for \({\mathcal {ATT}}\) to obtain the secret keys of the communicating participants. Hence, the proposed protocol avoids the DoS attack.

  • Privacy-Preservation: In the proposed protocol, UE transmits the SUCI to the ARPF followed by AMF as SUPI can’t be transmitted over the communication channel and SUCI is applied to form this. The ARPF decrypts the SUCI value by SIDF. Hence, the identity of the UE is achieved in the proposed protocol. In addition, the \({ID_{gNB}}_{s}\) is never transmitted from AMF to UE for computing the \(Key_{{gNB_{s}}}^{UE}, RHI_{UE}\). Suppose, \({\mathcal {ATT}}\) computes the \({ID_{gNB}}_{t}\) transmitted from \(gNB_{t}\) to UE and attempts to compute the bogus \(MAC_{{gNB_{t}}}\). However, an attacker can’t derive the private keys due to the computationally infeasibility assumption of ECDLP. Therefore, only legitimate UE can accept the \({ID_{gNB}}_{t}\) from \(gNB_{t}\).

  • Replay Attack: In the authentication stage of handover mechanism, replay attack couldn’t be initiated as each corresponding message has the chosen private keys. Let consider, \({\mathcal {ATT}}\) transmits duplicate informations to \(gNB_{t}\)/UE. Then, the communicating participants instantly verify that the information is achieved previously by them as secret/random keys are unique in every communication of handover. Also, \({\mathcal {ATT}}\) couldn’t obtain the genuine \(Key_{{gNB_{t}}}^{UE}\). Therefore, the protocol dodges the replay attack.

  • Redirection Attack: The \({\mathcal {ATT}}\) can initiate the redirection attack if it masquerades/impersonates UE or maintains the bogus gNB correctly. Moreover, no \({\mathcal {ATT}}\) could decrypt the identity of UE excluding the ARPF. Therefore, it can’t obtain the original identity of the UE. Also, \({\mathcal {ATT}}\) fails to obtain identity of \(gNB_{t}\) and compute \({MAC_{gNB}}_{t}\). \(gNB_{s}\) sends the LAI to \(gNB_{t}\) when the UE arrives in the range of \(gNB_{t}\). Hence, protocol averts the redirection attack from the 5G network.

  • MitM Attack: \({\mathcal {ATT}}\) can’t implant the MitM attack at the authentication stage of protocol. It is noted that the \(Key_{{gNB_{t}}}^{UE}\) is verified at UE and \(gNB_{t}\) successfully. Suppose, \({\mathcal {ATT}}\) corrupts the \(N_{UE}\), \({N_{gNB}}_{t}\) and generates the \({N_{UE}}_{{\mathcal {ATT}}}\), \({{N_{gNB}}_{t}}_{{\mathcal {ATT}}}\), where \({N_{UE}}_{{\mathcal {ATT}}}\) = \({n_{UE}}_{{\mathcal {ATT}}}.P\) and \({{N_{gNB}}_{t}}_{{\mathcal {ATT}}}\) = \({{n_{gNB}}_{t}}_{{\mathcal {ATT}}}.P\). Therefore, \({\mathcal {ATT}}\) generates the \({N_{UE}}_{{\mathcal {ATT}}}\) at \(gNB_{t}\) but, the \({Key_{{gNB_{t}}}^{UE}}_{{\mathcal {ATT}}}\) is not generated correctly as \({Key_{{gNB_{t}}}^{UE}}_{{\mathcal {ATT}}}\)= \(KDF(Key_{{gNB_{s}}}^{UE}||ID_{gNB_{t}}||inau_{gNB_{t}}||{N_{UE}}_{{\mathcal {ATT}}}.n_{gNB_{t}})\). Similarly, \({\mathcal {ATT}}\) obtains \({{N_{gNB}}_{t}}_{{\mathcal {ATT}}}\) at UE but, the \({Key_{{gNB_{t}}}^{UE}}_{{\mathcal {ATT}}}\) is not generated correctly as \({Key_{{gNB_{t}}}^{UE}}_{{\mathcal {ATT}}}\) = \(KDF(Key_{{gNB_{s}}}^{UE}||ID_{gNB_{t}}||inau_{gNB_{t}}||n_{UE}.{{N_{gNB}}_{t}}_{{\mathcal {ATT}}})\). As, the \({\mathcal {ATT}}\) doesn’t have the information of UE’s/\(gNB_{t}\) secret key, so it is not possible for to obtain valid \({MAC_{UE}}\)/\({{MAC_{gNB}}}_{t}\). Hence, \({\mathcal {ATT}}\) can’t achieve the authentic handover message to execute MitM attack in the network.

  • Eavesdropping Attack: In the handover establishment stage, the UE and AMF authenticate to each other. AMF transmits the \(Key_{{gNB_{s}}}^{UE}\) to \(gNB_{s}\) and then \(gNB_{s}\) broadcasts \(RHI_{UE}\) to the UE. The chosen secret keys are private in all over the handover operations. Hence, \({\mathcal {ATT}}\) couldn’t compute the secret session keys even though he/she calculates the universal/public specifications of the UE and \(gNB_{s}\). In the handover authentication stage, the universal and handover specifications are transmitted between \(gNB_{t}\) and UE in the public channel.

The analysis of SEAI handover AKA protocol and existing 5G protocols is presented in Table 2 based on numerous security characteristics. It can be defined that the current 5G handover protocol achieves the mutual authentication between the communicating participants in the authentication mechanism. Although, the protocol doesn’t obtain the KFS/KBS and deteriorates from authentication complication. Also, the protocol fails to avoid DoS attack. The Cao’s-AKA protocol doesn’t obtain the KFS/KBS and defeats from DoS, redirection, and eavesdropping attack. Also, Sharma’s-AKA protocol fails to achieve the key secrecy and avoid system complexity. Additionally, the protocol is vulnerable to redirection attack. Zhang’s-AKA protocol can’t preserve the identity during the handover authentication; hence, it is susceptible to several security attacks. Similar to Zhang’s protocol, Han’s-AKA protocol has numerous security weaknesses and can’t establish identity privacy preservation. Furthermore, Kumar’s-AKA protocol obtains most of the security characteristics but can’t prevent the MitM and eavesdropping attack from the communication network. Different from the current protocols, the proposed SEAI handover AKA protocol executes the key procedures adopting the ECC. The protocol accomplishes the KFS/KBS in the authentication mechanism. Moreover, the protocol resist all the potential attacks and free from the authentication complication. Therefore, the proposed protocol is relatively better compared to the existing protocols as it gains all the crucial security characteristics.

Table 2 Comparative scrutiny of the handover protocols

6 Performance Estimation

The performance of the proposed SEAI handover AKA protocol is estimated for the existing 5G handover schemes in terms of computation, communication, and transmission overhead. Additionally, we compute the handover delay, key size, and energy consumption for the handover protocols based on various parameters. The analysis represents that the proposed protocol gains all security objectives with adequate competence.

6.1 Computation Overhead

For the estimation of computation overhead of handover protocols at the handover authentication and initialization stage, elapsed time of various security functions is executed at OpenSSL written in C library [31] operating on 4 GB memory machine with Intel Core i5-7200U 4 GHz processor for gNB and 2.50 GHz processor for UE. Hence, the elapsed time (in ms) is: point multiplication (\(T_{pmul}\))= 0.441, hash (\(T_{hh}\))=0.0087, AES encryption/decryption (\(T_{aes}\))=0.071, modular exponentiation (\(T_{moe}\))=0.629, arithmetic operation (\(T_{art}\))=0.0021, multiplication operation (\(T_{mul}\))=0.0033 (for gNB); \(T_{pmul}\): 1.023, \(T_{hh}\)=0.0194, \(T_{aes}\)=0.109 ms, \(T_{moe}\)=1.277 ms, \(T_{art}\)=0.0074 ms, \(T_{mul}\)=0.0091 ms (for UE). The computational overhead of current and proposed handover protocols is presented in Table 3. Also, the graphical presentation is shown for the comparison of handover protocols in terms of computation overhead in Figs. 7 and 8.

Table 3 Estimated analysis of handover protocols
Fig. 7
figure 7

Computation overhead of handover protocols at UE

Fig. 8
figure 8

Computation overhead of handover protocols at gNB

The current 3GPP-5G handover protocol accepts the hash operations and symmetric cryptography that generates the overhead at each communicating participant in inter-gNB handover. However, the protocol fails to avoid the de-synchronization that derives the DoS attack and complex handover process. In the Cao’s-AKA protocol, UE and base-station execute the hash operation for integrity and AES for encryption/decryption operations. The protocol shows less overhead compared to the proposed scheme however, Cao’s handover protocol is not secure against eavesdropping and redirection attacks. Also, the Han’s-AKA protocol has less computation overhead compared to the SEAI handover AKA protocol as it executes only hash operations during handover operations but suffers from DoS and MitM attack. Both the Zhang’s-AKA and Kumar’s-AKA protocol operate the handover authentication using point multiplication, arithmetic, and multiplication operations. Moreover, the Sharma’s-AKA protocol execute the handover authentication by time-consuming modular exponentiation operations. Hence, these protocols aren’t recommended for the development of efficient handover authentication protocol in the 5G communication network. Different from above schemes, the proposed SEAI handover AKA protocol establishes mutual authentication and key agreement between the \(gNB_{t}\) and UE by adopting point multiplication operation. Moreover, the protocol avoids the loss of key secrecy and potential security susceptibilities. Hence, it obtains a significant security & privacy compared to the current handover schemes with competitive overhead.

6.2 Communication Overhead

In order to measure the communication overhead of the current and proposed protocols, we fix |p| = 1024 and |q| = 256 because the ECC key indicates identical security. The |n|=\(|\# E(F_{n})|\) = 256 and \(E(F_{n})\):\(\# E(F_{n})\) = 256 bits prime order q. Moreover, Table 4 represents the specification list and their costs/value [32]. To estimate the overhead, we measure the broadcasted information between the communicating participants in the current and proposed handover AKA protocols. In Table 3, the overhead of the protocols is measured. Also, the graphical presentation is shown for the comparison of handover protocols in terms of communication overhead in Fig. 9.

Table 4 Specifications for communication overhead
Fig. 9
figure 9

Communication overhead of handover protocols

Although, the overhead of the SEAI handover AKA protocol is larger than the 3GPP-5G handover mechanism. However, the 3GPP-5G protocol deteriorates from key negotiation issue, DoS attack, and authenticity complexity. In the Cao’s-AKA protocol, UE communicates to the target and future base-station for accomplishing mutual authentication respectively. The UE and base-stations share the message authentication codes, capability messages, and handover tickets in 1884 bits. Although, the protocol incurs less communication overhead during the handover initialization stage compared to SEAI handover AKA scheme because keys and identity are generated directly from the handover module. Also, the protocol suffers from lack of forward key secrecy and DoS attack. In Sharma’s-AKA protocol, the terminal and new/previous hub communicate with each other during handover authentication. The terminal transmits the sequence number, message authentication code, and various handover request/response. At the same time, the authentication server communicates with new and previous hubs in 2978 bits. Han’s-AKA protocol follows the EAP-AKA scheme during the initial authentication of UE and base-station. In the handover stage, the UE and base-station obtain the authentication parameters and use additional counter hash values. Also, the protocol fails to preserve the identity during the authentication process.

The Zhang’s-AKA protocol establishes mutual authentication between the communicating participants. Firstly, UE transmits its one-time trapdoor hash key, secret, public keys, expiration time, and identity. Then, the target base-station sends its handover specifications to the UE with a shared secret key, and UE approves handover acknowledgment by transmitting the secret key. Similar to Zhang’s-AKA protocol, Kumar’s-AKA protocol accomplishes mutual authentication between the communicating participants. Firstly, UE transmits its secret, public keys, passwords, and pseudo-identity. Then, the target base-station sends its random number, secret keys, and public parameters to UE with a shared secret key, and UE accepts the handover message successfully. The prime objective of the proposed SEAI handover AKA protocol is to avoid the overhead at the communicating participants and evolve the security capabilities at the time of handover. Hence, we designed the handover protocol by adopting the ECC procedure. Our protocol setups the session key secrecy and \(Key_{{gNB_{t}}}^{UE}\) is attained between \(gNB_{t}\) and UE without any ambiguous handover system. The UE and \(gNB_{t}\) maintain the secure mutual authentication in the protocol and there is no transmission of the secret session key in the public channel. Thus, the protocol is very efficient and secure compared to the current handover schemes.

6.3 Transmission Overhead

It is studied that the conventional cost of the message authentication between i) \(gNB_{s}/gNB_{t}\) and UE is \(\rho \) unit; ii) \(gNB_{s}\) and \(gNB_{t}\) is \(\sigma \) unit; and iii) AMF and \(gNB_{s}/gNB_{t}\) is \(\Delta \) unit to measure transmission overhead of the proposed and current handover protocols. As the gNB is implanted a very long distance from AMF; hence the overhead of \(\sigma \) unit has the scope as \(0<\sigma <\rho \). Also, the overhead of \(\rho \) is greater than the cost of \(\Delta \). The transmission overhead of proposed and existing handover AKA protocols is demonstrated in Table 5. Hence, it is noticed that the overhead of proposed SEAI handover AKA protocol is less compared to most of the existing protocols. Although, Kumar’s scheme has less transmission overhead but suffers from huge communication and computation overhead because of additional point multiplication operations during handover. In the handover authentication stage of proposed protocol, 3 communication messages are required between \(gNB_{t}\) and UE. Although, only 2 messages are enough to establish mutual authentication between \(gNB_{t}\) and UE. The third correspondence message is transmitted from the UE to approve the handover key agreement with \(gNB_{t}\).

Table 5 Transmission overhead of protocols

6.4 Handover Delay

In this section, the handover delay is computed for the proposed SEAI handover AKA protocol and other existing schemes when the user is executing various handover between base-station/nodes. The handover delay for each handover scheme in A by parameter \(HD^A\) as \(f^{*}_{HD{^A_{m}}}(s)= {\sum _{t\in T^A}} P_{t}f^{*}_{HD{^A_{t}}}(s) \) [33, 34]. In this scenario, t is the is the authentication or re-authentication process that is executed in each scheme. \(P_{t}\) is the ratio for executing the mechanism t, and \(T^A\) is the handover scheme. Here, suppose A is the \(A_{5G}\) then \(T^A=\lbrace gNB_{s},gNB_{t},gNB_{s},gNB_{t},\ldots . \rbrace \), and A is the \(A_{Cao}\) then \(T^A=\lbrace BS_{2},BS_{3},BS_{2},BS_{3},\ldots . \rbrace \), and A is the \(A_{Sharma}\) then \(T^A=\lbrace pHub,nHub,pHub,nHub,\ldots . \rbrace \). Also, A is the \(A_{Zhang}\) then \(T^A=\lbrace AP_{t},AP_{t},AP_{t},AP_{t},\ldots . \rbrace \), and A is the \(A_{Han}\) then \(T^A=\lbrace BS_{t},BS_{t},BS_{t},BS_{t},\ldots . \rbrace \), A is the \(A_{Kumar}\) then \(T^A=\lbrace AP,MBS,AP,MBS,\ldots . \rbrace \), and A is the \(A_{SEAI}\) then \(T^A=\lbrace gNB_{t},gNB_{t},gNB_{t},gNB_{t},\ldots . \rbrace \). Furthermore, \(\phi ^{A}_{t}\) is the set that has the delay factors in the protocol A. The Laplace transformation of \(HD^A\) is \(f^{*}_{HD{^A_{t}}}(s)= f^{*}_{\sum _{i\in \phi ^{A}_{t}}} HD_{i}(s) =(\Pi _{{i\in \phi ^{A}_{t}}}f^{*}_{HD_{i}})\). The Laplace transformation of \(HD^{5G}\) is \(f^{*}_{HD{^{A^{5G}}}}(s)= f^{*}_{HD{^{A^{5G}}_{gNB_{s}}}}(s)+ {HD{^{A^{5G}}_{gNB_{t}}}}(c)\), \(HD^{Cao}\) is \(f^{*}_{HD{^{A^{Cao}}}}(s)= f^{*}_{HD{^{A^{Cao}}_{BS_{2}}}}(s)+ {HD{^{A^{Cao}}_{BS_{3}}}}(s)\), and \(HD^{SEAI}\) is \(f^{*}_{HD{^{A^{SEAI}}}}(s)= f^{*}_{HD{^{A^{SEAI}}_{gNB_{t}}}}(s)+ {HD{^{A^{SEAI}}_{gNB_{t}}}}(s)\). Additionally, the Laplace transformation of \(HD^A\) can be written as \(E(HD^A)= \int _{0}^{\infty } f_{{HD}^A} \,(x)dx \) [35]. For the handover AKA protocols, it can be written as \(E(HD^{5G})=- \frac{d}{ds}f^{*}_{HD^{5G}}(s)\vert s=0\).

Figure 10 represents the handover delay of the SEAI handover AKA protocol and existing schemes concerned by increasing the hop count between the base-station/nodes and server. The handover delay of the proposed protocol is far less compared to the existing schemes because of executing a similar re-authentication process in each hop. Figure 11 shows the performance of the SEAI handover AKA protocol compared to the existing schemes in terms of the number of users and handover delay in milliseconds. As the number of users is increasing in each scheme, the handover delay is also increased. The proposed protocol obtains comparatively less handover delay to the Kumar’s, Sharma’s, Cao’s, and Zhang’s handover schemes. The proposed SEAI handover AKA scheme reduces the handover delay by 14%, 25%,30%, and 60% compared to Kumar’s, Sharma’s, Cao’s, and 3GPP-5G handover AKA schemes respectively.

Fig. 10
figure 10

Handover delay with hop count

Fig. 11
figure 11

Handover delay with number of users

6.5 Key Size

In this section, the size of the key is determined which are computed at the execution of handover AKA schemes. The size of computed and transferred keys has an important impact on the storage overhead as other parameters such as private/public key pair, time-stamp, identification parameters have a similar impact compared to an alternative approach. The sum of the key size is calculated for all the handover AKA protocols based on hop count as shown in Fig. 12. From, the Fig. 12, it is observed that the SEAI handover AKA protocol has a very competitive key size with an increasing number of hop counts compared to Han’s protocol. The key size of the SEAI handover AKA protocol will be the same with an increasing number of hop counts. In the Kumar’s, Cao’s, and Sharma’s handover AKA schemes, the key size is larger compared to the other protocols, and key size is increased at the following re-authentication processes. Additionally, in the Kumar’s, Cao’s, and Sharma’s handover AKA protocols, the users roam to the previously visited base-station/node (hops (H) 2 to 8), and some additional keys may be generated in the home server and during the re-authentication process. Also, the keys are generated and stored at every hop count.

Fig. 12
figure 12

Key size with hop count

Similarly, the Figs. 13 and 14 represent the key size of the handover AKA protocols for the number of users and user movements. Also, it can be noticed that the SEAI handover AKA protocol has far better key size results compared to the existing handover schemes.

Fig. 13
figure 13

Key size with number of users

Fig. 14
figure 14

Key size during user movements

6.6 Average Handover Cost

To evaluate the average handover cost of the handover AKA protocols, the wireless network model and mobility model are adopted as per [36, 37] respectively. It is considered that the network model is the 5G, WLAN-5G inter-networking domain and sizes of each subnet are similar. The average handover rate (\(\alpha _{j}\)) is calculates as \(\alpha _{j}=(v.P(i))/(\Pi .L(i))\), where j is the user group index, v is the UE’s average velocity (varies from 2 to 4km/h) in the 5G and WLAN-5G communication network. The perimeter P(i) of the respective network can be computed as \(P(i)=(12i+6).R\). Here i is the cells number, R is the radius of subnet. The roaming area L(i) is computed as \(L(i)=(2.6R^{2})(3i(i+1)+1)\). Therefore, the average handover cost (AHC) can be calculated as \(AHC_{t}=\alpha _{j}.C_{t}\). The cost of each scheme \(C_{t}=C_{t,s}+ C_{t,p}\), where \(C_{t,s}\) and \(C_{t,p}\) is the signaling and processing cost respectively. The \(AC_{t,s}\) for each scheme can be computed for each handover protocol as:

$$\begin{aligned} SEAI_{C_{t,s}}&=3C_{ws}+1H\\ 5G_{C_{t,s}}&=5C_{ws}+2H\\ Cao_{C_{t,s}}&=8C_{ws}+2H\\ Sharma_{C_{t,s}}&=12C_{ws}+2H \\ Zhang_{C_{t,s}}&=4C_{ws}+1H \\ Han_{C_{t,s}}&=8C_{ws}+1H \\ Kumar_{C_{t,s}}&=3C_{ws}+2H \end{aligned}$$

where \({C_{t,s}}\) is the transmission cost of wireless links. The calculation of each scheme \(C_{t,p}\) is the execution cost of each node \(C_{n,p}\). For example, \(C_{t,p}\) for 3GPP-5G handover scheme can be shown as \(C_{5G,p}=C_{UE,p}+C_{gNB_{s},p}+C_{gNB_{t},p}\), where,\(C_{UE,p}=4C_{Key}+C_{Enc}+C_{Dec}+C_{Ver}\), \(C_{gNB_{s},p}=2C_{Key}+C_{Hash}\), and \(C_{gNB_{t},p}=C_{Key}+C_{Enc}+C_{Dec}+C_{Ver}\). The \(C_{Key},C_{Enc},C_{Dec},C_{Ver},C_{Hash}\) are the costs of key computation, encryption, decryption, verification, and hash operation respectively. Therefore, \(C_{t,p}\) for all the handover AKA schemes can be computed as:

$$\begin{aligned} SEAI_{C_{t,p}}&=3C_{Key}+C_{Enc}+C_{Dec}+2C_{Ver}+7C_{Hash}\\ 5G_{C_{t,p}}&=7C_{Key}+2C_{Enc}+2C_{Dec}+2C_{Ver}+C_{Hash}\\ Cao_{C_{t,p}}&=7C_{Key}+3C_{Enc}+3C_{Dec}+3C_{Ver}+7C_{Hash}\\ Sharma_{C_{t,p}}&=8C_{Key}+2C_{Enc}+2C_{Dec}+2C_{Ver}+8C_{Hash} \\ Zhang_{C_{t,p}}&=5C_{Key}+2C_{Enc}+2C_{Dec}+2C_{Ver}+4C_{Hash} \\ Han_{C_{t,p}}&=6C_{Key}+2C_{Enc}+2C_{Dec}+2C_{Ver} \\ Kumar_{C_{t,p}}&=7C_{Key}+2C_{Enc}+2C_{Dec}+2C_{Ver}+6C_{Hash} \end{aligned}$$

The value of i is considered 10,\(C_{ws}\) is set to 10. The costs such as \(C_{Key},C_{Enc},C_{Dec},C_{Ver},C_{Hash}\) are set to one unit. The results achieved from the handover cost evaluations of each schemes are shown in Figs. 1516, and 17 at varying value of v from 2 to 4km/h. Also, the value of R is 0.1 km and H is 1 to 7 hop count. As the values of v and H increase, the average cost of existing handover AKA schemes is also increases compared to the SEAI handover AKA protocol. Therefore, the proposed protocol can be recommended for the IoT-enabled services in various handover scenarios as the handover cost is significantly reduced. Moreover, the AHC increases from 60 to 357 when H increases from 1 to 7 in the 3GPP-5G handover AKA scheme. However, the AHC remains the same with varying values of v and H in the proposed scheme. The reduction of handover cost in the SEAI handover AKA scheme raises 34%, 23%, and 15% compared to the 3GPP-5G, Cao’s, Sharma’s handover AKA protocol respectively.

Fig. 15
figure 15

Average handover cost at v = 2 km/h

Fig. 16
figure 16

Average handover cost at v = 3 km/h

Fig. 17
figure 17

Average handover cost at v = 4 km/h

6.7 Energy Consumption

The current cellular networks manage massive users; hence, the computation of energy consumption is one of the essential performance estimation metrics. The reduction of the computed keys and exchanged messages at the authentication process represent the energy consumption [38, 39]. Generally, the total energy consumption in wireless networks can be computed as \(Total_{Energy}=N.M+FC\), where N is the total bits transmitted/received by the UE, M is the incremental value, and FC is the fixed cost. The fixed and incremental value are coefficients which are obtained in [40]. The energy consumption is computed as per number of bits received and transmitted by the UE as \(Energy_{trans}=0.48N+431\); \(Energy_{rec}=0.12N+316\). The above-mentioned equations are adopted to compute the energy consumed by UE in each user movement. The calculations are utilized in the proposed and existing handover AKA protocols. For instance, the energy consumption of SEAI handover AKA scheme is \(Energy_{trans}\)=1088; \(Energy_{rec}\)=928. Figure 18 shows that the energy consumption in the previously proposed handover schemes is increased when UE roams into another base-station/node (inter/intra handover) in the 5G or WLAN-5G communication networks. Moreover, the proposed handover AKA scheme reduces the energy consumption 78%, 31%, and 54% compared to the Cao’s, Sharma’s, and Kumar’s protocol respectively.

Fig. 18
figure 18

Energy consumption in user movement

7 Conclusion

In this article, we introduced the secrecy and efficiency aware inter-gNB handover AKA protocol in 5G communication network to avoid the potential security susceptibilities as key negotiation, DoS & bogus base-station attack, and huge authentication complexity. In the proposed SEAI handover AKA protocol, mutual authentication is accomplished with a secret key between gNB and UE. Also, the protocol forms the forward/backward secrecy and averts the network complexities. In addition, simulation of the protocol is presented by the AVISPA tool to prove the correctness. To obtain the session key secrecy, confidentiality, and integrity, the formal security proof of the protocol is carried out by the ROM. The security analysis is examined with corresponding numerous security specifications and obtains the security across potential attacks. The performance estimation clarifies that the protocol is far valuable compared to the current 5G handover schemes based on various overhead analysis. Also, the handover delay, key size, and energy consumption of the proposed SEAI handover AKA protocol are very much competitive compared to the existing handover schemes. Hence, we expect that the proposed protocol will enhance the performance and security of the 5G communication network in numerous handover applications.