Keywords

1 Introduction

The Internet can be considered as a necessity to make our life easier. It becomes very crucial for users as part of their daily needs. Regardless of any ages, the people are relying on the Internet a lot to gain information, keep in touch with others and entertainment. By 2018, almost 3.6 billion people are using the Internet regardless of any platform either using computers, smartphones or table device [1]. With such technologies, people are not aware of possible menaces. The threats become very contagious and spreadable cross the world. According to [2], there are many types of attacks which pose threats to the user if they are not aware of the security warning such as malware, account hijacking, vulnerabilities and etc. Therefore, to prevent the threats occurred to the minimum level, the security warning plays a role as the line of defence to tell users that they are potentially exposed to harm. According to [3], a security warning is a warning system to remind computer users from the security breaches. In fact, [4] claimed that warning is meant to designed to prevent the people from harm. The security warning is crucial to the user because it gives a message to alert of any security threats recognized and protect the threats damage the computer system.

Although the security warning conveyed an alert to the users regarding the potential harm that might expose to them, the users tend to disregard the security warning because of some reasons. [5,6,7] claimed that users tend to disregard neither read nor understand the security warnings and fail to attract their attention. Also, most users believe that decision that they made when they encountered security warning is a false positive or their computer is safe against any attack because they think that antivirus is enough to protect their computer [6]. [7] stated that the users interpret a warning message with an optimistic way after encountering a dialogue repeatedly. On the other hand, [8] and [9] revealed that the usage of technical terms in security warning is also one of the reasons the users disregard the security warning. Thus, it can be noted that previous works revealed various issues on how end users perceive security warnings. This paper determines to reaffirm the issues by interview session in order to gauge deeper comprehension. This paper is structured as follows: Sect. 2 presents an overview of warning in a computer environment and summaries the problem and challenges in security warning; Sect. 3 describes the methodology utilising the interview session; Sect. 4 presents the result and findings; Sect. 5 highlights the discussion; finally ends with the conclusion in Sect. 6.

2 Related Work

2.1 Warning in Computer Environment

A computer security warning is to serve as a reminder to computer users from the security breaches [3]. According to [10], a security warning is a common method of alerting users from any harms and avoiding them from dangerous acts. A security warning is used in various application to inform the user of a security risk and it encourages users to take secure action to prevent becoming a victim of malware infection or information leakage [7]. However, the users tend to ignore security warning even though security warning conveys messages that tell users that the computer is exposing to various threats. According to [11], the users find that the security warning as an annoyance although it defends the system from harm.

Security warning can be encountered while installing application, open emails’ attachment and restarting the computer [12]. Nevertheless, security warnings in a computer can be displayed while the battery is low, the caps lock is on and opening a file. Apparently, security warning can be categorized into the balloon, in-place warning, notification, dialogue box and a banner [13]. Figure 1 presents some of the examples of security warning that available on the computer. Then again, Table 1 describes the usage of each type of security warning message based on the given example.

Fig. 1.
figure 1

Examples of security warning interface [13]

Full size image
Table 1. The description for the type of security warning interface [13]
Full size table

The focal point of this study is on the dialogue box context. One of the rationales choosing this context is because the implication of making the wrong decision on warning in dialogue box is more severe comparing the other context of warnings. In addition to that, most of the findings also utilizing the dialogue box in their studies [8, 9, 11, 12]. Therefore it is important to have an appropriate guideline in designing the warning from the usability perspective. A guideline is designed to produce a proper security warning interface based on the type of security warning [13]. It explains the design concept that needs to put, capitalization, concision of sentence, icons and terminology. This guideline provides a complete example to distinguish correct and incorrect warning message. However, with the current implementation, security warnings are designed in various way depending on developer’s viewpoint. The next section discusses regarding the problem and challenges of the current security warning encountered by the users.

2.2 Problem and Challenges

There are many approaches to improve the usability of the existing security warning. In most cases, the users had difficulty to understand the usability correctly and not able to utilize it. The users do not know the way to interact with the security tools and technologies which later might lead them in making the wrong decision.

Table 2 presents the eight classifications of problems and challenges towards the security warning implementation. According to [7], the users pay less attention to the warning upon encountered with various dialogue in certain frequency as the dialogue looks similar to them. Apart from that, [15] evaluated the respondents’ comprehension of technical terminology during the interview and online survey and the outcomes show that advanced user is generally able to understand the technical terminology while most beginner respondents unable are not. It shows that these problems can be considered severe because in reality the vast majority of users are from the beginner background. This work determines to reaffirm the problems experienced by the users as presented from the literature works.

Table 2. The problems and challenges
Full size table

3 Methodology

The interview was conducted on a one-to-one basis to discover the trends of user’s comprehension and perception and to analyze in-depth understanding of issues among the users towards security warning.

Previous research by [16, 17] chose an interview session to carry out the experiments and to gain more details about the security warning. This interview was targeting the participant from a technical and non-technical background. A person from the technical background means a person who has studied or involved in computer related field while the non-technical background means who have a slight knowledge of computer related field. Apart from that, [29] categories non-technical group from non-computer related college majors and technical participants had computer-related college majors. In this research, the technical background was identified as the one who has or currently majoring in courses such as computer science and engineering while the non-technical background is the one who majoring in courses not related to computer science and engineerings such as art, management, biology, education, accounting and communication.

The interview was well promoted via social networks such as Facebook, Twitter and WhatApps and word of mouth. Most of the participants were from the Universiti Sains Malaysia, Pulau Pinang main campus. The interview was held in a closed room in order to have better communication and comfortable surroundings. A previous study by [3, 10, 11, 13, 15] were also recruiting the students to be their participants for their interview session.

A simulation of warning was used as a prototype in the interview session. The prototype was developed using Microsoft Visual Studio 2017 with C# language and.Net framework. The database used in this experiment was Microsoft SQL Server Management Studio 2017. The prototype was used in this interview because it is able to gather basic information faster and to present the scenario and context of warning in a presentable manner. A prototype gives optional information such as user’s profile data, able to highlight the information the data site is given and rate the percentages of the information entered. This work was supported by [10, 30] where they conducted a semi-structured interview using the prototype to test their experiment. On the other hand, [7] also interviewed to investigate experimental design matter, user’s reaction matter and usability matter in their studies. On the contrary, [17] was interviewed to investigate the comprehension and attention of the participants toward symbolism which is a signal icon and signal words in security warning.

In the prototype, the participants were presented with three tasks that they have to perform. They needed to imagine that they were performing the task in each scenario and accomplished the task as they have done before. The interview was conducted 20 to 30 min.

Before the interview session started, the participants were given some brief instruction by the principal investigator regarding the experiment. The interview was aimed to gain a participant’s comprehension and response based on the script prepared [9]. The time for the participants made the decision and the conversation of the experiment was recorded for analysis (i.e. automatically via interaction between user and prototype). The actions (i.e. users’ decision, time accomplish one task etc.) were recorded in the database. In addition, audio conversations were also recorded with the consent from the participants. The audio interviewed were then coded, transcribe and analyzed. The coded were done by another researcher to avoid any biases.

4 Results and Findings

The interview gained a total of 60 responses where all the participants have been briefly instructed on how the flow of the interview and given their consent for the data and the recording to be used in this research. Previous research by [17] also recruited 60 participants while [35] recruited 30 participants. This can be concluded that the 60 participants in the research can be considered a sufficient sample size. The responses were treated as confidential as possible where no real information about the participants were revealed. The vast majority of the participants were from the Universiti Sains Malaysia, Pulau Pinang as this work had been well promoted and conducted in the university. Table 3 illustrates the result for the demographics section of the participants.

Table 3. Demographics
Full size table

4.1 Practical Tasks and Interviews

In this part, participants were required to undergo three scenarios of security warnings. For each task, the participants need to imagine that they were performing the task and accomplished the task as they have done it before. The participants need to decide for each task. After that, they need to answer interview questions based on the security warning dialogue presented to them. In the previous study, five of security warning dialogues were presented to the participants in order to gain a clear understanding of the user’s knowledge in security warning [9]. On the other hand, [3, 15] also used three scenarios in their interview study with similar aims like the current work. Series of questions were asked in order to find out their understanding about the signal icons, to understand consequences of actions, the difficulties that they encountered upon receiving the warning and to probe more details about the attractiveness of current version. These questions were set based on previous research by [17] that also conducted an interview session to gain more details perception and comprehension among the participants. Table 4 illustrates the questions asked in each scenario.

Table 4. Question for interview session
Full size table

4.1.1 Scenario 1

In Scenario 1, the participants need to imagine that they have downloaded the file and trying to open a file named SyncBackSetupDE.exe in their computer. The file located at D:\Incoming in their computer. When the participants open the file, the security warning dialogue pops up in their computer.

Security warning in Scenario 1 prompted the participant to make a decision. The participants need to make a decision either to ‘RUN’ or ‘CLOSE’. The decision time for all the participants was recorded. Upon completing all the tasks, the participants need to answer an interview question based on a security warning in Scenario 1.

First, the participants were asked on their first action upon they received this warning. It can be noted that the majority of the participant would read the message first and make their decision to ‘RUN’ the file which was 47% of them. Some of the reasons from the participants were:

  1. i.

    “I read first and decided to ‘RUN’ because of trusted publisher” (P13)

  2. ii.

    “I am making sure to read to ensure that the file is not harmful” (P31).

However, there were some of the participants who decided to run straight away (30%) because they always see the warning and they feel no worry to open the file. This indicates that some of the participants have no attention towards the warning and tends to habituate with the situations [6, 18]. The next question is to know the comprehension of the signal icon in the yellow shield. The question was also asked either the icon is necessary or not. Majority of the participants understands that the icon is a warning icon and the icon is necessary to be placed because it alerts the user to read the message. Figure 2 shows the security warning that encountered by the participants in the interview session.

Fig. 2.
figure 2

Security warning 2

Full size image

The next question is to know the comprehension of the file icon. According to [15], the icon means the application/file exe icon. The majority (56%) of the participants understand well that the icon is application/file exe icon in security warning. Also, the majority of the participants think that it is necessary to put the icon in the warning.

Next, the question was asked to the participants to identify the difficulties they had when they received this warning. Most of the participants have difficulties in understanding technical term such as.exe, type, publishers you trust. When we probe further, the participants claimed that terminology was too technical for them and they did not understand why it needed to be there. In general, and on balances, participants were able to decide due to the previous exposures to the warnings.

4.1.2 Scenario 2

Scenario 2 was depicted in Fig. 3. The participants need to imagine that they have downloaded the document file from Microsoft Outlook named test.docx in their computer desktop. When they are trying to open the document, the security warning pops up.

Fig. 3.
figure 3

Security warning 2

Full size image

Security warning in Scenario 2 prompted the participant to make a decision. The decision time for all the participants was recorded. Upon completing all the tasks, the participants need to answer an interview question based on a security warning in Scenario 2. The first question asked in this section was the first action upon receiving this warning. It can be noted that most of the participants would decide to open the file which 70.3% of them because it does not look dangerous for them. Also, only a minority of them would decide to cancel. However, some of them would read the message carefully before deciding either to open or cancel.

These are amongst the reasons the participants decide to open the file:

  1. i.

    “The icon does not look dangerous” (P8)

  2. ii.

    “The software is trusted because it is from Microsoft” (P48)

  3. iii.

    “It is just a document” (P60).

It can be noted that most of the participants feel that the colour of the icon affects the level of dangerous in the security warning. Hence, it is important to use a suitable colour to convey the message is dangerous or not. However, one of the reasons they decide to open the file which is “it is just documented”. It can be noted that in this situation, the participants underestimate the risk in the security warning. This behaviour can lead to a problem such as being hacked, loss of potential assets and privacy [13].

Next, the question was asked regarding the word icon and either the icon is necessary or not. The icon means Microsoft Word icon [13]. It is one of the Microsoft office products. Majority of the participants answered it correctly even though the majority of them were from a non-technical background and they answered that the icon is necessary to put in the warning message.

The other question was asked to the participant about the blue icon and either it is necessary or not. The blue icon means question mark icon which indicates help [13]. Majority of the participants answered that the icon means question mark icon and it has confused them to decide as it looks safe. Thus, it posed conflict from what users’ perceive and the real meaning of the question mark icon.

After that, the next question was to probe the difficulties that the participants faced when encountered with this warning. Again, most of the participants (62%) do not understand the technical term as well as some of the icons. Also, they do not understand the function of checkbox in the warning. One claimed that it should be automated from the warning to decide on behalf of the users. The last question was asked to the participants either the warning attract their attention or not. Majority of the participants which 55% of them claimed that the warning message attracts their attention because they were not always encountered with the warning.

4.1.3 Scenario 3

Similar to the previous scenarios, the security warning in Scenario 3 prompted the participant to make a decision as well. The participants need to make a decision either to ‘Install’ or ‘Do not install’. The decision time for all the participants was recorded. Upon completing all the tasks, the participants need to answer an interview question based on a security warning in Scenario 3. The first question was asked regarding the first action upon encountering the warning based in Scenario 3. It can be noted that most of the participants which are 53.3% would read the warning carefully before proceeding to any decision. However, some of the participants would decide not to install the application which is 23.3% and 18.3% of them decide to install the application. These were the feedback on the reason’s participants read the message carefully:

  1. i.

    “The red icon looks dangerous” (P3)

  2. ii.

    “I am afraid if something bad happened to my computer” (P40)

  3. iii.

    “It stated that the publisher is unknown in bold font” (P51) (Fig. 4).

    Fig. 4.
    figure 4

    Security warning 3

    Full size image

Also, the comprehension towards the signal icon was asked in this warning. The question was asked about the globe icon and either it is necessary or not. It can be noted that the majority of the participants (83%) do not understand the icon because they have not seen the icon before. The next question was asked regarding the red icon and either it is necessary or not. The red icon means the error icon which indicates something wrong and high possibility of unsafe [15]. Most of the participants misinterpret the icon as a warning icon (71%). Only a minority of them answered as error icon.

On the other hand, the question was asked regarding the difficulties that the participants faced when they encountered this warning. Again, most of the participants have difficulties in understanding the technical term, signal icon and hard to make a decision. However, a minority of them had no difficulties at all. Lastly, the question was asked the attractiveness of the message to the participants. It can be noted that the majority of the participants claimed that this message attracts their attention. Some of the reasons are stated as follows:

  1. i.

    “The red icon looks dangerous” (P8)

  2. ii.

    “The publisher is unknown” (P13).

4.1.4 General Questions

In this section, the interview study presents questions related to the improvement needed and another opinion regarding the current implementation of the security warning. The questions were aimed to gather more opinion from the user. Table 5 indicates the questions that had been asked. It can be noted that most of the participants claimed that the term is easier to understand mostly by those who were from the technical background users compared to non-technical background users. They also claimed that the technical terms sometimes tend to make things more complicated. It should be clear to the reader to ease their burden in deciding. The next question was asked about the presentation of current security warning implementation. 92% of the participants claimed that the security warning should be further improved. These were the reasons that express by the users:

Table 5. General Questions
Full size table
  1. i.

    “People do not like many words” (P1)

  2. ii.

    “The words should be bold if it dangerous” (P4)

  3. iii.

    “Use layman term so the most user would understand the message” (P7)

  4. iv.

    “The colour is dull” (P52).

5 Discussion

Based on the results and findings discuss in the previous section, it can be noted that there is a corresponding need to improve the current implementation of security warning regarding the usability. In this study, 92% of the participants suggested that the current implementation of security warning can be further improved. The most notable difficulties face by the users were the difficulties in comprehending the technical terminologies used in this warning. It tends to make the interface of the security warning plain, dull and less user-friendly. In all the three scenarios, users prefer to use the layman term or less technical term so that both backgrounds either technical or non-technical users could understand the message very well. Our work confirmed with the previous findings where it had been revealed by [9, 14] that users experienced significant problems with technical terms. Our results also suggested that users experienced significant issues with the usage of signal icons and words use as a claim in [17]. Besides that, according to [5, 19], most of the participants did not pay attention when the security warning popped up. Our work revealed that even though the details of information in security warning is short, the users still have difficulties in understanding the warning and making a decision. They tend just to ignore it and proceed with an action without reading it where the consequences may be severed. The resulted behaviour such as ignoring the warning is correlated with the user’s attention [6]. On the other hand, it is not surprising that some of the participants claimed that they had no difficulties at all with the current context of warning as they can clearly explain the meaning of the terminology used in the security warning as well as signal icon. The colours and the presentation of warnings may affect users’ understanding and attention towards it. Apart from that, the similar security warning appears repeatedly which leads to habituation effects. Previous experiment conducted by [36] proved that the static security warning obtained high habituation among the participants. Thus, it is crucial to improve the security warning especially from the usability context so that users can pay more attention towards the security warning to prevent them from any harmful threats. The summary of the work is presented in Fig. 5.

Fig. 5.
figure 5

Summary of discussion

Full size image

6 Conclusion

In conclusion, there are corresponding needs to improve the security warnings based on the continuous studies on usability issues in the warnings. It is crucial to discover the problems that arise in the computer security field for safety purposed in term of a personal asset, identity and also financial data. Our work has reaffirmed the previous studies and our work mapping the problems in further details as depicted in Fig. 5. The usage of technical terminologies, lack of explanation and the warning were not attractive were amongst the most highlighted findings in this study. Therefore, security warnings should be designed to map with the problems above and challenges experienced by the users.

On the other hand, there are some limitations to this work. This work is more focusing on dialogue box because most of the users encounter it in our daily life while using a computer and not the other contexts such as notification, balloon, in-place and banner. Apart from that, this research gained insights from mainly student’s perspective where the majority of them were from the Universiti Sains Malaysia.

For future works, the enhancement to improve the current version of security warning is needed to tackle the severe issues such as excessive usage of technical jargons, lack of attractiveness and hard to make a decision. It is expected that with the new design of security warnings, it will be able to increase attractiveness that leads the user to heed the security warning and increase the comprehension of the message context of the warnings to the users.