Keywords

1 Introduction

The expression ‘Early Warning’ is used in many fields to mean the provision of information on an emerging dangerous circumstance where that information can enable action in advance to reduce the risks involved. A universally accepted definition of an EWS does not yet exist and most probably never will. An EWS can be defined as “a social process for generating maximally accurate information about possible future harm and for ensuring that this information reaches the people threatened by this harm, as well as others disposed to protect them from the harm” [2]. An ‘Early Warning Information System (EWIS)’ can be understood as a set of institutional and technical solutions designed and implemented in a coherent way to make available, to a wide range of users and more particularly to decision makers, information useful to carry out vulnerability analyses, to evaluate and manage the risk of a hazard that can become a disaster, and to manage disasters from prevention to recovery and rehabilitation [3]. The objective of EWIS is to generate accurate information to empower individuals and communities threatened by hazards to act in sufficient time and in an appropriate manner to reduce the possibility of personal injury, loss of life and damage to property or the environment. An effective information system is essential to the success of any business or organization. The effectiveness of information systems (IS) is a complex variable. The literature on organizational effectiveness suggests that it may not be possible to find a precise measure of IS effectiveness and the criteria for effectiveness may vary from one organization to another [1]. A good architecture for an information system attempts to balance the conflicting criteria of functionality, usability, reliability, performance, portability, and maintainability. Any information system (IS) derives its value from the impact it has on the performance of an organization [4]. The literature on organizational effectiveness suggests that it may not be possible to find a precise measure of IS effectiveness and that the criteria for effectiveness may vary from one organization to another [2]. System effectiveness is a function of system availability, capability, and dependability [5]. Evaluating the effectiveness of an Information System is a complicated process due to its multidimensionality, its quantitative and qualitative aspects, and the multiple and often conflicting evaluator viewpoints [4]. Researchers agree that IS effectiveness depends on the organizational effects produced by an information system [5]. Information systems will improve decision-making, by providing more timely and more accurate information, stimulating more interaction among decision-makers, and offering better projections of the effects of decisions [7]. The effectiveness of IS can be determined by comparing performance to objectives or by attainment of a normative state [8]. Effective early warning systems not only save lives, but also help protect livelihoods and national development gains [9].

2 Characteristics of Effective Early Warning Information Systems

A good structure is one that is useful, robust, and beautiful. This simple 2000 year-old definition attributed to the Roman architect Vitruvius, is also true of Information Systems. Users of Information Systems will judge its effectiveness by these 3 criteria, and achieving a balance between the 3 is the mark of a good Information System [10]. The good information system can also be identified by the following characteristics: Accessibility, Accuracy, Simplicity, Flexibility and Security [11]. Other researchers have characterized good information systems as being: understandable, relevant, complete, available, reliable, concise, timely and cost-effective [12]. A system’s usefulness depends upon its functionality, usability, and maintainability. A system’s robustness depend upon performance and reliability. A system designer must be able to select the criteria applicable to solve the posed problems more effectively [10]. The ISO 9126, an open standard for software quality, provides 6 primary architectural criteria that system designers can use to find the best fit. These criteria, and some of their primary concerns, are as follows: Functionality, Usability, Reliability, Maintainability, Performance and Portability. According to Carlson (1974), IS effectiveness is concerned with those effects on an organization that result from the development and use of an information system. Every organization should determine the main criteria for a given system, and to determine the sequence in which each priority will be addressed. From last review of previous studies, we have concluded that the effectiveness of the EWIS is determined using a number of factors that differ from one organization to another.

3 Validating the Effectiveness of EWIS

According to the effectiveness criteria previously explained in this research, we have obtained the following results:

1- Integration:

The implemented EWIS sub-models are integrated into a unified model (Generic Model), The generic model is divided into set of sub-models with specific inputs and outputs. Every sub-model consists of a number of processes and functions. Each process has a number of identifying attributes: such as process name, input, output, executed by the user or the system, etc. The researchers have divided the generic model into a number of sub-models in order to facilitate and accelerate the process of development and construction of the EWIS, in addition to making it easier to evaluate each sub-model. The generic model is constructed and refined through the incremental prototyping concept (iterative series of prototype).

2- Availability:

An Effective EWIS is largely dependent upon the availability of current and reliable information, which is processed to facilitate decision-making and enhance productivity. The literal meaning of ‘availability’ is “the readiness to be obtained or used”. It is a measure of the time duration when the system under consideration not only remains up without any outages, but also performs the desired function efficiently and optimally and remains accessible to its desired users. There must be some kind of metric associated with the system’s availability. Researchers generally measure availability do it in terms of Latency [13], Mean Time to Fail (MTTF) and Mean Time to Repair (MTTR) [14], also known as Mean Time to Restore & Mean Time Between Failure (MTBF). These criteria measure the amount of time an information source was unavailable, but from different perspectives. MTBF and MTTR are the most commonly used metrics [15]. MTBF measures the average amount of time between failures, on the other hand MTTR measures the average time required to repair/restore a failed system. Availability may be measured, as the ratio of the time a system remained available at the time it should have been available [16]. The ratio is calculated using both MTBF and MTTR

$$ {\text{Availability = MTBF/}}\left( {\text{MTBF + MTTR}} \right) $$

From the above-mentioned equation, it is evident that by keeping MTBF greater than MTTR we can achieve a healthy percentage of availability. The availability depends upon the reliability and downtime of a system [17]. The EWIS has been observed for two years (see Table 1) and it has achieved high availability ratios in the year (2013).

Table 1. Calculation of availability in year 2012, 2013
Full size table

3- Predictability:

is defined as the degree to which a correct prediction or forecast of a system’s state can be made either qualitatively or quantitatively [18]. An EWIS is highly predictive and capable of forecasting the crisis before it occurs. The EWIS uses five prediction models (Linear, Quadratic, Exponential Growth, Moving average and Cubic) to forecast future data trends. In addition, the system selects the best forecasting model according to accuracy and correlation between the variables. The result of the prediction is double-checked using mathematical software (Minitab) to ensure the correctness of the prediction models. The EWIS can use the five models at the same time to predict future trends. In addition, the system chooses the best prediction model according to the some criteria such as lowest error rate and highest correlation between data.

4- Accuracy:

The accuracy of the system is defined as the freedom from error (correctness), or closeness to truth or fact, resulting from the exercise of painstaking care or due diligence. Accuracy depends on how the data are collected, and is usually judged by comparing several measurements from the same or different sources [19, 20]. The EWIS is tested using different types of testing such as system testing and module (Unit) testing. In Module testing the EWIS is divided into smaller chunks known as modules, these modules are tested individually to find out if they are working individually or not. System Testing is carried out only after the module testing and integration testing are completed. In System testing the same production environment is simulated and test cases are executed in the same environment as a client environment. In system testing we perform testing based on both functional and non-functional requirements. Each unit or screen is validated and the testing includes every unit present on the screen (like controls, fields, data length, etc.) It is basically done in the development environment itself. If a number of defects found, we normalize them to a unit amount of code, 1000 lines of code, or “KLOC” which is often used as a standard base measure (see Table 2). The higher the defect density, the more defects we are uncovering. It is impossible to give an “expected” value for defect density. Mature, stable code might have defected densities as low as 5 defects/KLOC; while new code written by junior developers may have 100–200 defects/KLOC [20, 21]. Defect density is a measure of conformance to customer requirements. That is, the number of defects per lines of code is a measure of quality. In fact, defect density is by far the most popular measure of quality in the field of software engineering [20]. In order to measure the accuracy of the EWIS, we can use the following equations:

Table 2. EWIS accuracy measurements
Full size table

  • Density of Test (Number of test cases per KLOC) = Number of Test Cases/size of the project (Total Lines of Code) * 1000 = (966/44953) * 1000 = 21.5 test cases per Thousand Lines of Code

  • Total Lines of code = 44953

  • Defect Density (Bug Rate per KLOC (Kilo Lines of Code)) = Number of defects/Lines of Code = (73/44953) * 1000 = 1.6 defects per KLOC

  • Error Discovery Rate (It is defined as the ratio of defects per test cases) = Total number of defects found in application/Number of test cases or scripts executed = 73/966 = 0.0755 Defects/Test cases

  • Quality = 1 – (defects/Lines of Code) * 100% = 1 – 73/44953 * 100% = 99.8%

5- Flexibility:

In order to be effective, an information system (IS) needs to be flexible, that is, it must be able to accommodate a certain amount of variation regarding the requirements of the supported business process [22]. The flexibility of information systems has been the subject of much research for during the past decade or two. A major area of research on information system’s flexibility, focused on exploring the effects of change on the alignment of organizational systems and information systems [23]. Flexibility is defined as the capability to adapt to new, different, or changing requirements [24] or in other words, flexibility is regarded to be the capability of reacting to perceived stimuli. An EWIS should be flexible and should be capable of expanding its activities to include different varieties of risks. The EWIS can be used in different organizations with different events or hazards. The EWIS is flexible to change; the system has been designed using the incremental prototyping approach. The system started to work with only two mathematical forecasting models, then the number of models used by the system increased thereafter to become five mathematical models by the end of the project, and now it is flexible to accommodate more mathematical models in the future. The system also started to work with a monolingual setting (Arabic Language) and during the data entry process, the user requested to add another language (English Language) because of the presence of non-Arabic speakers work as among the data entry personnel working on the project, so hence the system was flexible to change and now was transformed into a bilingual system.

6- Functionality:

The term function is defined in the IEEE Standard 610.2 as “A defined objective or characteristic action of a system or component. For example, a system may have inventory control as its primary function” [25]. This sense of function emphasizes the dynamic aspects of software, and is complemented in the normative dimension by the definition of functional requirement as “A requirement that specifies a function that a system or system component must be able to perform”. To complement these definitions, functionality is defined in the Merriam Webster’s online dictionary as “the particular set of functions or capabilities associated with computer software or hardware or an electronic device” [26]. The EWIS has succeeded in achieving the following functionality:

  1. 1.

    An EWIS is used in capturing, detecting and analyzing the event/crisis information, determining a set of mathematical indicators for each event/crisis that should be measured frequently, providing future forecasts depending on the data calculated according to previous indicators, and finally sending warnings (alerts) to users if there is an imminent danger.

  2. 2.

    The EWIS provides a useful, time-saving and acceptably accurate solution to a specified task or problem.

  3. 3.

    The user interface is effective in explaining the data requirements and describing the sequence of calculations to the user. At the same time, the user interface is easy and is not too complicated.

To prove that the EWIS is highly functional, the researchers designed a checklist to evaluate the functionality of the system. The researchers chose 20 participants to fill in the form (see Table 3). The checklist consists of 20 questions that measure different functional aspects of the EWIS. The researchers have analyzed the answers and the result is shown in the Table 3. The average result of the EWIS functionality checklist was 93.55%, which means that around 94% of the EWIS users are satisfied with the functionality of the system and around 6% are not satisfied. Using these results and the feedback from users, the researchers made more enhancements to the system, taking into consideration most of the users’ suggestions to increase the former percentage.

Table 3. EWIS functionality checklist result
Full size table

7- Usability:

refers to how well users can learn and use a product to achieve their goals and how satisfied they are with that process [27]. ISO 9241-11 defines usability as: “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use”. An EWIS is a highly usable system. It offers benefits to both users and businesses. The primary benefits to users are that they are able to achieve their tasks easily and efficiently. This sounds simple, but the feeling of achievement that people get when they use a computer system without frustration should not be underestimated. The EWIS is very easy to use and is simple to understand, different kinds of users were able to use the system with ease, following a brief training course whose duration ranged from 8 to 12 h. The researchers measured the usability of the EWIS by carrying out a usability test that identifies the key usability problems in the system (enabling them to be fixed); and/or collects quantitative measures of efficiency, effectiveness and satisfaction before release. The participants involved are existing users of the system. Participants attempt to carry out tasks that they would normally perform on the system. The usability test was set up in a way that is as close to the normal context as possible. The researcher chose (25) participants to fill in the form (see Table 4), which consists of (20) questions that measure the usability of the EWIS and the result was used to enhance the usability of the EWIS in later stages. The researchers then analyzed the answers in the form and the result is shown in Table 4. The EWIS average result of the usability checklist was 92.8% of all the questions in the form, which means that around 93% of the EWIS users are satisfied with the usability of the system and around 4% are not satisfied, therefore the researchers used these results as a guideline for making more enhancements to the system, also taking into consideration most of the user suggestions to increase the former percentage.

Table 4. EWIS usability checklist result
Full size table

8- Reliability:

is defined as the probability of performing without failure, a specific function under given conditions for a specified period of time. Moreover, according to the business dictionary on the internet, it is the ability of a computer program to perform its intended functions and operations in a system’s environment, without experiencing failure (system crash) [28]. The Wikipedia also defines software reliability as the probability that software will work properly in a specified environment and for a given amount of time [29]. Software Reliability is also defined as the probability of failure-free software operation for a specified period of time in a specified environment. Software Reliability is also an important factor affecting system reliability. It differs from hardware reliability in that it reflects design perfection, rather than manufacturing perfection. The high complexity of software is the major contributing factor of Software Reliability problems [30]. According to ANSI, Software Reliability is defined as: the probability of failure-free software operation for a specified period of time in a specified environment [31]. Although Software Reliability is defined as a probability function, and comes with the notion of time, we must note that, it is different from traditional Hardware Reliability, Software Reliability is not a direct function of time. Electronic and mechanical parts may become “old” and wear out with time and usage, but the software will not rust or wear-out during its life cycle [32]. The software will not change over time unless intentionally changed or upgrade [32]. Software Reliability is an important attribute of software quality, together with functionality, usability, performance, serviceability, capability, installability, maintainability, and documentation. Software Reliability is hard to achieve, because the complexity of software tends to be high [33, 34]. While any system with a high degree of complexity, including software, will be hard to reach a certain level of reliability, system developers tend to push complexity into the software layer, with the rapid growth of system size and ease of doing so by upgrading the software [34]. While the complexity of software is inversely related to software reliability, it is directly related to other important factors in software quality, especially functionality, capability, etc. Usually, failure metrics are based upon customer information regarding failures found after the release of the software. The failure data collected is therefore used to calculate failure density, Mean Time between Failures (MTBF) or other parameters to measure or predict software reliability [34]. The EWIS has proven to be consistent and has produced correct outputs and this was measured frequently through statistical mean value for error-free applications which is called Mean Time Between Failures (MTBF), The system has been observed for two years and it has a downtime of 39 h during the year 2013 which is calculated as [(39/8640) * 100 = 0.45%)], where total working hours per year = 8640 (24 h * 30 days = 720 h * 12 months), and we have calculated the 12-months average which equals to 99.55% availability this year.

4 Conclusions

A comprehensive and effective information system is essential to the success of any business. The effectiveness of Early Warning Information systems is a complex variable. The literature on organizational effectiveness suggests that it may not be possible to find a precise measure of EWIS effectiveness and the criteria for effectiveness may vary from organization to organization. The effectiveness of the EWIS can be determined according to a number of factors that differ from one organization to another. In our case study, which was applied in the law enforcement sector, the most important factors contributing to increase the effectiveness of the EWIS were determined through 8 pillars as follows: Integration, Availability, Predictability, Accuracy, Flexibility, Functionality, Usability and Reliability, So the researchers, implemented an effective EWIS that accurately supports critical functions and provide efficient and effective value added services to decision makers as well as maximizing the profitability and outcomes of the organization.