Quality Assessment

Abstract

Quality is a first-class citizen in eG4M. Users of services do not want to lose time in their interactions with administrations, they do not want to suffer to provide information which is already present in public administration databases and they do not want to be bothered by inefficiencies and errors in administrative processes. Figures say that the time period a company has to wait in Italy to be registered in the official public registry and start to operate is ten times the time needed in Great Britain and France. Yet, time is money, and such inefficiencies deeply influence business development. The same for citizens, who greatly appreciate fast, efficient, proactive and transparent administrations. All the previous aspects are captured by the concept of quality. In eG4M the quality assessment phase aims to identify and measure the most relevant qualities of the different eG4M layers defined in Figure 5.1. Among them, the most important ones are service qualities. Considering only service qualities in the planning activity hides issues that may deeply influence the nature of new projects, since services are produced by processes, that are performed in administrations, whose functions are defined by laws.

This chapter is authored by Carlo Batini and Gianluigi Viscusi.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Quality is a first-class citizen in eG4M. Users of services do not want to lose time in their interactions with administrations, they do not want to suffer to provide information which is already present in public administration databases and they do not want to be bothered by inefficiencies and errors in administrative processes. Figures say that the time period a company has to wait in Italy to be registered in the official public registry and start to operate is ten times the time needed in Great Britain and France. Yet, time is money, and such inefficiencies deeply influence business development. The same for citizens, who greatly appreciate fast, efficient, proactive and transparent administrations. All the previous aspects are captured by the concept of quality. In eG4M the quality assessment phase aims to identify and measure the most relevant qualities of the different eG4M layers defined in Figure 5.1. Among them, the most important ones are service qualities. Considering only service qualities in the planning activity hides issues that may deeply influence the nature of new projects, since services are produced by processes, that are performed in administrations, whose functions are defined by laws.

In Sect. 7.1 we define the concepts related to quality and introduce typical methods existing in the literature to observe and measure service-related qualities. In Sect. 7.2 we describe the eG4M quality registry that represents and defines qualities for all the layers discussed before. Such qualities pertain to the categories of efficiency, effectiveness, accessibility, and accountability. Section 7.3 investigates the correlations defined among qualities to represent qualities as a system of related issues. Section 7.4 recapitulates all the concepts introduced in previous sections providing a practical methodology to perform quality assessment. Finally Sect. 7.5 applies the methodology to the running example.

1 Introduction to Quality

According to the ISO [113], a quality of an artifact, a product or a service is the degree to which a set of inherent characteristics of the artifact, process, or service fulfills requirements. Qualities can be classified in terms of characteristics and sub-characteristics ; instead of characteristics/sub-characteristics, we will use the general term quality dimensions or, simply, dimensions . Two modalities are used to assign a value to quality dimensions, namely

• a measure , performed with a measurement procedure that results in a metric , namely a value in a domain and

• the evaluation of perception of users with questionnaires and focus groups.

According to the first modality, a measurement procedure consists in a sequence of steps that leads to measuring a value for the dimension in a specific domain. For instance, an efficiency dimension for services is temporal efficiency, which corresponds to the time related to service provision. A first possible metric for temporal efficiency is the user time, namely, the time the user has to spend in (i) service request, (ii) possible further interactions with the administration, and (iii) service acquisition. Another metric for temporal efficiency is the service provision time, the time passed from the time stamp of the request to the time stamp of the delivery. This interval of time can be measured easily and with low cost if the administration involved in service provision manages a workflow procedure that captures the request time and the delivery time. In case the macro-process involved in service provision crosses more than one administration, it is also necessary to identify the inputs and outputs associated with the different processes, in such a way that they can be univocally associated with the service request.

Notice that temporal efficiency can also be evaluated through user perception. In this case we are looking for an average perception, since we can find intolerant users and patient users, so the user’s sample has to be chosen in such a way to be representative of the universe of users. For methods related to the choice of samples and segmentation of users, see [81, 135]. Not all perceived dimensions can be measured with a measurement procedure, e.g., the kindness of the front-office personnel of an administration cannot be measured, it can only be evaluated through the perception of users or else of a third-party human observer.

The literature on quality is characterized by many proposals on how to model and assess the quality of services. One of the most popular proposals referring to the user’s perception concerns Parasuraman [168, 235]. In the approach of Parasuraman (see Fig. 7.1) user’s service expectations are compared to user’s service perceptions; this is done using questionnaires whose questions refer to a set of quality dimensions. In the original approach such dimensions pertain to the following categories:

• Tangibles, namely the appearance of physical facilities, equipment, personnel, and communication materials.

• Reliability, the ability to perform the promised service dependably and accurately.

• Responsiveness, the willingness to help customers and provide prompt service.

• Assurance, the knowledge and courtesy of employees and their ability to inspire trust and confidence.

• Empathy, that is, caring, individualized attention the firm provides its customers.

Fig. 7.1

The gap model of the methodology adapted from [167]

Fig. 7.2

quality dimensions considered in the gap model adapted from [169]

The gap between perceptions and expectations is the input to a process performed by the organization delivering the services, which analyzes such a gap in order to understand how to react by improving the organization’s service standards and performance. Such an effort of improvement has to be communicated to users, thus closing the life cycle of the quality measurement and improvement activity.

A more recent classification of service quality is provided in [167], where the focus is on the quality of the web site that manages requests of users and delivery of services. The different quality dimensions of web site quality are defined in Fig. 7.2.

Although the approach of Parasuraman is quite popular, in this book we prefer to propose a different approach where the quality of services is seen as intrinsically related to the quality of administrative processes, the quality of public administration organization, the quality of laws and norms, and, finally, the quality of ICT support framework. In the following we discuss this approach.

2 The eG4M Quality Registry

In this section we introduce and define all the quality dimensions considered in eG4M. Since administrative processes and the organization of the public administration are strictly related, we will refer generically to qualities of organization/processes. All the quality dimensions considered are represented in the eG4M quality registry , see Fig. 7.3, clustered in terms of the eG4M layers considered relevant in an eGovernment planning activity.

Fig. 7.3

The eG4M quality registry

Dimensions belong to four general categories:

• 1. Efficiency: the amount of resources (including time) needed for service provision.

• 2. Effectiveness: how close is the service provided to user’s expectations.

• 3. Accessibility: how easy and feasible it is for the user to request the service, in terms of technological resources available and friendliness of the interactions.

• 4. Accountability: the assumption of responsibility for actions, products, decisions, and policies of the administration. It includes the obligation to report, explain and be answerable for resulting consequences of service provision.

All of the above categories are meaningful for all eG4M layers. Each one of the above categories is refined in the following for each layer in terms of dimensions and in some cases in terms of metrics.

2.1 Efficiency

2.1.1 Legal Framework Efficiency

Legal framework efficiency is achieved when the legal framework is produced with a limited use of resources [112]; such resources can be human, financial, or temporal resources. A sub-dimension of legal framework efficiency is redundancy : a legal framework is not redundant when the set of laws is minimal, namely, each issue dealt with in the legal framework is regulated by one and only one law (or part of law) [73]. Besides the whole legal framework, these qualities can be defined for specific laws, parts of laws, or for a set of laws related to a specific domain. For what concerns the level of enforcement of quality dimensions, it is important to note the difference between legal technique (singular), pertaining to the correct production of the text of a law, and legal techniques (plural), these latter encompassing the analysis of administrative procedures and the appropriateness of a law [112].

2.1.2 Service Efficiency

Service temporal efficiency refers to efficient use of time in service production and provision. As anticipated in Sect. 7.1, it can be expressed in terms of two metrics:

• The user time : the average time spent by users to obtain the service.

• The service provision time : the average time spent by organizational units to produce the service.

To measure the user time for services provided with a traditional desk, we have to add

• 1. the time required to go to the desk;

• 2. the waiting time in line;

• 3. the service time at the desk.

In case of services which request citizens to collect information, such as certificates from other administrations, we have to also consider the time spent by users in obtaining this information.

The service economic efficiency concerns the costs sustained for service provision and their trends in time. This economic dimension can be specialized by relating the cost to the output which the service produces, e.g., for services that produce one or more documents in output, we can specify the service economic efficiency by means of a metric expressed as cost/output.

2.1.3 Organization/Process Efficiency

Organization/process temporal efficiency corresponds to time execution of the macro-process which provides the service. The organization/process procedural efficiency concerns the level of bureaucratic simplification. Procedural efficiency refers to obligations and constraints that laws impose on the administrative processes and on the interactions between administrations and external users. We associate a sub-dimension with the procedural efficiency, consisting in the level of simplification , and a metric, the number of interactions required by users to provide useful information in order to complete the service.

2.1.4 ICT Infrastructure Efficiency

ICT infrastructure temporal efficiency can be measured by means of the response time , the usual metrics considered for ICT infrastructures; the measure is simply the time interval that the ICT resources need to execute transactions and/or queries; it can be expressed referring to an application load expressed in terms of several benchmarks; see also [211] for a full introduction to this issue.

2.2 Effectiveness

Effectiveness concerns the closeness to user needs and expectations. Sometimes, it is not easy to distinguish between efficiency and effectiveness. Furthermore, to evaluate efficiency we can ignore user expectations and focus on the output produced by the service and the resources used. To evaluate effectiveness, we have to know precisely the user’s expectations. We can say that effectiveness tends to “do the right things” while efficiency has the objective of “doing things right.”

2.2.1 Legal Framework Effectiveness

Effectiveness of the legal framework is achieved when the legal framework has been conceived so that its concrete enforcement produces the results and outcomes initially planned [53]. Sub-dimensions of legal framework effectiveness are as follows:

• Accuracy : laws address precisely and non-ambiguously the domain of interest and do not give rise to misleading applications in the domain [134].

• Coherence : laws enacted over time are not contradictory to each other and the legal terms used within different legislative domains of intervention refer to a common coherent lexicon [32].

• Completeness : the legal framework addresses all the issues relevant for the considered domain of enforcement [32, 85], in our case the eGovernment plan.

2.2.2 Service Effectiveness

Service effectiveness expresses the property that the service achieves users’ expectations. Proper metrics of effectiveness can be the figures of complaints, and the users’ perception of the usefulness and reliability of the provision of the service [168]. A sub-dimension associated with service effectiveness is service correctness , namely the coherence of the service behavior with the requirements.

2.2.3 Organization/Process Effectiveness

At this level we are interested in effectiveness of personnel, which results in capability to face and proactiveness. Capability to face is the ability of personnel in the relationship with the user and in the resolution of all types of problems arising in process execution. Proactiveness corresponds to the ability to foresee and anticipate events and act in advance to deal with an expected difficulty.

2.2.4 ICT Effectiveness

At the data layer, effectiveness results in several dimensions, which we have examined in Chap. 2. Due to the relevance of the data resource in administrative activities, we discuss in detail possible metrics for accuracy and completeness.

Two kinds of accuracy can be identified, namely a syntactic accuracy and a semantic accuracy. Semantic accuracy of a value v is the closeness between v and the correct value v′ that v aims to represent. In the following we do not address semantic accuracy; the interested reader can find further details in [23].

Syntactic accuracy is the closeness of a value v to the elements of the corresponding definition domain D. In syntactic accuracy we are not interested in comparing v with the true value v′; rather, we are interested in checking whether v is any one of the values in D, whatever it is. So, if v = Jack, even if v′ = John, v is considered syntactically correct, as Jack is an admissible value in the domain of persons’ names.

Syntactic accuracy is measured and achieved by means of functions, called comparison functions , which evaluate the distance between v and the values in D. Edit distance is a simple example of a comparison function, taking into account the minimum number of character insertions, deletions, and replacements to convert a string s to a string s′. More complex comparison functions exist, for instance, taking into account similar sounds or character transpositions. So, given a comparison function C, we may define a measure of syntactic accuracy of a value v with respect to a definition domain D, as [1 − mindistance/n] where mindistance is the minimum value of C, when comparing v with all the values in D and n is the maximum possible value that the comparison function may have.

Notice that the definition of syntactic accuracy can be used, besides measuring the degree of accuracy, also for correction of inaccurate values, which can be changed into the closest value in the definition domain D.

Concerning completeness, we introduce a simple metric for tables, the typical data structures adopted in the relational data model. We assume that null values, namely, values that exist in the real world but for some reason are not available, denote the presence of an incompleteness. Thus, the measure of completeness of a table is $[1-\mathit{nv}/n]$ where $\mathit{nv}=$ the total number of null values in the table and $n=$ the total number of values in the table]. A more extensive discussion on metrics for completeness can be found in [23].

As an example of evaluation of accuracy and completeness, let us consider the relation Movies introduced in Chap. 2 (see Fig. 7.4).

Fig. 7.4

A relation Movies

The value Rman Holiday in movie 3 for Title is syntactically inaccurate, since it does not correspond to any title of a movie. The accuracy of the term Rman Holiday is equal to 1 − 1/15 = 0.9, since the closest movie title is Roman Holiday, and 15 is the maximal length of items in the table. The completeness value of the table is 3/24, namely, 0.13.

2.3 Accessibility

Dimensions associated with accessibility take into account all the issues related to the existing service access barriers for different kinds of users, such as elderly or disabled people, rural or suburban population, illiterate people, linguistic minorities, people without a sufficient level of technological skills; furthermore, accessibility also refers to technological barriers related to ICT networks and infrastructures needed to connect with the system providing the service.

2.3.1 Legal Framework Accessibility

The legal framework accessibility is achieved when the users of the legal framework can easily access the whole set of norms related to their rights, duties, and obligations [85]. The legal framework of a state is the result of years and years of enacting of new laws that are added to previous bodies of law. The new law usually expresses the abrogations and updates to previous laws with explicit references such as “law 325/98 is abrogated” “articles 1 and 2 of law 325/98 are abrogated” or else with implicit references such as “all laws referring to cattle raiding are abrogated.” Notice that in both cases it is not easy to deduct the consequences of abrogation or update. It is also evident that the more the legal framework is updated with new laws, the higher the risk is for the legal framework to have a “spaghetti style.” A possible metric for the legal framework accessibility is the complexity of the legal framework, measured by the number of implicit and explicit references divided by the number of different topics addressed, where the implicit references have a higher weight w.r.t. explicit ones. Other measures can be obtained using questionnaires to be filled by a sample of users, especially judges, legal offices of public administrations, and lawyers.

2.3.2 Service Accessibility

Service temporal accessibility, defined at the service layer, expresses the interval of time, usually in a day or in a week, during which the service can be requested. Focusing on a traditional service desk, it can be measured through the amount of business/opening hours of the desks. Service cultural accessibility concerns the diffusion among users of skills and capabilities required for an autonomous usage of service and the attitudes, preferences, and perception (e.g., trust, ease of use) expressed by users toward the usage of different technological channels which support the provision of service. A number of metrics can be used to measure this dimension, such as

• the percentage of words whose meanings can be understood by an average user;

• the number of languages in which the service is provided;

• the media richness of the channels [57]; and

• the perceived usefulness and perceived ease of use of the access channels [1, 59, 198].

Service physical accessibility measures the ability of the user to access the service from his/her physical status/functions. Particularly important in the social domain is physical accessibility for disabled persons . Metrics and guidelines for physical accessibility have been proposed, among others, by the World Wide Web Consortium [218] that defines the individuals with disabilities as subjects who

• 1. may not be able to see, hear, move, or process some types of information easily or at all;

• 2. may have difficulty reading or comprehending text;

• 3. may not have to or be able to use a keyboard or mouse;

• 4. may have a text-only screen, a small screen, or a slow Internet connection; and

• 5. may not speak or understand a natural language fluently.

An example of metrics for blind persons is the presence/absence of equivalent alternatives to visual content in a web site or an application used to access the service; such alternative is a text equivalent content . In order for a text equivalent content to make, e.g., an image accessible, it can be presented to the user as synthesized speech, braille, and visually displayed text. Each of these three mechanisms uses a different sense, making the information accessible to groups affected by a variety of sensory and other disabilities. In order to be useful the text must convey the same function or purpose as the image. For example, consider a text equivalent for a photographic image of the continent of Africa as seen from a satellite. If the purpose of the image is mostly that of decoration, then the text “Photograph of Africa as seen from a satellite” might fulfill the necessary function. If the purpose of the photograph is to illustrate specific information about African geography, such as its organization and subdivision into states, then the text equivalent should convey that information with more articulate and informative text. If the photograph has been designed to allow the user to select the image or part of it (e.g., by clicking on it) for information about Africa, equivalent text could be “Information about Africa,” with a list of items describing the parts that can be selected.

2.3.3 ICT Accessibility

Infrastructural accessibility , defined at the technological layer, concerns the diffusion of the infrastructure and technologies which support the service provision: proper measures are the number of territorial desks per inhabitant and the average distance from the territorial desk for different kinds of users. Channel accessibility focuses on the existence of different channels for service access and delivery, such as the desktop, cell phone, TV. Channel accessibility can be measured in terms of the number of available access channels for the service considered. Data accessibility refers to the possibility of access data involved in the service. Sub-dimensions of data accessibility are inter-administration data accessibility and data accessibility for disabled persons.

Inter-administration data accessibility corresponds to the possibility for administrations to access data involved in service provision and managed by a different administration by means of an inter-administration back-office. Data accessibility for disabled persons can also be seen as a particular case of service physical accessibility to which we address the reader.

2.4 Accountability

With regard to accountability, legal framework accountability is achieved when the legal framework provides rules which govern the clear responsibility and transparency to be provided when a law is enforced [32]. At the organization/process level we define three sub-dimensions: responsibility, transparency, and control. Responsibility is the obligation of the organization to make decisions and take actions which will enhance the interests of internal and external users.

At the process level, transparency evaluates the availability of information on the state of the administrative macro-process when the user asks for it. A feasible metrics for transparency is the percentage of processes on which there is information available for users. Control is the attitude of the organization to govern all events related to the correct and efficient functioning of the process.

Dimensions referring to accountability, defined at the service level, are transparency and feedback. Transparency concerns the volume of information that the PAs provide to users on the characteristics of the service and on what they could expect or claim using the service. Feedback refers to the effective level to which users’ opinions reach the PAs in charge of the service and influence the provision [2]. Metrics for this dimension are

• 1. the percentage of offices for which users can make a formal complaint in case of error or failure of the service;

• 2. the presence of two-way interaction channels between users and PA (the so-called public relation offices) ;

• 3. the percentage of responses to complaints;

• 4. the average response time to complaints.

3 Dependencies Among Dimensions

Quality is a multifaceted concept; dimensions/metrics introduced above provide different points of view on quality characterization and evaluation. While focusing on different aspects, quality dimensions are not always independent of each other and are related by reciprocal dependencies. Such dependencies may be positive, when an increase in the value of a dimension d1 results in an increase in the value of a second dimension d2, or negative, when the opposite occurs. Furthermore, dependencies may be inter-layers, when they concern different layers, intra-layer otherwise. The most relevant dependencies among dimensions in the quality registry are shown in Fig. 7.5.

Fig. 7.5

Inter- and intra-layer dependencies among dimensions

The user time is positively influenced by the channel accessibility (inter-layer dependency), since the availability of channels reduces the time needed to reach traditional service desks. It is also influenced by the level of simplification, since a higher level of simplification reduces the need for interactions. Finally, it is positively influenced by the inter-administration data accessibility, which impacts on query performance in the back-office.

The service provision time is positively influenced by the level of simplification and by the accuracy of data. Service provision time is also dependent on the level of simplification, the inter-administration data accessibility, and the response time of the ICT infrastructure. Completeness also influences service provision time, since in most administrative activities incomplete data force time-consuming investigations to complete them. Service effectiveness correctness is influenced by data accuracy, since inaccurate data result in user dissatisfaction. Physical accessibility of a service by disabled persons is positively related to the existence of an alternative text.

Knowledge on dependencies can improve the efficiency and effectiveness of the planning methodology, since the effort needed to improve actual metrics into target ones is lower than in the “independence of dimensions” assumption, e.g., considering again the accuracy and completeness at the data layer, the relevance of the data layer for the effective service provision is often disregarded and seen as a purely technological issue, concerning the efficiency of query management. On the contrary, since data are the major resource used in most organizations, an improvement of process/service qualities can often be effectively achieved only acting on data quality.

4 A Methodology for the Assessment Activity

The eG4M methodology for the quality assessment step is shown in Fig. 7.6.

Fig. 7.6

A methodology for the assessment step

In the first step we have to extract the dimensions related to knowledge collected in the state reconstruction step from the quality registry. In this extraction we need to be selective and focus on the most critical issues, e.g., if a service has turned out as very frequently used and at the same time is provided in a context with low technological eReadiness, we may focus on user time, service delivery time, and channel availability. As a second example, for administrative processes with a high number of administration offices involved and a high number of related services, we may focus on legal framework efficiency and organization – process procedural efficiency. At this stage we can also entrust a panel of users and ask them for most relevant service qualities.

Note that in this activity we could need to extend the quality registry with new dimensions, e.g., in order to measure the quality of a list of taxpayers, we could reach the conclusion that besides accuracy and completeness, we have to measure the currency dimension, which we defined in Chap. 2. Currency can be measured with respect to last update metadata, which correspond to the last time the specific data were updated. Another typical measure of currency is the time interval between the acquisition of data in the organizational system and the update of data in the database. We can also add weights to dimensions, associating major importance, say, to user time than to service delivery time. The output of activity 1 is a list of quality dimensions and related service, organization/processes, ICTs the dimensions refer to, plus, in case, weights to highlight the relevance of each dimension.

Activity 2 focuses on metrics. Evaluating metrics has a cost, and this cost should not be overcharged with respect to the value of the information we obtain on quality, e.g., the service delivery time of a service can be measured through a sophisticated realization of a workflow system that observes all the phases of service production and delivery. This means that all offices involved have to be equipped with a software application which traces the activities related to the process and measure the time interval needed to complete the activity. With such a workflow application we obtain precise measures of the average service delivery time, its distribution among offices and among time intervals such as the working hours of the day and the days of the week. However, the cost could be prohibitive; so we may reach the conclusion that it is worthwhile to be less precise, while at the same time obtaining a good approximation. As an example, we could trace service delivery time as a black box, measuring for each service the time stamp of the request and the time stamp of the delivery. In this case, in order to be able to recognize the service in input to the black box and the service in output we have to mark the service with its identifying information. As further approximation, we could focus on a sample of services delivered and measure time intervals by hand.

Activity 3 considers perceived evaluation of users, e.g., if the service delivery time is considered as the most relevant dimension, one could ask users to perceive their level of satisfaction w.r.t service delivery time choosing from several levels such as [very low, low, neutral, high, very high]. At the end of activity 3 we have to refer to a common scale of numerical values of metrics obtained in activity 2 and qualitative values obtained in activity 3; we can do so by associating each one of the values [very low, low, neutral, high, very high] with a value in the [0,…,1] numerical interval used for numerical values.

Activity 4 considers dependencies among dimensions. The dependency graph has to be simplified, considering only dependencies that relate dimensions selected in previous steps. Notice that a deeper analysis of the system considered could result in adding new dependencies, e.g., a project focused on improving the responsibility of public administration employees could investigate the influence of transparency and feedback at the service level on responsibility.

Activity 5 exploits the fact that at this stage we obtain a vivid image of the system around services and causes of low quality; thus, this phase of planning process is the right moment to define a first set of improvement initiatives, e.g., we could have discovered that the completeness of the registry of taxpayers is very low or else that the average currency is equal to, say, 6 months. It is not necessary to proceed to further phases of the methodology to reach the conclusion that a project leading to the digital transmission of tax declarations has to be set up.

5 Running Example

At the end of the chapter, we apply the methodology to the running example. For the sake of clarity, in the example we distinguish between service issues (and other service-related eG4M layers) and legal framework issues.

Quality dimensions for services (activity 1) can be selected in this case entrusting a panel of users and asking them which are the most relevant quality dimensions to be considered. The users may conclude that the most relevant dimensions and corresponding layers are

• 1. at the service layer, temporal accessibility, user time, and service provision time;

• 2. at the organization/process layer, the level of simplification; and

• 3. at the ICT layer, channel accessibility.

We may also conclude that such dimensions are considered relevant for all the services considered. At this point (activity 2), we have to identify metrics for selected quality dimensions and evaluate the cost of measuring the metric. We can draw the following conclusions:

• 1. Assuming that only physical desks are available, temporal accessibility can be measured in terms of hours per week the offices are open, this measure has a low cost.

• 2. User time can be measured according to the metrics proposed above by means of interviews given to a sample of users. To obtain an accurate value, we need a sufficiently large and differentiated sample; in order to restrain costs telephone interviews are needed.

• 3. Service provision time can be calculated with the black-box method described above, leading to reasonable costs.

• 4. At the organization/process layer the level of simplification can be evaluated with interviews to users, the same carried out for the user time. Another possibility is to make an analysis performed on service/process and process/organization matrices described in Chap. 5;

• 5. At the ICT layer, channel accessibility can be evaluated by means of the same interviews performed for user time and level of simplification, leading to further savings due to a unique interview instead of three.

We may now (activity 3) measure the values of quality dimensions, leading to the table shown in Fig. 7.7.

Fig. 7.7

Layers, qualities, values

With regard to the legal framework, we may focus on completeness and accountability. With reference to completeness, we may decide to involve domain experts having juridical background and entrust them in the evaluation of completeness level. We may assume that the juridical team assigns on a scale [very low, low, good, very good] a “low” level to the legal framework, due to the scarcity in the definition of rules for digital signature, which has been considered in Chap. 4 as a leading technology to be adopted. Furthermore, we also assume a “low” level for accountability, since the considered law 59/97 and the decree with the force of law 396/2000 do not define the public administration(s) or the agencies that have responsibility on the control of legal requirements, the validity of information flows and of the data/documents exchanged.

We have now (activity 4) to find dependencies among dimensions. Only a subset of dependencies in Fig. 7.5 has to be considered; by deleting dependencies between quality dimensions not involved among those selected, we obtain the quality registry of Fig. 7.8. Looking at the dependencies, due to the relevance of user time and the presence of channel accessibility among the dimensions considered, we can extend the channels available to request the service, thus influencing two different dimensions.

Fig. 7.8

Dependencies in the running example

Finally (activity 5) we proceed to identify a first set of quality improvement initiatives. Here we focus on initiatives related to the legal framework, see Fig. 7.9. The level of completeness of the legal framework can be improved by enriching it with

• 1. a law which introduces the legal validity of electronic documents with legal signature;

• 2. a law which defines rules and guidelines for the digital signature;

• 3. a decree with the force of law that introduces the obligation for local public administrations to exchange data in electronic format through the network by adopting the digital signature.

Fig. 7.9

Initiatives for improving the legal framework

Higher levels of legal framework accountability can be achieved, e.g., by assigning the responsibility for the enforcement of the laws for data exchange to a central agency which governs the legal validity of electronic documents and the digital signature, also providing the standard requirements for the adoption of the digital signature by public administrations. Figure 7.9 shows how the legal framework can be improved by introducing general rules on digital signature and certification services; the new rules must be enacted with new technical rules defining the guidelines for their enforcement. The new technical rules substitute and complete previous ones. Indeed, the improvement of legal framework enabled by the adoption of digital signature allows the innovation at the technological and organizational levels in initiatives such as the redesign of the office processes of records.

6 Summary

The eG4M quality registry introduced in this chapter and all the mechanisms in terms of the categories of quality, dimensions, metrics, dependencies defined for the different eG4M layers are to be seen as powerful yet adaptable means of assessing the current state of service provision and of all the service-related eG4M layers. Such mechanisms have to be chosen carefully, paying attention to costs and focusing on the most relevant issues. At the same time, it may happen that during the assessment activity new dimensions and metrics not considered before have to be investigated; the openness and the extensibility of the eG4M methodology guarantee flexibility and adaptability to unforeseen situations.