Keywords

1 Introduction

Securing public investment for the production of public services is one of the main tasks of the public sector in all modern countries. In the last 40 years, however, these systems have undergone significant reformative changes. These changes have been mainly focused on public management and the implementation of various elements of business management (new public management) and on individual principles of good government (Osborne and Plastrik 1997; Lynn 1996; Denhardt and Denhardt 2000; Fernandez and Rainey 2006). In the public sector, elements such as decentralization and deconcentration occur, the administration’s uniformity is disrupted and the role of non-profit organizations is made stronger with an emphasis on effectiveness and quality. The tendency to hand over the provision of public production into the hands of the private sector is becoming much more frequent (the tensions between the emphases on decentralization promoted in the market model and the need for coordination in the public sector; Peters and Savoie 1996). Some theories describe these “neo-managerial” tendencies as “public entrepreneurship” (Fernandez and Rainey 2006).

In the last two decades of the twentieth century, scholars and practitioners in Europe and elsewhere have frequently attempted to analyse systems for providing public production and to make suggestions for increasing productivity and finding alternative service-delivery mechanisms based on public-choice assumptions and perspectives (Dunleavy and Hood 1994). In variously named managerial trends, a focus on accountability, effectiveness and high performance has also been recommended in these areas. Additionally, in order to attain these goals, it is necessary to redefine the organizational and production mission, to limit the influence of bureaucratic agencies and to allow for the privatization of certain public functions. This was a huge revolution in the existing understanding of the state’s role in the economy (Brown and Osborne 2012). In the last decade of the twentieth century, Janet and Robert Denhardt published an approach called the new public service. This contains a range of elements from new public management and is considered a normative model, which differentiates it from other models. Precursors of the new public service are the theory of democratic citizenship, models of community and civil society as well as organization humanism and discourse theory.

This approach is focused on and is accountable to citizens; it is aimed at ensuring maximum prosperity by using new approaches for providing public production. Building coalitions of public, non-profit and private agencies is used for these purposes (Pestoff et al. 2013). Individual units are engaged in creating a suitable legal environment, they contribute to the creation of high-quality values in society and they further create standards for individual services according to the needs of the consumers—the citizens (Bao et al. 2013). At the same time, it draws these consumers into the production chain—both actively and passively (Bach and Kolins Givan 2011).

In their article, the creators of the new public service approach themselves state that it is necessary to create a suitable policy in order to apply this approach: one that will be a complex system featuring complex governance networks comprising a plurality of actors—public, private and non-government—each bringing their own special interest, resources, and set of expertise (Denhardt and Denhardt 2015). They emphasize the change in the public sector’s role—from the service delivery role (a rowing role) to policy development (a steering role). However, it is necessarily to convincingly resolve one problem in order to accomplish this. Public administrators have long struggled with how to measure outcomes of public programs, because performance measurement tools have traditionally neglected them (Slater and Aiken 2014).

The aim of this paper is to present a method for determining the perceived value of public services used by the consumer and to use this to measure the effectiveness of these services with the help of the standard methods of cost-benefit analysis or return on public investment. This paper presents evidence that the analysis of the effectiveness is practical for the public sector and is not limited to analysis of the economy.

The structure of this paper is divided as follows: Sect. 1 consists of theoretical background on measuring the provision of public service. Section 2 focuses on measurement methods and approaches. Section 3 is dedicated to describing our methodology, data set and calculations. The last section recapitulates the main results and conclusions.

2 Theoretical Background

All the research on public services will therefore be used as a backdrop for discussion on how to evaluate or measure the utility of public service consumption. Many papers and studies revolve around both providing services cost effectively and creating societal wealth. However, value is more complex in the public sector than in the private sector and can therefore be harder to measure (Bloch and Bugge 2013). The business sector can measure output by existing indicators (sales, added value). However, these do not exist (or they do not have a corresponding value) in the public sector. Therefore, other indicators are used for measuring public sector performance (usually, the macroeconomic perspective). For individual services or organizations in the public sector, the microeconomic perspective is necessary, taking into account costs as well as benefits derived from the needs of the collective good. Evaluation potentially provides the key to improved effectiveness at both organizational and policy levels as defined in terms of the capacity to satisfy the needs and improve the quality of life of citizens (Sanderson 1996).

For evaluating public service outputs or outcomes, input-output economic methods are used. These analyse either one criterion by itself or more criteria, often just the costs as inputs or outputs in the form of benefits. There are primarily three methods for the economic evaluation of non-market goods or effects (e. g., externalities) based on consumer surplus: the travel cost method, the hedonic price method and the contingent valuation method (Marella and Raga 2014). The problem with these methods is again the difficulty of measuring outputs or results (Modell and Wiesel 2008; Hajek and Stejskal 2014, 2015) and the need for direct interaction with the consumer. Other methods replicate the procedures commonly used in the private sector, e. g., return-on-investment (Kaufman and Watstein 2008). The goal is to provide a clearer picture of the benefits and costs of the service producer. These methods can be used both for analysing the effectiveness of individual providers as well as for a region or the overall system a country uses for a selected type of service (McIntosh 2013).

All the methods for further analysis mentioned here use service values as perceived by customers. These methods are based on the principle of contingent valuation (Cummings and Taylor 1999), which originated as early as 1947. The essence of the methods established on this principle is the valuation of the willingness-to-pay of public service customers. The contingent valuation method is a non-market valuation method that is widely used, especially in the areas of environmental cost (Venkatachalam 2004), health care (Klose 1999) and public libraries (Stejskal and Hajek 2015). The CV principle is the basis of the method that is currently used in practice—the contingent valuation method (CVM). The CVM is a survey-based technique generally accepted as a meaningful tool used to estimate the value of various non-market goods (Lee and Chung 2012); it reflects altruistic motivation, a major component of non-use value in contingent valuation. This method gained popularity after the two major non-use values namely, option and existence values were recognized as important components of the total economic value (Venkatachalam 2004). For a methodology of contingent valuation, see (Russell et al. 1995; Wedgwood and Sansom 2003). In literature, critics of this approach have appeared. Variations on the contingent valuation method place value on goods or services that are far removed from any market pricing mechanism. Second, the valuation rests on subjective notions of value, rather than market values, with little regard for level of income or the trade-offs with other goods and services (IBRC 2007 in McIntosh 2013). Also contributing to the criticism is Matthews, who proposes using the CVM method results as “fairly low” estimates of value (Mathews 2011).

Experiments are often used for acquiring perceived values sometimes suitably formulated questionnaires or direct interviews are also used. Contingent valuation studies ask questions that help to reveal the monetary trade off each person would make concerning the value of goods or services (by means of a questionnaire, a hypothetical market is described where the service in question can be traded). The researcher must obtain a so-called “stated preference” from the customer. Such surveys are a practical alternative approach for eliciting the value of public goods, including those with passive use considerations. Results from contingent valuation studies are used for many purposes in benefit–cost studies (Marella and Raga 2014; Carson 2012; Merickova and Stejskal 2014).

The evaluation techniques discussed have also been used for library services to analyse the “return” on public investments. There are three main groups of approaches:

The first studies were generally “efficiency” or output-oriented studies and demonstrate the value of libraries in operating efficiently in managing human and material resources, being financially responsible and, therefore, in providing value as a service, per se. In these studies, we can include costing library activities (how processes and services can be made more effective) and benchmarking (Wilson and Pitman 1999a, b). Second another approach to measuring the value of library services is aimed at demonstrating the success of the library in providing a financial return to the organization or to the region. The problem of these research approaches lies in establishing a dollar figure for the contribution of the library (Marshall 1993; Griffiths and King 1993).

Third, a new methodology came along in the 1990s—taking a broader view of libraries’ value and seeking to establish their value with stakeholders and clients. The balanced scorecard methodology was used, which enabled setting goals to split the hard numbers under consideration to determine which services should be changed and, also, to consider improvements in process (Kaplan and Norton 1996; Walsh and Greenshields 1998). The popularity of these approaches for the evaluation of public library services can be seen in current value studies using return-on-investment and contingent valuation. These methods are generally conducted to determine the economic benefit of public libraries for citizens and the economic benefit of particular services, such as national union catalogues and bibliographic services (Missingham 2005).

Regardless of the method used, it can be argued that analysis results can help to comment on both their effectiveness and their return.

3 Data and Research Methodology

The methodology of the completed research corresponds to the certified procedure for calculating CBA (OECD 1996). For analysing effectiveness, it is necessary to express the values in numbers for the benefits of individual services and the costs for their provision over a definite period.

The process of evaluating benefits for analysis requires empirical data. This paper builds on the results of a project focused on ROI analysis in municipal libraries in the Czech Republic. During 2012, an empirical survey was conducted. The data was obtained from the largest public library in the Czech Republic—from the Municipal Library of Prague (MLP).

The research was qualitative and representative, and it was conducted from July to August 2012 with the help of an on-line questionnaire (CAWI). A total of 11,397 randomly selected readers from the MLP were approached. These readers were over 15 years old, listed an email address in their application and used the library services within the last quarter previous to obtaining the questionnaire. The questionnaire’s return rate was approximately 20% and comprised a basic sample of 2227 respondents after the data set was cleaned.

The questionnaire determining the perceived value of selected services provided by the library was first subjected to pilot testing so that individual questions were understandable for readers and the questions were able to be answered. At the same time, it was drawn up so that neither the way questions were phrased nor their order influenced the readers; this ensured a high degree of predicative ability and that the valuation obtained for the individual services would be realistic. The experiences published by (Venkatachalam 2004) were used here. Questions on the perceived value of library services from the questionnaire are for example:

  1. 1.

    When you consider borrowed material or other information that you received during your last visit, has the library saved you money?

  2. 2.

    How much money has a library like this saved you?

  3. 3.

    If you could not use the library and had to use an alternative, what would be the cost?

  4. 4.

    If you could lower your taxes because of an annual contribution paid to the library, how much would you be willing to contribute to the library?

The questionnaire in its final form was used for obtaining evaluations (EVU) of individual library services from respondents (they expressed the perceived value using the WTP system). The average perceived values of individual services are listed in Table 1. The values observed were given in CZK and, for the purposes of publication, have been translated into EUR at the reported rate.

Table 1 WTP evaluation and quantification of library services (2013)
Full size table

Costs for providing the evaluated portfolio of public services were obtained from the MLP (listed in the necessary subdivisions in Tables 1 and 2). Financial resources in the form of MLP costs make up the public investment, because they come from the public budget at the given regional level. The result of the study presented in this paper is the ROI (or CBA) calculation of the allocation of public resources (sometimes also investments). In a cost-benefit analysis measuring secondary economic impacts, the library’s impact on the rest of the economy can be calculated, e.g., its contribution towards employment, income, consumption expenditures and state or local government revenue in the form of taxes. Economic impact studies are an established methodology in economics (Aabø 2009); the methods were mentioned above.

Table 2 Total costs and perceived utility of individual library services in 2013 (EUR)
Full size table

4 Results and Analysis

Based on the empirical examination described, the perceived value library services (per unit) was determined by the WTP research questions. The quantification of all the services provided by the MLP in the year 2013 is also presented in Table 1. Data for the quantification was obtained from the MLP database.

The details about regional investment (from the local authority’s budget) were obtained from the accounting department. The MLP accounting department is able track the cost (expenses) of every service category. Table 2 also shows calculations that are necessary when using the CBA method and that determine the ROI for all services.

The data in Table 2 shows the overall ROI of services provided by the MLP. The total ROI of the MLP is 1.80467 (a 1 EUR investment will bring 1.80467 EUR of benefit to the customer and will increase societal wealth in Prague).

The value of overall efficiency should be used as a general reference that can serve as a benchmark for comparing different libraries or as information for library management or donors from the public sector.

On the other hand, data on the partial efficiency of the individual library services exhibits significant ambivalence. Some of the services are seen as highly effective, some are only marginally greater than the limit value, and one of them is seen as highly inefficient. It is to be expected that the main service constituting the essence of the libraries’ existence (circulating loans) would exhibit a higher degree of efficiency. This is true to a similar extent in both libraries. Conversely, marginal services, which only complement the range of services provided by the libraries, are rated as inefficient. These include non-circulating loans, which are fairly labor intensive and, therefore, relatively expensive; it also include providing information or high-cost services such as residential stays in the library. When these results are interpreted by public authority management or those providing the investment, it is necessary to draw attention to the fact that inefficient services are often provided in the public interest and the resulting inefficiency is, therefore, just a consequence of entering the public sector.

A surprising finding is the high efficiency observed for digital services, both on-site and off-site. This is probably due to the current trend towards digitization and providing information through the Internet, e-books or various databases.

5 Discussion and Conclusion

The methodology of ROI calculation for public service systems is a very valuable tool for regional providers of public services and their investments. With this assistance, regional providers can better orient themselves when spending money and can make better decisions as to which services they will provide and to what extent they will provide these services. It will no longer be a question of making standard decisions under conditions of high uncertainty; applying this methodology will reduce the uncertainty.

The methodology also monitors the extent to which individual services are used and evaluated economically by the consumers themselves. This data can be used for the management of every library as well as for their owners and regional donors.

For the library analysed, it was found that its operation and provision of a selected range of library services is generally effective, but that the rate of effectiveness is relatively low, approaching a value of one. The library in question has a range of services that were defined but were not effective.

Wider practical use is hampered by a lack of high-quality data on the outputs of public service systems or, more precisely, their value as specified by consumers. This can be aided by the methodology presented here, which provides evidence that consumers are able to perceive the value of the consumed services as well as interpret it. This is the first research of its type for this branch of the public sector that has a high degree of representativeness and has been conducted in Central and Eastern Europe. The results and data cannot be transferred between countries, but it has been shown that such a survey can be conducted in other countries as well - with respect to the conditions for operation in that branch of the public sector in that particular country.