Abstract
To explore the determinants of global e-government performance, this paper examines the aggregate data of 163 different countries by conducting multivariate statistical analysis. The results of multivariate regression analysis indicate that the performance of digital government is likely to be determined by economic wealth, education, urbanization, civil liberties, government effectiveness, and the interaction between Internet usage and economic wealth, while the extent of internet penetration alone does not determine e-government performance. More importantly, this study indicates that government effectiveness is much more important than any other factors in determining global e-government performance. The countries with high e-government performance are likely to be the wealthy, developed, and Western countries or the rapidly developing Asian countries.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
The revolution of information technology and communication has been changing human behavior, management of corporations, and governance of states. Communication is, in fact, much easier via the Internet and mobile instruments. The high and low levels of government have increasingly utilized the Internet as a means of the provision and delivery of public goods and services. Some federal, state and local services are delivered more efficiently via online: for example, reporting taxes, renewing vehicle licenses, and applying for jobs, passports, or loans via government websites cut the costs dramatically. Government savings are estimated to yield nearly $117 billion per year from use of the web (Fountain 2001, 6). In addition, when contracting out public programs like social service programs with not-for-profit or for-profit agencies, some government agencies manage them more efficiently via the Internet or by using information technology.
What determines the performance of digital government? A variety of factors could determine e-government performance.Footnote 1 More precisely, what factors are more responsible for e-government performance? Each nation has a different level of information technology development, a different level of the application of information technology to the public administration, and a different level of electronic government. Less developed countries are significantly different from more developed countries with regards to information technology development, economic conditions, financial resources, administrative behavior, and political culture. To explore the determinants of e-government performance, this paper examines the aggregate data of 163 different countries by conducting multivariate statistical analysis and shows why some nations have made greater progress than others. This study is important in that it sheds light on how countries can improve the performance of digital government regardless of their status as developed or developing countries and Western or non-Western countries. After arguing the determinants of e-government performance theoretically, this paper conducts a multivariate regression analysis of the aggregate data of 163 different countries and discusses findings and policy implications.
Theoretical perspective
The literature demonstrates crucial determinants on the performance of digital government, including economic, social, political, technological, organizational attributes. First, the extent of internet usage, that is, the number of people online, is an important determinant of e-government performance on the grounds that the more people have Internet access the more people could access government websites and demand an efficient and responsible digital government. Some citizens are more likely to be involved in e-commerce, e-government, or e-politics as the number of people online increases. As the number of visitors to government websites increases, government is expected to improve the management of its websites as well as the delivery of public services. West (2004a) pointed out that “countries with more Internet users had more privacy protections on their government websites” (p. 20). Table 1 indicates that a majority of citizens in the Western European countries, the North American countries, and the emerging Asian countries, including Singapore, Taiwan, and South Korea, are online, while most citizens in some developing African and Asian countries are offline.
The global village has been severely divided by the extent of Internet access which provides a new way of information circulation and recreation of knowledge. Examining the aggregate data of 179 different countries, Norris (2001) found that Internet usage is likely to be determined by the level of research and development spending as well as economic wealth. A 2003 public opinion survey of 27,000 people in 27 countries undertaken by Dalziel revealed that Internet usage is remarkably different with regards to age and gender: in 2003, government online usage is 38% for those aged under 25 years, 41% for those aged 25–34, 36% for those aged 35–44, 29% for those aged 45–54, 19% for those aged 55–64, and 8% for those aged 65 and over; government online usage is 34% for men and 25% for women (Dalziel 2004, 7–8).Footnote 2
Second, the level of education is a crucial predictor of the performance of digital government. To access and utilize the Internet, individuals are required to have technical skills to use computers properly, reading comprehension, and the ability to search for, use, interpret, and evaluate information (Mossberger et al. 2004, 6). In other words, basic reading and writing skills as well as specialized education on the use of computer and the Internet are necessary for an efficient and effective use of the Internet. Twenty percent of Americans reported they need help using a mouse or keyboard and 37% said they need help navigating the Internet (Mossberger et al. 2003, 45). Table 2 indicates that whereas most people in the Western European countries and the North American countries are literate, while a majority of people in some developing countries are illiterate. For direct subsidies for computer-based services, especially for education and health, free access to electronic services through libraries, schools, and hospitals would be an alternative (Mechling 2002, 151).
Third, economic wealth is directly related to the ability to purchase computers and access the Internet. An investigation of 2,166 websites in 198 nations in 2003 undertaken by West showed that rich countries tend to have more electronic services on their government websites and “the most significant predictor of the number of online services is gross domestic product per capita” (West 2004a, 20). A study of the 18 OECD (Organization for Economic Cooperation and Development) members undertaken by Hargittai revealed that the best predictor of Internet penetration was economic wealth measured by gross domestic product per capita (Hargittai 1999). A study of Californian Internet access indicates that education or income influences upon Internet Access are on the rise in California, while racial influences are becoming increasingly insignificant in determining Internet access in California (Groper 2004, 296). While economic wealth is more important than anything else in Internet access, the successful management of e-government, however, would depend on more technical, organizational, and governmental attributes than financial resources.
Fourth, government effectiveness would be a crucial determinant of the performance of digital government. Government effectiveness refers to the quality of public service provision, the quality of the bureaucracy, the competence of civil servants, the independence of the civil service from political pressures, and the credibility of the government’s commitment to polices (Kaufman et al. 2003, 3). The fundamental conditions for the excellent performance of digital government rely basically on the competence of public agents on public service delivery, the sound structure of the bureaucracy, professionalism, and the separation of the civil service systems from politics. Unless these fundamental conditions are sufficiently established, even digital government with advanced government IT and information infrastructure would not be well performed.
Fifth, civil liberties are necessary for digital government in which citizens have the right to express their opinions, demand their needs, and participate in decision making processes. Lack of civil liberties would allow public officials to regard government websites only a means of service delivery and a billboard for one-way communication with citizens. When more voices and inputs from the public are allowed, or when citizens are allowed to participate in decision making processes via online, electronic government would be more responsive and accountable despite some confusion and inefficiency and despite influences of clienteles or interest groups. Digital government has the power to increase citizen input to government, improve official decision making, and increase the transparency of government transactions (Kamarck 2004, 35). Sixth, urbanization could be related to e-government performance. Urban Areas tend to provide better facilities and services for the Internet and mobile instruments than do rural areas. People in urban areas where the financial and services industries are concentrated are more likely to be interested in the Internet and information technology than their counterparts in rural areas where the manufacturing and agricultural industries mainly exist. Table 3 indicates that urbanization is more likely to be advanced in small rich countries than in large poor countries.
Finally, the level of information technology development could be another important predictor of the performance of electronic government. To deliver effectively and responsively public services via online, government agencies are expected to regularly update their websites by using advanced software and information technology or by establishing advanced information infrastructure. Information technology is, in fact, developing so fast that government IT is used to be outdated. Additionally, government IT systems should be cost-effective and competitive when developing government IT and outsourcing or contracting out government IT.
Methodology
Dependent variable
The dependent variable used in this study is e-government performance. West (2004a, b) showed the overall scores of global e-government performance by conducting a detailed content analysis of 2,166 government websites, including executive, legislative, and judicial offices, and major government agencies (i.e., most national levels and some state and local levels), in 198 countries, international public opinion surveys and an email responsiveness test in 2003. The overall e-government performance score in each country is the sum of scores from an evaluation of government websites based on two dozen different criteria, including the availability of publications, databases, foreign language access, disability access, privacy policies, security policies, audio clips, video clips, email contact information, automatic email updates, credit card payments, digital signatures, search capability, links to a government services portal, areas to post comments, PDA (personal digital assistant) accessibility, and presence of online services (West 2004a, b, 17). Though an overall e-government performance score is not either extensive or diverse as a measure, it is useful as the dependent variable.Footnote 3 Further, a global e-government readiness index published by the United Nations is considered in this study, but it is not used as the dependent variable because a global e-government readiness index does not exactly represent e-government performance. The United Nations ranked the 191 member states of the UN according to a quantitative composite index of e-readiness based on website assessment, telecommunication infrastructure and human resource endowment (UNPAF 2006).
Independent variables
The independent variables used in this study are Internet usage, education, economic wealth, urbanization, government effectiveness, and civil liberties that represent organizational, administrative, financial, and social characteristics and determine what separates stronger digital government performance nations from those that are weaker. Internet usage is measured as the number of Internet users per 1,000 people in each country; education is measured as adult literacy rate; economic wealth is gross domestic product (GDP) per capita in US dollars; urbanization is a percentage of the overall population that lives in urban areas; a Kaufmann measure of government effectiveness is used for government effectiveness,Footnote 4 and a Freedom House measure of liberalism is used for civil liberties.Footnote 5
The year 2002 data on those independent variables can be obtained in the UNDP (United Nations Development Programme) and the World Bank. Either the level of information technology development or the level of government IT development could be a significant independent variable, but those variables are excluded in this study due to insufficient data.Footnote 6 The unit analysis in this study is the country, and 163 different countries are examined cross-nationally by multivariate statistical analysis.Footnote 7
Statistical considerations for the model specification
There is the problem of multicollinearity in the independent variables. For example, Internet usage and economic wealth appear to be correlated (the Pearson’s correlation between Internet usage and economic wealth is .777). The more people are affluent, the higher possibility people use the Internet. In addition, those independent variables could interact with each other. For instance, Internet usage could interact with economic wealth. To minimize the problem of multicollinearity and to take account of the interaction effect between those variables, this study creates the interaction variable for Internet usage and economic wealth.
Analysis and findings
Table 4 displays the results of multivariate regression analysis on global e-government performance. The results indicate that the performance of digital government appears to be determined by economic wealth, education, urbanization, civil liberties, government effectiveness, and the interaction between Internet usage and economic wealth, while the extent of internet penetration alone does not determine e-government performance. The statistical analysis in this study appears to be reliable: the Durbin–Watson test (d = 2.05) shows no autocorrelation and the tolerance statistics shows no multicollinearity in the model.
Non-government factors
Tables 5 and 6 show that the high e-government performance countries are the North American countries (e.g., the United States and Canada), the Western European countries (e.g., Britain, Denmark, and Sweden), and the rapidly developing Asian countries (e.g., South Korea, Singapore, and Taiwan). The high e-government performance countries are wealthy. Past studies on electronic government or Internet penetration demonstrated the strong association between economic wealth and the performance of electronic government or Internet penetration. Affluent people tend to have greater opportunities to access the Internet than do poor people. While wealthy countries tend to have greater financial resources than do poor countries, poor nations have considerable disadvantages in the adoption and application of information technology to public administration due to insufficient financial and human resources. Whereas advanced information technology is most likely to be available in developed countries, many developing countries are limited in the application of advanced information technology to government agencies due to lack of information infrastructure and uncompetitive government IT. As Norris (2001), West (2004a, b), and Groper (2004) indicate, economic wealth is a necessary condition of e-government. Additionally, as Table 4 indicates, the interaction between economic wealth and internet usage plays a significant role in determining e-government performance, while the extent of internet penetration alone does not determine the performance of digital government.
To use the Internet efficiently and effectively, people should have reading comprehension and writing skills as well as technical skills on computers and the Internet, including how to use a mouse and keyboard and how to search and navigate the Internet. The uneducated and old generation are more likely to be afraid of accessing the Internet or using computers than the educated and young generation. Educated persons are not only more familiar with computers and the Internet, but they also could be more interested in what and how government agencies manage via online than are uneducated persons. Some countries provide not only mandatory education for citizens but also computer education in elementary and secondary schools for the young. People in the large number of countries, however, still do not have the benefits of mandatory education or computer education.
Personal access to the 24-h interconnected world still remains restricted to a minority of the world’s population, and the vast majority of people in the planet still get their signals not from cyberspace but from the national capital (Yergin and Stanislaw 2002, 396). People in the large number of countries do not have access to the Internet, and nearly 10% of the world’s population used the Internet in 2002 (UNDP 2004). The global village, in fact, has been severely divided by the extent of Internet access. Only four percentage of the population in developing countries used the Internet as opposed to almost half of the population in high- income OECD countries (UNDP 2004). Surveying 27 countries in 2003, Dalziel found that the high Internet usage countries were New Zealand (75%), Netherlands (72%), Demark (71%), Norway (69%) and the USA (68%), and the high government online usage countries were dominated by the Scandinavian countries, such as Denmark (63%), Norway (62%), and Finland (58%), followed by Singapore (53%), Netherlands (52%), Canada (51%), Australia (47%), New Zealand (45%), the USA (45%), and Hong Kong (43%) (Dalziel 2004, 4–6).
Further, 71% of the 1,782 government websites in 198 different countries in 2006 had no online services, while only 29% offered fully executable online services, including ordering publications online, filing complaints, applying for jobs or passports, and renewing vehicle licenses (West 2006, 4). A survey by the United Nations in 2005 found that of its 191 member states only 47% had one-stop portals and less than 25% offered online transaction services (UNPAF 2006, 93–96). More importantly, most government websites in the planet are still in either stage one (billboards) or stage two (partial service delivery). Government officials regard the government websites a billboard for one-way communication with the public and “they are not taking advantage of two-way features that provide citizens with a chance to voice their opinions or personalize websites to their particular interests” (West 2004a, 10 and 15). Only 33% of government websites in 2006 offered areas to post comments, the use of message boards, and chat rooms (West 2006, 7). Table 7 shows that the regions of the world offering the high percentage of online services are North America, followed by the Pacific Ocean Islands, Asia, and Western Europe, while Russia/Central Asia, Africa, Eastern Europe, and South America offer the low percentage of online services. The major use made of government online tend to be information seeking, followed by downloading government forms, consulting (for example, expressing a point of view), providing personal and household information, and transactions (for example, paying for government services or products) (Dalziel 2004, 6–7).
More people are moving to urban areas. Almost half of the world’s population lived in urban areas in 2002 (UNDP 2004). Three quarters of the population in the OECD (Organization for Economic Cooperation and development) members lived in urban areas, while approximately forty percentage of the population in developing countries lived in urban areas (UNDP 2004). Some small countries like Hong Kong and Singapore have a 100% urbanization, as Table 3 displays. Urbanization could lead urban residents to demand their distinct urban services through municipal governments, but it remains questionable whether urban governments allocate appropriately resources in innovating government IT or electronic government to satisfy urban needs and interests.
The literature on e-government indicates no linkage between civil liberties or political systems and the performance of electronic government. The type of party system, the level of party competition, the type of electoral system, and voter turnout rates may not be directly related to e-government performance. Table 4, however, indicates a significant linkage between the extent of civil liberties and the performance of digital government. Greater civil liberties could contribute to making e-government a means of two-way communication with citizens. Unless civil liberties are widely permitted, e-government would not perform beyond a billboard as one-way communication with the public, and citizens might be afraid of voicing their opinions and monitoring government programs and services. For an effective and accountable e-government, citizens are required to have civil liberties, including the right to express their opinions and their needs through e-government. As Kamarck (2004) pointed out, digital government has the power to increase citizen input to government and improve decision making.
Government factors
Table 4 displays that the slope coefficient of government effectiveness (i.e., 2.174) is much larger than other independent variables and the most significant predictor of e-government is government effectiveness. The performance of digital government depends on not only the effective management of government websites but also the quality of public bureaucracies for delivering better public services to the public. For an effective digital government, public officials are required to be competent, public bureaucracies are expected to be competitive, and advanced government IT and information infrastructure are expected to be widely and equitably available. If public bureaucracies lack the good qualities of making and implementing public programs and public policies, government websites would not be accountable enough to satisfy citizens’ needs. Compared to the Max Weber’s ideal-type of bureaucracy characterized as strict hierarchy, control, and chain of command, e-government not only remarkably reduces the distance between service providers and service demanders through cyberspace, but it also helps government organizations to be more flexible and organic.
Government effectiveness could significantly contribute to the development of web-based government-to-citizen (G2C) services, government-to-business (G2B) digital procurement processes and government-to-government (G2G) connectivity which Fountain (2001) mentioned. According to a 2000 public opinion survey conducted by Hart-Teeter on behalf of the Council for Excellence in Government, among institutional customers of the US federal government (businesses and nonprofit organizations) 56% said that the ability to communicate via the Internet has made it easier to do business with the federal government and among government officials 80% believed that their agency has done an excellent or a good job of using the Internet to improve efficiency and quality of its service (Council for Excellence in Government 2000).
Electronic government is expected to lead public management to be more transparent, more accountable, more responsive, and more efficient through structural and behavioral adjustments or adaptations, while reducing red tape, rigidity, secrecy, and corruption. According to a survey of US state employment agencies, most agencies felt that the introduction of a variety of technical online services is improving their ability to serve job seekers (Townsend 2001, 218). It, however, remains controversial how much public bureaucracies have become efficient, effective, productive, or accountable due to information technology and e-government. Utilizing information technology, citizens in Western nations are more likely to have access to government information, while citizens in non-Western nations are not equally accessible to government information (Welch and Wong 1998, 46). Public bureaucracies in many developing countries (for example, China, North Korea, and Cuba) have attempted to control or manipulate the distribution and circulation of government information to maintain their regime at the expense of the public interest.
Conclusion
The top performance countries of digital government are the wealthy, developed, and Western countries, including the United States, Canada, Britain, Denmark, and Sweden and the rapidly developing Asian countries, including South Korea, Singapore, and Taiwan. First, economic conditions and levels of education are directly related to e-government performance. It is undeniable that lack of resources constrains Internet access, education on computers and the Internet, and use of advanced information technology. However, financial conditions and levels of education are one thing, and how well government provides public services via online is another. Some countries (even countries with limited national wealth) pay more special attention to e-government and invest more money in developing government information technology as well as innovating the management of digital government than others. The rapidly developing Asian countries, for example, have invested enormous financial and human resources in developing information technology, government IT, and information infrastructure.
Second, government effectiveness appears to play a crucial role in determining e-government government. The performance of digital government may not be separable from the ability of the government, including making and implementing public programs and public policies efficiently, effectively, and responsively. Sound bureaucracy, competence of public officials, easy use of the government portals, user-friendly contents on the government portals, fast response of government agencies, and technical expertise of bureaucrats are necessary for the high performance of digital government.
Additionally, political and administrative support and leadership are necessary for an effective digital government. Top-level decision makers in government are required to have a clear vision, strategic information system plans, and strong leadership for the development of an effective e-government. US E-government Act of 2001 intends to enhance the management and promotion of electronic government services and processes by establishing a Federal Chief Information Officer (FCIO) within the Office of Management and Budget (OMB) and by establishing a broad framework of measures that require using Internet-based information technology to enhance citizen access to government information and services (US Congress 2002).
Finally, civil liberties appear to be related to digital government. Lack of civil liberties, including free expression of opinions about government activities or citizens’ demands via government websites, would lead e-government to a billboard for one-way communication with the public. For two-way communication with the public and to be interactive, civil liberties are not only widely permitted available, but government websites also should have enough areas in which citizens voice their opinions, make suggestions for better government services, provide comments and feedbacks about government actions, or monitor government programs and functions. Further, public officials are expected to appropriately respond to citizens’ suggestions, comments, feedbacks, and monitoring via online and equitably represent them in decision-making processes. However, it remains questionable whether civil liberties really contribute to a fair and responsible digital government because civil liberties or two-way communication with citizens have a room for interest groups with powerful resources to influence much more than ordinary citizens.
Notes
E-government, digital government, and the application of information technology in government are used interchangeably in this paper. E-government refers to the use of information technology and the Internet to provide public services.
Some studies found a significant linkage between gender, race, ethnicity, age, or geography and access to the Internet or information technology. For example, the wealthy, educated, young, male, and white are more likely to use the Internet at home and have a home computer than the poor, uneducated, old, female, African American, and Hispanic (for details, see Mossberger et al. 2003; US Department of Commerce 2002).
It seems that there is no consensus on the definition of e-government performance, what constitutes e-government performance, and how to analyze it. Some researchers focus more narrowly on e-government performance, including usability of the government portals, content on the government portals, and response time of government agencies.
A Kaufmann index of government effectiveness is based on responses on the quality of public service provision, the quality of the bureaucracy, the competence of civil servants, the independence of the civil service from political pressures, and the credibility of the government’s commitment to policies; “the main focus of this index is on “inputs” required for the government to be able to produce and implement good policies and deliver public goods” (Kaufman et al. 2003, 3). Additionally, we need to be cautious in interpreting the data regarding a Kaufmann index of government effectiveness in that “while the addition of data has improved the precision of our governance indicators relative to previous years, the margins of error associated with estimates of governance remain large relative to the units in which governance is measured” (Kaufman et al. 2003, 11).
As an organizational or administrative determinant, some studies used corruption as the independent variable. When corruption is included in the independent variables, the multivariate statistical analysis shows that corruption is not significant. Government effectiveness would be a more appropriate indicator than corruption using political or public power for private gain.
Some studies measured information technology development or government IT development as the number of researchers, including scientists and engineers, working in research and development (R&D) or the percentage of expenditures in R&D to gross domestic product (GDP). A large number of developing countries, however, have not reported data on research and development, and thus the size of cases is reduced to almost half if those variables are included in the independent variables. More precisely, the number of IT researchers or the extent of expenditures in the area of information technology would be more valid than all the researchers in R&D or all the expenditures in R&D.
Of 198 countries the West study published, data of 163 countries are available when all independent variables in this paper are considered. Unfortunately, data of many developing countries are not available. Further, a longitudinal study with a cross-national analysis would be ideal to make causal inferences, but cross-national longitudinal data of global e-government are currently rare.
References
Council for Excellence in Government. 2000. E-government: the next american revolution. Retrieved September 26, 2004, from http://www.excelgov.org.
Dalziel, D. 2004. Government online: a multi-country study of e-government usage. Retrieved August 16, 2004, from http://www.tns-global.com.
Fountain, J. E. 2001. Building the virtual state: information technology and institutional change. Washington, DC: Brookings Institution Press.
Groper, R. 2004. Digital government and the digital divide. In Pavlichev, A., & Garson, G. D. (Eds.). Digital government: principles and best practices. London: IDEA Group Publishing.
Hargittai, E. 1999. Weaving the western web: explaining differences in internet connectivity among OECD countries. Telecommunication Policy, 23: 701–718.
Kamarck, E. C. 2004. Government innovation around the world. John F. Kennedy School of Government. Harvard University (Faculty Research Working Papers Series: RWP04–101).
Kaufman, D., Kraay, A., & Mastruzzi, M. 2003. Governance matters III: governance indicators for 1996–2002. Retrieved August 5, 2004, from http://www.worldbank.org.
Mechling, J. 2002. Information age governance: just the start of something big? In Kamarch, E. C., & Nye Jr., J. S. (Eds.). Government. com: democracy in the information age. Washington, DC: Brookings Institution Press.
Mossberger, K., Tolbert, C. J., & Stansbury, M. 2003. Virtual inequality: beyond the digital divide. Washington, DC: Georgetown University Press.
Mossberger, K., Tolbert, C. J., & Gilbert, M. 2004. Race, place, and information technology. Paper presented at the annual meeting of the American Political Science Association, September 2–5, 2004, Chicago, IL.
Norris, P. 2001. Digital divide?: civil engagement, information poverty, and the internet worldwide. Cambridge, MA: Cambridge University Press.
Townsend, A. M. 2001. Supercharging the employment agency: an investigation of the use of information and communication technology to improve the service of the state employment agencies. In Abramson, M. A., & Means, G. E. (Eds.). E-government 2001. New York: Rowman and Littlefield.
United Nations Development Programme. 2004. Human development report 2004. New York: Oxford University Press.
United Nations Public Administration and Finance. 2006. Global e-government readiness report 2005: from e-government to e-inclusion. United Nations: New York.
US Congress. 2002. S. 803, e-government act of 2001. Washington DC: Government Printing Office.
US Department of Commerce. National Telecommunication and Information Administration (NTIA). 2002. A nation online: how americans are expanding their use of the internet. Washington, DC: Government Printing Office.
Welch, E., & Wong, W. 1998. Public administration in a global context: bridging the gaps of theory and practice between western and non-western nations. Public Administration Review, 58(1): 40–49.
West, D. 2004a. Global perspectives on e-government. Paper presented at the annual meeting of the American political science association, September 2–5, 2004, Chicago, IL.
West, D. 2004b. Global e-government, 2004. Retrieved September 13, 2004, from http://www.insidepolitcs.org.
West, D. 2006. Global e-government, 2006. Retrieved November 30, 2006, from http://www.insidepolitcs.org.
Yergin, D., & Stanislaw, J. 2002. The commanding heights: the battle for the world economy. New York: Touchstone.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kim, CK. A Cross-national Analysis of Global E-government. Public Organiz Rev 7, 317–329 (2007). https://doi.org/10.1007/s11115-007-0040-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11115-007-0040-5