1 Introduction: Development of Indicator Systems for Measuring Progress in Societies

The development and use of community indicators gained momentum in response to growing discontent with the use of traditional economic measures of progress such as Gross Domestic Product being used to reflect improvements in society, community wellbeing and quality of life. Globally, interest in these issues has built over many years beginning with social indicator measures in the United States, Canada and the western world during the twentieth century that have been summarized previously by Michalos (2011). Many international agencies have stimulated interest and action including: the United Nations with the Rio Declaration in 1992, the Decade of Education for Sustainable Development from 2005 to 2014, Habitat conferences on sustainable urban development and the Sustainable Development Goals (United Nations 2015); the launch of the OECD’s world forum on “Statistics, Knowledge and Policies” in Palermo 2004 that introduced international government statistics offices and organisations to the usefulness of indicator systems and applied use of statistics (OECD 2005); development of Redefining Progress and the Genuine Progress Indicator (Talberth et al. 2007); release of the Stiglitz–Sen–Fitoussi Report on economic performance and social progress (Stiglitz et al. 2009); and the launch of the OECD’s Better Life Initiative (OECD 2011, 2013). All of these examples incorporating economic, social and environmental measures of wellbeing for the measurement and monitoring of societal progress.

The driver behind the forementioned initiatives is to widen public administration’s understanding of progress or the ‘good life’ beyond traditional economic measurement, to support evidence based policy and planning, and to support the democratic process by increasing the availability of information to all members of the community. These foundations have led to interest and the adoption of indicators in many different settings with the inclusion of metrics to represent better accountability and transparency, particularly in government. Interest in these initiatives accelerated rapidly at the end of the last decade with the release of the Stiglitz–Sen–Fitoussi Report but the report itself was not without critique. Only 1 paper was cited from over 90 volumes of Social Indicators Research in the Commission’s report ignoring decades of learnings from social and community indicators research (Michalos 2011). It’s not only large scale indicator initiatives that create impact, and smaller scale community indicator systems have been operating for many years with much to share and learn from. In their simplest form indicators support the democratization of public data by making it available to all members of the public, but most importantly, indicator systems provide a measurable way of keeping issues of societal importance on the public and political agenda.

1.1 Indicators and the Australian Context

It took a long time for Australia to embrace indicators as measures of progress and to share administrative data for the development of indicators. Over the last 15 years, the Australian Unity Wellbeing Index has been the standout social indicator available in the country since 2001 (Cummins et al. 2003). The Index measures both Personal Wellbeing (subjective wellbeing) and National Wellbeing using surveys across Australia and is also replicated in many other countries through the associated International Wellbeing Group (Cummins et al. 2012) well known to the International Society of Quality of Life Studies.

The next most significant broad indicator initiative was developed by the national statistics agency, the Australian Bureau of Statistics (ABS), with the introduction of Measures of Australia’s Progress (MAP) which included social, economic, environmental and governance measures (Trewin and Hall 2005). MAP was released in 2009 and again in 2013 (Australian Bureau of Statistics 2009, 2013) before $50 million in national funding was withdrawn from the ABS in 2014. Following defunding of MAP at the ABS, the next most promising development in the social indicators movement within Australia is the evolution of the Australian National Development Index (ANDI). Much work has been dedicated to the development of this new Index which seeks to embark on a very large scale community engagement exercise to understand how people living in Australia define progress. ANDI is most similar to the Canadian Index of Wellbeing (Michalos et al. 2010) and has developed a business case and proposed conceptual framework but is currently still searching for a home and funding security.

These examples are the most significant developments in social indicators developed in Australia over the past 15 years and all have a national focus to their work. There has only been 1 long-term large scale community indicators system across this time in Australia—Community Indicators Victoria (CIV)—and is the case study focus of this paper. This reflective case study analysis is deemed particularly important with an increased appetite for indicators in Australia over the last 5 years, leading to a proliferation of new indicators on resilience, wellbeing, sustainability, urbanism and health which has also led to increased confusion about which indicators to use. Furthermore, many of the indicators being considered don’t consider local or community contexts and are focused at large scale national and international comparisons.

Another more recent trend of concern in Australia, is the production of indicators by government agencies where negative results and time series changes are withheld, delayed for release or made unavailable because of fear of political repercussions. These actions are inconsistent with long-standing established indicator systems with shared purpose of raising public awareness, improving evidence based decision making, addressing inequalities, monitoring progress and encouraging civic engagement (Government Accountability Office 2011).

This paper provides a case study of the CIV indicator system with the aim of demonstrating how the use of CIV has evolved since inception and to highlight key learnings from the process. The objective of this paper is to share methods and learnings of how indicator systems can be built and sustained across time to provide maximum impact for the communities that they seek to engage and improve. The case study of CIV is structured with an introduction describing an overview of community indicator systems and best practice principles, a method section describing the process of development and expansion of CIV, results describing the process and learnings from operating an indicators system and finally a summary with conclusions.

2 Overview: Community Indicator Systems

2.1 Community Indicators and Conceptual Frameworks

Community indicators are abbreviated or aggregated statistics (Briggs 1998; Davern et al. 2011) for a specific geographic area and generally relate to social, economic and/or environmental issues. The objectives of community indicators are to: monitor and report on the progress of society; engage and inform the wider community; highlight issues of importance to a community; identify trends and clarify plans; and stimulate discussion and action for improvement (Briggs 1998; Davern et al. 2011). Community indicators are designed to inform public policy and arose from the social indicators movement that strengthened and mobilised during the 1960s and 1970s (Cobb and Rixford 1998). They were first developed to provide more information on societal issues in an expansion beyond the dominant economic indicators of the time.

The inclusion of community in community indicators can refer to a place focus (Davern et al. 2011) or a group of people with common interest (Holden 2009). Indicators are developed from data which leads many to assume that they are objective and value free, but Holden (2009) has correctly identified that indicator frameworks are value laden in their scientific construction. Indicator frameworks form the backbone of measures and indicators included in an indicator system. These frameworks are built on many different perspectives and theories despite commonalities in measures that usually broadly include social, economic and environmental topics that measure progress towards shared goals (Besleme and Mullin 1997).

The majority of community indicator systems can be classified according to a ‘bottom up’ or ‘top down’ approach to development. Bottom up refers to a consultative community driven and involved model, while top down refers to a measurement system driven by government policy and planning officials with a lack of community involvement. Top down examples provided by Dluhy and Swartz (2006) include traditional economic indicators such as GDP and objective indicators of social wellbeing that began in the 1960s. In comparison, bottom up examples are aligned to quality of life and healthy communities models that proliferated in the 1980s and 1990s.

As discussed earlier, there has been a slow rise in interest in using indicators in Australia over the past 5 years but very few systems measure wellbeing according to a broad definition of progress that includes social, economic, environmental, cultural and democratic measures of wellbeing. In Australia, most commonly government departments that include indicators in their practice do so for their own internal purposes and select their own indicators without community consultation and without making results publicly available. CIV is a unique case study in Australia because it was developed in collaboration with community members and representatives, academics, non-government organisations and government stakeholders and is the first and longest standing community indicator system in the country. The system was preceded with a long consultation and community engagement exercise to determine what issues were important, relevant and needed to be measured to track community wellbeing in Victoria over time. These issues are discussed further in the results section of the paper after providing a review of best practice principles for indicators.

2.2 Characteristics of Successful Indicators Projects

The ultimate success of community indicator projects is the ability to connect evidence and knowledge to policy with the aim of developing better public policy (Dluhy and Swartz 2006). Indicator systems provide community level data with the objectives of stimulating change and improving society. A number of factors have been identified as important to the success of indicators projects for this purpose. Key learnings have been identified in previous reviews by Cobb and Rixford (1998) Dluhy and Swartz (2006), Hagerty et al. (2001) and Holden (2009) and best practice principles for indicators are summarized in Table 1.

Table 1 Summary of best practice principles for indicators
Full size table

All indicator systems should include these best practice principles in their development and operation and Table 1 forms a useful reference guide new and existing systems. This case study analysis reviews how these principles have been incorporated into the development and operation of the CIV indicator system to create influence, connect evidence to public policy and strengthen democracy. The analysis aims to understand the strengths and weaknesses of the CIV indicator system in terms of best practice advice summarized from Cobb and Rixford (1998), Dluhy and Swartz (2006), Hagerty et al. (2001) and Holden (2009).

3 Methods

Results in this reflective case study analysis of CIV are structured according to 3 key stages: firstly, the process of how the CIV conceptual framework was developed; secondly, system establishment and the importance of partnership with technical and practice communities; and thirdly, system expansion including application, teaching and knowledge translation activities that build knowledge and competency in the use of CIV, community indicators and evidence based approaches for improving community wellbeing. Materials used to inform this analysis include information available on the CIV website, academic publications, grey literature and CIV staff knowledge.

4 Results

4.1 Conceptual Framework Development

Differing theories underlying a conceptual framework for an indicator system influence both the community indicators developed and how to interpret changes in indicators across time (Besleme and Mullin 1997; Holden 2009; Sirgy 2011). The ideology guiding a community indicator framework should be embraced, acknowledged and clearly stated to gain maximum impact from using an indicator system. CIV was developed based on these principles and is underpinned by an integrated and theoretically informed framework that includes social, economic, environmental, cultural and democratic measures of wellbeing.Footnote 1 It is described as an integrated or systems-based approach to community wellbeing (Sonntag 2010) because changes in one domain (e.g. health indicators in the social domain) are influenced by changes in other domains (e.g. unemployment indicators in the economic domain). The holistic approach to the measurement of wellbeing is consistent with the approach recommended by leading organisations such as the OECD.

Indicator systems should be designed in conjunction with a theoretical standpoint but must also be relevant to the population and the area where wellbeing is being measured. This has led to great variation in indicator frameworks developed for communities and a major criticism of the field (Holden 2009). However, participatory citizen involvement is a crucial factor in determining the success of community indicator projects (Swain 2001) and this inevitably leads to the inclusion of indicators specific to a local area that can be more context specific and purpose driven rather than theoretically grounded.

The development of an indicator framework must try to balance competing tensions that indicators are guided by theory, yet relevant to a local area. In parallel, it is also important that indicators can be easily collated, replicated and compared with other areas to ensure data consistency and longevity of the indicator system towards their multifaceted purposes of engaging public interest, measuring trends, influencing policy and evaluating community wellbeing. The development and consensus of a conceptual framework that guides the indicators collected in a community indicator system is a large and complex task. The process will determine what factors are common to understandings of community wellbeing and societal progress and indicators that reflect this definition. Development of CIV’s framework was completed as an initial collaborative project across an 18 month period from 2005 to 2006 (Wiseman et al. 2006). Funded by the Victorian Health Promotion Foundation (VicHealth), the project capitalized on the expertise and ideas of many representatives from state government departments, local government representatives from 35 municipalities, the Australian Bureau of Statistics, the Municipal Association of Victoria, the Victorian Local Government Association, the Victorian Council of Social Services, academics, written submissions and attendees at public consultation forums held across Victoria. This approach balanced top-down and bottom-up contributions to the framework’s development.

The development of the CIV framework was a coordinated effort that brought together a diverse group of key stakeholders with a common agenda—to improve the community wellbeing of the residents of Victoria. The framework is the foundation of the system and aimed to bring about large scale social change with broad cross-sectional coordination of key stakeholders working in related areas. Future success of the system was built on this partnership based approach with CIV staff coordinating the efforts and providing centralised infrastructure needed to achieve an agreed upon agenda. Many other municipalities have since adopted the CIV framework of community wellbeing endorsing the relevance and agreed values across many different areas of Australia with the City of Sydney a large scale and influential example (Partridge et al. 2011).

4.2 Establishing Community Indicators Victoria

Following stakeholder and community discussions, the indicator system was proposed in 2005, with data and indicators developed throughout 2006 with CIV launched online in mid-2007. CIV has received long-term funding support from the Victorian Health Promotion Foundation (VicHealth) which is the world’s first health promotion foundation and an independent statutory authority with multi-partisan state government support. The organisation’s objectives are to promote good health and prevent chronic disease (Victorian Health Promotion Foundation 2012), consistent with the aims and theoretical foundations of CIV and key stakeholders.

Funding for CIV was initially attached to a larger grant that developed a new community wellbeing research centre at the University of Melbourne: the McCaughey Centre for the Promotion of Mental Health and Community Wellbeing. Locating CIV within the University of Melbourne’s School of Population Health (now Melbourne School of Population and Global Health) provided another theoretically aligned partner and a neutral home, particularly given that academia encourages independent thought without public or private biases. This is aligned with another principle of successful indicator programs i.e. the role of a neutral convener for indicator systems (Dluhy and Swartz 2006). However, it is also has implications for future business modelling and financial sustainability. Initially, VicHealth provided long term funding and the University of Melbourne provided infrastructure to enable investment in a long term process of social change. As a well-respected and large health promotion foundation, VicHealth had extensive reach into the target audience of CIV and is importantly governed by a multipartisan board structure. Similarly as a well-respected university, the University of Melbourne provided an independent academic environment, reputation, relationships with key stakeholders in population health, research opportunities for the development of indicators, and additional funding via competitive and philanthropic funding sources. Thus, the partnerships with VicHealth and the University of Melbourne not only provided opportunities to enhance reach and create impact, but also a neutral host, and additional funding opportunities to enhance the ongoing financial sustainability of the community indicators system.

The importance of funders and business models in the development of indicator systems are rarely discussed in the wider literature, yet they are very important to the long term sustainability and impact of the system. VicHealth have funded the core work of CIV maintaining the community indicator system in Victoria while additional funding has been sourced through philanthropic foundations, research grants and consultancy based research. These additional income sources have also ensured an additional level of independence for CIV and enabled the development of new measures in response to data availability, technological advancements as well as current and evolving community concerns that emerge over time. Being an independent statutory authority, VicHealth has more independence than other government departments but is also influenced by current political forces. Independence between CIV and VicHealth has been maintained through annual work plan development approval cycles ensuring that both funder and fundee are clear about deliverables related to the continuing operation of a free and publicly available resource. Obviously all funding contingent relationships raise questions about the true independence of any indicator system but that debate requires extensive discussion well beyond the scope of the current paper.

4.3 Data Access and Indicator Design

Data access is essential for any indicator development and the Australian Bureau of Statistics (ABS) remains a key partner and supporter of CIV which is good for parties. The establishment phase of CIV occurred between 2006 and 2008 and involved the sourcing and creation of indicators included in the agreed community wellbeing framework. The ABS seconded an officer to source data and assist with the construction and refinement of indicators and the partnership was important in the development of reputation and credibility and enhanced public dissemination of ABS data. This was a turning point in the development of CIV. The support of a national statistics agency has also been identified as important to the success of sister projects such as the Community AccountsFootnote 2 project in Canada which involves partnership with the Newfoundland and Labrador Statistics Agency. For CIV, the secondment and partnership with the ABS enabled the following: access to customized Census data on a 5 year cycle; better access and awareness of current and new data sources (particularly within state government); development of relationships with other seconded ABS officers across federal and state government departments; and the formation of the Australian Community Indicators Network (ACIN). This national network of community indicator practitioners was developed by CIV and the ABS to raise awareness of community indicators, share information, and develop coordinated action and best practice links with community, government and research sectors. Such partnership based activities between CIV and the ABS have influenced the outreach of CIV, provided the national statistics agency with additional sources of dissemination, and complements the national indicator suite Measures of Australia’s Progress discussed in the introduction (Australian Bureau of Statistics 2007, 2009, 2011, 2013).

The ABS and many other government departments are central data custodians for CIV. The majority of CIV indicators are derived from administrative data sources and only approximately a quarter are derived from new statewide surveys. Using survey methodology to create indicators is an expensive and resource intensive activity—both in terms of fieldwork administration, survey development and survey analysis. This is because indicator data needs to be sampled appropriately and aggregated to a consistent administrative unit (i.e. in the case of CIV, at the municipal level). Surveys have been necessary to fill data gaps, provide subjective measures and to create new indicators consistent with CIV’s framework but the use of existing administrative data is always the most cost effective method—this is also an important influence on long-term business modelling.

Spatial presentation and use of Geographic Information System (GIS) data are an alternate source of administrative data and are an additional and important part of the CIV indicator system. GIS is defined as a computer based system for data storage, manipulation, modelling and analysis of spatially or geographically referenced data (Jones 2007; Worrall and Bond 1997). It is an extremely useful application for the creation, visualisation and analysis of indicator data. New indicators are created from geographically referenced or geocoded administrative data and mapped results produce visual presentation and analysis of results (Davern 2014). GIS methodology has facilitated the construction of new indicators of the built and natural environment (Jackson 2003; Jackson et al. 2013) and has untapped potential for indicators that directly assess policy and planning guidelines. For example, CIV has previously assessed indicators of Public Open Space access and previous policy recommendations included in the 2013 Victorian Planning Provisions. These planning provisions stated that local parks should be located within 400 m safe walking distance of at least 95 % of all dwellings (Department of Sustainability and Environment 2013) yet over one-third of dwellings in Melbourne failed to align to this standard (Mavoa et al. 2014). The role of CIV is not to test and assess all policies but this example illustrates the alignment of CIV indicators to policy levers and their direct relevance to factors that influence community wellbeing.

4.4 Partnership and Theoretical Influences on the CIV Framework

Partnership, trust and cooperation with data custodians are all very important to accessing administrative data and the development of new indicators. CIV is built on respect, relationship management and strong collaboration with government administrative agencies. These partnerships take time to build and have been developed through regular meetings with key stakeholders over many years to recognize and appreciate common agendas, share information and assist with the gathering of new data sources. However, each of these stakeholders and their agencies are subject to changing political views and contexts. Privacy is a common concern for data custodians, and the data collected as part of everyday government process is often not intended for analysis and the creation of indicators. Accessing data from government officials and creating indicators within a culture of feared accountability on topics deemed controversial and politically loaded is not an easy task. These influencing factors are similar across countries, with similar experiences reported by Holden (2009) in Seattle and van Assche et al. (2010) in Flanders.

Partnerships that support data access have been challenging for CIV in the past, particularly access to government administrative data before the recent interest in indicators and the “big data” movement took off in Australia over the last few years. Difficulties in accessing data were a common problem across the Victorian Public Service in the past, and so much so, that in 2009 the Victorian Government held an inquiry into Improving Access to Victorian Public Sector Information and Data (Economic Development and Infrastructure Committee 2009). The inquiry resulted in 46 key recommendations and the establishment of an online directory of public service information held by government—a clear step in the right direction. However, a mismatch still exists between what data are listed on the register and what data are actually available.

Data access difficulties are regularly faced by CIV and other community indicator systems across the world and much of their success relies on developing trust and partnerships with data custodians and stakeholders. It is often easier for CIV to gain access to public infrastructure data in other jurisdictions (e.g. North America, Canada or Western Australia) than to find comparable data in Victoria. Hence, the development of partnerships, relationship management, stakeholder engagement, careful data management and a solid and trustworthy reputation are important to gaining access to data and expanding a community indicator system. One of the key learnings from CIV’s practice with data custodians is clear communication about how data are going to used and presented so there are no surprises to a custodian when they’re made publicly available. This could even include a warning to partners or data custodians about potential positive or negative media interest that could be associated with an indicator’s release. In Victoria at least, it would have been impossible to create the CIV indicators system without the cooperation of data custodians and this outweighs any counter argument about losing independence in working closely with these agencies. However, it is acknowledged that data custodian partnerships could also be interpreted as a weakness of strict separation and independence.

Partnership is essential for the creation of healthy communities because it requires community involvement and policy change across all levels of government, including public and private institutions (Marmot and Wilkinson 2003). This is because health is created and determined by multiple underlying factors including upstream (education, employment, income, living and working conditions) and downstream (physiological and biological) determinants. In a review of effective evidence-based strategies for reducing health inequalities, Marmot et al. (2010) and Marmot and Bell (2013) suggest that actions aimed at reducing health inequalities are a matter of social justice. This is because actions that focus on improving the health of a community aim to reduce the social gradient of health where lower social position is associated with poorer health outcomes.

The direct connection between community indicators, health equity and social justice is made clearer when the CIV framework is examined according to Sirgy’s (2011) understanding of theoretical frameworks guiding indicator projects. In a review of international, national and community Quality of Life (QOL) indicator projects, Sirgy (2011) identified 6 key theoretical perspectives guiding projects: socio-economic development (that economic development provides the foundation for social development); personal utility (include evaluations and expressions of satisfaction with various domains of life, conditions and community services and measures of quality of life); social justice (equality in basic rights and duties and inequality to benefit the least advantaged); human development (meeting a hierarchy of lower and higher order community needs similar to Maslow’s (1954) hierarchy of needs); sustainability (environmental wellbeing or the interrelationship between social, economic and environmental wellbeing); and functioning (freedom to choose abilities and situations considered important such as the United Nations Development Programme).

The consistent influence throughout the development of the CIV framework is social justice with its direct implication on health equity that aligned and focused all key stakeholders during the development phase of the indicator system. A just society is defined by Rawls (1971, 1975) as a society with equality in the assignment of rights and duties, where inequalities are justified if benefiting the least advantaged members of society. According to Rawls (1971, 1975) a community that meets these conditions would have a markedly reduced social gradient with more evenly distributed enhanced health outcomes.

Although the CIV framework is heavily influenced by social justice (e.g. social justice indicators across all domains), it is probably best defined as theoretically eclectic because it also draws on the theoretical perspectives of socio-economic development (e.g. social and economic domains), personal utility (e.g. social, built/natural environment, democracy), human development (e.g. select indicators across all domains) and sustainability and functioning (e.g. social, economic and environment domains). This eclectic theoretical influence is reflects the diversity of stakeholders involved in the framework development and the importance of measurement across multiple areas to address the multiple causes of community wellbeing. However, consistent with best practice principles of indicator development, involvement of diverse stakeholders also expanded the influence and impact of the indicator system after initial development.

A partnership based approach and consensus in theoretical orientation was needed for the development of the CIV framework for a number of reasons: (1) theory creates a common agenda, approach and understanding for all key stakeholders involved in the indicators system; (2) theory provides credibility and strengthens the meaning of the indicator system (Sirgy 2011); (3) theory is a direct influence on the measures chosen for inclusion as indicators; (4) theory is necessary to understand the meaning of changes in indicators across time and links this knowledge into policy directed action (Dluhy and Swartz 2006); (5) theory guides the development of new indicators over time; (6) theory enables each stakeholder to identify how their actions fit into an overarching plan and address multiple causes for complex problems; and (7) a theoretical orientation sharpens the focus for future indicator applications. Hence, although the CIV framework is eclectic it is anchored by social justice theory and guided by the principles of social determinants of health which was understood and common to all CIV partners and stakeholders. Agreement with, and acknowledgement of, these theories was very important for consensus building in the development of the CIV framework and the mutual goal that the system would be established as an evidence base for healthy, equitable and engaged communities in Victoria.

4.5 Imbedding Indicators Within a Policy Context

The foundation of an indicators project is the development of data and indicators. However, to create a successful community indicators system these indicators must be embedded within a policy context. Influence and traction can only be achieved by linking indicators to policy during indicator development and within decision making and action processes (Cobb and Rixford 1998; Holden 2009; Innes and Booher 2000). CIV has followed this approach when amendments to the Municipal Public Health and Wellbeing Act 2008 (Vic) required all 79 Victorian local government municipalities to prepare and submit Municipal Public Health and Wellbeing Plans to the Victorian Department of Health at the start of each cycle 4 year planning cycle. This Act stipulates that planning is guided by the principle of evidence-based decision making where priorities, resources and interventions are based on relevant and reliable evidence. From 2008, CIV provided a central data repository of indicators directly relevant to local governments requiring evidence for planning. Before the introduction of CIV, local governments were required to collate evidence from multiple sources using their own, often limited resources. The availability of comprehensive local level data and capacity building support provided state and local government, community organisations and residents with a resource of evidence to support action, consultation and monitoring. The immediate outcomes were the promotion of evidence in the development of policy and planning, strengthening accountability and widening accessibility to these data to local governments and all members of the public.

A very good example of the sphere of influence of indicators on municipal public health and wellbeing planning is the partnership developed between the City of Ballarat and CIV (Davern et al. 2011). The City of Ballarat is the largest regional city in Victoria and the municipality was an early adopter of CIV. Council was persuaded by the social determinants alignment of CIV and this was an important influence on its decision to use the indicators for planning and adopt the CIV framework of community wellbeing. The City of Ballarat began using CIV indicators in planning when support from CIV developed into a collaborative relationship with their municipal health planner who became a local champion within the organization. This led to the development of a range of integrated planning strategies for community safety, early years, youth, positive ageing and cultural diversity and confirmation that the use of community indicators enhanced the municipality’s public health planning. More recently, CIV has worked with a number of community health organisations seeking indicators as tools for community engagement, measurement, monitoring and both population and program evaluation. These long term integrated health promotion initiatives involve residents, municipal health planners, community organisations and local service providers in different locations across Melbourne and Victoria. CIV has assisted these projects by providing group facilitation to identify key project priorities and goals, measurement expertise, as well as indicator data and frameworks for project evaluation. In these examples, indicators are used to engage the wider community, identify strengths and weaknesses, effective program design and interventions, and provide tools for evaluation.

Establishing indicator systems within policy contexts and decision making processes is an ongoing task and the experience of CIV is consistent with the complexity theory argued by Innes and Booher (2000). Cities are made up of a diverse range of players including business owners, residents, elected officials and commuters who all influence the form and shape of a growing and adapting city. The applicability of CIV to health planning has extended the indicator system’s distribution to these players and agents across 79 local governments and a range of community health agencies and service providers. CIV key audience groups include local, state and federal government agencies of Victoria and Australia, as well as community organisations, philanthropic organisations and advocacy groups.

4.6 Applications, Building Capacity and Expanding CIV

The first 5 years of CIV focused on the establishment of the indicator system: sourcing data; building trust and relationships with data custodians; creating indicators; designing and improving methodologies; building the website and data dissemination services; supporting local governments and organisations in their use of indicators; and embedding the use of indicators within policy context. This is consistent with Innes and Booher’s (2000) estimate that it takes 5–10 years for indicators to be used by communities and to influence governance in areas that seeks to improve community wellbeing. Obtaining the data and creating the indicators is only the first step. The intentions of CIV and indicator systems are to give communities and organizations easy access to data that enable engagement monitoring and evaluation of community wellbeing. The next step is mobilizing the data into action where indicators are used for advocacy and evidence in changing and improving community wellbeing.

4.7 Building Capacity and New Partnerships Through Short Course Training

CIV continues to build the strength, value and influence of its indicator system by providing training and resources to both new users and experienced users of CIV to strengthen competencies and skills in using CIV and community indicators for planning. Training courses on the uses and applications of community indicators are key capacity building tool for indicator systems. CIV discovered early in its development as a system that to have the greatest impact and influence, people need to know how to maximize the use of the publicly available indicator system. In the earlier days of CIV this comprised of a “roadshow tour” around the state, alerting practitioners to the new resource and ongoing engagement with these audiences and key stakeholders across time. Later tutorials were added to the CIV website demonstrating how to customize data for specific geographic areas and later formal short course training began on applied topics including: an Introduction to Indicators; How to Use Indicators in Municipal Public Health and Wellbeing Planning; How to Measure Health and Wellbeing using GIS and Spatial Indicators; a Masterclass of Community Indicators for Planning (showcasing practical examples of how a range of organisations have used indicators in their work); and Using CIV Indicators to Promote Physically Active Communities. CIV also provides customized training courses according to demand. These capacity building opportunities also provide CIV with another mode of knowledge translation or knowledge brokering described by Choi et al. (2005) as the ability to synthesise, integrate and prioritise knowledge for policy makers and help them to identify the “lighthouse indicators” that are most important within planning context.

Short course training falls within CIV’s community engagement and policy focused capacity building role. Content input and delivery is shared by colleagues with skills in research and evaluation, urban and social planning, policy development, survey and indicator development and GIS. This ensures that the multidisciplinary skills and knowledge associated with the community indicator system is shared, and encourages a wider target audience for training courses. CIV training selects key indicators for demonstration in training according to their current relevance with policy and planning issues (i.e. hot topics in planning) complimented with recent research findings. Indicators provide a tip of the iceberg representation of issues that enable conversations to be started and further investigation into explanations and influencing factors. However, indicators don’t provide suggestions on interventions that could influence or improve results in the future and this is where research evidence is useful for community planning practitioners managing program development and evaluation. These interrelated relationships are summarized as the ‘indicator iceberg’ in Fig. 1 that is CIV has found to be very useful when working with service providers and practitioners that aren’t always comfortable with the world of data or research.

Fig. 1
figure 1

The “Indicator Iceberg” that connects CIV indicators to policy focused research

Full size image

The personal contact and delivery of knowledge via CIV is important because the absence of personal contact between researchers and policy makers is a common barrier to evidence based practice (Innvaer et al. 2002). Effective knowledge translation within CIV requires partnership within and across different audiences: it is the ability to bridge the expertise and research environment of academia with the practical and political contexts of planning and policy making practitioners. This requires the ability to speak both “languages” and select and translate relevant information for either party. The importance of context is vital during this process so that practitioners are provided with relevant, realistic and feasible research findings that can be incorporated into their work and policy focused research is informed by the current needs of practitioners and policy makers.

4.8 The Introduction of Indicator Application Methods for Practitioners

For the last 3 years, CIV has adopted and provided training on the Results Based Accountability (RBA) model developed by Friedman (2005). The RBA model starts with an end result and works backwards to develop a means or method and always involves external partnership to achieve maximum impact with fewer resources. It is a results-based decision making and planning method that is particularly useful for policy makers when much of their time is spent on answering the wrong question or figuring out what the question is (Bardwell 1991). One of the key components of RBA is the separation of population (i.e. people) accountability from performance (i.e. program) accountability. Friedman argues that this separation allays organizational fears about unrealistic outcome based responsibility because a single program or strategy led by an organisation (performance accountability) cannot be held responsible for the wellbeing of an entire population (population accountability). The distinction is necessary for the identification of appropriate data at the appropriate level, explains the history of an issue and can be used to measure future success or failure against a baseline. Population indicators and performance measures are used to clarify a decision making process to improve wellbeing, and actions are always developed in partnership.

There are obvious synergies between the RBA model with CIV and community indicator systems: the method relies on time series population based indicator data; it clarifies population wellbeing from processes/programs; is based on partnership with community and key organisations; and provides a method for using indicators in an evidence-based planning context. The strength of RBA is that it provides practitioners with a framework to apply indicators and measure progress in working towards outcomes. The model is based on measurable improvements (i.e. population based indicators) and is useful for practitioners who seek to turn talking (i.e. internal and external consultations and conversations) into measurable actions and strategies. Critique of the RBA model is directed towards a rigid or instrumental application, a restriction to quantitative measures (only measurable aspects are important) and a lack of innovation (Houlbrook 2011). However, it has been endorsed by many local and state government departments across Australia and New Zealand including the NSW Department of Community Services (Izmur 2004). CIV has found RBA to be a very useful tool for practitioners wanting to use evidence towards developing and improving programs and engage in evidence based planning. The RBA method can also be complemented with additional qualitative or participatory based methods or existing program logic models when developing evaluation plans for partners.

4.9 Indicators as Knowledge Translation Tools in Community Practice

Knowledge translation (KT) is a core activity of CIV. The Canadian Institutes for Health Research describe KT as a dynamic and iterative process of synthesis, dissemination, exchange and application of knowledge to improve health (Graham et al. 2007). It requires active participation from researchers (those who create the knowledge and add to the body of knowledge) and policy makers or research-users who seek to obtain popular support (Choi et al. 2005). However, a range of terms are used to describe KT and in a review of over 2500 articles McKibbon et al. (2010) found 100 individual terms were used to describe KT activities. Terms like implementation, adoption, dissemination, intervention, change, evaluation, innovation and diffusion were good discriminators of KT articles from non-KT articles, while other terms such as participatory research, communities of practice and action research provided low discrimination between KT and non-KT articles.

These findings are very relevant to the impact of community indicator systems and associated research reported in the academic literature. To create impact, applications and findings related to community indicator systems need to be shared with other indicator projects and those who seek to develop their own projects. However, one of the difficulties facing the field is the variability in KT methods. This is because knowledge and experience of many community indicator projects remains with practitioners who are busy working with citizens or, resources are published on web based resources (i.e. grey literature) that aren’t accessible via traditional academic literature searches. One of the most useful resources available to community indicator practitioners is the Community Quality of Life Best Cases series led by Sirgy et al. (2004, 2006, 2007, 2009, 2011) which describes best practice examples of community indicator projects. The series facilitates easily accessed shared knowledge and applications of how indicators can be used to create further impact.

The complexity of the CIV audience is that some practitioners using indicators are very skilled in how to use indicators in applied practice while others are new to the idea and sometimes even afraid of how to use quantitative information so many different approaches are suggested depending on the needs or competency of the audience. CIV advocates for a sociological approach to knowledge translation in the application of indicators which is described by Kitto (2012) as new knowledge created by mixing scientific knowledge with local and contextualized knowledge practices to produce appropriate change. As described earlier, community indicators are described well with an iceberg analogy within the sea of a bigger and broader community context (Fig. 1). An indicator is the tip of the iceberg communicating and transferring knowledge, reporting and monitoring on what is happening in a community (the what). However, when indicators are combined with partnership-based policy focused research, results can be linked to causal factors (the why) and effective strategies to address the issue (the how).

4.10 Community Indicators and Participatory Action Research

The direction of CIV was strengthened from 2011 with the development of the Place, Health and Liveability (PHL) research program at the University of Melbourne. PHL’s research examines the relationship between built and sociocultural environments and health to inform policy and practices that create healthy and liveable communities and is co-located with CIV. The goals of CIV and PHL research are aligned and guided by partnerships with practitioners in communities, government, and non-government organisations to develop well planned, healthy, happy and sustainable communities. Research is used to create new indicators that are consistent with the CIV framework. However, PHL extends CIV’s work by validating the indicators against health outcomes to inform locally relevant programs for change. This synthesis of research knowledge and practice is consistent with the recommendation from Cobb and Rixford (1998) who have argued that if indicators are to move from description to analysis; they need to understand the causes behind indicators and outcomes to create future action (i.e. the area below the iceberg in Fig. 1). Partnership and location within the PHL research program has also ensured that CIV indicators are developed according to best practice research design.

Much of the difficulty experienced by health and community planning practitioners in the use of evidence-based policy has been the professionalization of knowledge to ensure that public health practice and knowledge is consolidated as the preserve of professionals while ignoring local knowledge and experience (Green 2005; Warr et al. 2013). In an examination of health promotion and community engagement styles in disadvantaged communities, Warr et al. (2013) found that two distinct approaches used by workers when working with the communities on health promotion activities: cooperative and procedural styles.

Cooperative styles are based on commitment to partnerships—working with local people, harnessing their local knowledge, a willingness to listen to community concerns and working in partnership with residents and service providers. These styles embrace the abilities of workers to empathise with residents, support community engagement at all stages from priority setting to decision making and require workers to be innovative. It involves customizing actions to a local community’s context, working with the knowledge and experience of residents and organisations, and not directing expert knowledge “onto” the community. Cooperative approaches are aware of limitations or “off the shelf” evidence based health promotion strategies because they fail to account for socio-spatial difference in environments and people.

Procedural styles differ, most notably due to a lack of collaboration and partnership with residents or harnessing community knowledge. These approaches fail to acknowledge the local context, are influenced by judgmental assumptions about residents and tend to shift blame back to individuals (i.e. “lifestyle choices” as determinants of health) rather than trying to understand the limitations and influence of environment. Procedural styles are disempowering and make residents feel that workers are out of touch with their reality creating distrust, a lack of engagement and frustration by all.

CIV advocates and supports cooperative engagement styles and seeks to share or translate the knowledge of indicator based results into explanations that incorporate both academic and local knowledge (i.e. community based participatory approaches) to help planners devise the most effective strategies for their local context. Participatory action research seeks citizen or resident involvement in the research process to gain more impact in the community: it is about co-developing opportunities with people rather than for people (Kitchen and Muhajarine 2008; McIntyre 2008). Traditional academic research has tended to rely on the latter approach of projecting expert driven ‘knowledge transfer’ onto the community but CIV has been built on the collaborative and participatory approach of working with the local expertise of community members and practitioners. Two-way learning or knowledge exchange better describes the partnership approach employed by CIV in working with community and government partners because indicator development needs to be responsive to the needs of practitioners while being guided by sound research principles. CIV seeks to incorporate research knowledge and indicators as evidence in the development and evaluation of theoretically informed practice.

5 Summary and Conclusions

CIV indicators are designed to simply communicate technical data to monitor progress or wellbeing across areas and time. To be effective, community indicator systems need to translate and disseminate data to practitioners while contextualizing it within a policy framework and local community setting. A very important ingredient to the success of CIV has been partnership with key stakeholders and the broader community during development, establishment and ongoing application of the indicator system. CIV is now approaching a decade of operation with many lessons learnt about the importance of partnership and collaboration for the creation of a successful community indicator system.

This case study example of CIV describes multiple factors and identified best practice principles that have been embedded in the development of a sustainable community indicator system in Australia that has resulted in long-term impact and influence. CIV was developed with a clear purpose of improving community wellbeing by creating equitable, engaged, healthy and well planned communities through evidence based policy and planning, reporting and democracy. CIV indicators are directly linked to local and state government public policy and guided by social justice theory. These indicators have been developed according to a consultative framework of community wellbeing with good research design. Citizen engagement, democratic principles and participatory action research are central to CIV with balanced involvement from government, business and community and a neutral convener within an academic institution. Over the last 8 years CIV has also gained the support of many influential champions including small community based organisations, social planners within numerous Victorian municipalities particularly VicHealth, the City of Ballarat (Davern et al. 2011), the Victorian Department of Health and Human Services, and the Municipal Association of Victoria and the Lord Mayors Charitable Foundation who also partnered to produce the first Vital Signs report for Melbourne.

Indicators are just numbers unless they are proposed, developed, understood and applied in partnership with residents, multiple tiers of government, planners, service providers, advocacy groups, academics, community organisations and philanthropics in a broad range of settings. These partnerships are needed to create, measure and apply a valued framework of community wellbeing that can be used to monitor change, enhance participatory democracy and support evidence-based policy and planning. This experience of the CIV is provided for both emerging and established indicator projects to share lessons learnt and provide insights into sources of impact, future applications of community indicator systems around the world.