Introduction

The WHO and UNICEF Joint Monitoring Program has reported that the world has already surpassed the 2015 Millennium Development Goal target to improve drinking water (WHO and UNICEF 2014). However, this declaration of success ignores two important components: that improved water supply should be both safe and sustainable (Clasen 2012; WHO 2012). Sustained functionality of improved water schemes in rural Sub-Saharan Africa is a critical challenge. It is estimated that between 35 and 80 % of improved water supply systems are non-functional (Sutton 2004; Haysom 2006; Hoko and Hertle 2006), and up to one-third break within the first few years after installation (SustainableWASH.org 2012).

Some progress has been made in identifying factors associated with sustainability of community water supply schemes, but few use empirical data (Katz and Sara 1998; Behrens-Shah 2011; CARE 2012). Montgomery et al. (2009) proposed three key drivers of sustainability in rural Africa: (1) effective community demand, (2) local financing, and (3) operations and maintenance. Effective community demand includes appropriate technology and community participation in planning; both key features identified by other researchers as well (Harvey and Reed 2003; Mukherjee and van Wijk 2003; Giné Garriga and Perez-Foguet 2008; Koestler et al. 2010; Addai 2012). Long-term sustainability depends upon successful local financing and cost recovery at the community level, including fees to cover maintenance costs or cost-sharing options with NGOs or government agents (Katz and Sara 1998; Hoko and Hertle 2006; Giné Garriga and Perez-Foguet 2008; Carter et al. 2010). Successful operation and maintenance of water systems comprises multiple components, including well-trained local technicians, access to spare parts, clear management responsibilities, monitoring and evaluation systems, and ongoing outside support. Technical capacity and knowledge for operation and maintenance is often lacking and can lead to system failure (Godfrey et al. 2009; Kamruzzaman et al. 2013). Spare parts are frequently unavailable, resulting in poor maintenance and the inability to repair broken systems (Harvey and Reed 2003; Hoko and Hertle 2006; Ademiluyi and Odugbesan 2008; Godfrey et al. 2009). The roles and responsibility for repairs, and the associated costs, are often unclear; particularly when NGOs have handed off responsibility to the community. More research is needed to understand the role of water scheme governance on sustainability.

One key gap is a set of validated and consistent metrics of sustainability and functionality to assess programs and allow for comparability across studies. Studies assessing water scheme sustainability typically use a binary measure of current water availability, having water or not, often referred to as “functionality” (Carter et al. 1996; Harvey and Reed 2004; Haysom 2006; Giné Garriga and Perez-Foguet 2008; Whittington et al. 2009; Koestler et al. 2010; WaterAid 2010; Beyene 2012; Marks et al. 2012). There are limitations to this measurement as water schemes can work improperly, but still have water technically available (e.g., unusually slow water flow rates, unsafe water collected at broken pipe). Few studies have attempted to account for these scenarios by defining functionality in a variety of ways: functioning as “originally intended” (Carter et al. 1999; Ademiluyi and Odugbesan 2008; Behrens-Shah 2011), whether it yielded water “regularly” and was accessed daily (Jiménez and Pérez-Foguet 2011) or whether it was partially functional, where water was available, but at least one component of the water system was in need of repair (Behrens-Shah 2011). Using a snapshot of the system functionality at a single point in time to assess sustainability is practical for organizations monitoring the status of their water schemes. Of value would be a quantitative scale of functionality that could serve to identify leading indicators of sustainable water systems.

The aim of this study was to assess sustained functionality of community managed water points in two regions of Ethiopia and identify key governance components associated with water point sustainability. We propose an approach for developing measures of functionality that includes both a binary determinant of basic functionality, or “operational vs. non-operational,” and a continuous indicator assessing level of functionality. We developed the functionality score using readily available, observable and objective measures. We then compared functionality scores to measures of water committee management, finances and maintenance to assess what factors may be associated with higher functionality scores, and therefore potentially more sustainable water schemes.

Methods

Tool development

We reviewed articles, tools and reports that proposed frameworks of water point sustainability in low-income settings (SustainableWASH.org, Godfrey et al. 2009; Whittington et al. 2009; Lockwood and Gouais 2011; Moriarty et al. 2011; Rojas and Chatterley 2011; WaterAid 2011; CARE 2012; Adank and Kumasi 2013; Lockwood 2013; Pankhurst 2013). Through these resources, we identified contextually specific domains and indicators associated with sustainability and prepared a conceptual model to guide the development of our survey (Fig. 1). In this model, there are key aspects of the contextual environment—natural, physical, political and cultural—that should be considered when determining how best to implement a water supply system. These components are unchanging in the short term and need to be accounted for in program development. The implementation of the system, which includes capacity building, technology choice and community demand, is a one-time event critical in the ability of stakeholders to sustain the program. These components must be responsive to local contextual factors. One key function of the implementation, in areas where community systems are the norm, is to establish a system of governance that is responsive to the local context. The four pillars of governance—management, accountability, maintenance, and finance—have been proposed by a number of organizations and authors as critical for sustainability, though again, the specific aspects of governance necessary and sufficient for sustainability may be largely dependent on the local environment and competency of the implementation. As part of this study, we focused on aspects of implementation and governance that led to sustained system functionality. We developed our survey based on tools from peer-reviewed and gray literature and piloted and refined it for the Ethiopian context prior to data collection.

Fig. 1
figure 1

Conceptual model linking contextual conditions, implementation, system governance and components of water scheme functionality

Full size image

Study site, sampling and data collection

Data were collected in two regions of Ethiopia: Southern Nations, Nationalities, and Peoples’ Region (SNNPR; three districts) and Oromia (eights districts) where international non-governmental organizations WaterAid and Catholic Relief Services (CRS), respectively, had constructed or rehabilitated water schemes with their local partners. A map of the study areas is available in the supplementary material (Online Resource 1). Data were collected between 29th April and 24th June, 2013.

In SNNPR, water schemes were selected from a list of all water sources, supplied by the local government offices. In Oromia, water schemes were selected from a list of schemes that had been constructed or rehabilitated by CRS partners. Schemes were purposively selected from these lists according to age (5 years or more) and functionality status, with the intention of visiting water points with a range of functionality. We conducted direct observation of the water scheme, including functionality measures that consisted of (1) presence of water, (2) flow rate, (time to fill 20 L jerrican), (3) adequate drainage around scheme, (4) construction quality, (5) observed water quality (odor, color), (6) presence of a fence, (7) closing/opening schedule, and (8) whether it was functioning as intended—where water was collected from the main spout (e.g., not a broken pipe). A water scheme implies all types of water sources including deep wells, handpumps, and protected springs with and without distribution systems (Table 1). Each community had unimproved systems in addition to the improved systems we surveyed.

Table 1 Definitions of water scheme types for project areas, Ethiopia, 2013
Full size table

Data on management of the water system were collected through semi-structured interviews with water committee members, and included topics such as history of scheme, capacity of committee to perform repairs, maintenance, finances, outside support, service to users, and structure and activities of the committee.

Data analysis

Data were collected on paper surveys and double entered into Microsoft Excel (Redmond, WA, USA). Data were imported into SAS 9.3 (Cary, NC, USA) to compare entry sheets and correct any errors; data analysis was conducted in SAS and STATA v.11 (College Station, TX, USA). General frequencies were generated on water scheme characteristics in each of the two study areas. We used Mokken scaling techniques to non-parametrically identify unidimensional variables that shared a statistically significant relationship along a hierarchical, latent characteristic (van Schuur 2003). Composite functionality scores were then created by summing the individual characteristic scores. Details on the individual characteristics included in the composite score, categories, and their associated values are shown in Table 2.

Table 2 Description of variables observed for creating the functionality scores for community water schemes, Ethiopia, 2013
Full size table

Values for the summed functionality score ranged from 0 (not functional) to 8 (highly functional). Functionality scores were the primary outcome for our analysis and the intent was to model the complete score in a single regression model. However, the distribution of our functionality scores suggested a zero-inflated Poisson distribution and our sample size constrained our ability to accurately estimate parameters for such a distribution. Instead, modeling was done in two stages. First, scores were dichotomized at zero and categorized into operational (score ≥1) vs. non-operational (score = 0). Scores of zero meant there was no water. However, due to the method of scoring, a score ≥1 did not mean water was present. We assessed the independent relationship between variables from our survey and operational status using a logistic regression. We restricted our second analysis to operational schemes to assess what variables were associated with improved levels of functionality using linear regression.

Ethics

Before conducting interviews, all participants gave oral consent. This study was approved by the Institutional Review Board, Emory University and the Institutional Review Board at Jimma University, Ethiopia. Written permission to conduct research was also granted by representatives of the local government offices.

Results

General

Data from 89 water schemes were used in the analysis, though complete data were not available for every scheme. The age of the water schemes ranged from 5 to 44 years old, with a median of 9 years. Sixty-seven (75 %) of the schemes had water on the day of visit. The numbers and types of schemes were: 45 (51 %) handpumps, 21 (24 %) deep wells, 16 (18 %) protected springs with distribution system and 7 (8 %) protected springs. The flow rate ranged from 30 s to 22 min to fill a 20 L jerrican, with a mean of 1.6 min (median 1.4).

Seventy (80 %) committees reported that the community was consulted about the location and 58 (67 %) were consulted about the type of water scheme (Table 3). Schemes served between 25 and 7000 households, (median 219). In the wet and dry seasons, users on average spent 30 min collecting 50 L per household per day and 90 min, 80 L, respectively. Of the 64 water schemes that have needed repairs, 45 (70 %) were reported to be fixed by technicians from the local government office.

Table 3 Characteristics of community water schemes, Ethiopia, 2013
Full size table

Management and finance

The Ethiopian government and/or NGOs provided funds to establish all of the water schemes assessed. Some communities contributed labor or materials during construction. Seventy-seven (87 %) water schemes had a caretaker. Caretakers performed a number of duties such as collecting the fee, managing the line, and cleaning around the water scheme. Forty-eight (66 %) caretakers received some level of compensation [average 8USD (155 birr)/month]. Caretakers at 26 water schemes received no compensation; however, a number of these were in communities where households took turns as caretaker, rotating weekly or monthly. Fees were collected at 77 (88 %) water points: 37 collected fees per month and 38 collected fees per jerrican. The average monthly fee was 0.16USD (3 birr) and average fee charge per jerrican was 0.02USD (0.3 birr).

Nearly all committees (89 %) had bylaws that guided the use of the scheme, and 95 % of those reported that the community was aware of the bylaws. Ninety-nine percent of committees reported that community members were aware of the roles and responsibilities of the committee members and 78 % of committees reported that community members were knowledgeable about committee finances. Good financial records were kept by 66 % of committees and 65 % are periodically audited by local government officials (Table 3). All committees reported that they require additional training to better perform their duties.

Water scheme operations and functionality

Data on functionality were available for 82 of the 89 water schemes. Of these, ten (8.2 %) had a functionality score of zero and were defined as non-operational. We looked at a number of governance indicators such as committee management, system repairs and accountability and transparency with the community. We did not find evidence of any variables significantly associated with schemes being operational or non-operational (Table 4). None of the variables representing accountability and transparency, such as: committee elections, community knowledge of committee roles and responsibilities, or frequency of committee meetings with the community, were found to be associated with operational water schemes. The mean monthly fee charged at operational water schemes was found to be more than double that of non-operational water schemes 0.12USD vs. 0.05USD (2.3 birr vs. 1 birr), however, this was not statistically significant.

Table 4 Univariate logistic regression analysis of variables associated with operational community water schemes, Ethiopia, 2013
Full size table

There were a total of 72 water schemes in our analysis classified as functional (scores >1; range 1–8). The average functionality scores according to type of water scheme were: protected spring: 4.5, protected spring with distribution: 4.7, handpump: 4.9, deep well: 6.2.

Six indicators were positively associated with higher functionality scores: having a caretaker [β 1.82, 95 % confidence interval (CI) 0.46, 3.17], the caretaker receiving compensation (β 1.30; 95 % CI 0.17, 2.42), higher monthly fees (β 0.48; 95 % CI 0.12, 0.85), keeping good records (β 2.60; 95 % CI 0.72, 4.42), periodic auditing of records (β 2.69; 95 % CI 1.44, 4.22), the committee having the capacity to make minor repairs (β 3.00; 95 % CI 1.03, 4.96) and the committee meeting regularly (at least every 3 months) (β 1.27; 95 % CI 0.20, 2.33).

Although not statistically significant, committees with the capacity for major repairs, and those that increased fees over time had higher average functionality scores. Functionality scores were slightly higher, though not statistically significant for committees with bylaws (β 0.25; 95 % CI −1.47, 1.97), regular elections (β 0.45; 95 % CI −1.29, 2.19), schemes that charged a fee (β 1.10; 95 % CI −0.41, 2.63) and schemes where the committee had performed recent maintenance (β 0.84; 95 % CI −0.82, 2.50). Water schemes had lower functionality scores when members of the community were consulted about placement of the water scheme (β −1.40; 95 % CI −2.60, −0.19). We found no evidence of an association between functionality score and: the price per jerrican, holding regular meetings with the community, amount of compensation given to the caretaker, number of households served by the water scheme, and proportion of women on the water committee (Table 5).

Table 5 Univariate linear regression analysis of functionality score of community water schemes and associated variables, Ethiopia, 2013
Full size table

Discussion

We assessed community managed water schemes older than 5 years in two regions of Ethiopia to investigate factors associated with operational schemes and various levels of functionality for operational schemes. Understanding the determinants of functionality of rural water supply schemes in Sub-Saharan Africa is of great importance. To our knowledge, this is the first study to assess factors associated with a functionality score that is easily replicable. Though our sample size was limited, this is one of the few studies to develop empirical scores of water point functionality and assess components of governance associated with scheme functionality. A simple score of this type would support monitoring and evaluation of water supply, and enable applied research into water system sustainability. Components which were strongly associated with higher levels of functionality included: charging slightly higher fees, maintaining good records, holding regular meetings, having the capacity for performing minor repairs, having a caretaker for the water scheme, and awarding the caretaker with some level of compensation. Periodic financial audits of the committee’s records by a third party were also associated with a higher level of functionality.

In addition to the governance factors mentioned above, our study also considered factors specifically representing transparency and accountability between the committee and the community. Although there was some evidence of a positive association, we found no statistical evidence of an association between level of functionality and having bylaws, community knowledge of bylaws, higher proportion of women on the committee, community knowledge of committee finances, holding regular meetings with the community or holding regular elections of committee members. While these factors were not associated with improved functionality, factors of transparency and accountability may be fulfilled by committees that meet regularly, maintain good records, and have financial audits, as was seen in schemes with higher functionality levels. Committees that collect higher fees (to have money for spare parts) and have the capacity to make minor repairs are able to maintain functioning systems, which makes them accountable to the community. Caretakers—and specifically paid caretakers—may serve as a proxy for overall community support and ownership of the water scheme, a feature often cited to be vital for rural water supply sustainability (Harvey and Reed 2007; Montgomery et al. 2009; Whittington et al. 2009). Although access to spare parts is seen as fundamental to sustainability (Harvey and Reed 2003; Hoko and Hertle 2006; Godfrey et al. 2009), we did not find an association with functionality. One potential explanation is that water committees generally reported access to spare parts as well as a reliance on local government offices that could procure needed parts. As such, availability of spare parts may be necessary only where local governments do not play such a considerable role in management, as they do in some parts of Ethiopia.

Studies assessing the sustainability of water points tend to use presence of water during a site visit as a proxy for functionality. We found a wide range of flow rates, from 0.9 L per min to 20 L per min demonstrating the array of “functionality” that can be masked by a simple binary indicator of water availability. In our analysis, we found no variables associated with basic functionality, but some key variables associated with level of functionality. These findings justify the use of a functionality score that goes beyond water presence to represent levels of service to users (Kayser et al. 2013).

Good record keeping and having the knowledge for making minor repairs are factors beyond the capacity of committees without initial training and support from NGOs or government offices. Financial management, record keeping and basic maintenance are all part of the training committees received when a water scheme is installed. It is possible that when new committee members join they are not trained on these essential skills, eventually leading to poor performance of the committee and lower water scheme functionality. Financial audits are also outside the scope of committees and should be completed periodically by government or NGO partners.

Our data suggest that not consulting the community about the location of the water point improves functionality score. This finding conflicts with the literature, which consistently supports the need for community involvement in all stages of a rural water supply project (Katz and Sara 1998; Sun et al. 2010). One interpretation is that there are certain topics, such as the location of a water point, that do not require the involvement of the community—since professional technicians know more about the geological benefits of one site over another. An additional interpretation is that for different social contexts, community involvement occurs in different ways (Harvey and Reed 2007) and water committees will function in different ways. Of all the water systems needing repairs, 70 % of committees “knew who to contact” when repairs were outside their capacity—the same number who employed a government technician to complete the repairs on their behalf. Effective committee management, capacity for repairs and the involvement and engagement of the community are key aspects supporting improved water scheme functionality; yet the ways in which these sustain functionality will differ according to local context and government policies.

Monitoring the functionality of water schemes is essential to understanding how current and future investments in water infrastructures can better serve community members. A key output of this study is a functionality score that was developed using a reduced set of indicators that can be easily and quickly collected from water points. A score used across the sector would allow for comparability of studies and further understanding of factors related to water point sustainability. A simple, objective measure of water system functionality could also enable NGO or government staff to reach more water points and feel confident they are collecting reliable data on water service to users. While a retrospective study of the effect of various water point components on sustainability is not ideal, we believe our findings will be useful for identifying indicators for future prospective studies.

Limitations

There are a number of limitations to this analysis. Our sample was not random and therefore findings on prevalence of governance factors are not representative and our results are not directly generalizable. The functionality score we created had to utilize variables for which we had sufficient data available, meaning that a dataset without any missing data might have led to a slightly different set of components for the functionality score. The distribution of our functionality score was zero-inflated; however, our total sample size was limited to 89 water points. This hindered our ability to include all water points in a single regression model that accounted for a zero-inflated distribution and necessitated the two-step regression approach presented here. This two-stage modeling may have contributed to the high variance in our estimates, thus increasing the likelihood of type II statistical error.

Because so few schemes were considered to be non-functional, we had high variance in our estimates, meaning that some of the factors presented in Table 4 may be significantly important for functionality, but we did not find statistical evidence of that association. Future studies will include more water points, which may allow for one regression model to incorporate both the binary measure of water availability and level of functionality. We also had missing data for a number of our survey questions, which affected our ability to detect significance (or lack of significance) for those indicators. Though our approach could be replicated elsewhere, our current findings may not be useful outside the Ethiopian context.

Conclusion

There has been considerable focus on how to ensure rural water schemes are sustainable and how to monitor functionality. Studies have typically used the presence or absence of water at one point in time as an indicator for functionality, and a proxy for sustainability. In this study, we found a wide range of conditions for water schemes with and without water. We used simple, easily obtained measures and calculated scores representing various levels of functionality. Higher functionality scores were associated with higher monthly fees, good record keeping, regular committee meetings, financial audits, a paid caretaker and committee capacity for minor repairs. This score can be replicated for monitoring purposes by NGOs or government agents.