1 Introduction

According to the United Nations Intergovernmental Panel on Climate Change (IPCC) Fourth Assessment Report, global warming will lead to more water scarcity, and it is estimated that water stress will be the main concern for between 1.0 and 2.0 billion people by 2050s (Kundzewicz et al. 2007). While climate change impacts impair sustainability, decrease opportunities and decline rural societies (Mnisi and Dlamini 2012), many researchers and policy-makers think in terms of improving resilience of natural and social systems in an effort to mitigate disasters such as water scarcity (Gibbs 2009).

Resilience has been defined as the capacity of a system to absorb disturbance and reorganize while undergoing change so as to retain essentially the same function, structure, identity, and feedbacks (Adger et al. 2011). Management has an important role to destroy or build resilience, depending on how the system organizes itself in response to management actions (Folke et al. 2002).

Although there are theoretical advancements about resilience as a framework to sustain relationship between human and environment (Holling 1973; Gunderson and Holling 2002; Olsson et al. 2004a), it is a concept that as yet has not been well operationalized (Cumming et al. 2005), and its utility to practical management is still basically underdeveloped (Olsson et al. 2004b; Walker and Meyers 2004). Even in theory, as some authors (e.g. Berkes and Jolly 2001; Cumming et al. 2005) stated, the location of thresholds between desirable and undesirable states and therefore resilience is difficult to measure as the result of the complex and dynamic behavior of system overtime and the multidimensional, abstract and complex nature of the concept of resilience that makes it difficult to operationalize. Despite these difficulties, having standard scales to measure resilience is necessary to: 1) identify actions that change system’s resilience; and 2) identify policies that enhance or reduce specific qualities (Cumming et al. 2005). In fact, without developing standard scales, it is often difficult to validate the calculations resulted from using non-standard variables (Allen and Davis 2010).

Literature review indicates some attempts for selecting indicators and developing scales to assess resilience in some social systems and in confrontation to various disasters (e.g. Connor-Davidson Resilience Scale by Connor and Davidson (2003) and the Child and Youth Resilience Measure by Ungar and Liebenberg (2011)). But in spite of the availability of thousands reports on different aspects of farm water scarcity in the journals and books, and during the time that water has become an increasingly important determinant of agricultural sustainability, especially in arid and semi-arid areas of the world (Forouzani and Karami 2010), there are only few studies that have focused on measuring resilience under scarcity of this valuable resource. In addition, the majority of studies that have attempted to measure resilience under water scarcity does not have a clear and standard scale to assess resilience. Most of these studies simply rely on some variables extracting from literatures or qualitative interviews as proxies which in some cases these variables are more determinants rather than an resilience indicator. Some examples of these studies are Langridge et al. (2006) and Nuorteva et al. (2010). Langridge et al. (2006) in a study on the relationship between access and resilience to the threat of water scarcity defined resilience as community capacity to cope with and adapt to water scarcity, but they did not make it clear how they have measured resilience in their study. Also, Nuorteva et al. (2010) attempted to evaluate farmers’ resilience towards variation of water resources based on key informant interviews in a field research. The main issues discussed in the semi-structured interviews to evaluate resilience included the farmers’ recollections of past environmental shocks and the strategies which they have used for adaptation. Also, changes in the living standards during the past decades, farmers’ prediction of these standards 10 years to the future and their desire to stay in the village in contrast to migration to the cities. Finally, the farmers’ visions and their hopes for the future and possible strategies to improve capacities to adapt potential future challenges discussed.

Water scarcity affects various social-ecological systems, however farm households—for whom agricultural production is the primary source of their direct and indirect employment and income—are most affected by scarcity of this critical natural resource (Ringler et al. 2010). Therefore, measuring resilience of farm households is an essential step in sustainability planning. According to provided definitions of resilience by various authors such as Holling (1973) and Maleksaeidi and Karami (2013), we define farm households’ resilience under water scarcity as the capacity of a farm household to withstand water scarcity, moderating potential damages from it, maintaining its family and agricultural structure and still have the same identity as a farm household or possibly improve and advance in self-statue by learning, creativity, adapting and coping.

Considering the shortage of an organized and standard metric system to measure resilience in the condition of water scarcity, and with attention to this reality that farm households’ capacity to adapt the adverse effects of decreasing natural resources determines both the severity of impacts and the costs of adaptation to changes (Malone 2009), the aim of this study is developing a scale to measure resilience of farm households as one of the main elements in the social-ecological systems.

2 Material and methods

2.1 Study area

Parishan wetland in Iran was selected as the study area and farm households living around this wetland were used as the cases for the study. Parishan wetland is the largest fresh water ecosystem in Iran that is located in south of Famour mountains and in the Arjan and Parishan Protected Area (United Nations Development Programme 2012). The average depth of wetland in the years with low and high rainfall varied between 2 and 2.5 m. This wetland’s altitude is 820 m above the sea level. Its features are the warmth and dryness of the air in the summer and warmth in the winter (Mosleh et al. 2013).

Parishan wetland was declared as a Biosphere Reserve by UNESCO in 1976 and was a wintering ground for globally significant waterfowl as well as home to invaluable endemic species of fauna. This wetland has tremendous national and local importance, not only for biodiversity reasons but also for sustaining livelihoods of people living around it. There are about 21 villages around this ecosystem; most of them directly depended on the wetland’s water for agriculture, fishing, reed harvesting, and tourism (United Nations Development Programme 2012).

While over the last decade, many internationally-renowned wetlands and lakes have completely dried up (Keshavarz et al. 2013), Parishan wetland also has dried up at the result of long-term drought and overexploitation of groundwater. Therefore, the ability of the farm households to produce agricultural products has decreased. Before drying up, the wetland was being fed by rain falls, springs and canals. Araban and Pol-Abguineh were two main springs. In addition to providing water for wetland, they were sources of irrigation water for agriculture. Farmers were using the water of these two springs based on Mosha (traditional collective ownership) (Saghafi et al. 2010). Figure 1 indicates Parishan wetland before and after drying up.

Fig. 1
figure 1

Parishan wetland before (a) and after drying up (b)

Full size image

2.2 Initial framework and indicators selection

While an indicator is characterized as an observable variable assumed to estimate a dimension of a concept such as resilience, a scale is a synthesis of several effect indicators (Bunge 1975). Therefore, selecting indicators may be considered as the first step for developing a scale such as FHRS.

Complex concepts such as resilience have several dimensions. Thus, before developing a scale to measure resilience we require to distinguish its dimensions that should be considered and measured. Resilience can be understood through the reaction of system during disaster and after it. In a social system these reactions can appear in five dimensions cognitive, emotional/affective, behavioral, spiritual and physical (Kumpfer 1999). Therefore, these five dimensions were considered as an initial framework for developing FHRS. There are various indicators that can be used to measure each of these five dimensions. In this study, 16 indicators which based on the available literature were able to measure the reaction of social-ecological systems to water scarcity were selected. These indicators which covered the five dimensions of resilience are described in Table 1.

Table 1 Selected indicators to develop farm households’ resilience scale
Full size table

2.3 Questionnaire development and data collection

After selecting 16 indicators (Table 1), the next step was developing a questionnaire to measure these indicators. These indicators were measured through 130 questions which were selected through an extensive review of literature, several in-depth interviews and two focus groups with the farm households living around Parishan wetland. Questions were designed as a self-rating scale that required the respondents to indicate how much they agree (disagree) with each question on a five-point Likert-type scale (0 = strongly disagree; 4 = strongly agree). The questionnaire was used in a survey to collect data.

The statistical population of the study were all farm households living in the 21 villages around Parishan wetland (N = 2574). A random sample of 450 farm households were selected for the study. Household heads, who were the main decision-makers in the household, were considered as the representative of the households and were invited to provide information. The full sample was divided randomly into two samples for analysis. A slightly larger portion of the sample (n = 268) that complied with the minimum sample size requirement (Cattell 1978; Meyers et al. 2006) was used for principal factor analysis and the other portion (n = 182) was used for confirmatory analysis.

2.4 Analysis methods

This methodological overview summarizes the logic and procedures used in analyzing data of this multi-stage study. The questions related to measurement of each indicator were subjected to reliability test. Reliability is defined as the extent to which a set of questions produce the same results on repeated trials. In fact, it is the stability of scores over time or across raters (Borg and Gall 1989). The results of reliability test of the scale were used for evaluation of questions. Recommendation by Nunnally and Bernstein (1994) was used to set the standard of 0.3 item-total correlation coefficient for inclusion of questions. Item-total correlation coefficient indicates the degree of correlation between the score on an individual question and the sum of the scores on the remaining questions that measure an indicator. A high item-total correlation shows that the individual question and the remaining have a strong relationship and hence, are tapping the same common indicator (Shaw et al. 2000). Therefore, questions with item-total correlation coefficient of 0.3 or less were dropped from the scale. In the second phase, to explore the factor structure of FHRS, principal factor analysis using oblique rotation in Statistical Package for Social Science (SPSS) software was conducted on the 16 indicators in the exploratory sample (n = 268). An oblique rotation was used because correlation between factors was expected (Liebenberg et al. 2012).

In the third phase, confirmatory factor analysis was used to verify the structure of the FHRS in the second portion of the sample (n = 182). Therefore, a second-order confirmatory factor analysis was undertaken with maximum likelihood, using Amos Software version 21. In the fourth phase, farm households’ resilience score was estimated based on the confirmed structure of FHRS in the previous stage. We adopted a procedure that was recommended by Antony and Visweswara Rao (2007) for calculation of Human Development Index and Human Poverty Index. The first step for computation of farm households’ resilience score using this procedure was estimating factor scores as the scores of each case on each factor. Factor scores were estimated using the formula 1; where the FS(i) is the score of factor i for each case or household; L(j) are the factor loadings of variable jth (j = 1, 2, …) on factor i and Xj are the initial score for each sub-scale.

$$ \mathrm{FS}\left(\mathrm{i}\right)={\displaystyle {\sum}_{j=1}^n\mathrm{L}\left(\mathrm{j}\right)\times \mathrm{X}\left(\mathrm{j}\right)} $$
(1)

This method was applied to all the variables loaded on one factor. This calculation was carried out using SPSS software and factor scores were saved as new variables in the data set. A non-standardized score was calculated for each household using formula 2; where the NSS(k) is the non-standardized score for the kth case; V(i) is the explained variance by the ith factor and FS(i) is the factor score ith for each case.

$$ \mathrm{NSS}\left(\mathrm{k}\right)={\displaystyle {\sum}_{i=1}^n\mathrm{V}\left(\mathrm{i}\right)\times \mathrm{FS}\left(\mathrm{i}\right)} $$
(2)

Our primary goals was to extract a version of FHRS that was short and could easily be used in the future studies to measure resilience under water scarcity. Therefore, in the sixth phase, an additional factor analysis using an unrotated solution was conducted to extract those questions that load best on the first factor. Unrotated factor solution allows for the maximization of the sum of square factor loadings, where the first factor accounts for the largest share of the total variance in the data, yielding those questions of the measure that have the best perform (Ungar and Liebenberg 2011, p. 139).

Finally, in the seventh phase, content validity of FHRS was estimated using Lynn’s approach (Lynn 1986). Content validity is the degree to which the questions of a scale are representative of the construct of interest (Delgado-Rico et al. 2012). According to Lynn’s approach, the questions of FHRS were rated on a Likert-type scale with four possible responses. The responses included a rating of 1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, and 4 = very relevant. Using this approach is supported by researches specifying that rating of 1 and 2 are considered content invalid, while ratings of 3 and 4 are considered content valid (Wynd et al. 2003). Then a panel of 7 experts independently reviewed and evaluated the relevance of the questions to the concept of farm household resilience under water scarcity.

The Content Validity Index (CVI) which is a content measure of validity was calculated for each question by dividing the number of raters who have considered a judgment of 3 or 4 on the corresponding Likert scale by the total number of the raters (Delgado-Rico et al. 2012). Although calculating content validity index through this approach is advocated by various authors, but concern about the risk of chance agreement between raters in this technique led many statisticians to recommend Cohen’s Kappa coefficient for examining inter-raters agreement (Wynd et al. 2003). The Kappa coefficient describes the portion of agreement remaining after removing chance agreement (Haley and Osberg 1989) through formula 3; where P0 is the total proportion of agreement, Pe is the proportion of agreement expected to occur by chance alone, and (P0−Pe) is the observations for which there are real agreement versus chance agreement (Wynd et al. 2003).

$$ \mathrm{K}=\left({\mathrm{P}}_{\mathrm{o}}-{\mathrm{P}}_{\mathrm{e}}\right)/1-{\mathrm{P}}_{\mathrm{e}} $$
(3)

Whereas Cohen introduced Kappa coefficient to measure the agreement between just two raters, Fleiss in 1971 introduced a generalization of Cohen’s Kappa for the multi-raters cases. Like other versions of the Kappa statistic, Fleiss’ multi-raters Kappa takes the general form presented in above equation (Randolph 2005). In this study, Kappa coefficient was calculated using Online Kappa Calculator that developed by Randolph (2008).

2.5 FHRS development process

Briefly, the process for developing Farm households’ Resilience Scale in this study can be summarized by several steps, as presented in Fig. 2.

Fig. 2
figure 2

Farm households’ resilience scale development process

Full size image

3 Results and discussion

3.1 Item (question) analysis and developing the initial FHRS

The results of reliability test indicated 53 questions of questionnaire have correlated item-total correlation coefficient of less than 0.3 and are unacceptable for FHRS. This process resulted in 77 questions as an initial version of FHRS. The reliability test was conducted for the initial FHRS scale, the Cronbach’s alpha was estimated from 0.61 to 0.85.

3.2 Exploring factor structures in the FHRS

As pointed out in the section 2.4, to explore the factor structure of FHRS, principal factor analysis using oblique rotation was conducted on the 16 indicators. The Kaiser-Mayer-Olkin coefficient verified the sampling adequacy for the analysis (KMO = 0.86). Also, Barlett’s Sphericity test indicated that correlation between indicators were sufficiently large for an exploratory factor analysis (χ 2 (120) = 2764; Sig < 0.001).

An initial analysis was run to determine the factors that had eigenvalues greater than one. Each factor is a weighted linear combination of indicators being analyzed. Also, eigenvalue is the sum of squared distance of the indicators to the factor (Meyers et al. 2006). The three factors had eigenvalues greater than one and in combination explain 56 % of total variance. Table 2 shows the three-factor structure of FHRS, the pattern coefficient resulted from principal factor analysis for FHRS and the location of each indicator on the three-factor structure. The pattern coefficient is the unique effect of a factor on an indicator (Wu et al. 2014). The lagging indicators for each factor has shown through bolding their pattern coefficient on that factor (Table 2).

Table 2 Pattern coefficients for farm households’ resilience scale under water scarcity
Full size table

Factor 1 was named Adapting to water scarcity. This factor includes six indicators Creativity, Hopefulness, Positive attitude towards water scarcity, Adaptation, Learning and Spiritual beliefs and explained 35.34 % of the variance. Factor 2 that was named Retaining function and structure included six indicators Retaining agricultural function, Retaining agricultural structure, Retaining living standards, Family cohesion, Social cohesion and Health situation. This factor explained 10.52 % of the variance. Factor 3 or Retaining individual capabilities included four indicators Controllability of impacts of water scarcity, Decision-making abilities, Self-Esteem and High on emotion and explained 9.76 % of the variance. The results of this part of analysis indicated that Farm Household Resilience Scale includes three Factors or components Adapting to water scarcity, Retaining function and structure and Retaining individual capabilities.

3.3 Verifying the factor structure of FHRS

Second-order confirmatory factor analysis was used to verify the obtained three-factor structure of the FHRS. The tested model (Fig. 3) comprised from the four latent variables including resilience and three factors with their connected indicators as found in the principal factor analysis reported in the previous section (Table 2). To assess whether a model verifies by confirmatory factor analysis, we should see model fit statistics or indexes that confirmatory factor analysis gives us. Model fit statistics for the examined model are presented in Table 3. The Goodness of Fit Index (GFI) as a measure of fit between the hypothesized model and the observed covariance matrix was higher than 0.9 (GFI = 0.952) that showed an acceptable model fitness. Also, the Bentler-Bonett Normed Fit Index (NFI) and the Comparative Fit Index (CFI) were higher than 0.9 (NFI = 0.946; CFI = 0.980) and suggested a good fit to the data. NFI analyzes the discrepancy between the chi-squared value of the hypothesized model and the chi-squared value of the null model and CFI analyzes the model fit by examining the discrepancy between the data and the hypothesized model. Moreover, the Root-Mean-Square Error of Approximation (RMSEA) as another fit index showed a good fit (0.052). In fact, according to Meyers et al. (2006) the smaller the RMSEA (e.g., <0.06) indicates the better fit. Although, the Chi-square value was significant (χ 2 (53) = 96.3, p < 0.001), this apparent lack of fit is not surprising because very small differences between expected and observed correlations in large samples can lead to a significant χ 2 (Cole 1987).

Fig. 3
figure 3

Confirmatory factor analysis model of farm households’ resilience scale under water scarcity

Full size image
Table 3 Model fit summary statistics of farm households’ resilience scale confirmatory factor analysis
Full size table

3.4 Calculating farm households’ resilience

As pointed before, three factors explained 56 % of total variance. The first, second and third factors explained 35.34 %, 10.52 % and 9.75 % of the variance, respectively. Using these percentages as the weights of the factors, the non-standardized score (NSS) of resilience for each household was calculated for each case (k) as:

$$ {\mathrm{NSS}}_{\left(\mathrm{k}\right)}=(35.34)\left(\mathrm{Factor}\kern0.5em 1\kern0.5em \mathrm{score}\right)+(10.52)\left(\mathrm{Factor}\kern0.5em 2\kern0.5em \mathrm{score}\right)+(9.75)\left(\mathrm{Factor}\kern0.5em 3\kern0.5em \mathrm{score}\right) $$

To facilitate the interpretation of the resilience scores and make comparison between individuals, a standardized score (SS) was developed which its value can range from 0 to 100, using formula 4; where Resilience(K) is resilience score for the kth household; Resilience(min) is the minimum score for resilience in the sample; and Resilience(max) is the maximum score for resilience in the sample.

$$ \mathrm{SS}=\frac{\mathrm{Resilience}\left(\mathrm{k}\right)-\mathrm{Resilience}\left( \min \right)}{\mathrm{Resilience}\left( \max \right)-\mathrm{Resilience}\left( \min \right)}\times 100 $$
(4)

3.5 The final version of FHRS

The results of factor analysis using unrotated solution for each of three extracted factors revealed that 32 questions from 77 questions of FHRS loaded best on the first factors and reflected the three domains of resilience (Adapting to water scarcity, Retaining function and structure and Retaining individual capabilities). These 32 questions were included in the short version of the FHRS. Using Lynn’s approach (Lynn 1986) for measuring content validity, 32 questions were rated on a Likert-type scale with four possible responses (1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, and 4 = very relevant) and evaluated by 7 experts. While most researchers believe that the content validity index values should be ≥0.70 (Tilden et al. 1990), the results of calculating content validity of FHRS showed only one question (question 24) had insufficient content validity (CVI ≤ 0.70) (Table 4). Also, the results of estimating multi-rater Kappa coefficient for each of the 32 questions ranged from K = 0.33 to 0.99 (Table 4). A Kappa above 0.5 indicates moderate agreement, above 0.7 indicates good agreement and, above 0.8 indicates very good agreement among raters (Peat et al. 2002). The results of this content validity analysis are presented in Table 4 which indicates question 24 (I feel water scarcity has some positive side too) which has a low content validity (CVI = 0.62) and poor Kappa value (K = 0.33) should be eliminated in the final version of FHRS (Table 5) and other questions should be kept with little or no modification.

Table 4 The results of calculating content validity index and kappa coefficient for the farm households’ resilience scale
Full size table
Table 5 Farm households’ resilience scale under water scarcity
Full size table

4 Conclusion and application

During the time that water resources declining at an alarming rate is an important challenge to attain agricultural sustainability throughout the dry and semi-dry regions of the world (Forouzani and Karami 2010), farmers are faced with this question that how they can mitigate water scarcity to attain sustainability. Resilience as a new concept that have been incorporated into studies of sustainability can respond to this dilemma (Maleksaeidi and Karami, 2013). But in spite of the availability of thousands of reports on different aspects of farm water scarcity in the journals and books, there are only few studies that have focused on measuring resilience under scarcity of this critical resource. In addition, the majority of studies that have attempted to measure resilience under water scarcity does not have a clear and standard scale to assess this concept. For those reasons and whereas having standard scales to measure resilience is necessary to identify policies and actions that change system’s resilience, the importance of present study is its focus on developing a standard and organized scale to measure farm households’ resilience under water scarcity.

An initial version of FHRS with 77 questions was developed by selecting 16 effect indicators based on the initial framework, developing a questionnaire that represents the selected indicators and, collecting data by questionnaire from farm households living around Parishan wetland in Iran. Then, a principal factor analysis was conducted on 77 questions to explore the factor structure of FHRS. The results of factor analysis suggested a three-factor structure for this scale. Confirmatory factor analysis verified this structure. Through conducting an additional factor analysis on each of three extracted factors and then calculating content validity, a final version of FHRS with 31 questions was developed. The validity and reliability of this scale was tested and verified.

Considering the importance of resilience of socio-ecological systems, the scale developed in this study is a step forward towards understanding and improving the resilience of farm households. This reliable and valid scale provides a standard framework for assessing the ability of farm households to bounce back from water scarcity. In fact, without developing a standard scale, it is often difficult to validate the calculations resulted from using non-standard variables. Also, assessment of resilience using FHRS provides an opportunity for managers and policymakers to develop management solutions, identify the most effective combination of short and long term mitigation and adaptation strategies. Also, it helps understanding how to lift farm households out of cycles of poverty and hunger and how to generate and protect social-ecological well-being. All of these are based on the principle that before detecting the factors that make households resilient to a disturbance such as water scarcity, resilience must first be understood, and then strengthened.

Although, the present study was conducted among farm households around Parishan wetland, it has an application beyond the geographic limits of the study area. Researchers can use scale developed in this study for quantitative tracking of resilience of farm households in the conditions of water scarcity in the different areas. Moreover, they can use the development process of FHRS in the present study as a new pattern for developing sustainability measures and scales in their studies.