Abstract
Small-scale dairy systems are important contributors to national milk supplies in many areas of the world, and an option to ameliorate rural poverty in developing countries. In Mexico, they comprise over 78% of dairy farms. These systems must be sustainable in order to persist in the future. By applying several methods to assess the sustainability of farms, valuable information is collected on the practical, operational, and systemic requirements, as well as an insight into the difficulties in the use of each tool in practice. The objective was to assess the sustainability of small-scale dairy systems during the rainy season. Three methods were compared (IDEA, RISE, and SAFA) to evaluate their ability to deal with such systems in the Mexican context. Ten small-scale dairy farms were assessed from June to November 2018. Monthly semi-structured interviews were applied to collect economic, social, and environmental information. The three methods met criteria for on-farm assessments, with no large differences among them. The IDEA method was more applicable in the context of small-scale dairy systems because its indicators may be collected on-farm and were easy to measure. RISE requires more specialized technical information not always available at the small-scale farm level, and SAFA covered the largest number of indicators but is better suited for large-scale systems. The IDEA and RISE methods are adequate tools to assess the sustainability of small-scale dairy systems. The mean overall sustainability score over the three methods for the 10 assessed farms was 55.3±5.7 over 100. This medium level of sustainability indicates areas of opportunity to enhance the sustainability of small-scale dairy systems.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
There is a need not only to increase food production in the coming years to cope with demand from a growing population (FAO 2013), but agricultural production must be sustainable in its economic, environmental, and social dimensions. Sustainability of agricultural production is based not only in increased production of goods and services, but on its robustness, rooted in the local communities, being autonomous, and with a global responsibility (Zahm et al. 2015), within a holistic approach (Van Passel et al. 2007; Zahm et al. 2008).
Sustainable agricultural systems involve production following a good management of the environment and looking after the social context and wellbeing of farming families and their communities (Van Passel et al. 2007). However, assessing the sustainability of agricultural systems is complex, since it deals with dynamic and holistic issues that develop and evolve in a specific site and relies on the perspective of who undertake the assessment (Webster 1997).
Different methods have been developed for the assessment of differences and variations of the sustainability of farming systems (Hayati et al. 2010). These methods can be based on simple indicators, i.e., from information taken from records or simple questions, or on complex ones that require a higher degree of knowledge or specialized equipment (Bockstaller et al. 2015).
The Intergovernmental Panel on Climate Change (IPCC) (2014) states that greenhouse gas emissions by dairy production account to 4% of the world total, and 22% of all agricultural emissions. Thus, the assessment of sustainability in dairy systems is relevant since these have been pointed out as having a large environmental footprint (Flysjö 2012). All agricultural systems must engage in sustainable production, especially the small-scale dairy systems because they comprise the majority of dairy farms worldwide even in developed countries as the European Union where the mean herd size in 2016 was 18 cows (IFCN 2017). Therefore, small-scale dairy systems represent a large potential to reduce their environmental impacts and develop sustainable production (FAO 2014a).
In Mexico, 78% of specialized dairy farms are small-scale defined by a small agricultural land, with a herd size between 3 and 35 cows plus replacements (Prospero-Bernal et al. 2017). Milk sales are the main source of income in 90% of these farms (Martínez-García et al. 2012). The small-scale farms are considered as a viable option for territorial development, as they are a source of full-time employment, enabling rural populations to remain in their communities (FAO 2010). Farmers are linked to informal milk markets, and they have developed a strong relationship with milk collectors and artisan cheese producers, giving strength to the agri-food chain (Espinoza-Ortega et al. 2007). However, the economic disparities between prices paid for milk and costs of inputs put extra strain on the economic scale of their sustainability, which is the weakest part in these systems (Fadul-Pacheco et al. 2013).
On the other hand, the literature states the need to compare methods that provide information to integrate criteria for the assessment and understanding of the sustainability in a given context (Bockstaller et al. 2009). However, there are few reports that compare methods on-farm. De Olde et al. (2016) compared the IDEA (Indicateurs de Durabilité des Explotations Agricoles) (Vilain et al. 2008), RISE V 3.0 (Response-Inducing Sustainability Evaluation) (Grenz et al. 2016) and SAFA V-3 (Sustainability Assessment of Food and Agriculture systems) (FAO 2014b) in dairy and pig farms in Denmark, stating that RISE was the method better adapted to the Danish context; noting its relevance, easiness of use, understandable, and with advantages in the software. In addition, de Olde et al. (2016) identified out of 48 methods for the assessment of sustainability, that only IDEA, RISE, and SAFA met criteria for on-farm assessments, while Binder et al. (2010) out of 35 methodologies selected IDEA and RISE.
The three methods (IDEA, RISE, SAFA) allow the on-farm assessment of sustainability, through scientifically rigorous indicators, integrated by the ecological, economic and social dimensions of sustainability (De Olde et al. 2016). They also enable the self-evaluation of each farm and the comparison among farms and do not require an optimal or reference farm for comparison (Häni et al. 2003; Zahm et al. 2008; FAO 2013). Table 1 shows the characteristics of each of these methods.
The IDEA method is the most accessible and easiest to understand. RISE and SAFA require more complex technical data. The objectives of each method address different dimensions and topics, through indicators that comprise the holistic sense of sustainability (De Olde et al. 2016), with the end goal of guiding farms towards sustainable development (Zahm et al. 2015).
The IDEA method enables the assessment of the sustainability of individual farms with a score that may be compared against other farms within the same production system, or even compare among different systems (Zahm et al. 2008). There are reports of its successful application in developing countries such as Algeria (Ghozlane et al. 2006), Tunisia (M’Hamdi et al. 2009), and Uruguay in dairy systems (Tommasino et al. 2012), also, for sheep and goat systems in Algeria and Lebanon (Ghozlane et al. 2008; Srour et al. 2009).
In Mexico, Fadul-Pacheco et al. (2013) and Prospero-Bernal et al. (2017) assessed the sustainability of the small-scale dairy systems, applying for the first time the IDEA method; and Salas-Reyes et al. (2015) also applied the same method to assess the sustainability of dual-purpose small-scale cattle farms in a subtropical area. These studies showed that the IDEA method enabled the identification of areas for improvement in the economic scale which limits the sustainability of these systems, basically in the need to reduce feeding costs to enhance the profitability and economic viability of farms.
However, there were questions on the suitability of the IDEA method, developed in France, when applied in the Mexican context. Therefore, the need arose to evaluate different methods for the assessment of sustainability within and between farms and systems that may be better adapted to the context of small-scale dairy systems with low availability of data and that are easy to apply considering the limited time and financial resources for the assessment.
From these, the IDEA, RISE, and SAFA methods were applied in the work herein reported, selected for the quality of indicators, their scientific framework and that they can be applied at farm level, and in multiple systems. The IDEA method was included as a reference for comparison, given the previous experience of the research team with this method (Fadul-Pacheco et al. 2013; Prospero-Bernal et al. 2017).
The assessment of the sustainability of small-scale dairy systems with the three methodological tools was aimed at discerning their strengths and weaknesses in the Mexican context; as well as providing a better understanding of the sustainability dynamics in these farms to identify areas of improvement and support for decision making. Therefore, the objective was to assess the sustainability of small-scale dairy systems during the rainy season. Three methods were compared (IDEA, RISE, and SAFA) to evaluate their ability to deal with such systems in the Mexican context.
Materials and methods
Study area
The work took place in the central highlands of Mexico (Fig. 1), between coordinates 20° 06’ and 20° 17’ N and at 99° 40’ and 100° 00’ W and mean altitude of 2440; a sub-humid temperate climate with rains in summer, and a dry season with frosts in winter (INEGI 2009). Mean temperature was 16.4°C and mean rainfall 776.7 mm (SMN-CONAGUA 2019).
Almost 90% of dairy farms in the study area were small-scale dairy producers (INEGI 2007), who relied on milk sales for their livelihoods (Martínez-García et al. 2012). Farms were characterized by herds between 3 and 35 cows plus replacements, two milkings per day, and small land areas, that relied on family labour (Fadul-Pacheco et al. 2013).
Feeding is based on cut-and-carry of temperate cultivated pastures (ryegrasses with white clover), forages as oats or bought-in alfalfa hay, complemented with cereal straws (maise, oats, barley, and wheat) and commercial concentrates (Martínez-García et al. 2015a). Some farms graze native grasslands, and some have incorporated grazing their cultivated pastures and maise silage (Prospero-Bernal et al. 2017).
Selection of farms and data collection
Ten farms participated in the study. They have participated in the project to which this work belongs (Prospero-Bernal et al. 2017), initially selected by snow-ball non-probabilistic sampling (Goodman 2011; Sedgwick 2013). Farmers accepted to participate in the study voluntarily and were informed at all times of the objectives and scope of the work under a participatory rural research approach (Conroy 2005).
At the start of the assessment, the indicators of each methodology were revised to identify their specificity and applicability (FAO 2014b; Prospero-Bernal et al. 2017; Berbeć et al. 2018; Soldi et al. 2019). Data were collected with a structured questionnaire for each method (IDEA, RISE y SAFA) (Vilain et al. 2008; FAO 2013; De Olde et al. 2016).
Questionnaires included the indicators for the environmental, social and economic dimensions (Appendixes A1, A2, and A3), adapted to the study area (Zahm et al. 2015) to ensure an approach compatible with the Mexican context (Prospero-Bernal et al. 2017). Appendix B shows the indicators that were not included for each method.
Information was collected during monthly visits to each farm when questionnaires for the three methods were applied; always by the same member of the team to reduce potential bias.
Since farmers do not keep records, semi-structured questionnaires were also applied during each visit to collect information on the quantity of feeds, milk sales, and productive, reproductive and economic information from the previous month. Milk and feed samples were collected during those visits and analyzed in the laboratory for milk composition (milk fat and protein) and chemical composition of feeds (dry and organic matter, crude protein, neutral and acid detergent fiber, and in vitro organic matter digestibility) following Fadul-Pacheco et al. (2013) and Prospero-Bernal et al. (2017).
Data collection was during the rainy season from June to November 2018, so that there were 60 questionnaires and collected data from farms for each method. Previous research with the IDEA method was in the rainy season (Fadul-Pacheco et al. 2013), and the IDEA method was taken as reference given the experience of the research team. Also, budget constraints limited the duration of the study.
Indicators were adapted to current Mexican standards for milk composition and environmental issues, and those not applicable to the Mexican context were not considered as was done in previous work (Fadul-Pacheco et al. 2013; Salas-Reyes et al. 2015). The economic analyses followed Prospero-Bernal et al. (2017) through partial budget analyses, just considering the dairy operation as the basis of livelihoods.
Interpretation of sustainability level by IDEA, RISE, and SAFA
Results for each dimension (environmental, social, and economic) were from weighing scores where each dimension may get a score from 0 to 100. SAFA scores (0–5) were transformed to a 0-100 scale to compare methods.
Sustainability score in IDEA is from the dimension with the lowest score (limiting scale) (Zahm et al. 2019). Sustainability from RISE and SAFA was from the average of the three and four dimensions, respectively (FAO 2014b; Grenz et al. 2016).
Scores for each indicator (on a 0 to 100 scale) are classified as (Grenz et al. 2016): 0–33: problematic, 34–66: critical, and 67-100: positive (Appendix A). Mean results for each method are presented in radar graphs (Figs. 2, 3, and 4).
Statistical analyses
Descriptive statistics were applied to the 10 participating farms to justify that the sample is representative of farms encountered in the study area. Data from each method was organized following the guidelines from IDEA (Vilain et al. 2008), RISE (Grenz et al. 2016) and SAFA (FAO 2014b) (Table 2).
Indicators were analyzed for each dimension. The level of sustainability was described as suggested by Binder et al. (2010) and de Olde et al. (2017) since the number of indicators and topics are different for each method, so that results are presented as independent indicators without considering interactions between them.
The Shapiro-Wilk test recommended for samples under 50 observations did not show a normal distribution of data (Field 2013); therefore, the comparison within dimensions and the level of sustainability for each method were analyszed with the Kruskal-Wallis test and the Mann-Whitney U test to detect differences (Field 2013).
Results
Characteristics of participating farms
Table 2 shows the characteristics of the participating farms, with the largest variation in farmland size (ha) and the number of cows; with the lower variation for milk fat and protein content among farms. Farms rely on family labor, with temporal hiring of labor during harvesting of forages and crops like maise and oat.
Indicators classification by color code (green, amber, and red)
Table 3 shows the classification of indicators by score, identified as positive (green), critical (amber), and problematic (red) for each method. Appendixes A1, A2, and A3 show all indicators reported in terms of their maximum possible score, the mean score and maximum and minimum scores for farms for each method.
In the environmental scale, indicators for fertilisation have problematic scores in the three methods, with positive scores for the majority of indicators relating to animal production diversity and water management. Indicators for energy and materials use were problematic in IDEA and RISE, although positive in SAFA. However, all the others indicators in this scale having high scores, the environmental scale was classified as positive (see Appendices A1, A2, and A3).
Also, on the social scale, the majority of indicators in IDEA showed a positive score, while in RISE and SAFA most indicators are at a critical score. Indicators relating to health and safety at work have similar scores in the three methods, and it is in the social scale where fewer indicators are problematic. Most indicators are qualitative and with similar content in the three methods, which weigh scores similarly.
In the economic scale, most indicators in IDEA and SAFA showed positive scores, in contrast with RISE where most indicators were classified as problematic.
Assessment of sustainability by the three methods
Figure 2 shows results for IDEA. Two of the four themes (components) that had the lowest scores were for the economic scale: economic efficiency and viability. The other two components with a low score were the organization of space and quality of products of the land. The highest scores were for the environmental scale.
Figure 3 shows the results of the RISE method. Scores for economic viability were the lowest, similar to IDEA. The highest scores were those related to animal welfare (environmental scale) and farm management.
It is the RISE method that resulted in the lowest score for the environmental scale. This is due to more exhaustive and detailed indicators on soil management, as well as indicators on environmental protection and energy use. In contrast, IDEA takes into consideration more general indicators on crops, land areas, and the territory.
SAFA results (Fig. 4) showed that the indicator for local economy had the highest score; with overall high positive scores for the economic scale. One aspect valued by SAFA is local trade, which in the farms studied refers to the sale of milk destined to local small artisan cheesemakers obtaining a 100 score for this indicator which influence the overall high score for the economic scale. However, SAFA indicators for profitability and liquidity that reflect the economy of each farm were not high scores.
IDEA and RISE results showed that farms are at a critical (amber) level of sustainability. IDEA results indicated that the economic scale limits the sustainability of these systems. RISE and SAFA have the environmental and social scales with lower scores than for the economic scale.
Themes on product information and quality, responsibility, and land use showed the lowest scores in SAFA, which differ from IDEA and RISE in those scales.
Comparison of dimensions and sustainability level
Table 4 shows results for the scores for each environmental, social, and economic dimensions and the overall level of sustainability for each of the studied methods. There were highly significant differences among methods (P<0.001) for the environmental and economic dimensions with RISE showing the lowest score for environmental dimension.
There were no statistical differences among methods for the social dimension (P>0.05), and there were highly significant differences (P<0.001) among methods for the economic scale. SAFA had the highest mean score for the economic dimension, with similar scores between IDEA and RISE.
In terms of overall sustainability scores, there were highly significant differences (P<0.001). The SAFA score was the highest, while IDEA and RISE showed a similar sustainability score. In spite of differences, the three methods showed an overall medium (critical) sustainability score.
Discussion
Farm characteristics
Participating farms were similar to those reported by Romo-Bacco et al. (2014) and Prospero-Bernal et al. (2017) in small-scale dairy systems in two different areas of the Mexican highlands. Both works reported the reliance on family labour (by two family members), and about 10% of hired labour. Farms have between 6 and 7 ha of farmland, with 9 to 15 milking cows that yield between 14 and 16 litres of milk per day.
Assessment of the environmental, social, and economic components of sustainability
The three methods applied enabled the assessment of the sustainability of participating farms and were sensitive to detect problematic, critical, and positive points (Grenz et al. 2016).
In the environmental scale, the three methods identified problematic indicators in crop management, due to high fertilizer use and soil degradation. Farmers are aware of the high amounts of fertilizers applied, but few have reduced their use.
Given the low schooling level of small-scale farmers, they are generally unwilling to introduce changes in their practices (Martínez-García et al. 2015b), and changes happen usually until they are convinced by the influence of their social referents from whom they take advice (Martínez-García et al. 2018).
As positive indicators, the 10 farms use manure as organic fertilizer for their pastures, and having mixed grass-clover pastures is also a positive indicator. Other positive indicators were diversity, animal welfare, and water use.
The IDEA method showed the highest scores for the environmental scale, attributed to the indicators the method evaluates, centered in diversity, management, and the territory.
RISE and SAFA, on the other hand, evaluate very specific indicators on issues of air, water, and soil, with sub-topics and indicators for a detailed assessment that requires specific information that farmers do not have and are not easy to obtain, as the balance of greenhouse gases (that were not measured) in RISE, and a whole theme on the atmosphere in SAFA.
De Olde et al. (2016) and Berbeć et al. (2018) mentioned that RISE and SAFA have the largest number of specialized indicators. Under these methods, positive indicators were those related to animal and plant diversity. Jouzi et al. (2017) pointed to one of the advantages of small farms is the rational use of local resources.
The three methods utilized have strengths but also weak points. In the IDEA method, water is a weak issue, since IDEA only has an indicator for water management related in the studied farms to the availability of irrigation for pastures. RISE and SAFA, with similar scores, have water as a specific topic with indicators on measures for the saving and control of water, water quality and availability, and amounts of water used in the farm and for irrigation.
The IDEA method is general and does not consider important issues for the assessment of sustainability; while RISE and SAFA include more indicators that yield more reliable results. However, the inclusion of more themes to the assessment implies more specialized indicators that require more information and data that are not available in small-scale farms, as well as resources and time for the assessments.
In terms of the social component of sustainability, social indicators in the three methodologies are similar. IDEA, RISE, and SAFA established as positive indicators animal welfare, labor security, economic incomes above the community means, low generation of residues, and freedom to make decisions.
Indicators for the social dimension of sustainability are complex given the constant evolution of society, which makes it difficult to develop simple and precise indicators, and the fact that assessments take place at a specific moment in time (Vilain et al. 2008).
Hayati et al. (2010) stated that these indicators lead farms towards sustainable development. However, indicators as the intensity of work in IDEA are problematic due to the heavy workload, as has been identified in previous works (M’Hamdi et al. 2009; Fadul-Pacheco et al. 2013; Prospero-Bernal et al. 2017). Nonetheless, Moretti et al. (2016) mentioned that family labor strengthens farms making them more resilient to changes.
RISE identified a low quality of social relations, in contrast to IDEA and SAFA that identified strength in the relations among farmers. Even though social indicators have been developed since the inception of the sustainability concept (WCED 1987), methodologies have been negligent by diminishing their importance. Therefore, there is a need for the development of indicators to measure the creation of social capital (Vallance et al. 2011). In this work, social indicators (Table 3) and their objectives are similar in the three methods (Binder et al. 2010).
In the economic dimension of sustainability, positive indicators were the generation of economic incomes, adequate financial autonomy, low dependency of external subsidies, and the production of food for the community, key elements for farm resilience (Jongeneel and Slangen 2013).
Problematic indicators were low specialization of production, lack of available information and in the generation of information on the management of the farm. This affects decision making and results in a lack of knowledge of the actual processes, reducing economic efficiency as detected by IDEA and RISE.
SAFA results for the economic dimension agree with de Olde et al. (2016) who indicated that this method tends to over-evaluate economic indicators, yielding results that do not coincide with the reality of farms that are not economically efficient. In contrast, IDEA and RISE are based on indicators as cash flow, incomes, and investments, which are easy to measure.
RISE allows for the lack of data in farms, while SAFA allows some specific themes to be omitted that may be irrelevant in a given context, avoiding the need for indicators that require unavailable data, using in place indicators based on practice (FAO 2013).
The economic scale is relevant in farm resilience, on which the continuity of farms relies (Hayati et al. 2010). Economic viability was an indicator with low scores in the three methods, which can be attributed to the expenditure in cattle feeding (purchase of external inputs), purchase of fertilizers, and dependence on fossil energy (gasoline and diesel). Therefore, the economic scale limits the sustainability of small-scale dairy systems (Prospero-Bernal et al. 2017).
Overall assessment of the sustainability by three methods
The three methods (IDEA, RISE, and SAFA) showed variation in the content of indicators, reference values, and methods for scoring and aggregation. This variation is due to the differences in judgment values, the context, and priorities of those involved in the development of each method (De Olde et al. 2017).
Variability in the methods gave rise to differences in the assessment of the sustainability of the studied farms, although results presented are transparent both in the use of the methods and in the results generated (De Olde et al. 2017), so that adaptation and integration of the various indicators are feasible due to their inter-relationships given their similarities as the three are multi-criteria methodologies (Binder et al. 2010).
Score values are different as each method values differently the indicators, assigning different scores based on their specific norms or assessment protocols for the scoring of indicators (Marchand et al. 2014). There are times when there are many possible variables integrating an indicator, and it may be difficult to decide which is best. At other times, variables are not easy to measure, or there are no data and must be changed for other less reliable variables (Sarandón 2002).
These aspects must be taken into consideration for a good assessment of sustainability in order to have an objective and reliable result for the farms that enable decision making in relation to weak points that need improvement.
The limitation of the three methods was the lack of information that could not be collected as farms have little data available, and there were not sufficient financial resources to undertake all laboratory analyses needed for a complete assessment.
RISE and SAFA offer possibilities to overcome the lack of information. RISE gives the option of qualitative measurements of indicators if specific data is missing as for economic or life quality indicators. SAFA allows for indicators of practice to be changed for indicators of yield which are easier to obtain. IDEA has indicators closer to on-farm situations that make it easier to adapt to specific contexts.
An aspect to take into consideration is that when adopting an existing method, like IDEA, RISE, or SAFA, the number of themes, indicators, and assessment procedures are defined, and most of the method to apply is fixed.
IDEA and RISE were specifically developed for the assessment of farm sustainability, while SAFA has a broader application that encompasses agriculture, forestry and fisheries, as well as the assessment of companies at a world scale (FAO 2013).
SAFA also proved to be the least applicable method for its use in small-scale farming. Firstly, some indicators require economic data of more than five previous years which are not available in the small-scale farms. Secondly, SAFA was not developed for small-scale farms, and thirdly, there is a large number of specialized indicators that are not easy to measure for lack of instruments, or financial and time resources. The interest in including SAFA in the study is that it was put forward as a probable better method given its development by a global organisation as FAO.
The proportion of sub-themes form a method that is dealt with by the other two is called sub-theme coverage. SAFA has an intermediate to high indicator coverage at 89% for IDEA and 92% for RISE. RISE has a coverage of 67% for SAFA and 81% for IDEA, and IDEA covers 59% for SAFA and 76% for RISE (De Olde et al. 2017).
SAFA is the method with the largest number of indicators also employed by IDEA and RISE. Soldi et al. (2019) mentioned that SAFA requires specialized work in the collection of information and is aimed at regional assessments, which are less sensitive at farm level. On the contrary, IDEA and RISE were developed to assess the sustainability of farms (De Olde et al. 2016).
IDEA has well-defined indicators, easy to collect that can be used at farms with limited information. On the contrary, RISE, as SAFA, has very specialized indicators at the environmental scale, and requires more technical and intellectual infrastructure for the assessment compared to IDEA, RISE, and SAFA may be considered for sustainability assessments with ample financial and time resources.
There will always be variability in the assessment tools as well as in the results since each method is based on the context, availability of scientific data, and knowledge of values and norms of those who develop the methods (De Olde et al. 2016).
The IDEA method was better adapted to the sustainability assessment of small-scale dairy systems in Mexico as most of its indicators may be collected on-farm and at easy to measure, compared to RISE and SAFA. Therefore, it is suggested to continue using the IDEA method in future assessments of sustainability in small-scale farming systems.
Conclusions
The IDEA, RISE, and SAFA methods share in essence the concept of sustainable development from the holistic integration of the environmental, social, and economic dimensions of sustainability, and are sensitive so that it is possible to identify problematic indicators, to make decisions that may guide farms towards an enhance sustainability.
IDEA and RISE were identified as the stronger methods for on-farm assessments and did not show differences in the social or economic scales, nor in the overall sustainability score.
IDEA was the less demanding method for environmental indicators in contrast to RISE and SAFA that concentrate efforts in this dimension. In SAFA, the economic scale is ambiguous since indicators are aimed at communities or larger regions. When applied at the farm level, SAFA does not detect small variations, particularly on the economic scale.
These three methods enable an understanding of sustainable development by generating an interaction between research institutions and farmers. Even though there is not a strong culture of sustainability in the study area, work undertaken enable to raise awareness of farmers, their families and communities.
The mean overall sustainability score over the three methods for the ten assessed farms was 55.3±5.7 over 100. There were no large differences between the three methods, even though indicators vary in the way of their measurement; they share more than 70% of objectives. This level of sustainability places farms at a critical level (Amber) following the color code, although towards the higher end, opening opportunities to enhance their sustainability.
Data availability
The data that support the findings of this study are available from the corresponding author upon reasonable request.
Code availability (software application or custom code)
Not applicable
References
Berbeć, A.K., Feledyn-Szewczyk, B., Thalmann, C., Wyss, R., Grenz, J., Kopiński, J., Stalenga, J. and Radzikowski, P. 2018. Assessing the Sustainability Performance of Organic and Low-Inputs Conventional Farms from Eastern Poland with RISE Indicator System, Sustainability, 10, 1792. https://doi.org/10.3390/su10061792.
Binder, C., Feola, G. and Steinberger, J.K., 2010. Considering the normative, systemic and procedural dimensions in indicator-based sustainability assessments in agriculture, Environmental Impact Assessment Review, 30, 71-81. https://doi.org/10.1016/j.eair.2009.06.002.
Bockstaller, C., Guichard, L., Keichinger, O., Girardin, P., Galan, M.B. and Gaillard, G., 2009. Comparison of methods to assess the sustainability of agricultural systems. A review. Agronomy for Sustainable Development, 29, 223-235. https://doi.org/10.1051/agro:2008058.
Bockstaller, C., Feschet, P. and Angevin, F., 2015. Issues in evaluating sustainability of farming systems with indicators, OCL, 22, D102. https://doi.org/10.1051/ocl/2014052.
Conroy, C., 2005. Participatory livestock research: a guide, (ITDG Publishing, Bourton-on-Dunsmore, Warwickshire, U. K).
De Olde, E.M., Oudshoorn, F.W., Sørensen, C.A.G., Bokkers, E.A.M. and de Boer, I.J.M., 2016. Assessing sustainability at farm level: Lessons learned from a comparison of tools in practice, Ecological Indicators, 66, 391-404. https://doi.org/10.1016/j.ecolind.2016.01.047.
De Olde, E.M., Bokkers, E.A.M. and de Boer, I.J.M., 2017. The Choice of the Sustainability Assessment Tool Matters: Differences in Thematic Scope and Assessment Results, Ecological Economics, 136, 77-85. https://doi.org/10.1016/j.ecolecon.2017.02.015.
Espinoza-Ortega, A., Espinosa-Ayala, E., Bastida-López, J., Castañeda-Martínez, T. and Arriaga-Jordán, C.M., 2007. Small-scale dairy farming in the highlands of central Mexico: Technical, economic and social aspects and their impact on poverty, Experimental Agriculture, 43, 241–256. https://doi.org/10.1017/S0014479706004613.
Fadul-Pacheco, L., Wattiaux, M.A., Espinoza-Ortega, A., Sánchez-Vera, E. and Arriaga-Jordán, C.M., 2013. Evaluation of sustainability of smallholder dairy production systems in the highlands of Mexico during the rainy season, Agroecology and Sustainable Food Systems, 37, 882–901. https://doi.org/10.1080/21683565.2013.775990.
FAO - Food and Agriculture Organization of the United Nations, 2010. Status and prospects for smallholder milk production a global perspective, (FAO- Food and Agriculture Organization of the United Nations, Rome, Italy).
FAO - Food and Agriculture Organization of the United Nations, 2013. Organic supply chains for small farmer income generation in developing countries – Case studies in India. (FAO- Food and Agriculture Organization of the United Nations, Rome, Italy).
FAO - Food and Agriculture Organization of the United Nations Food and Agriculture Organization of the United Nations, 2014a. Greenhouse gas emissions from agriculture, forestry and other land use. (FAO- Food and Agriculture Organization of the United Nations, Rome, Italy).
FAO - Food and Agriculture Organization of the United Nations, 2014b. SAFA Sustainability Assessment of Food and Agriculture Systems. Guidelines Version 3.0. (FAO- Food and Agriculture Organization of the United Nations, Rome, Italy).
Field, A., 2013. Discovering statistics using IBM SPSS Statistics, Fourth Ed., (SAGE Publications, London, U.K).
Flysjö, A., 2012. Greenhouse gas emissions in milk and dairy product chains: Improving the carbon footprint of dairy products, (unpublished Doctoral Thesis, Science and Technology, Aarush University, Denmark).
Ghozlane, F., Haçene, Y., Mustapha, A. and Bouzida, S., 2006. Évaluation de la Durabilité des Exploitations Bovines Laitieres de la Wilaya de Tizi-Ouzou (Algerie), New Medit, 4, 48-52.
Ghozlane, F., Ziki, B., Abbadie, B. and Yakhlef, H., 2008. Évaluation de la durabilité des exploitations ovines steppiques de la wilaya de Djelfa. Livestock Research for Rural Development, 20, Article # 170, http://www.lrrd.org/lrrd20/10/ghoz20170.htm.
Goodman, L.A., 2011. Comment: On respondent-driven sampling and snow-ball sampling in hard-to-reach populations and snow-ball sampling not in hard-to-reach populations, Sociological Methodology, 41, 347-353. https://doi.org/10.1111/j.1467-9531.2011.01242.x.
Grenz, J., Mainiero, R., Schoch, M., Sereke, F., Stalder, S., Thalmann, C. and Wyss, R., 2016. RISE 3.0 Manual. Sustainability themes and indicators, (School of Agricultural, Forest and Food Sciences, Bern University of Applied Sciences, Zollikofen, Switzerland).
Häni, F., Braga, F., Stämpfli, A., Keller, T., Fischer, M. and Porsche, H., 2003. RISE, a tool for holistic sustainability assessment at the farm level. International Food and Agribusiness Management Review, 6, 78-90.
Hayati, D., Ranjbar, Z. and Karami, E., 2010. Measuring Agricultural Sustainability. In Lighthouse E. (ed) Biodiversity, Biofuels, Agroforestry and Conservation Agriculture, Sustainable Agriculture Reviews, 5, 73-100. https://doi.org/10.1007/978-90-481-9513-8_2.
IFCN – The Dairy Research Network, 2017. IFCN Dairy Report: For a better understanding of the dairy world, (International Farm Comparison Network, IFCN Dairy Research Center, Kiel, Germany).
INEGI, 2007. Censo Agrícola, ganadero y forestal 2007, INEGI - Instituto Nacional de Estadística y Geografía. http://www3.inegi.org.mx/rnm/index.php/catalog/219. Accessed 15 September 2018.
INEGI, 2009. Prontuario de Información geográfica municipal de los Estados Unidos Mexicanos. Aculco. México. INEGI - Instituto Nacional de Estadística y Geografía. http://www3.inegi.org.mx/contenidos/app/mexicocifras/datos_geograficos/15/15003.pdf. Accessed 24 September 2019.
IPCC, 2014. Climate Change 2014: Mitigation of Climate Change. Contribution of Working Group III to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC), (Cambridge University Press, Cambridge, U.K. ).
Jongeneel, R. and Slangen, 2013. Sustainability and resilience of the dairy sector in a changing world: A farm economic and EU perspective. In: P. de Jong (ed), Sustainable Dairy Production, (Wiley-Blackwell, London, U.K), 55 - 86
Jouzi, Z., Azadi, H., Taheri, F., Zarafshani, K., Gebrehiwot, K., Van Passel, S. and Lebailly, P., 2017. Organic farming and small-scale farmers: Main opportunities and challenges, Ecological Economics, 132, 144–154. https://doi.org/10.1016/j.ecolecon.2016.10.016.
M’Hamdi, N., Aloulou, R., Hedhly, M. and Ben Hamouda, M., 2009. Évaluation de la durabilité des exploitations laitières tunisiennes par la méthode IDEA. Biotechnologie, Agronomie, Société et Environnement, 13, 221–228. https://popups.uliege.be:443/1780-4507/index.php?id=3865.
Martínez-García, C.G., Dorward, P. and Rehman, T., 2012. Farm and socioeconomic characteristics of small-holder milk producers and their influence on the technology adoption in central Mexico. Tropical Animal Health and Production, 44,1199-1211. https://doi.org/10.1007/s11250-014-0724-0
Martínez-García, C., Rayas-Amor, A., Anaya-Ortega, J. P., Martínez-Castañeda, F. E., Espinoza-Ortega, A., Prospero-Bernal, F., and Arriaga-Jordan C. M., 2015a. Performance of small-scale dairy farms in the highlands of central Mexico during the dry season under traditional feeding strategies. Tropical Animal Health and Production, 47, 331-337.
Martínez-García, C.G., Janes Ugoretz S., Arriaga-Jordán C.M. and Wattiaux, M.A., 2015b. Farm, household and farmer characteristics associated with changes in management practices and technology adoption among dairy smallholders, Tropical Animal Health and Production, 47, 311-316. DOI https://doi.org/10.1007/s11250-014-0720-4
Martínez-García, C.G., Arriaga-Jordán, C.M., Dorward, P., Rehman, T. and Rayas-Amor, A.A., 2018. Using a socio-psychological model to identify and understand factors influencing the use and adoption of a successful innovation by small-scale dairy farmers, from central Mexico, Experimental Agriculture, 54, 142-159. DOI :https://doi.org/10.1017/S0014479716000703
Marchand, F., Debruyne, L., Triste, L., Gerrard, C., Padel, S., Lauwers, L., 2014. Key characteristics for tool choice in indicator-based sustainability assessment at farm level, Ecology and Society, 19, No. 3, Art. 46.
Moretti, M., de Boni, A., Roma, R., Fracchiolla, M. and van Passel, S., 2016. Integrated assessment of agro-ecological systems: The case study of the “Alta Murgia” National Park in Italy, Agricultural Systems, 144, 144-155. https://doi.org/10.1016/j.agsy.2016.02.007.
Prospero-Bernal, F., Martínez-García, C. G., Olea-Pérez, R., López- González, F. and Arriaga-Jordán, C. M., 2017. Intensive grazing and maize silage to enhance the sustainability of small-scale dairy systems in the highlands of México, Tropical Animal Health and Production, 49, 1537–1544.
Romo-Bacco, C.E., Valdivia-Flores, A.G., Carranza-Trinidad, R.G., Cámara-Córdova, J., Zavala-Arias, M.P., Flores-Ancira, E. and Espinosa-García, J.A., 2014. Gaps in economic profitability among small-scale dairy farms in the Mexican Highland Plateau, Revista Mexicana de Ciencias Pecuarias, 5, 273-290.
Salas-Reyes I.G., Arriaga-Jordán, C.M., Rebollar-Rebollar, S., García-Martínez, A., and Albarrán-Portillo, B. 2015. Assessment of the sustainability of dual-purpose farms by the IDEA method in the subtropical area of central Mexico, Tropical Animal Health and Production, 47, 1187 - 1194. https://doi.org/10.1007/s11250-015-0846-z
Sarandón, J. S., 2002. El desarrollo y uso de indicadores para evaluar la sustentabilidad de los agroecosistemas. In: Agroecología, el camino hacia una agricultura sustentable, (Ediciones Científicas Americanas, Buenos Aires, Argentina), 394-414.
Sedgwick, P., 2013. Statistical question: Snowball sampling, British Medical Journal, 347, f7511. https://doi.org/10.1136/bmj.f7511.
SMN-CONAGUA, 2019. Resúmenes mensuales de temperatura y lluvia. Servicio Meteorológico Nacional - Comisión Nacional del Agua. http://smn.conagua.gob.mx/es/climatologia. Accessed 20 Aug 2019.
Soldi, A., Aparicio-Meza, M.J., Guareschi, M., Donati, M. and Ortiz-Insfrán, A., 2019. Sustainability Assessment of Agricultural Systems in Paraguay: A Comparative Study Using FAO´s SAFA Framework, Sustainability, 11, 3745. https://doi.org/10.3390/su11133745.
Srour, G., Marie, M., and Abi Saab, A., 2009. Evaluation de la durabilité des élevages de petits ruminants au Liban, Options Méditerranéennes, Série A, 91, 21-35.
Tommasino, H., García-Ferreira, R., Marzaroli, J., and Gutiérrez, R., 2012. Indicadores de sustentabilidad para la producción lechera familiar en Uruguay: análisis de tres casos, Agrociencia Uruguay, 16, 166-176.
Vallance, S., Perkins, H.C. and Dixon, J.E., 2011. What is social sustainability? A clarification of concepts Geoforum, 40, 342 - 348. https://doi.org/10.1016/j.geoforum.2011.01.002.
Van Passel, S., Nevens, F., Mathijb, D. and Van Huylenbroeck, G., 2007. Measuring farm sustainability and explaining differences in sustainable efficiency, Ecological Economics, 62, 149 - 161. https://doi.org/10.1016/j.ecolecon.2006.06.008.
Vilain, L., Boisset, K., Girardin, P., Guillaumin, A., Mouchet, C., Viaux, P. and Zahm, F., 2008. La méthode IDEA: indicateurs de durabilité des exploitations agricoles: guide d’utilisation, 3a. Ed. (Educagri éditions, Dijon, France).
Webster, J.P.G., 1997. Assessing the economic consequences of sustainability in agriculture. Agriculture Ecosystems and Environment, 64, 95-102. https://doi.org/10.1016/S0167-8809(97)00027-3.
WCED - World Commission on Environment and Development 1987. Our common future, (Oxford University Press, Oxford, U.K.).
Zahm F., Viaux, P., Vilain, L., Girardin, F. and Mouchet, C., 2008. Assessing Farm Sustainability with the IDEA Method—from the Concept of Agriculture Sustainability to Case Studies on Farms, Sustainable Development, 16, 271–281. https://doi.org/10.1002/sd.380.
Zahm, F., Ugaglia, A., Boureau, H., Del’homme, B., Barbier, J.M., Gasselin, P., Gafsi, M., Guichard, L., Loyce, C., Manneville, V., Menet, A., and Redlingshofer, B., 2015. Agriculture et exploitation agricole durables: état de l’art et proposition de définitions revisitées à l’aune des valeurs, des propriétés et des frontières de la durabilité en agricultura, Innovations Agronomiques, 46, 105–125.
Zahm, F., A.A. Ugaglia, J.M. Barbier, H. Bourean, B. Del’homme, M. Gafsi, P. Gasselin, S. Girard, L. Guichard, C. Loyce, V. Manneville, A. Menet and B. Redlingshöfer. 2019. Évaluer la durabilité des exploitations agricoles, La méthode IDEA v4, un cadre conceptuel combinant dimensions et propriétés de la durabilité. Cahiers Agricultures 28: 5. https://doi.org/10.1051/cagri/2019004.
Acknowledgements
The authors express gratitude to the ten farmers and their families who participated in this study, whose privacy is respected by not disclosing their names. This work was undertaken thanks to funding by the Mexican National Council for Science and Technology (Consejo Nacional de Ciencia y Tecnología–CONACYT) through grant 129449 CB-2009 and the postgraduate grant for Estefany Torres Lemus and the postdoctoral grant for Fernando Prospero Bernal. Our thanks are also due to Ms Maria de Lourdes Maya-Salazar and Ms Laura Edith Contreras-Martínez for their assistance in laboratory analyses.
Funding
The Mexican National Council for Science and Technology (Consejo Nacional de Ciencia y Tecnología–CONACYT) through grant 129449 CB-2009 and the postgraduate grant for Estefany Torres Lemus and the postdoctoral grant for Fernando Prospero Bernal.
Author information
Authors and Affiliations
Contributions
Estefany Torres-Lemus (estefany_t_l@hotmail.com). Contribution: Investigation, laboratory analyses, writing—original draft.
Carlos Galdino Martínez-García (cgmartinezg@uaemex.mx). ORCID: 0000-0001-9924-3376. Contribution: Methodology, writing—original draft, review and editing.
Fernando Prospero-Bernal (fer_104_7@yahoo.com.mx). ORCID: 0000-0001-9109-1806. Contribution: Methodology, writing—original draft.
Carlos Manuel Arriaga-Jordán (cmarriagaj@uaemex.mx), ORCID: 0000-0002-6140-0847. Contribution: Conceptualization, resources, writing—review, editing and translation, supervision, funding acquisition.
Corresponding author
Ethics declarations
Ethics approval
The work with collaborating farmers followed guidelines accepted by Instituto de Ciencias Agropecuarias y Rurales (ICAR) of Universidad Autónoma del Estado de México.
Consent to participate
On-far work herein reported was carried out with ten collaborating farmers, who were aware of the objectives of the work, were duly informed, consulted and their decisions respected at all times, actively participated in the study, and their privacy and that of their families respected by not disclosing their names.
Consent for publication
Not applicable.
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix A-1
Appendix A-2
Appendix A-3
Appendix B. 2. Indicators not considered in the assessment of the sustainability of small-scale dairy systems by three methods.
RISE method
The only indicator not included was the greenhouse gas (GHG) balance, since there was no possibility to measure GHG emissions.
SAFA method
Rights and permissions
About this article
Cite this article
Torres-Lemus, E., Martínez-García, C.G., Prospero-Bernal, F. et al. On-farm assessment of the sustainability of small-scale dairy systems with three methods based on indicators. Trop Anim Health Prod 53, 208 (2021). https://doi.org/10.1007/s11250-021-02658-7
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11250-021-02658-7