Introduction

Although there is no an agreed definition of the term “socio-economic impacts” (SEI) in relation to genetically modified organisms (GMOs), its broad use involves the combination of different social and economic factors, with arguably the exception of particular ethical, health and environmental issues (Spök 2010). General SEI definitions adopted in country guidelines provide useful insight. For instance, the Interorganizational Committee on Principles and Guidelines for Social Impact Assessment (2003) of the United States of America describes social impacts as “the consequences to human populations of any public or private actions that alter the ways in which people live, work, play, relate to one another, organize to meet their needs and generally cope as members of society. The term also includes cultural impacts involving changes to the norms, values, and beliefs that guide and rationalize their cognition of themselves and their society” (p. 231). In this literature-review paper, we understand SEI of GM crops as both the direct and indirect changes in social and economic conditions from the introduction of agricultural varieties of modern biotechnology and their corresponding technological packages (e.g. necessary herbicides). Accordingly, multiple and cumulative effects, or trajectories, need to be considered, as well as case-by-case assessment.

The inclusion of SEI in GMO-biosafety policy-making (Bereano 2012; Binimelis and Myhr 2016; Catacora-Vargas 2012; Falck-Zepeda and Gouse 2017; Mackenzie et al. 2004; SCBD 2003), and in scholarly research (Falck-Zepeda and Zambrano 2011; Kleinman and Kinchy 2007; Stabinsky 2000) has been challenging. This is due firstly to the lack of a clear definition of what it constitutes as well as the different policy-making and research views on multiple aspects, such as scope, methods, indicators, and “endpoints” (Binimelis and Myhr 2016; Spök 2010). Second, to the general priority given to ostensibly precise scientific dimensions of knowledge for policy issues, which has left social dimensions of GM innovation neglected relative to economic assessments. One of the overall results of this is the limited quantity and quality of research and information in this field. Despite these challenges, GMO-SEI have received special attention in regulatory frameworks (COGEM 2009; European Environment Council 2008; Greiter et al. 2011; Spök 2010). By 2015, more than 40 countries from all continents have included them in their GMO biosafety legislation (Binimelis and Myhr 2016; SCBD 2014; Spök 2010).

Our analysis aims to identify and understand the present knowledge base on SEI of GM crops by reviewing the available relevant peer-reviewed scientific literature. For this purpose, our study takes into account both economic and social aspects occurring along the different phases of the full agricultural value-chain or at the aggregated societal level.

This research complements the few comprehensive and systematic reviews on this matter, such as the work of Smale et al. (2009), who analysed the economic studies literature providing empirical data from developing countries between 1997 and 2007, and Fischer et al. (2015), who analysed the impacts of GM crops at the farm level reported in peer-reviewed articles published from 2004 to 2014. In their work, impacts are classified in five categories: economic, distributional, access and ownership, wellbeing, and cultural heritage.

The following sections describe our methodological approach and findings, and offer an integrated analysis, quantitative and qualitative, of current gaps and methodological shortcomings in the literature reviewed. Finally, we suggest options for addressing existing knowledge gaps, biases, and methodological weaknesses in GMO-SEI research, to improve the relevance and quality of information for both regulatory and scholarly purposes.

Methodological approach

In this section are described the selection, organization and criteria for analysis of the papers included in the review.

Selection criteria

The peer-reviewed articles were selected according to subject matter and review status. For subject matter, selection focused on papers reporting, analysing or discussing social and/or economic impacts of GM crops based on empirical, methodological or theoretical research. In order to streamline the review to actual social or economic aspects of GM crops and on the understanding that public attitudes are not socio-economic impacts per se, articles on public perception were excluded, unless explicitly related to market demand, consumer choices, or other concrete GM crops/foods SE issues. Additionally, papers related to ethical analysis and regulatory processes were also omitted, unless dealing with concrete SEI.

Regarding review status, we concentrated on peer-reviewed journal articles published in English. We did this for three main reasons: first, the availability of research tools that allow comprehensive retrieval of such articles; second, the authoritative character of peer-reviewed literature; and third, the majority of internationally published research is in English. Despite our emphasis on English-language peer-reviewed articles, we acknowledge that they represent only part of the whole body of global literature, and that other publications (e.g. books and reports peer-reviewed or not, in English and other languages) based on solid research and analysis, are also relevant for informing regulation and research.

Sources of information

The articles reviewed were retrieved from Web of Science (formerly ISI Web of Knowledge) using the following terms: (i) related to the scope of the literature review, “soc*” or “econ*”; (ii) related to the general categorization of socio-economic impacts, “impact”/“risks”/“damage”/“harm”/“cost”/“benefit”; and (iii) subject “GMO”/“GM”/“transgen*”/“genetic engineer*”/“genetic modification”/“genetically modified”; all specific to agriculture.

Period of the retrieval

In order to identify articles on SEI of GM crops published before their commercial introduction in the mid 90s, the search did not impose a starting year. Hence, it included all possible peer-reviewed articles published by December 2014, with the last retrieval done on the 31st of January 2015.

Data extraction

A preliminary total of 1451 articles were selected from the original retrieval. From this, 410 fulfilled the selection criteria and were included in the review; nine were eliminated for being anonymous, and two were not accessible through our university libraries. The data-extraction was restricted to the papers retrieved by the search engine, and no additions of other articles was made, even when missing papers where identified. Inherently, this imposes limitations for analysing the full body of literature in SEI of GM crops. However, the total number of articles reviewed can be considered a sufficiently comprehensive sample. The analysis of each article was performed according to the following guiding criteria (Table 1): (i) disciplinary approach, (ii) type of article, (iii) crop, (iv) trait, (v) country, (vi) groups researched, (vii) scale of appraisal, (xviii) timing, and (ix) category and parameters of research. Three additional criteria were applied to ex-post empirical and econometric types of article: (x) period of time after the introduction of the GM crop when the research started, (xi) duration of research-observation, and (xii) comparator used. In order to record the information in the most comprehensive manner, the guiding criterion combined an a priori and an a posteriori categorisation of the possible findings. This approach aimed to show the multifaceted and mutually interactive complexity of possible SEI of GM crops.

Table 1 Guiding criteria for data collection, consolidation and analysis
Full size table

Data analysis

A posteriori grouping of the findings was performed, setting categories of data consolidation. The data extracted was arithmetically consolidated and graphically represented.

Finally, as an important remark on the methodology applied, we restricted our literature analysis to the generally accepted definition of SEI. The concept of “impact” implies a direct cause-effect relationship; thus, this focus within the agricultural value-chain overlooks impacts of a more complex and relational kind. For example: the marked concentration of global corporate control of R&D and innovation; greater concentration and reduced variety in seed and inputs markets; proliferation of private intellectual property rights culture; and related institutional rearrangements which have been encouraged by the advent of GM agrobiotechnologies (Ervin and Welsh 2006; Glenna et al. 2015; Rudy et al. 2007). These are vastly more far-reaching than the many case-specific ones that almost define the prevailing literature.

Results and discussion

Quantitative findings

Disciplinary approach and types of article available in the literature

The majority (55%) of the articles analysed have an economic focus, while 40% and 5% have a SE and exclusively social approach, respectively (Fig. 1). In terms of types of article, 42% are discussion papers based on secondary data or essays with no empirical evidence for their conclusions. Only 20% are based on first-hand empirical data. Econometric papers represent 38% of the total articles.

Fig. 1
figure 1

Distribution of articles reviewed according to disciplinary approach (social, socio-economic, or economic) and type of article (empirical, econometric, discussion or theoretical/methodological)

Full size image

Crops and traits researched

Of the total papers reviewed (410), 19% lack information on the type of GM crop or the trait researched (Fig. 2). Instead they refer to the crop in generic terms or using a broad trait-based description, such as GM crops or herbicide-tolerant crops, respectively. Another feature is the limited number of GM crops studied. As of mid-2016, the Biosafety Clearing House of the Cartagena Protocol on Biosafety (BCH-CBD 2016) registers 31 different GM crops (for food, feed and ornamental purposes). However, the published socio-economic research concentrates on only a restricted number of these: 81% of the 334 articles that specify the crop(s) studied refer to cotton, followed by maize, and soybean (Fig. 2). Consistent with Fig. 1, these top-three crops are mainly analysed from an econometric or discussion approach, and empirically to a much lesser extent.

Fig. 2
figure 2

Distribution of articles according to GM crops researched

Full size image

Of the 81% (334) articles which do specify the trait of the GM crop studied, 38% are discussion papers, followed by 35% econometric ones. Only 22% provide empirical information. From the total trait-specific articles, 17 (5%) deal with stacked-trait GM crops (Fig. 3), with only 1% (4) of these having empirical data. For the studies focused on single-trait GM crops, 79% (322 in total) relate to either insect-resistance (228 papers) or herbicide-tolerance (139 papers).

Fig. 3
figure 3

Distribution of articles reviewed according to GM traits researched

Full size image

Insect-resistance and herbicide-tolerance traits were introduced in the mid 90s, during the first generation of commercial GM crops. Since then, they have been the main focus of GMO SE research. Articles on GM traits intended for enhancing nutritional food-content are generally opinion-based and non-empirical (Asante 2008; Bouis 2002, 2007; Bouis and Welch 2010; Christou and Twyman 2004; Potrykus 2010). As noted above, new GM traits (e.g. stacked) either commercially available or in the R&D pipeline receive minimal attention in the SE literature (NASEM 2016), pointing to a knowledge-gap (and also of course, an inevitable lag-time in empirical research and publication, following cultivation) since these crops increasingly feature in agricultural production and regulatory discussions (Berger and Braga 2009; EFSA Panel on Genetically Modified Organisms 2012; Freese 2012; Nordgård et al. 2013; Taverniers et al. 2008).

Countries of research

From the total of articles reviewed, 65% (267) make explicit the country of research (Fig. 4). The remaining 35% either just provide a regional indication (128 papers), or no indication (15 papers).

Fig. 4
figure 4

Distribution of articles according to researched regions and countries

Full size image

The 267 total articles indicating the place of research include sixty countries, with different depths of analysis. Each article commonly refers to more than one country, with Asian (37%) and North American (31%) the most studied. However, the majority (78%) of such research is restricted to four: the USA (28% of total), India (22%), China (15%) and South Africa (13%). Most of these are econometric analysis based on secondary data, and the total empirical research into social, socio-economic and economic aspects is limited: 23 (9%) and 13 (5%) papers in the USA and India, respectively.

Groups researched

84% (340) of the articles indicate at least one type of group researched, with 10% being unclear (using generic terms only, such as “population” or “humankind”), and 3% giving no indication of the group analysed. The literature reporting the group studied concentrates on producers (77%) particularly in econometric (93 papers, 35%) and discussion articles (95, 36%), (Fig. 5). Among these the focus is on farmers in 261 papers (76%). For the period of retrieval of the literature, only one paper deals with beekeepers. The second most researched group is consumers (52 papers, 15%), followed by traders (6%). Studies on other actors involved in the value-chain of GM crops are scarce. A general sector that receives attention is the agrifood and agribusiness sector (14% of the articles analysed). In the whole body of the SE literature appraised, one paper alone relates to indigenous communities through a discussion type article (i.e. Satterfield et al. 2013). This is surprising, particularly in light of the 2007 UN Declaration on the Rights of Indigenous Peoples (UN 2007), and the Cartagena Protocol (SCBD 2000), which specifically indicates the relevance of SE considerations to indigenous and local communities.

Fig. 5
figure 5

Distribution of articles according to groups researched

Full size image

We further analysed the empirical and econometric research on farmers (23% and 36% of the papers reviewed, respectively, a combined total of 154 articles) in terms of the scale of the agricultural systems studied. The largest group (40%) refers in general terms to “small-scale” systems, and to a lesser extent to farmers engaged with “medium” (8%) and “large-scale” production (15%). Most of the papers provide no details on what these categories entail.

In relation to the scale of the farming system the percentages shown in Fig. 5 exceed 100% because it is common that each article deals with more than one type of farming system. A total of 53% articles (16% of the empirical and 37% of econometric papers) related to farmers provides no indication of the scale of the system analysed, resulting in no distinction in the agricultural system and the corresponding production dynamics where the GM crops are researched. This information is frequently absent: in 41% of empirical and 61% of econometric papers describing SEI on farmers (Fig. 5).

Scale of appraisal

Consistent with the findings on groups researched, the largest portion of the literature (48% or 195 papers) relates to farm scale impacts. Sectorial analysis (either agricultural production or trade) also receives attention (22% of the total). A restricted number of papers have a household (5% of the articles), or a community (2%) approach (Fig. 6).

Fig. 6
figure 6

Distribution of articles according to the scale of analysis

Full size image

Large-scale national and global level analysis (25 and 22% of the total articles reviewed, respectively) is common especially in discussion and econometric papers. Empirical research is limited: only eight articles were based on empirical research compared with 56 with an econometric approach, and 97 discussion-type articles. Hence, there is minimal empirical evidence on SEI of GM crops at national and regional scales, pointing to a key knowledge gap.

Timing of the research

The analysis of the timing of the research is applied to empirical and econometric types of article. From the former (empirical), 90% are ex-post; while from the latter (econometric), 53% are ex-ante. In both cases, the predominant papers are of economic type (Fig. 7).

Fig. 7
figure 7

Distribution according to the timing of the research in relation to the type of article and disciplinary approach

Full size image

Given their relevance in regulatory processes, ex-post empirical and econometric types of articles are analysed in relation to the period and length of the research, as well as the comparator used. Accordingly, these complementary examinations included a total of 133 articles (32% of the total articles reviewed).

Period of time after the introduction of the GM crop

Figure 8 shows that in the case of ex-post empirical research, 44% (of 133 articles) report short-term findings, from 1 to 3 years after the introduction of the GM crop to the area of study. Another 26% is research carried out in a mid-term period, from between 4 to 7 years after the GM crop introduction. Together these proportions underline the predominant short-term quality of the data, analysis and reporting, and leaves the potentially different medium or long-term effects quite excluded from examination (Stone 2011). As Stone implies, such research as published could be misleading for policy-making. Important part of the ex-post papers—empirical (19%) and econometric (55%)—does not provide information on the year of introduction of the GM crop in the area of research. The absence of this information makes the findings difficult to interpret.

Fig. 8
figure 8

Distribution of ex-post empirical and econometric types of articles according to time of research after introduction of GM crop

Full size image

Duration of the research

The analysis of the temporal duration of research is performed on all the 133 ex-post empirical and econometric types of article. Among empirical papers, 96% effectively report the years of research; however, a much lesser percentage (59%) of econometric articles provide this information, meaning that the remaining 41% of this category do not specify the length of the research. The majority of the articles specifying the duration of the research are of short-term after the introduction of the GM crop: 58% and 44% in empirical and econometric research, respectively (Fig. 9). The largest portion (20%) of articles missing this information follow the economic disciplinary approach.

Fig. 9
figure 9

Distribution of ex-post empirical and econometric type of articles according to duration of research

Full size image

Comparator

In this review, we analyse the comparator used in ex-post empirical and econometric research at farm scale (89 articles in total). From these, 74% (66 papers) are clear on the comparator, thus a quarter lacks this information (Fig. 10). Of the 66 papers making explicit the comparator, 91% use conventional agriculture while only a minimal portion indicates organic farming (6%), subsistence agriculture (8%) and other kinds of agricultural systems (3%). In this case, the percentages shown in Fig. 10 add to a number larger than 100% because some articles refer to more than one farming system. We discuss the implications of using conventional agriculture as the dominant comparator in the section on qualitative findings.

Fig. 10
figure 10

Distribution of ex-post empirical and econometric types of article according to comparator used in research

Full size image

Categories, criteria and indicators

From the literature reviewed, we find 72 different SE criteria, giving a total of 30 categories, which we group under five dimensions (Table 2), as proposed by the Ad-hoc Technical Expert Group on Socio-Economic Considerations of the Cartagena Protocol on Biosafety (SCBD 2014). This broad range of criteria has also been noted by Spök (2010).

Table 2 Categories, parameters and indicators reported in literature
Full size table

The majority of parameters analysed (82%) are economic, mostly comprising income/revenue, productivity, and production-costs. The second-largest dimension of analysis is the social (44% of the articles), which includes few parameters, particularly food and nutritional security followed by agricultural management. SE factors related to ecological and public health issues are covered in 28 and 20% of the papers, respectively. Cultural aspects are minimally studied (2%) (Fig. 11).

Fig. 11
figure 11

Distribution of articles reviewed according to dimension of analysis

Full size image

Qualitative findings

Mono-disciplinary dominance from a research bias towards neo-classical economics

In the literature reviewed there is a clear predominance of articles following a monetary economic approach (Fig. 1), mainly recognising direct changes to income, revenue, productivity and production costs (see Table 2; Fig. 11). These studies aim to measure effects on profits and savings for producers and consumers resulting from the adoption of GM crops. This restricted economic focus occurs more on econometric and discussion types of paper, being found less frequently in empirical research. The current SE literature’s domination by economic impact papers is also reported by Fisher et al. (2015) and Smale et al. (2009), who note their typical use of derivative (previously published) empirical evidence (e.g. Knezevic 2007; Klümper and Qaim 2014; Smale et al. 2009). Other non-economic SEI receive less attention, for example anthropological aspects of the agricultural introduction of GM crops (Stone 2010). This methodological economic approach is not only applied for SEI of GM crops, but broadly used in agriculture and related supply- and value-chains research, reflecting the hegemonic post-war paradigm of development economics (Leach et al. 2010; Lélé 1991; Vanloqueren and Baret 2009), in which capital- and technology/science-led modernisation of agriculture is essential to maximize profit-accumulation in order to invest in manufacturing (secondary sector) and then in services (tertiary sector) technologies and economic developments. Historically, this view has been integrated into local and regional cultural as well as agricultural and environmental conditions (Thompson et al. 2007), translating them into global systems of export distribution, trade and marketing. Under this rationale, the market model strongly favours economic values and parameters while the more complex, but substantive social and cultural factors and impacts are yet further excluded (for examples see DFID 2014; The World Bank 2008).

The prevailing entrenched narratives discussed above, of agricultural “modernisation”, made to mean GM as required driver of all further economic growth and technological innovation, therefore seems also to have shaped the core approach for GM crops SE scholarly work and publication (Vanloqueren and Baret 2009). This is reflected in framing of the research questions, methods, and interpretation of findings, even the definition of terms such as “beneficial/positive” or “adverse/negative” impacts (Brooks 2005; Thompson and Scoones 2009; Zadoks and Waibel 2000). Within this particular framing, productivity and monetary income-generation are presented as the sufficient criterion for appraising the possible SE advantages of GM crop adoption. This makes it difficult to achieve equal status for non-monetary and other broader socio-cultural factors, which are also relevant in the assessment and regulation of agriculture (Stone 2010, 2011; Dibden et al. 2013; Rivera-Ferre 2008), and other fields of technology assessment. This neglect of other relevant and complementary monetary and non-monetary factors creates knowledge gaps on many important SEI such as employment-generation, institutional arrangements, the agricultural matrix, land-tenure, financial independence and debt-alleviation, farmers-choice, autonomy, access to local seed varieties (other than GM), as well as SE impacts of pest-resistance, patent-restrictions and knowledge-concentration, among many others. Then, our findings show that current SE research of GM crops suffers important systematic conceptual biases and dominance of a mono-disciplinary approach rooted in neo-classical economics.

Under this general approach, SEI research of GM crops reflects a linear and deterministic logic: first, GM crops are presented (or justified) with an overestimated capacity for raising productivity by controlling specific biotic or abiotic stresses due to the introduced genetic change. Second, higher monetary income is expected and described due to the claimed increase in productivity or decrease in costs by controlling the production stress targeted by the genetic modification. Third, the higher income reported is assumed to reduce poverty and raise purchasing power, hence improvements in living conditions and food security, respectively, are expected. Fourth, GM cultivation—as other technological innovations—is presumed to be scale-neutral and to benefit equally small-scale and large-scale farming systems and their interlinked livelihoods (Fischer 2016). This common line of analysis has several failings (Brooks 2005). Among them is that multi-faceted SE impacts are framed and depicted as a one-dimensional matter of supply (either more production or more currency). Paradoxically, greater yield from GM crops has not been a goal of genetic modification (Heinemann 2009; IAASTD 2009), given that productivity is determined by a combination of factors beyond the crop’s genetic make-up, even less by a single trait alone (Zamir 2008). Moreover, the analysis of specific cases (e.g. Bryant et al. 2003; Pray and Naseem 2007; Qaim and Zilberman 2003; Spielman 2007), long-term trends, and comparative performance (e.g. Catacora-Vargas et al. 2012; Gurian-Sherman 2009; Heinemann et al. 2014; Hilbeck et al. 2013; Pretty 2001), indicates that GM crops did not generate long-term increases of productivity for first generation GM crops.

As mentioned before, the economic approach common in the literature emphasises econometrics (Fig. 1) restricted to data lacking direct empirical observation. The resulting measures forecast accumulated economic impacts of GM crops in the medium and long-term, particularly future production volumes, profits at various levels, effects on savings, and trade opportunities. As long as they are understood for what they are, such calculations are useful as a proxy for possible forthcoming scenarios. However, their common framing embraces methodological limitations that jeopardize their possible validity. Furthermore, their apparently precise quantitative appeal is misleading since most such econometric research relies upon untested assumptions and normatively weighted built-in choices. The limitations of such research include:

  • Questionable—and usually unacknowledged—assumptions about the conditions of the most marginalized (Glover 2010a), to whom non-monetary factors are often even more important;

  • Neglect of variable, interactive and changing contexts, since econometric calculations attempt to describe and quantify a non-existent equilibrium as well as static and implicitly standardized conditions (Leach et al. 2010), disallowing adjustment to variation and change (NASEM 2016);

  • Scarce attention to the human factors and human variations in differentiated temporal and geographical scales (Danish Council of Ethics 2012); and

  • Use of data which is not empirically observed, but is derived from hypothesis and secondary source-estimations. This results in what Phillips (2003, p. 24) described as “metaphors of reality, sometimes amounting to a very basic set of relations that are easily refuted by the data [from observation]”.

These shortcomings together with the absence of grounded empirical data make most of the current econometric research on SEI of GM crops of limited value for informing policy and decision-making on GM cultivation.

Research lacking analysis of the context and drivers of outcomes

Failure to consider and report the context in which GM crops are introduced and interact (from R&D to consumption) is common in the literature reviewed. This results in an unduly narrow perspective which omits aspects relating to: (i) the contextual factors influencing, for instance, seed-choice available to farmers; (ii) the rates and underlying drivers of GM crops adoption, such as new contractual relations controlling farmers’ decisions on seeds; (iii) the scales and management affecting GM crop yields in the short and long term; (iv) the structure and dynamics of markets foreseen to absorb the GM harvest; (v) the institutional networks that influence and control the production, trade, and commercialization of GM crops; (vi) the conditions and circumstances in which GM crops could influence food-access and nutrition; and (vii) the kinds of market (for example, global commodity or local food) for which crops are envisaged and controlled; among other overlooked aspects.

In contrast with the narrow perspective of most SEI research, some studies do include historical or contextual analysis. For instance, empirical research reported by Binimelis (2008), Glenna et al. (2011), Glenna et al. (2015) and Glover (2010a, b), and analytical papers by Adi (2006), Ervin et al. (2011), Ervin and Jussaume (2014), and Mascarenhas and Busch (2006), among some others, adopt a relational and institutional approach describing the technological and SE context of adoption of GM varieties in specific cases, showing the interplay of several and diverse local drivers in creating impacts. This allows acknowledging the complex and long-term realities, and the multiple interacting factors, which play into real outcomes and findings. It also provides relevant information for identifying policy alternatives. This perspective accommodates synergistic and cumulative interactions in the assessment perspective, which are generally missing from economistic SEI studies (Felt et al. 2007; Harremoës et al. 2001; Wynne 2005). Generally speaking, such contextual factors are absent because “scientific” risk assessment calls for quantitative precision. Yet, in real-world complexity, precise measurement requires exclusion, reductionism and short-termism. Therefore, precision as a scientific norm in risk or impact assessments often conflicts with comprehensiveness or realism—also legitimate scientific criteria (Wynne and Stirling 2007). A comprehensive science (including social science) for policy would acknowledge this inherently normative epistemic tension, and inclusively deliberate its resolution case-by-case. Thus, the more integral and multi-factorial studies, which are missing from SE research of GM crops, may have compromised apparent precision, but for the benefit of greater realism and reliability.

The lack of context-description in the literature analysed often results in the suggestion of universal and static circumstances of GM crops-introduction, -adoption, and -use. However, the generalizability of such empirical data is questionable. This is illustrated by Glover (2010a), who examines the institutional context of Bt cotton adoption in South Africa, previously reported by Gouse et al. (2005). Glover notes that a local seed company and a credit agency facilitated the initial adoption of Bt cotton by providing free technical, financial and marketing support to producers, resulting in the high adoption-rate by small-scale and poor farmers (Glover 2010a). Yet, adoption stopped when the “contextual” support was suspended. Gouse et al. (2005) claims that this situation can be considered as a technical triumph but an institutional failure, ignoring the basic point that technology adoption and apparent success was highly driven by particular institutional conditions. This example illustrates how including the historical and contextual aspects of the GM crop uptake would allow better understanding of the actual factors (e.g. availability of credits, affordable inputs, access to markets) (NASEM 2016) influencing farmers’ decisions. This is particularly important considering that most of the literature reviewed mistakenly assumes that farmers’ choices are fully informed and influenced by free market factors alone.

Given that the outcomes of GM crop-introductions—as for any other agricultural innovation—are context-dependent because they are part of the fuller socio-technical environment, they are subject to and generative of multiple interactions and pathways (Asdal and Moser 2012; Herrero et al. 2015; Leach et al. 2010; Pavone et al. 2011; Thomas et al. 2008). Corresponding with the co-production thesis of science and technology studies (Jasanoff 2004), this indicates that: (i) it is not possible accurately to determine the trajectory and socio-economic impacts of a specific GM crop introduction a-priori; and (ii) as mentioned previously, the exclusion or over simplification of the context analysis results in debatable conclusions, as it allows exaggeration of the generalizability of what are localized and conditional findings. Consequently, the validity of the reported findings and of conclusions may be limited, if researchers do not acknowledge the interrelations among impacts attributed to the GM crop, and the diverse situational drivers as well as local circumstances.

Use of conventional agriculture as “universal” comparators

In all fields of research on technology-impacts, the selection of the comparator or, when applicable, “null-condition”, is crucial for the generation and validity of the results, particularly for defining change (Spök 2010; COGEM 2009; Kaphengst et al. 2011; Smale et al. 2009; Mannion and Morse 2013). Because of this, the formulation of the research question and the corresponding methodology should involve a cautious choice of the study’s comparator for identifying the effects of the agent of interest, here GM crops. This selection also needs to be transparent and justified by reference to alternative options. A thorough scientific study would also conduct a sensitivity analysis of the findings to variation in the chosen comparator. However, this is rarely found in the SE literature of GM crops.

As shown in Table 1 and Fig. 10, the GM-SEI literature analysed refers mostly to conventional agriculture as comparator, to a limited extent to organic and subsistence farming, and there is no consideration of agroecology and integrated pest management. This finding is also consistent with the analysis of the National Academies of Sciences, Engineering and Medicine of the United States (NASEM 2016). The problem here is the lack of comparative analysis between truly dissimilar production systems, which can be distinguished as follows: Conventional agriculture is in general terms a monocrop production system that relies on a technological package composed by commercial non-GM seeds, synthetic inputs (mostly fertilizers and pesticides), fossil fuels and different levels of mechanization. Integrated pest management (IPM) is an approach introduced in some conventional agricultural systems that first relies on managerial, physical and biological methods for pest control, using synthetic pesticides as a last resource. Subsistence agriculture (mostly found in indigenous and traditional rural communities) is a farming systems exclusively intended to supply food to the farmer’s household with little, if any, surplus for selling except for only very local exchange. Organic farming replaces synthetic inputs with biological ones, excludes GM seeds, and in different degrees may involve natural resource conservation practices. Agroecology, as production system, excludes external systemic inputs and GM seeds, and focuses on the design and management of the whole agroecosystem to enhance social-ecological functions that guarantee stable and biodiverse yields.

Thus, a fundamental limitation of the majority of the SEI of GM crops literature having conventional agriculture as comparator is the overestimation of possible benefits, masking the substantial differences with other non-conventional and non-GM based agricultural systems, especially when sustainability is a key analysis criterion. For instance, GM crops are usually introduced in industrialized mono-cropped conventional agriculture, which relies on singular pesticides for controlling weeds and pests, but facing self-induced development of resistant species. Consequently, it is expected that adoption of an herbicide-tolerant or insect-resistant GM crop will have positive short-term impacts by temporally decreasing the use of specific agrochemicals; however, as reported in the literature (e.g. Pemsl et al. 2011; Powles 2008; Wang et al. 2008; Benbrook 2012, among many others), this may turn into negative long-term consequences due to emergence of secondary effects (i.e. resistant weeds, insects or other non-target organisms). Accordingly, positive changes in pest management—with the corresponding economic savings and environmental advantages—from using a GM crop are time-dependent due to the biological dynamic within the agroecosystem: the new GM trait introduced into a highly simplified system as conventional monocrops (Heinemann 2009) becomes an advantage until the biological interactions surpass the artificially introduced agronomic characteristic (Altieri 2005). Pursuant to this, Pretty (2001) illustrates the relevance of including different comparators in GM crops research for opening-up the spectrum of alternatives that may be more effective, particularly in the long term. He reports lower effectiveness of insect-resistant GM crops when compared with integrated pest management, organic and agroecological farming because the co-evolving ecological inter-relations in biologically complex agroecosystems are more ecologically successful over time than GM and chemical control (Altieri and Nicholls 2003).

These technical examples illustrate that although comparisons between conventional and GM-based farming have value, they prevent policy-making from identifying or testing long-term alternatives for different pathways of food production beyond business-as-usual. In conclusion, the use of conventional agriculture as universal comparator is inadequate, and can lead to biased results, portraying GM crops as “the solution” where no other alternative form of agriculture than high external input agriculture is deemed possible (Brooks 2005; Zadoks and Waibel 2000). The exclusion of alternatives in agricultural research, nonetheless, is not restricted to the study of the SEI of GM crops. They have been systematically ignored in mainstream agricultural and food R&D, shaping a technological and economical regime that favours industrialized types of production system (Glenna et al. 2011; Vanloqueren and Baret 2009).

Knowledge generated mostly by short-term studies

Most of the published empirical research on SEI of GM crops has a short-term approach, up to 3 years (Fig. 7). This is consistent with other reviews (e.g. NASEM 2016) that indicate the dearth of long-term studies.

As indicated above, short-term analyses have limitations for understanding the impact-trajectory of GM crops, particularly in relation to: (i) the time-sensitivity of social-ecological parameters, such as resistance-emergence, income-generation, productivity-change, production costs, volume of pesticides applied, etc.; and (ii) the cumulative and combinatorial interactions within dynamic and complex systems when a change is introduced, which become manifest only in the longer term (Leach et al. 2010; Pavone et al. 2011).

The literature also exposes inadequacies in long-term econometric analysis that uses only data from short-term studies on the performance of GM crops. This is true especially for the most common aspects characterized in SE research (Table 2; Fig. 11), namely productivity, production costs and reduction of herbicides (Benbrook 2012; Catacora-Vargas et al. 2012; Gurian-Sherman 2009; Heinemann et al. 2014). For example, Areal et al. (2013) use short-term data collected on various GM crops in different countries in an econometric analysis that indicates a decrease in pesticides use, among other effects. Conversely, Benbrook (2012), using records on herbicide use in six major herbicide-tolerant and Bt GM crops in the USA from 1996 to 2011 (16-year period), concludes that the adoption of these GM crops has led to a 239 million kg increase in herbicide use in the USA over the period of analysis, while Bt crops have reduced specific insecticide applications by 56 million kg. On balance, this country’s pesticide use increased by an estimated 183 million kg. This underlines the need for empirical studies and data collection over longer time frames, and also the more careful use of short-term date for long-term analyses.

Application of untested assumptions and extrapolations

As explained previously, socio-technical systems are complex, dynamic and context-dependent (Leach et al. 2010). Because of this, prognostic methods using assumptions, and exporting localised results without proper contextualization of research and findings, are unrealistic in most prevailing SE research of GM crops, and in technology assessment in general (Callon and Law 1982; Flyvbjerg 2005; Rip 2002).

Assumptions are central to any paradigm, theory and research. They are instrumental in the framing of any study, in the selection of methodologies, argumentation, evidence-generation, and in interpreting findings and reaching conclusions (Nkwake 2012).

In general, we identified two styles of assumption in the SEI research of GM crops, reflecting the types of scholarly work they served:

  • Assumptions in quantitative research used for setting the scene of appraisal and impacts of GM crops. These are mainly applied in econometric analysis. For example, Anderson and Jackson (2005) apply an econometric model to analyse the potential effects of GM crop-adoption in Sub-Saharan Africa in relation to the EU market. The model assumes non-realistic conditions such as perfect competition; no negative environmental risks associated with GM crops; no loss or reduction in international market; improvement of countries’ reputation as producers of clean, green, safe food upon adoption of GM crops; and full employment of all productive factors. Moreover, the authors conclude that adoption of GM crops will generate significant gains, particularly to the mostly poor countries. Yet no reflection is provided on the validity of these founding premises and their possible impact upon these assertions. Additionally, the analysis includes a limited group of countries, among them some outside the region of study (Sub-Saharan Africa) and not regarded “as mostly poor”, such as Argentina and China. The conclusions also contain statements on health and improved economic performance although these parameters were not analysed.

    Another questionable assumption is described by Glover (2010b), concerning the quantification of farmers’ labour when studying cost-benefits from GM crop adoption in small-scale production systems. In those quantifications, a decrease in labour costs is frequently reported as result of the reduction in pesticide applications with the cultivation of GM herbicide tolerant and insect resistant crops. However, in reality self-sustained farmers from small-scale production systems in developing countries are not self-paid for their working hours. Therefore, reduction in working time does not imply actual economic savings and higher income, particularly in contexts where employment opportunities to occupy and reward the “freed” time are scarce.

  • Assumptions in qualitative analysis when describing large-scale impacts. These are common in discussion papers, although also found in the framing and conclusions of other types of quantitative and qualitative empirical economic and econometric SE research. For instance, Christou and Twyman (2004) and Park et al. (2011) make untested and over-emphasized assumptions about the singular capacity of GM crops to solve complex SE and geo-political issues, such as hunger, poverty and sustainability. This kind of analysis is often connected to the notion that increased productivity and income is distributed equally between different scales and types of farms and households, without considering SE, environmental and even managerial differences that influence productivity, income generation, resources distribution, and livelihood decisions. Additionally, productivity and monetary income generated by GM commodity crops are presumed to be distributed equally, to remain in and thus come from local markets. Such assumptions, according to the deterministic view described in a previous section, are frequently interpreted as a factor of poverty alleviation, although no such evidence for this is provided in the relevant papers. Critical analyses of these assumptions are presented by several authors (e.g. Francescon 2006; Jansen and Gupta 2009; Richards 2010; Walters 2006). Other common unquestioned presumptions are that higher purchasing power will automatically generate more food-access, and that undernourishment is only an issue of nutrient intake. Articles on bio-fortification follow this technological determinism (e.g. Bouis 2007) by proposing that health problems (including mortality), cognitive capacities and economic possibilities will improve with only the availability and consumption of GM nutritionally enhanced food.

    In parallel with such problematic underlying assumptions are untested extrapolations of results from observation in the original conditions studied at a specific point in time, into broader and more general prospective outcomes. This involves the implicit assumptions of uniformity and unchanged conditions over time, when it is widely recognized that the social-ecological contexts of GM crop-introduction and -use comprise a large variety of structures and dynamics and, accordingly, also of effects (NASEM 2016). These assumptions allow frequent extrapolations of results beyond the original sample size, locality, crop-type, period of research and others. This is a particular problem for the economic and econometric studies. Exaggerated generalizations of specific research findings mainly occur from small-scale and short-term studies to large-scale and long-term contexts (for instance, Barwale et al. 2004; Mugo et al. 2005; Qaim 2003, 2005); and from specific farm-(e.g. Kolady and Lesser 2008) and field-trials (e.g. Espinoza-Esquivel and Arrieta-Espinoza 2007) to national and regional conclusions. Such papers usually include normative policy recommendations based on their untested assumptions and misleading results.

Unclear methods and incomplete results

A significant proportion of the literature reviewed does not report or is not clear on the object or criteria for analysis. For example: GM crop (19% of the articles, Fig. 2); trait (21%, Fig. 3); country of research (35%, Fig. 4); group researched (13%, Fig. 5); time-period after the introduction of the GM crop (19% of ex-post empirical and 55% of ex-post econometric articles, Fig. 8); duration of the research (4% of ex-post empirical and 41% of ex-post econometric articles, Fig. 9); and comparator used (26% of ex-post empirical and econometric research, Fig. 10). Without better information about the object or criteria of analysis the quality of the findings reported is weak. More specifically, these omissions obliquely assume that the characteristics and potential impacts of whichever specific GM crops and traits studied are interchangeable across diverse countries, crops, groups, and temporal and farming conditions. The need for clear description of the details of the methods applied is therefore evident.

These problems are portrayed, for example, in the article entitled “Yield effects of genetically modified crops in developing countries” by Qaim and Zilberman (2003) that reports significant increase in yields and decrease in pesticide use from on-farm field trials of Bt cotton in different Indian states. Confused by the results and the missing information in the original article (e.g. the Indian states where the on-farm and the non-Bt counterpart field-trials were conducted), Arunachalam and Ravi (2003) report the following shortcomings identified after direct communication with the paper’s corresponding authors: (i) the empirical sample is not restricted to on-farm field trials as indicated in the article, but includes additional production plots randomly chosen without consideration of the agroecosystem differences; (ii) the major Indian cotton-producing state is not included in the research, raising questions on the applicability of the findings to national cotton production; (iii) variations of the net plot area are masked by calculating yield based on uniform gross plot size; (iv) the comparators are chosen deliberately by farmers, most of them unknown to the authors; (v) high variations of management between the production plots and the controlled on-farm field trials are not taken into account, yet the authors applied statistical analysis to all plots together, as if they had received the same treatment; and (vi) as a result of the foregoing factors, the coefficient of variation reported is very high, invalidating the results. Despite these methodological limitations, the conclusions are extrapolated to the whole of India and developing countries.

Conclusions

Our review exposes limited empirical SEI research (particularly on social impacts) of GM crops, and an imbalance in knowledge and framings used. This effectively defines the realities influenced by the introduction of GM crops according to few selective economic parameters. Other shortcomings are the focus on short-term studies; lack of contextual analysis; the application of untested or unjustified assumptions and extrapolations; incomplete information on relevant research parameters; and, the use of conventional agriculture as the “universal” comparator, which masks other alternatives, both for research and policy. Hence, although there are some critical studies that assess more broadly defined SEI of GM crops, the mainstream literature is methodologically predetermined to favour agricultural innovations designed on neo-classical economic principles and to reproduce large-scale industrial agriculture systems linked to external markets. Thus, excluding agricultural and food alternatives that involve a wider spectrum of considerations than monetary economics alone.

The limitations identified force us to reflect on what the existing literature cannot tell us, with the aim to offer some warning against over-reliance on the current mainstream SEI scientific literature in further research and for policy-making. While many of these SEI literature deficiencies may be justified by inadequate research resources (including funding and timeframes) allocated to such studies (NRC 2010), this does not excuse the methodological limitations described above, nor the presumption of extrapolation from particular conditions to general projections. Probably the most substantive shortcoming is the economistic bias at the expense of more attention to social dimensions and effects (Fischer et al. 2015; Smale et al. 2009). This deficiency is worth paying attention to considering that technical innovations, apart from being economic, are always also social (Thomas et al. 2008), even if this is neglected by the continuing prevalent demand for exclusively quantitative measures for describing reality.

The neglect of attention to social aspects of GM crops in the SEI research, especially in the medium and long term, creates a crucial knowledge-gap for drawing reliable conclusions; understanding the systemic effects of GM crops on the food systems and related institutional dynamics; and, consequently, for the identification of alternatives. This is particularly relevant when the more authoritative quantitative “scientific” form is achieved by standardisation and aggregation, and by negating substantive and observable non-quantified factors (mostly social and cultural) that defy such manipulation (Glover 2010a). This selectivity is a typically neglected normative dimension of what are assumed to be the objects of scientific attention. This in turn facilitates a mono-disciplinary epistemic culture, where the excessive extrapolations of empirical quantitative findings from limited sites, conditions and time-periods are made mistakenly to stand as universal realities.

The above is connected with the dominance and limitations of a case-by-case approach for assessing impacts in general. While this is a necessary part of overall SEI research, just as for GM ecological risk assessment, it is not alone an adequate approach, since single cases cannot cover interactive, multiple or cumulative impacts. For these, trajectory-type analyses are necessary (Pavone et al. 2011), in addition to or integrated with case-by-case studies.

The problems in the methodologies and in the corresponding reported results of the SEI literature on GM crops have biosafety political and policy implications. Among them is the ubiquitous ontological as well as epistemic a priori bias in the dominant agri-food and rural development model, which assumes that so-called “modern”, industrial and highly “technified” agriculture deserves exclusive promotion because it is taken as more productive than other agricultural systems (WEF 2012). Even though this dominant worldview has been critically assessed (e.g. Hobart 1993; Scott 1998), its overriding influence persists, in part due to the related demand from policy, commercial and academic hierarchies for quantitative and universalist data. Accordingly, the concentration of SEI research on only certain types of GM crops and impacts together with the overestimation of their possible benefits, decreases the possibility of identifying, testing and potentially strengthening alternative socio-technical pathways for agriculture and the food systems (Stirling 1999; Vanloqueren and Baret 2009). The biases we identify in dominant forms of SEI research also tacitly promote a corresponding policy and political-economic bias.

Unavoidably, these final remarks raise the biosafety normative questions that have been buried in the papers reviewed, because this literature often includes such normative choices and commitments without explicitly acknowledging them. In a policy system rightly aspiring to be informed by reason and evidence, such commitments cannot be legitimate unless transparently disclosed, tested and debated. Thus, scientific peer-review production requires that authors recognise and make explicit their normative assumptions, and leave them open to deliberation and further research. Above all, we argue that the large sample of existing peer-reviewed literature on SEI research for GM crops has systematically reported policy conclusions without enough properly qualified empirical data. This becomes evident from specific examples as: normative recommendations based on short-term increases in yield extrapolated to identical long-term conclusions; short-term decreases in herbicide-uses presented as constant over the long run; reduction of labour equated automatically to lasting reduction in costs analysed from the economic benefit lens without considering the inherent social costs; the commercial monopoly over seeds portrayed as economic competitiveness only, without careful attention to the associated contractual dependencies and the surrounding political economy.

Adequate SEI scientific practice related to GM crops will require acknowledging the limitations of single-discipline economic, econometric and related methods, and—even when social dimensions are investigated—the short-term quality of most current research. To advance on this, towards more realistic in-context trajectory evaluations (Ely et al. 2014; Herrero et al. 2015; Leach et al. 2010; Pavone et al. 2011) a key step would be to overcome the inconsistency of appraising long-term global development goals (e.g. hunger- and poverty-reduction) by using only short-term studies (Ervin et al. 2011). By doing this, SEI research, publication and debate will develop more legitimate authority for itself, contributing also to identifying and answering further biosafety policy-relevant research questions, such as “what are the real social and economic effects of GM crop-adoption?” “on which groups and in which ways?”, “under which local conditions?”, “for how long time?”, “who gets excluded and why?”, “what are the indirect social and environmental costs?”, and perhaps most crucially, “could GM crops bring real, sustained social benefits if governed and developed under a different political economy?” Addressing these questions will also require public and open deliberations with a broader range of informed policy actors and stakeholders than has hitherto prevailed.