Introduction

In 1989, Cyrille Fijnaut (1989:75) contrasted the ‘very small’ academic interest in organised crime with an increasing state and popular concern. The academic study of organised crime has, however, expanded considerably since the end of the Cold War: There now exist dedicated organised crime textbooks and handbooks, journals and research groups. Many undergraduate and postgraduate social science degrees have organised crime modules and there are a small number of postgraduate programmes dedicated solely to organised crime and more combined with terrorism studies.

While the field is growing, it has had very few systematic health checks. In the UK, when you turn 40, you receive an invitation from the British National Health Service to attend a routine health check-up to ensure your aging body is functioning as it should. The academic study of organised crime is almost 90 years old, if we start with Landesco (1929) and Thrasher (1927), yet its health has yet to be reviewed.

The aim of this paper is to assess organised crime research by systematically reviewing the data collection and analysis methods employed in articles published in the two main organised crime dedicated academic journals. This technique has a long history in terrorism studies (Schuurman 2018; Silke 2001, 2004, 2006) – a field which shares certain characteristics and methodological challenges (Windle et al. 2018) – but has yet to be explored in organised crime studies.

Organised crime research

The main function of research is to create new knowledge: to provide new insights and, greater awareness and understanding of a phenomenon. According to Colin Robson (1993), this research-generated knowledge can be categorised as: exploratory, descriptive or explanatory. Interpretative can be added as a fourth type. Exploratory research may be concerned with basic questions, often trying to understand what is happening by researching an emerging issue with little existing knowledgebase. It may also explore new data collection methods or sources. The next level attempts to describe what is happening, or what has happened, including who is involved and how the phenomena might work. Exploratory and descriptive research does not move beyond the descriptive to explain causation nor interpret meaning sufficiently. Exploratory and descriptive research often employs qualitative research methods, often relying on smaller case studies. The methods may not be overly concerned with issues of reliability and validity, as the primary importance is to set the scene. That is, to identify what may be the main forces at work. A second function may be to lay down a useful ‘mark in the sand’ for research grant applications or larger studies.

Explanatory research attempts to identify causes and sometimes to forecast what might happen in the future, often building on foundations set by more exploratory and descriptive studies. At this final level, the research methods used are more rigorous and more intensive than at the previous stages, partly because the researcher must address greater concerns that findings are clearly reliable and valid (Silke 2001). Interpretative understanding also goes beyond the descriptive, however, instead of seeking causes of a phenomena, the research will seek to understand the meaning of the action within a particular context (Bottoms, 2010). While explanatory research is often identified as quantitative, and interpretative as ethnographic, well designed and executed qualitative research can be explanatory and vice versa.

Klaus Von Lampe (2016:45), a leading expert on organised crime and former editor of Trends in Organised Crime, acknowledges that while a large body of ‘conceptually and theoretically ambitious research’ exists, ‘much of contemporary research on organized crime is primarily descriptive’. Von Lampe’s statement came 50 years after Donald Cressey (1967:102) noted that ‘social scientists have tended to write about organized crime only in descriptive terms’.

Research is an iterative endeavour. The aim of our field should be progression from one level of understanding to the next: Exploratory and descriptive studies are valuable for their capacity to inform future explanatory and interpretive works. All fields, however, require a significant number of studies at the explanatory level. Subject areas which fail to make the transition into explanatory are left with gaps in their knowledgebase, a fatal uncertainty over the causes of events and what are the truly significant factors at work, and may be constrained by a failure to predict future events. Such a field may, however, appear relatively active, especially in an applied field such as organised crime. Research that has a real-world focus will nearly always have outlets for exploratory and descriptive research.

It is acknowledged that organised crime can be a difficult topic to research. Cressey’s (1967) early observations on researching organised crime remains, more or less, intact:

The secrecy of participants, the confidentiality of materials collected by investigative agencies, and the filters or screens on the perceptive apparatus of information’s and investigators pose serious methodological problems for the social scientist who would change the state of knowledge about organized crime (101).

To this, one could add that researching organised crime can be costly, time consuming, potentially risky and present ethical barriers (see Galliher and Cain 1974; Hobbs 2000; von Lampe 2016; Windle 2018). At the very least it can be difficult to convince risk adverse research ethics committees that the research will not harm the researcher, participant or university.

William Chambliss (1975:36), however, suggested that criminology and sociology suffer a ‘myopic research vision’ about the possibilities of collecting data on organised crime. Drawing from his own ethnographic research experience, he argued that such data is ‘more available than we usually think. All we really have to do is get out of our offices and onto the streets’ (39).Footnote 1 While von Lampe (2016:50) simply highlights that previous research endeavours demonstrate that ‘there are no insurmountable obstacles for examining’ organised crime. Research may often need just a little more patience, planning, foresight and social capital than less clandestine and risky phenomena (see Felbab-Brown 2014; Hobbs 2000; von Lampe 2012).

Von Lampe (2002) proposed three problems haunting organised crime research. The first and second are interlinked: (1) the problem of definition and (2) conceiving organised crime as a distinct field. Neither will be discussed at length here, but conceptual confusion continues to mire the study of organised crime. Researchers cannot agree on what is meant by organised crime, organised criminal or organised crime group. Varese’s (2010) position that organised crime should be split into three categories (enterprise crime, organised crime groups and mafias) based upon governance of markets is not only a strong one but also demonstrates the breadth of the organised crime field: research subjects have ranged from solitary dodgy car dealers to well organised criminal fraternities performing quasi-state functions. Furthermore, there can be a fuzzy overlap between organised crime and other fields of study, notable street gangs, corporate crime and terrorism. It may, however, be that ‘organized crime’ is best thought of as representing an ‘open, multi-dimensional and dynamic concept to mark out a field of study’ (von Lampe 2002;195). In practice, the parameters of this field may well be set by the very existence of Trends in Organized Crime, Global Crime and the various organised crime textbooks. This article is not, however, concerned with conceptual debates, but rather research activity.

The third problem is that media, public and professional perceptions of organised crime – including ‘established facts’ and ‘common knowledge’- are often challenged by empirical evidence (von Lampe 2002; also Hobbs and Antonopoulos 2014). Indeed, a number of studies have shown how offenders and criminal justice practitioners perceive particular phenomenon quite differently (Decker and Kempf-Leonard 1991; Windle and Briggs 2015). This presents the researcher with two problems. First, policy makers, practitioners, the media and public may not want to hear alternative evidence and analysis which challenges their view (or agenda). This may lessen some researchers’ prospects of securing funding and having meaningful impact outside of academia. Second, it highlights the difficult issue of relying on data drawn from the state and media, a potential problem which will be returned to in the results section below.

The current paper is concerned with how data on organised crime is collected and analysed. The practical nature of research on organised crime. The ultimate aim of research is to arrive at a level of knowledge and understanding which allows us to explain why particular events have happened and, to predict the emergence and outcome of similar events in the future. It will be argued below that organised crime research has failed to attain such a level of knowledge. This article will examine what measure of responsibility for this failure rests with the activities of the research community itself, particularly in how it gathers data and in the level of analysis to which it submits harvested data. We do not argue that any one method is superior, but rather that for a field to thrive it should be generating new data, explaining rather than describing the phenomena under investigation and employing a range of methods.

Methodology

There are two primary ways of assessing the state of the art of organised crime studies: sending a questionnaire survey to active researchers, a method which has previously been employed in terrorism studies (see Schmid and Jongman 1988). A clearer method is to examine the published literature produced by researchers. This method has previously been employed by one of the authors to evaluate terrorism research (Silke 2001, 2004; see also Schuurman 2018). Only one organized crime study has employed a somewhat comparable method: over 40 years ago, John F. Galliher and Cain (1974) reviewed 102 Criminology textbooks (1950–1972) to identify sources cited.

The review of published research is feasible because there are currently two well-established peer-reviewed academic journals dedicated to primarily publishing research on organised crime: Trends in Organized Crime and Global Crime.Footnote 2 They have different publishers, separate editorial teams and largely separate editorial boards (though with some overlap). While we acknowledge much organized crime research is published outside of these two journals, taken together they provide a reasonably balanced impression of research activity in the field during the period under review. Peer-reviewed journals are considered particularly important as the peer review process acts as an important quality check to ensure that the published work meets minimum standards in terms of scientific quality and reliability. While peer review has its flaws, no alternative system has been developed so far which can exceed or even match peer review in terms of maintaining the quality of scientific literature. As a result both national research assessment frameworks and academic employment and tenure panels place a heavy emphasis on research publications in peer-reviewed journals, generally weighting them heavier than any other research outputs.

In order to better understand the current trends in research activity in the field for the past 14 years, this article presents the results of a review of the published output of the two primary journals in the area between 2004 and 2018. Each article published in the two journals for this period was reviewed (N = 463), this included introductions to special issues and book review essays but excluded erratum’s, book reviews and extracts from official reports. We acknowledge that some may have issue with the inclusion of book review essays and introductions to special issues, however, these articles tend to provide new knowledge and are often cited by other authors.

Following the categories in Silke's (2001) review of terrorism research, the following data was recorded for each article: full citation; first author; second author; first author’s institution; types of data source; data collection method; whether paper collected primary dataFootnote 3 or relied solely on a secondary document analysis; type (if any) of statistical analysis conducted. In addition to Silke's original categories, we also collected data on the number of research participants and whether the paper included a methodology.

Data was collected between May 2017 and May 2018, by the first author. The following procedure was followed for each article: The title, abstract and key words were initially read. The methodology (if available) was then read and the paper was skimmed for any tables or charts which would indicate statistical analysis. The reference list was then scrolled through to identify sources used. If methods remained unclear, then key words were used to search the article.Footnote 4

We recognise that categorising data collection methods can be subjective and another researcher reviewing these 463 articles may have categorised some methods differently. As with Bart Schuurman (2018:5) we utilised a ‘low inclusion threshold’ in order to avoid making subjective judgements about the quality of methods. For example, participant observation ranged from brief mentions of field work to in-depth ethnographies, while a pie chart was sufficient for the descriptive statistics box to be ticket in the dataset.

Limitations

Organised crime is a large and diverse field and we acknowledge that the two journals do not represent the entirety of organized crime research. Many of the most influential studies on organized crime have been published in other academic journals, edited volumes, monographs and research reports. The two journals do, however, represent a good sample of the literature. Most researchers with a significant interest in organised crime publish in Trends in Organized Crime and/or Global Crime at some point in their career. Furthermore, identifying and reviewing all published organised crime research would be an unmanageable task which would involve a degree of subjectivity in choosing what is and what is not organised crime. Here the parameters of organised crime have been set by the editors and reviewers of the two journals.

Results

Figures 1 and 2 provide breakdowns of the data collection methodologies and data sources currently being used by published researchers. A minor cause for concern with Fig. 1 involves the finding that 1.30% (N = 6) of articles published in the journals give no indication for the source of their information. Most of these articles were written by criminal justice or security practitioners, and the assumption is that the article is based on their own personal experience. It is, however, impossible to evaluate such articles in terms of reliability and validity, and as a result their value in research terms is contentious. This said, this type of paper became increasingly less common during the period under review.

Fig. 1
figure 1

Data collection methods in organised crime research, 2004–2018

Full size image
Fig. 2
figure 2

Data sources in organised crime research, 2004–2018

Full size image

Document analysis

Figures 1 and 2 clearly show that research on organised crime is dominated by secondary data analysis of open-access documents. Figure 1 shows that 64.1% (N = 297) of all research on organised crime is based solely on data gathered from academic literature, the media, state or non-governmental published documents.Footnote 5 Discouragingly, 56.5% (N = 262) of papers failed to include a methodology. Organised crime studies may legitimately be regarded as an interdisciplinary field of research and different academic disciplines can vary in terms of standards for the presentation of new research. Nevertheless, the lack of a methodology in so many papers raises concerns about the quality and reliability of much research. Given the heavy reliance on document analysis, important concerns may be raised over: the lack of detail on how documents were found, combined with an absence of discussion on the documents’ reliability and validity or assessment of the potential limitations to their use. Ultimately, as John Scott (1990) argues, documents should be subjected to the same principles guiding primary data collection.

Figure 2 shows that 96.7% (N = 448) of all published articles included at least one academic reference. The next most used source was open-access government documents (60%, N = 278) followed by, media and news sources (41.4%, N = 192). Conversely, the most common primary source - interviews with government sources –– was used in just 17.9% (N = 83) of published articles.

These figures seem to indicate a field overly dominated by ‘integrators of [the open-access] literature’ (Schmid and Jongman 1988:180). Most of the documents involved in these studies are open-access and neither classified nor accessed via government archives: just 4.3% (N = 20) of the 463 articles reviewed included an archived source.Footnote 6 While many more included state documents which may be closed - most often law enforcement and judicial documents - the access status of these documents was seldom apparent; and this basic information tells the reader much about the validity of the document.

Closed/restricted and archival sources can be difficult to access and, time and resource intensive. Historian Marc Trachtenberg (2007:147) has, however, argued that closed/restricted documents - which later become available in government archives - are ‘far and away the best’ document type. As confidentiality allows authors to express themselves more freely than they would in public they tend to be more reliable and less distorted than open-access documents. Closed/restricted documents are:

… generated for a government’s own internal purposes, and what would be the point of keeping records if those records were not meant to be accurate? It’s just hard to believe that a major goal … would be to deceive historians thirty years later … you can be reasonably sure that it’s not a pure fabrication (Trachtenberg 2007:147).

Of course, any researcher being given access to closed/restricted-documents needs to question why they are being granted access (see Cressey 1967), why these documents survived if others did not and whether they are being shown a representative sample of the available documents (Scott 1990).

There are a number of advantages to secondary data analysis. First, documents can be superior to other methods when investigating the past, if subjected to systematic scrutiny, for they represent the ‘traces which have been left by the thoughts and actions of’ the author without the limitations of hindsight (Langloid and Seignobos 1908:17; also Scott 1990). Second, document research and literature reviews can be cheaper, easier and less risky than primary data collection. Consequently, secondary data analysis is often used to research topics that are particularly difficult to gather primary data on. Its dominance in organised crime research is, therefore, not overly surprising. Third, document research can be valuable in establishing the wider context in which a phenomenon is occurring, in illustrating the potential complexity of the various factors that may be involved and establishing a foundation for future empirical research. Finally, documents can provide important ‘verifiable supplementary materials to evaluate and interpret more accurately’ data generated by other methods such as interviews and participant observation (Bernasco 2010:3).

As with all methods, there are disadvantages. In order to illustrate some of these limitations, the following section will consider the case of information gathered through media sources, the second most common source in organised crime research. As state-generated data is created through both secondary and primary data collection methods, this will also be discussed at length below.

Media and news stories as a data source

Documents from the media can be useful. Quality investigative journalism can produce very good and reliable data (Rawlinson 2008) on issues and in areas which academics may find hard to access, while news journalism provides the ‘only continuous public source of information about essentially secret subjects’ (Fijnaut 1989:77). As crime news is often based on police or court press releases, or underworld informants, it can represent a convenient conduit between the researcher and underworld or criminal justice system (Fijnaut 1989).

Nevertheless, there are three main concerns with research that draws heavily from media sources. The first is accuracy. News media reports tend to ‘produce a very superficial and very selective image’ (Fijnaut 1989:81). It is well established that even reputable media outlets frequently make factual errors in their reports, not to mention unintentional technical errors and the promotion of propaganda (Macdonald and Tipton 1993:191). The news media can also be heavily influenced by works of fiction and, use familiar terms and images which may not be a true reflection (Antonopoulos 2008).Footnote 7

The second issue is bias. Media reports rarely aim to be entirely neutral on any subject. Consequently, an element of distortion enters the coverage of any event or phenomenon. This distortion may reflect the commercial, cultural or ideological preferences and objectives of the proprietor, editor, journalist or even photographer. Distortion can also arise because of the need to compress a story into the available space and publish ‘newsworthy’ stories (see Jewkes 2004). For example, feuds between drugs dealing gangs are often over-reported due to their newsworthiness (actors are often given catchy nicknames, such as the General or Ice Man), while non-commercial and non-violent drug dealers are seldom given space in national or even local newspapers. This can skew the perception of drugs markets as overly violent. Distortions may also result from the journalists’ increasing reliance on press releases from governmental and intergovernmental bodies, and reciprocal relationships between the media and state. This can result in newspapers echoing official versions of events (Reiner 2000). As such, as investigative journalism becomes more difficult to conduct, and consequently less common, the usefulness of media reports may be declining.Footnote 8

A third concern with media reports is that of audience context. A researcher’s interpretation of an account is a social construct (Erikson 1973) and foreign language, technical terminology or unfamiliar terms, not to mention cultural norms, jokes and irony, may prevent an understanding of what the author was attempting to communicate (Scott 1990). In short, if the researcher is out of this loop, then serious misinterpretations and misapprehensions can be made (Silke 2001). Overall, while the media can represent a useful source, it is worth keeping in mind Yvonne Jewkes (2004:37) warning that the ‘media is not a window on the world, but a prism subtly blending and distorting our picture of reality’.

These are just some of the concerns with a reliance on media reports, but the same problems are inherent in documents created by state bodies, non-governmental organisations, private sector organisations and, organised criminals and groups (a particularly useful document source seldom used by organised crime researchers, see Fig. 2). As a result, there are concerns over the reliability and validity of research that depends heavily on such sources.

Some of the issues inherent in using documents can, however, be lessened by following the ‘quality control criteria’ developed by Scott (1990) to assess authenticity, credibility and meaning, coupled with Langlois and Seignobos’s (1904) criteria to identify distortions, and by triangulating documents from multiple sources (Windle 2016). Many of the studies reviewed for this paper may well have critically assessed the documents they used and compared multiple data sources, however, there is no way to tell as few papers employing secondary document research included a methodology.

Primary sources

As Fig. 3 indicates interviews are the most common primary data collection method, followed by participant observation, surveys and focus groups. The following section will discuss the strengths and limitations of interviews and participant observation.

Fig. 3
figure 3

Primary methods used in organised crime research, 2004–2018 (excluding document-reviews)

Full size image

Participant observation

Participant observation is a method of data collected involving the researchers’ observation of phenomena. This can range from shallower field work observations to in-depth ethnography whereby the researcher immerses themselves in their surroundings. The strength of participant observation is that it can provide a level of detail and nuance about an area, crime or group, within a broad social and cultural context, which may be missing in other data collection methods. Furthermore, by exploring the participants’ perspective, participant observation can humanise offenders and victims. The method has led to the establishment of typologies, which have then informed studies using other methods, and resulted in the collection of quantifiable data.Footnote 9 The primary weaknesses of participant observation are that, as a case study, it tends not be generalizable or replicable (see Antonopoulos 2008; Hobbs 2000; Hobbs and Antonopoulos 2014; Ritter 2006), immersion in the field and humanisation of participants can weaken researchers’ objectivity (Hobbs and Antonopoulos 2014) and the presence of the researcher can change the situation being observed. The limitations inherent in interviews (discussed below) also tend to apply to participant observation.

While organised crime research has a well-recognised tradition of participant observation, originating with Thrasher (1927) and other early Chicago School researchers, this review supports von Lampe’s (2012, 2016) observation that the method is actually infrequently used in organised crime research: just 5.6% (N = 26) of total articles mentioned some form of participant observation (Fig. 2) and the method was used in 16% (N = 26) of articles which collected primary data (Fig. 3). Moreover, just 50% (N = 13) of these articles involved observations of offenders and few directly observed offences. That researchers seldom witness actual crimes is not surprising considering the risk posed to both the researcher and participant (i.e. the researcher becomes both a potential witness to be called before court and an accomplice who could be prosecuted). The research objective is, however, often to observe the lived experiences and social processes around the crime and then use interviews to extract data on the offence itself.

This said, it is likely that the growing trend towards virtual ethnography will result in an upsurge in participant observation studies of organised crime. This method reduces some of the resource costs and risks associated with traditional participant observation. Further, the anonymity provided by the internet, especially via the ‘dark web’, may ease some of the concerns offenders and victims have about participating in research with academics from outside of their social network.

Interviewing

Of the 160 articles which used primary data, 79% (N = 127) included at least one interview (Fig. 3). Of the total 463 articles reviewed, researchers interviewed a combination of: state officials (17.92%, N = 83), non-state actors (12.5%, N = 58),Footnote 10 offenders (9.5%, N = 44) and victims (1.07%, N = 5) (Fig. 2).

This does not, however, mean that the article was primarily based on interviews, but rather that the researchers had conducted some interviewing in an effort to gather information. Several of these articles used interviews not as the main source of data but to supplement other sources, such as documents. This is often apparent in articles where interviews were referenced as footnotes and basic methodological information was absent.

In general, samples sizes are reasonable. Of the 70 articles which provided participant numbers, the median sample size was 47.Footnote 11 However, 35.7% (N = 25) had less than 20 participants and 61.4% (N = 43) had less than 40 participants (see Fig. 4). This is not particularly worrying: relatively modest numbers are to be expected in organised crime research and even very small samples can be fruitful (i.e. Sutherland 1937).

Fig. 4
figure 4

Number of interview participants in organised crime research, 2004–2018

Full size image

Of greater concern is that 34.6% (N = 44) of studies containing interviews failed to specify participant numbers and 62.2% (N = 79) failed to state whether the interviews were conducted in a structured, semi-structured or unstructured manner. One of the key tenants of quality research is that future researchers should be able to replicate the study. Replication is impossible without basic information on the number of participants and interview method. Some argue that qualitative studies are un-replicable, due to the individual people’s uniqueness, and do not need to be replicable due to being interpretative rather than explanatory. Even if this position is accurate, it remains difficult to assess the quality of the information provided when such basic methodological information is absent.

There are a number of advantages to using interviews as a way of gathering data. First, it is a very flexible method, especially in the unstructured or semi-structured form that dominates organised crime interviews. This allows interviewers to probe for additional information when interesting or unexpected avenues emerge. Second, interviews provide the researcher with a good measure of control. They can ensure that full answers are provided to specific questions; and if answers are not forthcoming, the researchers can judge whether avoidance of the question is a deliberate decision on the part of the respondent rather than an oversight. Interviews also tend to have good response rates and can produce a great deal of extra and unanticipated information. Indeed, supplementary information gathered in interviews can be valuable in establishing wider context and, can result in exciting avenues for future research and analysis.

Again, however, there are disadvantages. First, interviewing is an expensive method both in terms of finance and time, and, unless supported by large grants, researchers are usually severely limited in the number of interviews they can conduct: a problem exacerbated by the difficult target populations organised crime researchers must contend with and by the fact that organised crime researchers tend to work alone. Second, there is a risk of interviewer bias entering the data. The flexibility of interviews allows room for the interviewer’s personal influence and bias. For example, the way a particular question is phrased can influence the type of answer the respondent gives, as can the location, the interviewer’s skill and experience, and the recruitment method. The result is that the same individual, when interviewed by two different researchers about the same topic, could provide noticeably different answers. Indeed, the same researcher interviewing the same participant on different days may get noticeably different answers.

A third issue, common to all methods discussed above, relates to truthfulness and the validity of participant accounts. Wim Bernasco (2010:5) suggests that offenders ‘lie or misrepresent information’ for a ‘myriad of reasons’.Footnote 12 Henk Elffers (2010:14) proposes three broad ‘validity threats’:

  1. 1.

    Misinformation - participants pass on information they do not have access to because they are not well informed, did not observe the event or have memory problems;

  2. 2.

    Misunderstanding - the meaning of the question or answer is lost during the interviewFootnote 13;

  3. 3.

    Misleading - the participant misleads the researcher by ‘knowingly returning an incorrect answer’ (Elffers 2010:14), possible by giving replies which they think will please or not offend the interviewer, including exaggeration or downplaying of criminal activities.

This said, a benefit of interviewing is that the researcher can use tried and tested techniques to identify threats to validity, such as repeat interviews, participant observation and, triangulation with other participants and sources. Indeed, as Clifford Shaw (1930; cited in Bernasco 2010) observed, distortions can even provide important information if recorded and classified as such.

A further problem with interviews in organised crime research is that most studies employ opportunity sampling: many use pre-existing contacts or are referred to participants through information provided by law enforcement (von Lampe 2012, 2016), and then employ snowballing (Hobbs and Antonopoulos 2014). This means that the interviews are carried out with conveniently available groups or individuals with no systematic sampling. This is not entirely surprising: opportunity sampling is common when dealing with difficult to access groups or individuals and may often be the only option. This sampling method does, however, pose serious limits to the generalizability of findings to wider populations (Burns 2000:93). It can be difficult for researchers to know if they are dealing with biased samples that are noticeably different from the population of interest. The sample may be representative, but then again it may not. A consequence of this uncertainty, is that opportunity sampling tends to be limited to more exploratory and descriptive research in the social sciences. The dominance of the method in organised crime research, may therefore, raise a question mark over the reliability of the information being generated.

State-generated data

Figure 5 is a redrawing of Fig. 2 to highlight our fields overreliance on data collected from state agencies.Footnote 14 The influence of the state increases when we consider that 41.4% (N = 192) of papers used data collected from media sources and that many media sources are reliant upon press releases from the police and other state bodies (Reiner 2000). This can be compared with the offender perspective: just 9.5% (N = 44) of all papers interviewed an offender, 1.07 (N = 5) interviewed a victim, and 0.43 (N = 2) surveyed victims. To put this in stark perspective, more papers contained no sources than interviewed or surveyed victims. In short, while the state and media have a significant voice in organised crime research, the offender voice is hushed and victims are all but silent.

Fig. 5
figure 5

Data source of organised crime research, by type, 2004–2018

Full size image

Interviewing and observing offenders is useful as they are viewed by many as being the most knowledgeable informants (Zhang 2010) with ‘the richest source of information on their crimes and on their lives’ (Bernasco 2010:3). There can be little doubt either that the victim is able to provide the richest source of information on their lived experience of being victimised.

The difficulties of recruiting offenders to participate in research is often seen as a key reason for the dominance of state data (Zhang 2010). While victims are seen as even more ‘reluctant to talk to researchers’ (Hobbs and Antonopoulos 2014:96), some studies on human trafficking (see Kim et al. 2009; Tsutsumi et al. 2008) and terrorism (see Argomaniz and Lynch 2014) have interviewed or surveyed victims.

There are issues with using data generated by the state, whether gathered second-hand through documents or first-hand by interviews and surveys. First, some data is restricted to those with ‘connections’, notably researchers on government funded research (Cressey 1967) or those ‘well-respected’ by law enforcement or other state agencies (Fijnaut 1989). Second, state data can be limited by partiality (Hobbs 2000). For example, data and intelligence reports on drug production and trafficking have been manipulated to highlight the success of a programme or policy, to show increased trafficking in order to attract foreign aid and for diplomatic objectives, such as undermining an enemy on the international stage (Windle 2016). Closer to home, governments have - often alongside the media - perpetuated alien conspiracy myths to deflect attention from their own failing policies (Galliher and Cain 1974; Hobbs and Antonopoulos 2013, 2014). Related to this, law enforcement case files, a common source, are ‘rhetorical devices’ whose primary purpose is to secure conviction in court. As such, they tend to project an image of

… the one-dimensional “criminal” and feed the notion of an underworld of exclusively deviant intent, driven by economic motivation, yet drained of cultural context (Hobbs and Antonopoulos 2014:99).

Third, there are often gaps between police intelligence and organised crime realities (Windle and Briggs 2015). Law enforcement and other state employees perception of events can be influenced by the media (Decker and Kempf-Leonard 1991), political rhetoric (Windle and Briggs 2015) or may view the world from their own institutional and cultural viewpoint (Hallsworth and Young 2008; Windle and Briggs 2015).

Fourth, Elfers (2010) three validity threats, discussed above, apply as much to state employees as they do to offenders and ‘neither the reliability nor validity of what is said [by state employees] … should be taken for granted’ (Fijnaut 1989:80). Most practitioners, no matter how informed, only have a partial picture of the phenomena through fragmented experiences and there will often be elements of the picture they are unable or unwilling to disclose. Furthermore, law enforcement data can itself represent second-hand information from informants who may or may not have witnessed the event: The informant may be selective in what she tells the officer, who may be selective in what she tells the researchers or puts in the records to be read by the researcher.

The result of this overreliance on media and state sources can be the prolongation and dissemination of ‘unchecked folklore regarding organized crime’ (Galliher and Cain 1974:73) and the employment of ‘politically motivated’ and ‘constantly shifting’ conceptions of which activities are to be included as organised crime (Hobbs and Antonopoulos 2014:98). Whereas, what:

… is needed [of organised crime research], or at least appears to be desirable, is a concerted effort by interested scholars to confront media and politically induced imagery with well researched and sober analyses (von Lampe 2002:189).

Overall, Fig. 1 shows that researchers are very heavily dependent on easily accessible sources of data and only about 34.5% (N = 160) of articles provide substantially new knowledge which was previously unavailable to the field. Even when serious concerns exist with the manner in which data is collected, researchers can still take steps to address this when they progress to analysing the gathered data, and it is to this issue that the focus of this article now turns.

Analysing the data

From the 1950s, all social science disciplines experienced a rapid increase in the use of statistics. As people and groups of people are extremely complex, social science researchers typically have to work with very ‘noisy’ data where there are potentially a vast number of factors exerting an influence on any one behaviour, event or trend. Statistical analysis has emerged as a way for researchers to determine which factors genuinely are important and which are less so.

Descriptive statistics enable the researcher to summarize and organize data in an effective and meaningful way. Inferential statistics allow the researcher to make decisions or inferences by interpreting data patterns. Inferential statistics are regarded as particularly valuable as they introduce an element of control into research that can help to compensate for the use of relatively weak data collection methods. Moreover, inferential statistics provide an indication of how confident we should be that our results were not arrived at by chance, or methodological error, and whether they are important (statistically significant) or not. This provides insight into the generalisability and representativeness of our findings (Frankfort-Nachmias and Nachmias 1996).

Experimental designs are often identified as particularly powerful. Here control is normally achieved by randomly assigning research subjects to experimental and control groups. This can, however, often be very difficult to achieve in real-world research, and consequently the lack of control throws doubt on any association between variables which the research claims to find. Inferential statistics can help to introduce a recognized element of control, so that there is less doubt and more confidence over the veracity of findings (Frankfort-Nachmias and Nachmias 1996).

Figures 2 and 3 show that even though organised crime researchers tend to rely heavily on uncontrolled data-gathering methods – secondary document analysis, unsystematic interviews and participant observation - very little effort has been made to balance this by the use of statistical analysis.

Figure 6 puts the trend seen in Figs. 2 and 3 in a solemn context. It shows that from 2004 to 2018 just under 10% (N = 43) of research papers in the two primary organised crime journals involved the use of inferential analysis; and just under 70% (N = 322) had no statistical analysis of any type.

Fig. 6
figure 6

Statistical analysis in organised crime research, 2004–2018

Full size image

This article is not arguing that statistical analysis is superior to qualitative research, nor that it should dominate the field or feature in most piece of organised crime research. Rather we argue that, as it appears that organised crime research suffers a serious qualitative imbalance, greater effort is needed to address this imbalance. Statistics alone are not the way forward, but neither is avoiding their use to the degree that organised crime research apparently does. Indeed, the most effective, well-rounded and influential studies will often involve a mix of qualitative and quantitative analysis. Furthermore, research is an iterative process and qualitative methods can be productively employed to better understand and further explore quantitative results, and vice versa. For example, Elija Anderson’s (2000) ‘code of the street’, a theory derived from ethnography, has been subjected to a great deal of quantitative testing and exploration.Footnote 15

Of course, raw statistical data does not come out of thin air. The majority of quantitative articles reviewed here drew data from published official statistics. The limitations of official crime data are well documented (Bottomley and Pease 1994) and criminologists often propose victimisation surveys as a corrective. Although there are issues here also. For example, surveys often miss hidden populations (including those individuals most vulnerable to exploitation by organised crime) and the flexibility of the term organised crime can result in a lack of awareness by participants about whether they have been the victim of ‘organised crime’ or not (Hobbs and Antonopoulos 2014). Indeed, the local drug wholesaler or hard-man extorting businesses may not fit the public perception of predatory thugs in dark suits ‘whacking’ their rivals. While there are challenges in conducting victimisation surveys of organised crime, that the method has seldom been employed in organised crime research (for an exception see Tilley and Hopkins 2008) represents a cavernous hole in our knowledgebase.

Conclusion

It is difficult not to be pessimistic when presented with the above data. Some of the observations by Cressey (1967:102) are as relevant today as they were 50 years ago:

Social scientists have tended to write about organized crime only in descriptive terms, taking their clues from the reports on Congressional hearings, rather than in analytical terms.

On a more positive note, we can contest Cressey’s (1967:102) suggestion that the study of organized crime requires ‘methods not ordinarily utilized by social scientists’. The 160 papers which collected new data reviewed here used the full range of social science data collection and analysis methods. The question now is not whether typical methods can be utilized but rather why are some so rarely utilized?

From this review we can identify a number of key issues limiting organised crime research. First, while organised crime researchers have used a considerable range of data collection methods and sources, secondary analysis of open-access documents has overwhelmingly dominated the field. In addition, data analysis has been predominantly qualitative. The lack of statistical analysis may partly reflect both the absence of primary survey data collection and, the paucity and weakness of official statistical data on organised crime.

The second, and most damming limitation, is the overuse of data from the state and media, and underuse of offender and victim perspectives. The result is a field lacking balance, which appears to ‘obediently follow the beaten track of popular imaginary and official parlance’ (Von Lampe 2012:192). This is not to say that we should not interview state officials or use state documents and data. Indeed, many classic and scientifically rigorous studies have relied partially or solely on official sources: some of the strongest studies have triangulated accounts from a range of perspectives.Footnote 16 Nevertheless, for the field to thrive it must balance the official and media version of events with that of offenders and victims. In particular, the distinct lack of victimisation surveys may be the most pressing concern here.

Third, the field appears somewhat lackadaisical about scientific rigour. This is most apparent in the lack of a described methodology in over half of all published papers, coupled with the failure to elucidate interview type for the majority of studies involving some interviews, and the failure to specify the number of participants for many more. The lack of basic methodological discussion prevents readers from critically assessing the validity of the research or hinders attempts to replicate the study.

This article is not lobbying for any one method of data collection or analysis. A healthy field will employ a wide range of methods, both qualitative and quantitative. Nor is it trying to minimise the scholarly importance of document research or state-generated data (both authors have used these sources in their own work). Equally, empirical research or inferential statistics are not markers for success: poorly designed and executed research contributes little to the knowledgebase, and could produce unintended negative consequences. Our findings are not critical of any one published article, but rather of a field of research which when considered en masse appears unbalanced by a reliance upon a small number of methods and sources. Rebalancing the field requires more organised crime researchers to speak to offenders and victims, to employ greater use of statistical analysis and to apply more rigour to our methodologies.