Abstract
The use of Big Data in the law enforcement sector turns the traditional practices of profiling to search for suspects or determining the threat level of a suspect into a data-driven process. Risk profiling is frequently used in the USA and is becoming more prominent in national law enforcement practices in Member States of the European Union. While risk profiling creates challenges that differ per jurisdiction in which it is used and vary along the purpose for which the profiling is deployed, this technological development brings fundamental changes that are quite universal. Risk profiling of suspects, or of large parts of the population to detect suspects, brings challenges of transparency, discrimination and challenges procedural safeguards. After exploring the concept of risk profiling, this chapter discusses those fundamental challenges. To illustrate the challenges, the chapter uses two main examples of risk profiling: COMPAS and SyRI.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
Zouave and Marquenie 2017.
- 3.
Fundamental Rights Agency 2018.
- 4.
Fundamental Rights Agency 2018.
- 5.
Mittelstadt et al. 2016.
- 6.
- 7.
Marks et al. 2017.
- 8.
For example predictive policing software was first introduced in the USA before it was used in European countries.
- 9.
- 10.
Hildebrandt 2008, p. 23.
- 11.
Hildebrandt 2008, p. 19.
- 12.
- 13.
- 14.
Swedloff 2014.
- 15.
O’Neil 2016.
- 16.
Van Brakel 2016.
- 17.
Mittelstadt et al. 2016.
- 18.
Clavell 2016.
- 19.
Clavell 2016.
- 20.
Van Brakel 2016.
- 21.
With the use of PredPol software.
- 22.
Besluit SUWI, Staatsblad 2014, 320. Available only in Dutch at: https://zoek.officielebekendmakingen.nl/stb-2014-320.html. Last accessed 30 September 2018.
- 23.
Besluit SUWI, Staatsblad 2014, 320. Available only in Dutch at: https://zoek.officielebekendmakingen.nl/stb-2014-320.html. Last accessed 30 September 2018.
- 24.
Besluit SUWI, Staatsblad 2014, 320. Available only in Dutch at: https://zoek.officielebekendmakingen.nl/stb-2014-320.html. Last accessed 30 September 2018.
- 25.
Information about the pending court case in English is available at: https://pilpnjcm.nl/en/dossiers/profiling-and-syri/. Last accessed 30 September 2018.
- 26.
Information about the pending court case in English is available at: https://pilpnjcm.nl/en/dossiers/profiling-and-syri/. Last accessed 30 September 2018.
- 27.
Robinson 2017.
- 28.
Brkan 2017.
- 29.
Ferguson 2016.
- 30.
Angwin et al. 2016.
- 31.
The issues raised in the petition are: (1) Whether it is a violation of a defendant’s constitutional right to due process for a trial court to rely on the risk assessment results provided by a proprietary risk assessment instrument such as the Correctional Offender Management Profiling for Alternative Sanctions at sentencing because the proprietary nature of COMPAS prevents a defendant from challenging the accuracy and scientific validity of the risk assessment; and (2) whether it is a violation of a defendant’s constitutional right to due process for a trial court to rely on such risk assessment results at sentencing because COMPAS assessments take gender and race into account in formulating the risk assessment.
- 32.
Loomis v. Wisconsin, docket no. 16-6387, available at: http://www.scotusblog.com/case-files/cases/loomis-v-wisconsin/. Last accessed 30 September 2018.
- 33.
Dressel and Farid 2018.
- 34.
Angwin et al. 2016. Together with their report, the researchers of ProPublica made several files publicly available, such as a list with the factors that COMPAS uses in scoring.
- 35.
Hildebrandt 2008, pp. 21–22.
- 36.
Hildebrandt 2008, pp. 21–22.
- 37.
Hildebrandt and Koops 2010.
- 38.
Rauhofer 2008.
- 39.
Leese 2014.
- 40.
Mittelstadt et al. 2016.
- 41.
- 42.
Koops 2009.
- 43.
Koops 2009.
- 44.
Brinkhoff 2017, p. 68.
- 45.
- 46.
Ferguson 2015.
- 47.
- 48.
Simmons 2016.
- 49.
Hildebrandt and Koops 2010.
- 50.
- 51.
Directive (EU) 2016/680 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data by competent authorities for the purposes of the prevention, investigation, detection or prosecution of criminal offences or the execution of criminal penalties, and on the free movement of such data, and repealing Council Framework Decision 2008/977/JHA, L 119/89.
- 52.
Brkan 2017.
- 53.
Rauhofer 2008, p. 192.
- 54.
Koops 2009.
- 55.
- 56.
Leese 2014.
- 57.
Angwin et al. 2016.
- 58.
Angwin et al. 2016.
- 59.
Dressel and Farid 2018.
- 60.
Data & Society 2015.
- 61.
Leese 2014.
- 62.
Leese 2014.
- 63.
Kosta 2017.
- 64.
Such as the court case pertaining to SyRI in the Netherlands.
References
AI Now Institute (2018) Litigating algorithms: Challenging government use of algorithmic decision systems. https://ainowinstitute.org/litigatingalgorithms.pdf. Last accessed 30 September 2018
Angwin J, Larson J, Mattu S, Kirchner L (2016) Machine Bias: There’s software used across the country to predict future criminals. And it’s biased against blacks. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing. Last accessed 30 September 2018
Brayne S, Rosenblat A, Boyd D (2015) Predictive Policing. https://datacivilrights.org/2015/. Last accessed 30 September 2018
Brinkhoff S (2017) Big Data Data Mining by the Dutch Police: Criteria for a Future Method of Investigation. European Journal for Security Research 2:57–69
Brkan M (2017) Do algorithms rule the world? Algorithmic decision-making in the framework of the GDPR and beyond. https://ssrn.com/ab-stract=3124901. Last accessed 30 September 2018
Broeders D, Schrijvers E, Hirsch Ballin E (2017) Big Data and Security Policies: Serving Security, Protecting Freedom. WRR-Policy Brief. https://english.wrr.nl/publications/policy-briefs/2017/01/31/big-data-and-security-policies-serving-security-protecting-freedom. Last accessed 30 September 2018
Christin A, Rosenblat A, Boyd D (2015) Courts and Predictive Algorithms. https://datacivilrights.org/2015/. Last accessed 30 September 2018
Clavell GG (2016) Policing, Big Data and the Commodification of Security. In: Van der Sloot B et al. (eds) Exploring the Boundaries of Big Data. Amsterdam University Press, Amsterdam, pp 89–116
Data & Society (2015) Data & Civil Rights: A New Era of Policing and Justice. http://www.datacivilrights.org/pubs/2015-1027/executive_summary.pdf. Last accessed 30 September 2018
Dressel J, Farid H (2018) The accuracy, fairness, and limits of predicting recidivism. Science Advances 4; eaao5580
Ferguson A (2015) Big Data and predictive reasonable suspicion. University of Pennsylvania Law Review 163:327–410
Ferguson A (2016) Predictive Prosecution. Wake Forest Law Review 51:705–744
Ferguson A (2018) Illuminating Black Data Policing. Ohio State Journal of Criminal Law 15:503–525
Fundamental Rights Agency (2018) Big Data: Discrimination in data-supported decision making. http://fra.europa.eu/en/publication/2018/big-data-discrimination. Last accessed 30 September 2018
Hildebrandt M (2008) Defining Profiling: A New Type of Knowledge? In: Hildebrandt M, Gutwirth S (eds) Profiling the European Citizen. Springer, Dordrecht, pp 17–45
Hildebrandt M, Koops EJ (2010) The Challenges of Ambient Law and Legal Protection in the Profiling Era. Modern Law Review 73:428–460
Joh EE (2016) The New Surveillance Discretion: Automated, Suspicion, Big Data, and Policing. Harvard Law & Policy Review 10:15–42
Keats Citron D, Pasquale F (2014) The Scored Society: Due Process for Automated Predictions. Washington Law Review 89:1–33
Koops EJ (2009) Technology and the Crime Society: Rethinking Legal Protection. Law Innovation and Technology 1:93–124
Kosta E (2017) Surveilling Masses and Unveiling Human Rights - Uneasy Choices for the Strasbourg Court. Tilburg Law School Research Paper No. 2018-10. https://ssrn.com/abstract=3167723. Last accessed 30 September 2018
Leese M (2014) The new profiling: Algorithms, black boxes, and the failure of anti-discriminatory safeguards in the European Union. Security Dialogue 45:494–511
Mantelero A (2016) Personal data for decisional purposes in the age of analytics: From an individual to a collective dimension of data protection. Computer Law & Security Review 32:238–255
Marks A, Bowling B, Keenan C (2017) Automatic justice? Technology, Crime and Social Control. In: Brownsword R, Scotford E, Yeung K (eds) The Oxford Handbook of the Law and Regulation of Technology. Oxford University Press, Oxford, pp 705–730
Mittelstadt BD et al (2016) The ethics of algorithms: Mapping the debate. Big Data & Society 3:1–21
O’Neil C (2016) Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Crown Publishers, New York
Pasquale F (2015) The Black Box Society: The Secret Algorithms That Control Money and Information. Harvard University Press, Cambridge
Rauhofer J (2008) Privacy is dead, get over it! Information privacy and the dream of a risk-free society. Information & Communications Technology Law 17:185–197
Robinson D (2017) The Challenges of Prediction: Lessons from Criminal Justice. I/S: A Journal of Law and Policy for the Information Society. https://ssrn.com/abstract=3054115. Last accessed 30 September 2018
Simmons R (2016) Quantifying Criminal Procedure: How to Unlock the Potential of Big Data in our Criminal Justice System. Michigan State Law Review 2016:947–1017
Swedloff R (2014) Risk Classification’s Big Data (R)evolution. Connecticut Insurance Law Journal 21:339–373
Taylor L, Floridi L, Van der Sloot B (eds) (2017) Group Privacy: New Challenges of Data Technologies. Springer, Dordrecht
Van Brakel R (2016) Pre-Emptive Big Data Surveillance and its (Dis)Empowering Consequences: The Case of Predictive Policing. In: Van der Sloot B et al. (eds) Exploring the Boundaries of Big Data. Amsterdam University Press, Amsterdam, pp 117–141
Zarsky T (2014) Understanding Discrimination in the Scored Society. Washington Law Review 89:1375–1412
Zouave ET, Marquenie T (2017) An Inconvenient Truth: Algorithmic Transparency & Accountability in Criminal Intelligence Profiling. European Intelligence and Security Informatics Conference. https://ieeexplore.ieee.org/document/8240764. Last accessed 30 September 2018
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 T.M.C. Asser press and the authors
About this chapter
Cite this chapter
van Schendel, S. (2019). The Challenges of Risk Profiling Used by Law Enforcement: Examining the Cases of COMPAS and SyRI. In: Reins, L. (eds) Regulating New Technologies in Uncertain Times. Information Technology and Law Series, vol 32. T.M.C. Asser Press, The Hague. https://doi.org/10.1007/978-94-6265-279-8_12
Download citation
DOI: https://doi.org/10.1007/978-94-6265-279-8_12
Published:
Publisher Name: T.M.C. Asser Press, The Hague
Print ISBN: 978-94-6265-278-1
Online ISBN: 978-94-6265-279-8
eBook Packages: Law and CriminologyLaw and Criminology (R0)