Politics of Machine Learning Evaluation
Workshop organised by Dieuwertje Luitse, Anna Schjøtt Hansen and Tobias Blanke | Thursday 16 (afternoon) & Friday 17 November 2023, at the Institute for Advanced Study (IAS).
Event details from: Politics of Machine Learning Evaluation
- Start date
- 16 November 2023
- End date
- 17 November 2023

Is the data good enough for training purposes? Does the model perform accurately enough? Is the error rate low enough? Such questions of ‘good enough’ are at the very core of the process of Machine Learning (ML) evaluation and can also be considered a highly political process in the development of ML systems. There is already a growing interest in the political implications of ML in relation to, for example, dataset construction and the political capacities of specific ML models or foundational algorithmic techniques. However, there has been less focus on the politics of evaluation practices and techniques in ML.
To further explore this issue, the aim of this workshop is to collectively engage in conceptualising or historicizing ML evaluation as a politically contested practice, provide methodological approaches to the study of evaluation techniques or empirical examples of ML evaluation in practice. Invited contributions engage with conceptual, methodological, and political questions in relation to topics, such as but are not limited to:
- Dataset construction
- Data labelling practices
- Ground truths and benchmarks
- Biases in evaluation
- Metrics
- Errors and error analysis
- ML Evaluation techniques
The workshop is organised by Dieuwertje Luitse, Anna Schjøtt Hansen and Tobias Blanke, and it will take place on Thursday 16 & Friday 17 November 2023, at the Institute for Advanced Study (IAS) located in the city centre of Amsterdam.
The workshop will feature three keynotes by Florian Jaton (Tech Hub, Graduate Institute of International and Development Studies, Geneva), Nanna Bonde Thylstrup (University of Copenhagen) and Claudia Aradau (Kings College London).
The workshop is funded by the Amsterdam School for Cultural Analysis (ASCA), the Amsterdam school for Heritage, Memory and Material Culture (AHM) and the Research Priority Area Human(e) AI.