Canada Research Chair in Data Ethics

Tal, Eran (2024)

The Routledge Handbook of Philosophy of Scientific Modeling

https://doi.org/10.4324/9781003205647-23

The subfield of history and philosophy of measurement has seen a revival of scholarly interest over the last two decades. Much of this recent scholarship is driven by the recognition that models play important roles in measurement. This chapter surveys three uses of models in measurement: (i) the use of models in mathematical logic to study the properties of measurement scales; (ii) the analysis of measurement data using statistical models; and (iii) the use of theoretical models of the measurement process to predict the behaviors of measuring instruments. The centrality of models to measurement has given rise to the idea that measurement itself can be characterized as a model-based activity. This chapter concludes by outlining the model-based account of measurement and using it to clarify key measurement concepts, including accuracy, calibration, and objectivity.

Nancy E. Mayo, Mohammad Auais, Ruth Barclay, Joan Branin, Helen Dawes, Ida J. Korfage, Kim Sawchuk, Eran Tal, Carole L. White, Zain Ayoubi, Fariha Chowdhury, Julia Henderson, Mae Mansoubi, Kedar K. V. Mate, Lyne Nadea, Sebastian Rodriguez & Ayse Kuspinar

Quality of Life Research (2024)

Part 1: Content Development https://doi.org/10.1007/s11136-024-03714-z

Part 2: Validity Evidence https://doi.org/10.1007/s11136-024-03720-1

Many older persons do not think of themselves as “patients” but as persons wishing to live as actively as possible for as long as possible. However, most health-related quality of life (HRQL) measures were developed for use with clinical populations. The aim of this project was to fill that gap and to develop, for international use, a measure of what matters to older persons as they age and seek to remain as active as possible, Older Persons for Active Living (OPAL). This study underscores the significance of listening to older adults and highlights the importance of considering linguistic and cultural nuances in measure development.

Sebastian Rodriguez Duque
Eran Tal
Skye Pamela Barbic

Measurement 225 (2024) 113993
https://doi.org/10.1016/j.measurement.2023.113993

In the natural sciences, measurement is taken as a reliable source of knowledge because it requires a special kind of rigour. This is usually understood as an epistemic kind of rigour. We argue that in psychometrics, the science of measuring mental traits, attitudes, and experiences, the quality of knowledge supplied by measurement procedures must be established through ethical as well as epistemic justification. We reject views that restrict the role of ethics in measurement to guarding against potential harmful consequences and show that ethical and social value judgments are intrinsic to the design, validation, interpretation, and use of psychometric tools. We propose a five-step procedure called ‘ethical iterations’ that allows researchers, decision makers, and other interested parties to ensure that measurement practice is aligned with their aims and values. We substantiate our claims with evidence from our work with Foundry, a youth health organization in British Columbia, Canada. 

Tal, Eran (2023)
Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society
https://doi.org/10.1145/3600211.3604678

Bias in applications of machine learning (ML) to healthcare is usually attributed to unrepresentative or incomplete data, or to underlying health disparities. This article identifies a more pervasive source of bias that affects the clinical utility of ML-enabled prediction tools: target specification bias. Target specification bias arises when the operationalization of the target variable does not match its definition by decision makers. The mismatch is often subtle, and stems from the fact that decision makers are typically interested in predicting the outcomes of counterfactual, rather than actual, healthcare scenarios. Target specification bias persists independently of data limitations and health disparities. When left uncorrected, it gives rise to an overestimation of predictive accuracy, to inefficient utilization of medical resources, and to suboptimal decisions that can harm patients. Recent work in metrology – the science of measurement – suggests ways of counteracting target specification bias and avoiding its harmful consequences.

Tal, Eran (2021)
Perspectives on Science
https://doi.org/10.1162/posc_a_00391

Axiomatic measurement theories are commonly interpreted as claiming that, in order to quantify an empirical domain, the qualitative structure of data about that domain must be mapped to a numerical structure. Such mapping is supposed to be established independently, i.e., without presupposing that the domain can be quantified. This interpretation is based on two myths: that it is possible to independently infer the qualitative structure of objects from empirical data, and that the adequacy of numerical representations can only be justified by mapping such qualitative structures to numerical ones. I dispel the myths and show that axiomatic measurement theories provide an inadequate characterization of the kind of evidence required to detect quantities.

Tal, Eran (2020; substantive revision of 2015 entry)
Stanford Encyclopedia of Philosophy
http://plato.stanford.edu/entries/measurement-science/

Measurement is an integral part of modern science as well as of engineering, commerce, and daily life. Measurement is often considered a hallmark of the scientific enterprise and a privileged source of knowledge relative to qualitative modes of inquiry. Despite its ubiquity and importance, there is little consensus among philosophers as to how to define measurement, what sorts of things are measurable, or which conditions make measurement possible. This entry surveys the central philosophical standpoints on the nature of measurement, the notion of measurable quantity and related epistemological issues.

Tal, Eran (2019)
Philosophical Studies
https://doi.org/10.1007/s11098-018-1216-2

When discrepancies are discovered between the outcomes of different measurement procedures, two sorts of explanation are open to scientists. Either (i) some of the outcomes are inaccurate or (ii) the procedures are not measuring the same quantity. I argue that, due to the possibility of systematic error, the choice between (i) and (ii) is underdetermined in principle by any possible evidence. Consequently, foundationalist criteria of quantity individuation are either empty or circular. I propose a coherentist, model-based account of measurement that avoids the underdetermination problem, and use this account to explain how scientists individuate quantities in practice.