Users often do not realise that a language model is making certain assumptions. Subtle but systematic differences can emerge in how men and women are described, without this being immediately visible. Because such biases are so hidden, they can spread unnoticed and influence the outcomes of AI systems in ways that are difficult to detect.

A central problem that Van der Wal addresses in his thesis is that existing measurement methods do not sufficiently reflect real-world use. Many of those methods look for explicit stereotypes in a model’s output, such as direct statements about population groups. In practice, however, biases are rarely so explicit: they are tied to the context in which a model is deployed. Van der Wal points to the use of AI in healthcare as an example. When training data contains outdated or incorrect assumptions, for instance around the discredited concept of race in relation to disease, a model may continue to repeat those assumptions. If the medical data used to train a model comes primarily from research conducted on men, the model may interpret women’s symptoms differently or less seriously, and may produce different risk assessments.

To examine whether realistic scenarios expose different errors than simple tests, Van der Wal presented language models with a range of medical cases. He asked the models to provide diagnoses, risk assessments and advice, varying the ethnicity of the patient each time. The research revealed subtle but consistent differences in outcomes, differences that had remained invisible in standard tests. Precisely because the setup closely resembled real-world practice, it became clear how biases can affect medical decision-making.

In addition to measuring bias, Van der Wal also examined what happens inside a language model during training. He tracked, step by step, how a model learns to store information. When the word “doctor” frequently appears alongside “he” in training data, and “nurse” alongside “she”, the model picks up on those associations. Over time, the model was found to store that information increasingly in specific locations, reinforcing the gender bias internally. Biases therefore arise not only from the data a model is trained on, but also from the way the model structures that information.

Bias in language models cannot be resolved with a single simple fix, but Van der Wal demonstrates that targeted approaches can be effective. Once it is known where in a model biases are stored, those specific locations can be addressed. He tested this by comparing a model before and after a targeted adjustment, in which the model was trained not to replicate identified gender biases. Although this approach already appears to work in specific cases, Van der Wal argues that more research is needed to extend the method to more complex forms of bias. He will defend his thesis at the University of Amsterdam on 29 April.