13 January 2026

University of Amsterdam

AI in healthcare is not inclusive enough

Health & Care

AI is increasingly embedded in healthcare, from applications that analyse skin conditions to generative systems such as ChatGPT that provide health advice. Yet these technologies are not equally effective for everyone. According to Mahsa Shabani, associate professor of Law for health and life at the University of Amsterdam, this raises significant legal and societal concerns. She received a Vidi grant for her work in this area.

Shabani observes that AI systems are increasingly taking on responsibilities traditionally held by healthcare professionals. This shift affects the relationship between patients and providers and raises questions about liability, trust and fairness. At the same time, not all groups benefit equally from these technologies. Access to digital infrastructure, levels of digital literacy and age matter, as does the way AI systems are developed. Many medical AI tools are trained on datasets that largely consist of people of European ancestry, which can make outcomes less reliable for other populations.

According to Shabani, this can result in poorer care for minority groups and erode trust in the healthcare system. When people realise that AI tools do not work equally well for everyone, they may lose confidence in healthcare altogether and even avoid seeking care. Trust, she argues, is a fundamental condition for effective healthcare, and AI should reinforce that trust rather than undermine it.

In her work, Shabani focuses on how these challenges can be addressed from a legal and regulatory perspective. She argues that regulators should play a stronger role in assessing medical AI systems. Issues of inclusivity and discrimination should be explicitly considered in approval procedures, for example through mandatory fairness checks. A clearer legal understanding of fairness is central to this approach.

The aim is to develop a practical, legally grounded framework that regulators and developers can use to make AI in healthcare more inclusive. Shabani also seeks to raise awareness and stimulate a broader societal debate on the implications of data-driven technologies in healthcare.

Read also

UvA Researchers Created an Evaluation Checklist for Datasets

17 May 2024

Databricks expands major operations in Amsterdam with new 11‑storey office

3 March 2026

D66 Amsterdam Aims for 100,000 AI-Literate Residents by 2030

2 October 2026

Subscribe to our newsletter

Would you like to stay informed about what is happening within the Amsterdam AI ecosystem?