Srestha Mitra

Problem Statement

AI is transforming healthcare with promises of speed and efficiency, yet it has systemic flaws. For decades, medical research has focused on male bodies, leaving women underrepresented and misdiagnosed, for example in heart disease, where women are 50% more likely to be misdiagnosed during a heart attack (British Heart Foundation, 2016; Criado Perez, 2019, p.240) . These biases, embedded in clinical records, are now being replicated into AI systems. Because AI is not neutral it reflects the biases it is trained on, we risk creating a bigger systemic level problem in the UK healthcare, that is integrating AI models into it’s practice. If not addressed immediately, this systemic bias will not only prevail, it will worsen.

(Mitra.S,2025)

Research Question

How might we build user-centered design strategies to improve AI model training in healthcare, to ensure adequate representation of cisgender women and reduce diagnostic bias in AI-driven tools within the UK healthcare system?

(Mitra.S,2025)

Abstract

This project addresses the persistence of gender bias in healthcare artificial intelligence (AI) by developing a bias-awareness toolkit for AI engineers. Cardiovascular disease was selected as the primary case study, as it remains one of the leading causes of death among women, yet AI algorithms for cardiovascular risk and disease prediction perform significantlyworse in women (Achtari et al., 2024) and is frequently misdiagnosed due to datasets historically centered on male physiology. Employing an iterative, user-centered research approach, the toolkit was co-designed with input from AI developers and medical practitioners to ensure both technical feasibility and clinical relevance. The final solution is structured into six phases, providing engineers with symptom-specific resources, a dataset bias scanner, reflective checkpoints, and model testing protocols incorporating diverse female patient profiles. Delivered as a web-based resource, the toolkit is designed for seamless integration into existing development workflows and offers scalability to additional medical conditions. By enabling engineers to identify and mitigate bias during data labeling, training, and evaluation, the toolkit positions AI as a supportive tool for clinicians, enhancing diagnostic accuracy while contributing to the creation of more equitable and inclusive healthcare systems.