This explainer is a collaboration between the Tony Blair Institute for Global Change, Stanford Healthcare Innovation Lab, and the Women’s Brain Project, an international not-for-profit studying the sex and gender determinants of brain and mental health along with partners such as Altoida.
Precision medicine promises to deliver therapeutic and preventative interventions that are tailored to each patient. To truly understand an individual’s unique health profile, researchers must consider not only patients' genes but also their environment and lifestyle. Exciting technological innovations–such as wearables and portable digital devices, remote monitoring tools, health data infrastructures and cloud databases allow the collection and storage of vast quantities of data that give insights into an individual’s health and wellbeing—from information on cognitive function and social behaviour to real-time brain scans. Artificial intelligence (AI) allows the analysis of these troves of data so that researchers and clinicians can use it to develop truly individualised healthcare.
It seems clear that individualised healthcare should consider biological factors like sex and socio-cultural factors like gender—these factors, after all, are intimately tied to our physical and mental health and wellbeing. But unfortunately, the medical and biomedical fields have a long history of sex and gender bias.
Researchers and policymakers must ensure that AI-based digital health tools being developed today do not further entrench these biases, but rather offer healthcare that is truly tailored to each patient’s needs. Although algorithmic bias is a risk with AI, AI can also be leveraged to ensure healthcare works for everyone.
The Historical Sex and Gender Bias in Medicine
Even though we know sex and gender play an important role in health outcomes, our current clinical and biomedical understanding of disease, diagnosis, treatment, and prevention is largely derived from studies mostly done on males (humans as well as mice).
The historical preference for male subjects is not only attributable to the now-debunked theory that male and female cells are biologically equivalent but also because male and female subjects present differently. Ovarian cycles were long considered to complicate studies, so researchers preferred the relative “ease” of studying males. In addition, it is generally considered unethical to include pregnant women in clinical trials—which, practically, precluded a great many women from research. In fact, in the United States all women of childbearing age were explicitly excluded from clinical trials until 1993.
Even now that women are more often represented in clinical trials, researchers often fail to analyse study results by sex—in 2019, only 42% of studies that used male and female subjects analysed data by sex. In doing so, they are potentially ignoring important distinctions in diagnostics and treatments.
Consequently, women and girls have suffered from a lack of appropriate care. For example, the diagnostic criteria for attention deficit hyperactivity disorder is based on how the disorder is manifested in young boys; because young girls often present different symptoms, they are often undiagnosed or under-diagnosed. Similarly, while heart disease impacts men and women equally, symptoms present differently. However, until 2020 federal guidelines in the US characterised women’s symptoms as “atypical,” due to diagnostic criteria rooted in studies conducted largely on men. By contrast, guidelines characterised men’s symptoms as “typical.” As a result, women are more likely to report that doctors ignore their symptoms and are at greater risk of emergency cardiac hospital visits.
To achieve truly individualised healthcare, it is essential to move toward an accurate and inclusive collection of sex and gender data and a careful examination of their impact on health and disease.
AI Can Help Deliver Better Care
AI is a powerful tool in precision medicine. It allows researchers and clinicians to analyse large volumes of information, identify patterns in patient data, discover new biomarkers, leverage insights in genomics and even repurpose drugs. It can help support real-time clinical decision making, including treatment plan design, patient diagnosis, and population health management.
Bias is a very real issue in algorithmic design. Most of the currently used biomedical AI technologies do not account for sex and gender bias detection. New research shows that even ostensibly agnostic algorithms are anything but. A recent study showed machine learning models could discern a patient’s self-reported race by analysing only the clinician’s notes in which all references to race had been redacted.
The biomedical and medical fields can interrogate whether sex and gender bias might affect the efficacy of digital tools. At the same time, they can also leverage sex and gender-specific characteristics to improve diagnostics as well as the efficacy of digital tools.
For example, the “typical” vs. “atypical” heart disease diagnostic guidelines discussed above? Those outdated distinctions were dissolved thanks to AI. A study by the Center of Complex Interventions used machine learning to show that these criteria were outdated. Consequently, they were replaced with diagnostic criteria that more accurately reflect men and women’s symptoms. This breakthrough required additional research that included more women and considered that even when men and women have the same symptoms they might describe them differently–that is, the model leveraged both a non-biased algorithm and subtle sex and gender distinctions in pathology.
But reworking guidelines is less than ideal. As we enter the next stage of precision medicine it will be critical to create diagnostic models that are inclusive of women in the research and in the analysis from the outset. If we don’t include diverse subjects in every new AI study and design (along sex, gender, race, socioeconomic status and culture lines), we may soon end up with highly accurate precision medicine tools that work for only a subset of society.
Developing objective digital biomarkers that can pick up subtle biological and social distinctions will help ensure individuals are diagnosed earlier and receive optimal treatment. For example, a recent study by the Women's Brain Project and Altoida showed that AI and augmented reality can depict sex-based differences in cognitive, functional, and motoric performance using digital biomarker data collected from a cognitive assessment test.
Policy Recommendations
First, policymakers must understand both the necessity for sex and gender-sensitive precision medicine (including digital biomarkers) and the complex tools required to realize its potential. This will require a multi-stakeholder approach in which academics, technologists, patient advocates and civil society engage with policymakers to share learnings. The Women’s Brain Project, for example, holds regulatory roundtables to educate stakeholders and plans to open a sex and gender precision medicine research institute.
Next, researchers should incorporate analyses of sex and gender into their work at every stage — from study design to gathering data, analysing those data and drawing conclusions. Pre-clinical trial research should be done on female and male rodent and cell models. A diversity of sex, gender, ethnicity and age must be represented in clinical trials, which might require active outreach on the part of researchers to overcome the "trust gap" in medicine. The resulting data must be disaggregated along sex and gender and other identifiers to ensure precision medicine applications for men and women.
The EU’s clinical trial regulations can serve as a model–they require that participants in clinical trials represent the population groups that are likely to use the intervention under study, and non-inclusion must be justified. Outcomes must be analysed and results presented by age groups as well as sex. Similarly, to obtain funding under the Horizon Europe Strategic Plan or the Innovative Health Initiative, researchers must include sex and gender analysis in research design. The United States’ Women’s Health Research Roadmap and England’s Women’s Health Strategy also take steps toward sex- and gender-specific research and care.
Now is the time to lay the groundwork for inclusive AI-based digital health tools so that 21st century healthcare does not repeat the mistakes of the 20st century. By acting now, while these tools are nascent, policymakers can ensure a more equitable–and effective–medical paradigm. Steps that can be taken include:
Ensure algorithms are trained on data sets reflecting diversity across sex, gender, age, ethnicity, and other factors. For example, genomic data sets built primarily on the genetic information of white people entrench existing inequities, while diverse data sets provide insights into how health conditions impact many populations. This will require a clear path to obtaining representative data that includes outreach to women and marginalised groups.
Develop tools to detect and mitigate algorithmic bias, such as explainable algorithms in data analysis and drug development.
Ensure algorithmic and data transparency by encouraging developers to release datasheets and model cards alongside data sets and models, respectively.
Encourage the medical and biomedical fields to leverage patient-specific characteristics including sex, gender, age, and ethnicity to develop tailored digital biomarkers.
Incorporate key ethical considerations including inclusivity, explainability, and transparency during every stage of technological development. While developers have often cited these goals, in practice AI has not lived up to these ideals. Therefore, regulation is key. To this end, the European Commission proposed the first-ever legal framework on AI, which adopts a risk-based approach to regulation. Canada is following suit, while the United Kingdom is exploring a more explicitly pro-innovation approach.
Precision medicine has the potential to offer individualized healthcare to everyone–so long as its tools are developed conscientiously. Regulators have made progress to ensure clinical trials are representative of diverse populations; now is the time to ensure the digital health tools being developed today are as well.
Photo by National Cancer Institute on Unsplash.