The Challenges Related to Artificial Intelligence for Women’s Health,Rights,Privacy,Autonomy,and Safety Need to Be Addressed: A Global Call to Action

The Artificial Intelligence Promise in Medicine

Artificial intelligence (AI) is ushering in a new era of rapid diagnostics, precision medicine, and health analytics and is increasingly prominent in digital health for improving patients’ quality of life. In a recent article published in The Lancet Digital Health, Sau and colleagues (2024) showed that AI may help predict cardiovascular disease and mortality risks. Oren and colleagues (2020) provided evidence of the performance of AI in diagnosing endometriosis and analyzing medical images to detect abnormalities, tumors, and other pathologies using deep learning and machine learning algorithms. Beyond diagnosis, AI contributes to personalized medicine and treatment plans calibrated to each patient’s unique characteristics (genetic profiles, health history, lifestyle) (Sau et al., 2024).

Evidence of AI Use for Intimate Partner Violence Reduction

Evidence also suggests that AI has the potential to address gender-based violence. Advanced AI models have been developed using textual or voice signal data and annotated clinical notes to detect women at risk of intimate partner violence (IPV) (De Boer et al., 2024). In Chile, SOF + IA has been designed to assist survivors experiencing technology-facilitated gender-based violence (TFGBV) on social media platforms—collecting incident data, offering advice on digital self-care, and referrals to appropriate support services (Peña et al., 2023). Finland’s AinoAid (Crofts and Hansson, 2023)—a platform linked to AI—expands this model by providing confidential first-line assistance to women survivors of IPV. This feminist chatbot, which won the 2025 Security Innovation Award, is currently being tested in France and other European countries. These AI systems eliminate barriers to access to IPV resources, overcoming geographical and cultural limitations that traditionally impede women’s help-seeking. When embedded within wider social service systems, they can reduce response times, streamline case management, and ultimately save lives.

Multiple Threats of AI Misuse

Most studies related to AI emphasize its benefits. However, despite how AI is revolutionizing digital health practice and accelerating progress on gender equality, it can pose a threat to the health, rights, privacy, autonomy, and safety of certain social groups, notably women and girls.

In terms of health threat, most AI models are trained on historical health data that inadequately represent women—particularly those from marginalized racial, ethnic, or socioeconomic groups (Panteli et al., 2025). Therefore, medical use of such AI represents an important risk for marginalized patients. For example, algorithmic bias can thus result in misdiagnoses or suboptimal treatment recommendations, perpetuating systemic health inequalities. AI can also hallucinate, therefore their use without active expertise can lead to inappropriate medication. The disparity is measurable: vision-language foundation models used in chest radiograph diagnosis consistently underdiagnose female and Black patients relative to board-certified radiologists, with the highest error rates in intersectional subgroups such as Black women (Jin et al., 2024; Seyyed-Kalantari et al., 2021).

Women’s rights can be easily violated by inappropriate AI use. Actually, AI is increasingly intruding into intimate relationships with serious consequences. Annually, millions of women and girls worldwide experience TFGBV, including cyberbullying, stalking, doxxing, and image-based sexual abuse (UN WOMEN, 2025). For instance, IPV perpetrators exploit AI-powered tools to gather sensitive personal data from social media and then apply facial-recognition software to map survivors’ movements or identify new photos shared without consent (UN WOMEN, 2025). Additionally, advanced analytics on wearable sensors can infer mood fluctuations or stress levels, enabling near-constant digital surveillance of partners (De Boer et al., 2024). Such AI-enhanced monitoring can increase coercive control, erode personal boundaries, and magnify psychological trauma for survivors. This constitutes a violation of survivors’ human rights. More seriously, the latest can face social exclusion and even auto exclusion from any legal assistance.

In terms of privacy violation, in 2023, an 18-year-old woman was killed in Pakistan in the purported name of “honor crimes” after a picture of her sitting with her boyfriend, edited with AI, went viral online (CBC News, 2023). This case demonstrates how synthesized content can directly cause severe harm offline and AI’s potential to not only violate privacy but also to inflict fatal consequences. A 2025 UN Women report warns us that the number of partners or ex-partners using generative AI to create nonconsensual images to denigrate women on social media and manipulate them is increasing at an alarming rate (UN WOMEN, 2025). Generative adversarial networks, a primary AI tool in the creation of deepfakes, can seamlessly graft women’s faces onto explicit material without their knowledge or consent. Over 90% of documented nonconsensual deepfake cases target women, weaponizing digital intimacy to shame, blackmail, and socially isolate survivors (Dunn, 2021). Such social exclusion is harmful for the survivors who may suffer a lack of autonomy even in their community of origin. The gendered disparity is clear. A 2023 analysis found that 98% of 95,820 deepfake videos online were pornographic and 99% targeted women, a 550% increase from 2019 (Security Heroes, 2023), whereas men, including prominent politicians, account for a negligible share. Moreover, the gap is partly technical, as many deepfake tools are trained on women’s bodies and perform poorly on male subjects (Jankowicz, 2023), and partly social, with more than half of U.S. deepfake victims reporting suicidal ideation (UN Women, 2026).

Another multidimensional negative consequence of inappropriate AI use is the survivors’ autonomy. Autonomy is a core value worldwide, with multiple contemporary accounts such as competency, authenticity, freedom or independence, opportunities, and social support (Catena, 2026). Women, but also any other vulnerable social group or individual, affected by online denigration facilitated by malicious use of AI, may suffer a reduction of autonomy in multiple aspects of their daily life. Unethical use of AI can negatively affect survivors’ self-determination in terms of their decision-making possibilities in different contexts such as recruitment opportunities, professional limitations, freedom of movements, among others. In medicine, even appropriate use of AI can have side effects such as reduction in patient autonomy, while in other circumstances, AI may improve medical decision-making (Keskinbora, 2019).

Finally, malicious use of AI can heavily affect people’s safety, particularly in healthcare, where AI misuse or hallucination can lead to fatalities. The beneficial applications of AI might not hide the risk of malicious use of AI (Brundage et al., 2024). Two aspects of safety worth attention when it comes to considering AI. They are related to the designers’ and users’ intentions. From the victim’s point of view, both the intentional misuse of AI and the unintended dysfunction of AI lead to harmful outcomes. In the context of vulnerable social groups such as women and girls’ healthcare, prevention of such technological challenges calls for special attention, as the consequences concern the whole of society.

A Call to Global Action for Better AI Use

Empirical research quantifying the prevalence and impact of AI-enhanced IPV remains extremely limited, highlighting an urgent need for interdisciplinary studies that integrate computer science, public health, and gender-based violence expertise. Amid the rapid deployment of AI tools without robust ethical guardrails, emergent harms have outpaced the development of corresponding legal frameworks and support infrastructure to address the threat of AI in IPV perpetration. For instance, legislation varies widely, and enforcement is inconsistent globally. While some countries and local administrations criminalize nonconsensual sexual imagery, others prohibit harassment and privacy invasion (Council of Europe, 2011). The governance challenge echoes climate change: a globally distributed harm produced by diffuse actors, with regulatory capacity concentrated in a few jurisdictions and enforcement lagging behind the technology (Maas and Villalobos, 2023). The absence of binding international instruments has left a patchwork of national and regional initiatives with incompatible scope and penalties, enabling jurisdictional arbitrage whereby perpetrators host content in permissive jurisdictions while survivors seek redress elsewhere. Existing instruments illustrate the gap. The EU AI Act (2024) imposes transparency obligations on deepfake creators but provides no remedies for victims, no penalties for malicious deployers under Article 99, and broad exemptions for “creative” and satirical content (Romero Moreno, 2024). Globally, fewer than half of countries have laws addressing online abuse, and fewer still specifically cover AI-generated content (UN Women, 2026). The Istanbul Convention (Concile of Europe, 2011) remains nonuniversally ratified and predates the generative AI era. Closing these gaps will require binding international standards, domestic enforcement capacity, and platform accountability, not transparency alone. In like manner, social media platforms implement ad hoc content moderation protocols that rely on user reports rather than rigorous detection and enforcement models, further prolonging IPV survivor exposure to harmful material (Council of Europe, 2011). Such fragmented enforcement and implementation gaps leave many survivors without legal recourse and protection.

AI can contribute to supporting the UN Sustainable Development Goals by improving population health and reducing social and gender inequalities. Urgent, effective, and sustainable action is needed. Balancing the transformative potential of AI in healthcare with its capacity to enable violations of personal rights and freedoms is critical and requires a multifaceted approach and an ecosystem of actors and cooperation. Developers and users of AI in digital health must be aware of the issues related to data nonrepresentativeness and incorporate intersectional design principles into their models, which are crucial to prevent further harm under the guise of technological progress. AI algorithms may inherently contain errors or flaws that can lead to unintended consequences and perpetuate inequities across economic and racial lines (Keskinbora, 2019). Policymakers must streamline AI regulations and impose accountability on technology platforms to prioritize respect for individuals’ privacy, safety, and well-being. They must constantly raise awareness among the population, practitioners, and researchers about the responsible use of AI tools. It is also imperative to accelerate ratification and implementation of the Istanbul Convention (Council of Europe, 2011) and introduce targeted deepfake laws that criminalize nonconsensual synthetic media and mandate swift removal of abusive content from social media platforms (Yavuz, 2025). By embedding robust ethical, legal, and technical guardrails, key interest holders and policymakers can ensure the safe use of AI tools to advance women’s and other vulnerable groups’ health, safety, and autonomy, while minimizing their harmful potential.

Authors’ Contributions

All the authors contributed to the article by writing, reviewing, and editing.