Abby Reproductive Health Chatbot

Accurate, empathetic & secure support for abortion/mental health navigation via anonymous chat.

Problem & Motivation

Accessing comprehensive reproductive healthcare in the United States faces increasing barriers and complexities. State laws vary drastically and change frequently, creating confusion. Individuals seeking services like abortion, contraception, or general support must navigate a fragment

ed landscape of multiple websites to find accurate legal information, verified providers, funding options, and emotional support. This burdensome process (often taking hours to days) can lead to critical delays in care, causing significant stress and potential harm during an already vulnerable time. Existing resources often lack integrated support, real-time legal updates tailored to specific situations, or consistent empathetic guidance. The goal of our project is to develop a solution centered on providing transparent, accessible, and personalized support for individuals seeking abortion healthcare and associated mental health resources, directly addressing the confusion and distress caused by the current fragmented system.

Our Solution

We are excited to introduce you to Abby, an AI-powered Reproductive Healthcare Companion. Abby integrates accurate, verified medical and legal information with empathetic guidance. The interface is designed to be welcoming and easy to use. Our core value proposition is simple: to provide transparent, accessible, and personalized support for individuals navigating the complexities of abortion healthcare and related mental health needs.

Data Pipeline

Our technical approach utilizes a Retrieval-Augmented Generation (RAG) architecture built with LangChain, employing OpenAI’s GPT models for natural language understanding and response generation. User queries received via a React frontend are processed by a Python Flask backend. LangChain orchestrates the retrieval of relevant information from curated data sources based on semantic similarity to the user’s query. This retrieved context is then passed to the LLM, enabling Abby to generate informed, accurate, and empathetic responses grounded in reliable data, all while prioritizing user privacy and security within the environment.

Data Sources

To address this critical gap, we developed Abby, an AI-powered chatbot designed as a centralized, private, and accessible platform for reproductive healthcare navigation. Abby integrates information from multiple vetted sources, including AbortionFinder.org API for state-specific policy information, American College of Gynecology and NIH data for maternal mental health and pregnancy and Planned Parenthood for abortion related questions.

Model Selection

Selecting the appropriate model architecture for Abby was guided by the dual requirements of providing factually accurate, grounded information while maintaining an empathetic and natural conversational style. Given the critical nature of reproductive health information, minimizing factual errors and ensuring responses were based on verified sources was paramount. This led us to employ a Retrieval-Augmented Generation (RAG) framework, leveraging a powerful generative model combined with retrieval from our curated knowledge base.

The performance of this chosen approach is validated by our evaluation metrics:

• Semantic Accuracy (BERTScore): The high BERTScore results (Precision: 0.81, Recall: 0.76, F1: 0.79) demonstrate that Abby's responses align well semantically with the intended meaning of reference answers. This indicates the model effectively understands and conveys the correct information, fulfilling a core requirement, even when phrasing differs. This validates the use of a capable generative model within the RAG framework for nuanced understanding and expression.
• Factual Grounding (Faithfulness): The strong Faithfulness scores (Mean: 0.80, with ~75% scoring ≥0.75 and ~65% being entirely supported) are crucial. They show that the RAG architecture successfully constrains the generative model, ensuring responses largely adhere to the information retrieved from our verified knowledge sources. This directly addresses the risk of hallucination and grounds the chatbot's advice in reliable data, a key reason for selecting RAG over a standalone LLM.
• Response Style (ROUGE): The moderate ROUGE-1 (0.40) alongside lower ROUGE-2 (0.11) and ROUGE-L (0.18) scores indicate that while Abby incorporates relevant keywords, it generates responses with different phrasing and structure compared to the reference answers. This reflects the generative nature of the underlying LLM, allowing for more varied and potentially more natural-sounding conversation than simple information retrieval, while the high BERTScore and Faithfulness confirm the core message remains accurate and grounded.

Collectively, these metrics support our selection of the RAG architecture. It strikes the necessary balance: leveraging a generative model for semantic understanding and fluent response generation (validated by BERTScore and ROUGE patterns) while ensuring factual accuracy and reliability through grounding in verified data (validated by Faithfulness scores). This approach best meets the project's goal of providing trustworthy, empathetic, and informative support.

Key Learnings & Impact

Developing Abby provided significant learnings, particularly around the challenges of ensuring real-time accuracy for rapidly changing legal and provider data, mitigating LLM limitations like hallucinations through the RAG approach, the complexities of prompt engineering for sensitive topics, addressing potential biases, and navigating the critical ethical considerations involved. 

The intended impact of Abby is to substantially improve the experience of seeking reproductive healthcare. By consolidating reliable information, provider connections, and emotional support into one user-friendly interface, Abby aims to reduce search time and stress, empower individuals with accurate knowledge for informed decision-making, build trust through a secure platform, and ultimately shorten delays in accessing essential care and support, especially for those facing significant barriers.

Future Work

Our current version of Abby demonstrates the feasibility of integrating critical reproductive healthcare information - legal guidance, provider details, and empathetic support - into a single, accessible platform. Based on our development process and user feedback, we have identified several key areas for future enhancement to further increase Abby’s utility and impact:

• Enhancing user Interface & Experience: While functional, further refinement is needed to address minor formatting inconsistencies and optimize the user interface across different devices and browsers. This involved rigorous bug fixing and usability improvements to ensure a seamless, intuitive, and reliable user experience.
• Deepening Support Resource Integration: Currently, Abby primarily provides information and recommendations. A significant next step is to deepen the integration with verified support resources to facilitate direct action. This could include features like:
  • Appointment Scheduling: Exploring partnerships or API integrations with clinics to allow users to initiate appointment requests directly through the chatbot interface.
  • Direct Hotline Connections: Implementing functionality to seamlessly connect users with verified emotional support or counseling hotlines, directly from the chat, potentially via click-to-call buttons or embedded communication widgets.

• Expanding Knowledge Base & Nuance: Continuously updating and expanding the underlying knowledge base with more granular legal details, additional funding sources, and specific provider service nuances will enhance the accuracy and relevance of Abby’s guidance.

• Advanced Personalization & Follow-Up: Investigating secure methods for offering personalized follow-up reminders or check-ins, while maintaining strict user privacy and data security protocols, could provide ongoing support.

Our Contributions

• Designed and deployed a privacy-conscious mental health chatbot with real-time NLP and sentiment understanding.
• Created a lightweight, low-latency dialogue system deployable on mobile and desktop.
• Established a pipeline for emotional adaptive dialogue generation using transformer-based models.

Acknowledgements

We extend our heartfelt thanks for our capstone mentors, Cornelia Paulik and Ramesh Sarukkai who guided us through our project. Additionally, we want to thank Professor Ushma, a Professor in University of California, San Francisco, who offered transformative feedback in our chatbot approach. Special thanks to the individuals who participated in our pilot program, offering vulnerable and invaluable feedback.