CareAssist: An AI-Driven Case Prioritization Tool for Foster Care

Motivation

There are over 400,000 children in the U.S. foster care system at any given time, and roughly 38% of placements end in disruption—meaning the child is uprooted and moved again. Each move compounds trauma, severs relationships with teachers, therapists, and peers, and disrupts the stability children need to heal. For our team, this problem is deeply personal: one of our teammates, Samantha Townsend, grew up as the eldest sister in a foster home and witnessed firsthand how placement instability and fragmented information systems affect children and the families who care for them. Her experience directly shaped how we approached every design decision in CareAssist.

Social workers are the frontline defense for these children, yet they manage caseloads of 15–30+ cases using monthly visits, scattered notes, and county-specific systems that do not communicate with each other. When a child moves placements, sometimes across county lines, their medical history, behavioral data, school records, and prior case notes do not automatically follow them. Critical context is lost at the exact moment it matters most.

CareAssist addresses this gap with a unified, AI-powered platform that preserves a child's complete history, surfaces machine-learning-driven early warnings of placement disruption, explains why each child is flagged using SHAP-based feature attribution, and provides an AI assistant that lets caseworkers query their caseload in natural language—helping them intervene before placements fail, not after.

Data Source

The primary data source is the federal AFCARS (Adoption and Foster Care Analysis and Reporting System) dataset, containing case-level records submitted by all 50 states to the U.S. Children's Bureau. We used approximately 5.76 million de-identified records spanning fiscal years 2020–2024, with 350,000–400,000 new records per year. Key fields include placement history, removal reasons, demographics, disability and behavioral indicators, permanency goals, and case outcomes.

Data Science Approach

Predictive Modeling

CareAssist predicts placement disruption—whether a child's current living arrangement will break down within 12 months—using a weighted ensemble of four models: XGBoost (35%), LightGBM (35%), CatBoost (20%), and a multilayer perceptron (10%). Each model was independently tuned with 50 Optuna trials and 5-fold cross-validation across 65 features (20 baseline AFCARS fields plus 45 engineered features including interaction terms, severity composites, and nonlinear transformations). The weighted ensemble outperformed all individual models and alternative combination strategies including stacking and rank averaging.

Ensemble Architecture

Weights were optimized through 50 Optuna trials and 5-fold stacking cross-validation:

Individual Model Evaluation

Each model was independently tuned with 50 Optuna trials and 5-fold cross-validation. LightGBM narrowly emerged as the strongest standalone performer, with XGBoost close behind. The MLP, while outperforming the baseline, reflected the relative difficulty of capturing feature interactions with a shallow neural architecture compared to tree-based methods.

Ensemble Model Evaluation

We evaluated several ensemble strategies to determine the most effective way to combine predictions. While all approaches produced strong results, the weighted averaging method consistently achieved the best overall performance, slightly outperforming stacking and simple averaging. Compared to stacking, the weighted approach provided a more stable and interpretable aggregation while effectively leveraging the strengths of each base learner.

Performance Summary

We optimized for high recall because missing a child at risk is worse than a false alarm.

Features (65 total)

20 baseline features from AFCARS + 45 engineered features including: age², disability severity composite, abuse severity composite, substance abuse score, LOS ratio, interaction terms (age × disability, removals × abuse), one-hot placement types, permanency goals, and removal reason flags.

Top Feature Importances (Ensemble-Level)

Feature importance analysis reveals that placement trajectory variables—length of stay, placement type, and prior removals—dominate predictive signal. The relative importance of these core features remained consistent across models, suggesting that predictive performance was driven by effectively capturing a stable set of high-signal variables.

SHAP Explainability

A risk score alone is not enough—caseworkers need to understand why a child is flagged to trust and act on the prediction. We integrate SHAP (SHapley Additive exPlanations) to decompose every prediction into per-feature contributions. Each factor is shown with its direction and magnitude: red for risk-increasing drivers (e.g., multiple prior placements, group home setting) and green for protective factors (e.g., kinship care, short length of stay). Workers can verify the model's reasoning against their own clinical knowledge, building the trust necessary for adoption in high-stakes environments.

AI Case Assistant

Beyond prediction, CareAssist integrates a containerized Ollama/Llama 3.2-based AI assistant backed by a SQL data layer. Caseworkers can ask natural-language questions like "Which of my cases have the highest risk?" or "What are the active flags for Child X?" and receive grounded, data-driven answers pulled directly from the case database—not hallucinated text. The assistant is restricted to queryable data, ensuring responses are always factual and auditable.

The Application: Four Roles, One Platform

CareAssist is a production-grade, full-stack web application built with Angular 17 (TypeScript, standalone components, lazy-loaded routes) on the frontend, FastAPI (Python, async, Pydantic schemas) on the backend, and SQLite via SQLAlchemy async ORM (swappable to PostgreSQL) as the data layer. It is deployed on AWS (Lambda, API Gateway, S3, CloudFront) and serves four distinct user roles, each with a tailored experience:

Evaluation

Methodology

We evaluated model performance using AUC-ROC, recall, precision, average precision (AP), and F1 score to capture both overall predictive power and practical decision-making performance. AUC-ROC assessed the model's ability to distinguish between disruption and non-disruption cases across all thresholds, while AP summarized precision-recall performance in the presence of class imbalance. We intentionally prioritized recall over precision—a decision directly informed by practicing social workers, including veteran worker Otoya Brown, who emphasized that no child should fall through the cracks. A missed at-risk case carries far greater consequences than an extra review. The final operating threshold was validated in collaboration with caseworkers to balance sensitivity with operational feasibility.

Results

Our final model, a weighted ensemble of tuned tree-based methods, achieved the strongest overall performance: AUC 0.9205, 92% recall, and 63% precision. This represents a clear improvement over the baseline XGBoost model (AUC 0.9041, 91% recall, 58% precision) and outperformed all individual models and alternative ensemble strategies including stacking and rank averaging.

Performance gains were driven by combining complementary model signals rather than increasing complexity within a single model. Predictive power was concentrated in a stable set of high-impact features, with prior placements emerging as the dominant driver of disruption risk. SHAP explanations expose these drivers at the individual case level, making each prediction transparent and actionable.

Key Learnings

Acknowledgements