PhishNet: Check Before You Click

The Problem

Every year, older adults lose billions of dollars to email scams and phishing attacks, and the problem is accelerating. According to the FTC, fraud losses reported by adults over 60 reached $2.4 billion in 2024, a four-fold increase since 2020. Because fraud is deeply underreported due to stigma, the real figure is almost certainly higher. This isn't just a financial problem, either. Three in four older adult fraud victims experience significant effects on their mental and physical health, with the impact even more severe for those living alone or managing cognitive decline. Despite all of this, 6 in 10 older adults say technology simply wasn't designed with them in mind, and no widely available free tool exists to help them navigate their own inbox safely. With 63.6M U.S. adults aged 55+ using Chrome on desktop or tablet, and AARP projecting a $2 trillion AgeTech market, this is both a meaningful human problem and a large, underserved market.

The Transparency Gap

Existing tooling falls into three failure modes. Simple classifiers give users a binary label with no explanation and no fallback if a scam email gets through. Black-box models hide their reasoning entirely, leaving users unable to learn or make informed decisions, building dependency rather than literacy. And the tools like LifeLock or SeniorShield.ai that do offer some transparency are either enterprise-priced, reactive rather than preventative, not senior-specific, or simply unavailable. The result is that older adults are left relying on intuition at exactly the moment when scams are becoming indistinguishable from legitimate emails.

Our Solution

PhishNet is a free Chrome extension for Gmail that closes this gap in three ways: it identifies linguistic, structural, and behavioral scam patterns; it surfaces clear, interpretable risk signals rather than just labels; and it keeps older adults in control of their own inbox. We designed for two primary personas. The first is the independent older adult who wants protection and education but doesn't want to feel monitored. The second is the trusted family member who wants peace of mind without being on call 24/7.

When a user opens an email, PhishNet assigns one of three risk tiers (safe, suspicious, or dangerous) and surfaces the reasoning in a sidebar widget. Safe emails get a closed green indicator. Suspicious emails turn yellow and expand on click to reveal the signals that drove the verdict. Dangerous emails partially auto-open with full details one click away. The Risk Analysis tab explains which psychological tactics were detected, including authority, scarcity, social proof, and the rest of Cialdini's weapons of influence. An Ask tab lets users pose follow-up questions about any email. A Settings tab allows users to add a trusted contact and forward a summary with one click, and an Analytics tab shows usage and cost trends in real time. Throughout, the system is privacy-first: we only store the user's email address, an encoded identifier for message IDs, the risk scores, and the explanations. We never store personal email content unless the user explicitly allows it.

Architecture

At the core is a FastAPI service running on AWS App Runner, fronted by AWS and Google Cloud API Gateways and called from the Chrome Extension over HTTPS. When new emails arrive or Gmail is opened, the extension checks for untagged threads. Those events are routed through a GCP-hosted Pub/Sub layer that asynchronously triggers calls to the backend. Messages are scored on Amazon Bedrock, where our two-stage RAG classification pipeline runs. A Bedrock Knowledge Base (vector store) retrieves relevant context from trusted sources before classification. Results, user settings, and feedback are persisted in DynamoDB, with S3 holding vectorized AARP and IC3 source material. OAuth handles user identity so the extension only listens to the account the user explicitly authorizes, keeping accounts separate even on shared devices.

Two-stage scoring flow: When an email arrives, the backend first retrieves related context from the vector store and runs an initial classification. If the email is safe, the flow stops there, keeping latency and cost low for the majority of inbox traffic. If the email is suspicious or dangerous, a second stage generates a plain-language explanation of why it was flagged. This lazy two-stage approach delivers fast results for most emails while still providing detailed reasoning when risk is detected. On activation, we score the user's last 90 days of email, with customizable backfill planned.

Populating the knowledge base: A weekly scraper job pulls articles from trusted sources like AARP's scam library and the FBI IC3 Scam Alert Map, normalizes them, stores them in S3 as JSONL, and incrementally upserts them into the vector store via an embedding pipeline. This ensures content is updated rather than duplicated.

Feedback loop: When a user gives a thumbs-down response, we capture their suggested verdict and any notes in DynamoDB. That data is exported as a labeled evaluation set, run through model-as-judge and RAG evaluations, and used to iterate on prompts and retrieval quality. This production feedback loop was added directly in response to user testing and complements our manual review and bulk evals.

Datasets

A core challenge was the lack of elder-specific phishing datasets. We assembled five sources to address this. Nazario (elder-adapted) contributed roughly 1,500 real phishing emails, with about 1,300 paraphrased by Claude Sonnet 4 into elder-targeted variants that preserve original attack structure while adapting language and targets. SpamAssassin provided 5,809 balanced ham/spam emails (4,091 ham, 1,718 spam) from a shared real-world corpus ideal for cross-evaluation. We also generated two synthetic datasets of 1,000 emails each from Claude Sonnet 4 and Amazon Nova Pro, specifically designed to cover all seven Cialdini weapons of influence, with validated sender domains and a deliberate mix of malicious, legitimate, and "near-miss" samples (reciprocation, liking, and authority lookalikes). We used two generator models to reduce single-model bias. Finally, over 400 AARP and FBI IC3 articles and elder-fraud reports serve as the live RAG knowledge base.

EDA on the Nazario corpus revealed that the highest TF-IDF tokens in the original emails were "account" and "org," but after elder-targeted paraphrasing these shifted to "medicare," "benefits," and "security," confirming the paraphrasing meaningfully retargeted the language. SpamAssassin analysis surfaced scammer tactics like domain impersonation with subtle misspellings (for example, "hotmaill.com" with two L's). Across the synthetic datasets, we identified 23 unique elder-specific scam patterns spanning Medicare fraud, prescription drug offers, romance scams, and more.

Modeling Journey

We evaluated three approaches. Traditional ML (Logistic Regression, Random Forest, XGBoost) used five semantic Cialdini features extracted via Claude Sonnet 4. It reached roughly 85% accuracy but with very low legitimate-class precision, letting too many phishing emails pass through as safe. LLM fine-tuning improved accuracy to 93% by giving the model the full email body rather than just five features, but it requires retraining as scams evolve. That led us to RAG, which let us ground classification in a live, updatable knowledge base of real elder scams and produced our best results.

One-step to two-step: Using a single prompt for both classification and explanation made the model overly cautious and dropped accuracy. Our solution was to split it into two prompts and expand from binary to three risk tiers (safe, suspicious, dangerous). We benchmarked GPT-4o-mini, Claude Haiku 4.5, Nova Lite, and Nova Pro on the same pipeline across 800 to 1,000 emails. Nova Pro achieved 96% overall accuracy with 100% phishing recall when used for both stages, and an equivalent 96%/100% when paired with Claude Haiku 4.5 for explanation.

Evaluation

Binary accuracy alone doesn't tell us whether the risk tier is appropriate or whether the explanation is comprehensible to older adults, so we layered two additional evaluations on top. For manual review, we randomly sampled 100 suspicious and dangerous emails and assessed label accuracy and explanation quality. Nova Pro was succinct and accurate, occasionally flagging legitimate SpamAssassin emails as suspicious. We considered this acceptable given our caution-first stance and the dataset's age. Claude Haiku 4.5 was overly verbose and often disagreed with Nova Pro's labels.

For the LLM-as-a-judge evaluation, we used GPT-4.1 with two custom metrics: phishing tier classification quality (scored 1 to 3) and elder explanation quality (scored 1 to 5), plus Bedrock's faithfulness, coherence, and relevance metrics. Nova Pro scored 2.73/3 overall on tier quality (2.88 for dangerous, 2.06 for suspicious) and 4.75/5 on elder explanation quality, with 4.88 faithfulness, 4.90 relevance, and 4.96 coherence.

Benchmarking

The closest comparable published work is Combatting Senior Scams Using a Large Language Model-Created Rubric(DiMario et al., 2024), which reached 98% accuracy on Nazario using GPT-3.5 with a static rubric but performed lower on other datasets and relied on out-of-the-box GPT training rather than elder-specific evaluation data. PhishNet achieves 96% overall accuracy, 99% on the elder-paraphrased Nazario subset, and 100% phishing recall, while also staying current via RAG knowledge base updates and surfacing tiered, interpretable explanations directly to the user. These are three advances the benchmark does not provide.

Roadmap

The current MVP serves the "no oversight" persona. Phase 1 (reactive oversight) adds a conversational agent for email Q&A and one-click sharing of fraud alerts and explanations with trusted family members. Phase 2 (proactive oversight) introduces a dashboard and additional models to detect concerning interaction patterns and share flagged emails with family. Phase 3 (heavy oversight) adds enhanced alerting, the ability for trusted contacts to review and release emails, and full protective support. We are also exploring locally hosted models to further strengthen privacy guarantees.

Why This Matters

Older adults bear a disproportionate burden from email scams. Median losses are 2 to 3 times higher than for younger adults, and growing. Existing spam filters weren't built for this problem: they catch keyword spam but miss the psychological manipulation tactics like authority, urgency, scarcity, and liking that make elder-targeted scams so effective. Scammers constantly adapt, too. Grandparent scams, Medicare fraud, SSA impersonation, and new variants emerge faster than any retraining cycle can keep up with. PhishNet's RAG-based approach updates automatically as new AARP and IC3 reports are published, so detection improves without retraining. It's a structural fit for a structurally shifting threat.

Acknowledgements

We would like to acknowledge our professors, Korin Reid and Todd Holloway, whose guidance, feedback, and encouragement were essential to the success of this project. We also extend our gratitude to AARP and early testers of our tool whose work made this possible. Lastly, thank you to the team, for believing in the vision of this project and for dedicating many hours and weekends to develop this capstone project.