we_have_detected_potential_grooming_behavior_in_your_childs_gaming_chat_environment._blog_banner.png
MIDS Capstone Project Spring 2024

chat compAnIon

Our mission is to revolutionize online safety for children in the face of a rapidly evolving digital landscape. We work to empower parents to stay ahead of child grooming and predator detection.

At chat compAnion, we serve as a critical ally for parents by seamlessly integrating an Applied AI classification model into gaming chat rooms. This innovative technology swiftly identifies and alerts to potential child grooming activities, empowering parents to actively engage in protecting their children. Operating with unmatched speed, our system analyzes text in real-time and delivers predictions within milliseconds, ensuring parents or guardians—the first line of defense—are promptly informed and able to take immediate action.

Click here to visit our public website

The Problem

In today's digital age, the safety of our children online is facing unprecedented challenges. Described by the New York Times as "AN INDUSTRY WITHOUT AN ANSWER," the landscape is fraught with dangers. With over 500,000 online predators active daily, the prevalence of child grooming and predatory behavior poses a significant threat to our children's well-being.

Lack of Transparency and Data Accessibility

One of the most pressing issues is the alarming lack of transparency and data accessibility surrounding incidents of child grooming and predation online. Many online platforms and corporations choose to keep such information confidential, citing privacy concerns or legal liabilities. This lack of comprehensive data hinders our ability to understand the scope of the problem fully and develop effective prevention strategies.

Limited Parental Involvement

Compounding the problem is the limited involvement of parents or guardians in addressing online risks. Instead of alerting families directly, it's often large corporations that handle these situations internally, leaving parents unaware and uninformed about potential threats to their children's safety. This disconnect prevents timely intervention and leaves children vulnerable to exploitation.

Urgent Need for Action

It's clear that urgent action is required to address these challenges effectively. Greater transparency and accountability from online platforms and corporations are essential to ensure that incidents of child grooming and predatory behavior are properly documented and addressed. Additionally, parents must be empowered with access to relevant information and resources to protect their children online.

Collaborative Solutions

By fostering collaboration between technology companies, law enforcement agencies, policymakers, and parents, we can work towards creating a safer digital environment for our children. Through education, technological innovation, and advocacy, we can confront the problem of child grooming and predator detection head-on, ensuring a brighter and safer future for the next generation.

Our Work

Advanced Detection System

chat compAnIon utilizes an advanced, multi-stage resampling pipeline that feeds into a deep learning model, enabling it to navigate the nuances of communication among youth and detect even the subtlest signs of predatory behavior. Facing data scarcity, our team improved the data through various sampling techniques, extensive exploratory data analysis (EDA), and testing different probabilistic loss functions to address class imbalance. Through this process, we developed multiple models and ultimately landed on our final BERT-CNN-NN model with an F3 score of 0.82, which outperforms any other research papers tackling similar tasks in this domain. You can explore our model development journey below:

Proactive Alert System for Parents

Recognizing the critical need for parental involvement in safeguarding children online, we've developed a robust alert system tailored to meet the needs of today's families. Our system is designed to ensure that parents remain informed and equipped to take swift action against potential online threats.

Email Notifications: Upon detecting suspicious activity or grooming attempts, our alert system promptly sends parents email notifications, providing them with crucial information about the concerning situation.

Temporary Chat Room Blocking: In addition to alerting parents, our system takes proactive measures to protect children by temporarily blocking access to chat rooms or online platforms where potential risks have been identified. This temporary restriction allows for a cooling-off period and prevents further interaction with potential predators.

Parental Confirmation: We understand the importance of parental discretion and control. That's why our alert system includes a feature that requires parents to confirm or deny the alert, ensuring that they are actively involved in the decision-making process regarding their child's online safety.

Resource Provision and Next Steps: In conjunction with alert notifications, we provide parents with comprehensive resources and guidance on how to handle the situation effectively. From reporting suspicious activity to seeking support from relevant authorities, we equip parents with the necessary tools and knowledge to navigate challenging online scenarios.

Collaborative Initiatives 

In alignment with our commitment to fostering collaborative solutions, we have taken proactive steps to share our model and engage with key stakeholders in the fight against online grooming and predatory behavior.

Open-Sourcing Our Model: Recognizing the importance of transparency and accessibility, we have made our model publicly available on platforms such as GitHub and Hugging Face. By open-sourcing our technology, we aim to empower developers, researchers, and organizations worldwide to leverage our framework in their efforts to protect children online.

Exploring Collaboration Opportunities: By engaging with key stakeholders, we seek to identify synergies and opportunities for collaboration. We are committed to working hand-in-hand with law enforcement and industry leaders to develop effective strategies and solutions for enhancing online safety for children.

Last updated: April 11, 2024