MIDS Capstone Project Fall 2023
Shine With Eyes: Classifying Eye-Tracking Scanpaths to make Autism Spectrum Disorder (ASD) Diagnoses More Accessible
Team members:
- Problem & Motivation:
- Half the children in the US are currently being screened for ASD, with 5 million children at risk of developmental delay. Negative symptoms can worsen over time if untreated, including reduced motor abilities, impaired communication skills, and trouble socializing with their communities. Untreated adults can find employment difficult, sometimes resulting in severe depression or lower life expectancy. The public cost of caring for Americans with autism costs >$268 billion today and is projected to reach $461 billion by 2025.
- Data Source & Data Science Approach:
- By utilizing a public eye-tracking scanpath dataset and machine learning technology, we've deployed a Provider Portal prototype product to augment the ASD screening process. Our tool consists of a client user interface that is separated into Provider’s View (a HIPAA compliant demo view that requests the healthcare provider’s physician ID, the test ID that the provider is requesting results from as well as the date of the test) vs. Developer’s view (live view where a scanpath image is loaded by the user).
In the live developer’s view, the uploaded image is sent to our image classification API in Amazon EKS. The trained model processes the image, performs a prediction, and returns the findings alongside the explanation for the classification to the provider portal page. An intuitive, seamless, upload interface will allow medical professionals to interpret session results quickly to make decisions more efficiently.
- By utilizing a public eye-tracking scanpath dataset and machine learning technology, we've deployed a Provider Portal prototype product to augment the ASD screening process. Our tool consists of a client user interface that is separated into Provider’s View (a HIPAA compliant demo view that requests the healthcare provider’s physician ID, the test ID that the provider is requesting results from as well as the date of the test) vs. Developer’s view (live view where a scanpath image is loaded by the user).
- Evaluation:
- With a dataset of approximately 530 eye-tracking scanpaths, our team explored various machine learning models to find the most effective approach to distinguishing between scanpaths of children with and without ASD. We finally settled on a vision transformer model. We pre-trained and fine-tuned the ML model, along with utilizing advanced data augmentation techniques, we were able to achieve a 98% validation accuracy.
- Key Learnings & Impact:
- Moving from a CNN model and Language Transformer model to a pre-trained Vision Transformer Model ended up being the best model for accuracy. In addition, due to small data size, our model was overfitting. Unfreezing the encoder layer alone doesn’t help to mitigate it. Data-augmentation fixed the over-fitting problem and significantly improved the model performance.
- Research has shown that early intervention techniques, such as applied behavioral analysis sessions, are most impactful for children under the age of 5, further highlighting the importance of early diagnoses. By introducing eye-tracking machine learning classification alongside in-person diagnostic screenings, families can begin to receive diagnoses for their child well before the current 4-year-old average, resulting in more effective treatment, societal inclusion, and benefiting communities as a whole.
- Acknowledgements:
- We would also like to thank our professors and fellow students at U.C. Berkeley, as well as the subject matter experts, psychiatrists, psychologists, and data scientists for their invaluable feedback, guidance, and insight.
Course
Data Science 210. Capstone, Fall 2023
