Master of Information and Data Science (MIDS) alums Katya Aukamp, Keta Desai, Nichol Flowers, and Clara Rhoades are the winners of the Hal R. Varian Capstone Award, for their project, Eye2Voice.
The MIDS team developed a gaze-to-speech communication tool that gives people with physical disabilities a voice using only their eye movements.
We spoke to the team to learn more —
What inspired your project? How did you decide on the concept?
Katya: This project was inspired by personal life experience. A childhood friend of mine experienced a traumatic brain injury and suffered nearly full-body paralysis. When I’d come to visit him and his family in the hospital, I’d watch as they struggled to communicate, relying on blinking to either say yes or no. It was so frustrating and heartbreaking to watch. I wanted to build something that would help them and other families and individuals in the same position.
What was the timeline or process like from concept to final project?
Katya: The timeline was all about seeing how far we could get. We mocked up plans for a simple version of our application and a more comprehensive one. Depending on how well our model performed, we agreed that if we could get our model to a good place we’d push for the more comprehensive version. We managed to get a stable version of our model by week 9. At that point, we shifted our focus from experimentation to product development. We expanded the scope, built the real-time communication pipeline, integrated gaze-based interaction with text-to-speech capabilities, and developed the end-to-end user experience. The final weeks were dedicated to refining the system, improving responsiveness and usability, testing different scenarios, and preparing the platform for demonstration. Reaching that stretch goal was one of the most rewarding parts of the project.
How did you work as a team? How did you work together as members of an online degree program?
Keta: We were fortunate that most of our team members were located in the Central and Eastern time zones, so coordinating meetings and collaborating in real time was relatively straightforward. As members of an online degree program, we were already accustomed to remote collaboration from the very beginning of MIDS. What made the experience especially rewarding was that every team member was highly motivated and genuinely invested in building the best possible solution. There was strong synergy across the team, which helped us focus on a shared goal.
We relied on regular video meetings, shared documentation, cloud-based development environments, and frequent communication through Slack to stay aligned throughout the project. Responsibilities were assigned based on individual interests, strengths, and learning goals, while maintaining a highly collaborative approach. Team members worked independently on their respective tasks while keeping everyone informed of progress and challenges. During our weekly meetings, we reviewed milestones, assigned ownership of upcoming work, and made key decisions collectively.
This balance of ownership, accountability, and collaboration allowed us to move quickly, achieve our stretch goals, and successfully deliver a functional minimum viable product (MVP).
“MIDS gave me more energy, curiosity, and connection than it ever took away.”
How did your I School curriculum help prepare you for this project? Which course in the program was uniquely helpful in preparing you for your capstone project and why?
Nichol: Eye2Voice drew on much of what we learned throughout MIDS. Applied Machine Learning was a foundational course for this project. It’s where we learned the core of training a model to generalize to new data, from the fundamentals up through the convolutional neural network architectures that our gaze model is built on, along with how to evaluate whether a model is actually working.
Machine Learning at Scale gave us the tools to work with datasets that were too large to process locally, while Machine Learning Systems Engineering informed the cloud architecture and deployment pipeline that allowed us to serve the model in real time.
Modern AI Strategy shaped how we approached Eye2Voice as more than a machine learning project. This course encouraged us to think critically about the problem we were solving, the stakeholders affected by it, and how our solution compared to existing assistive technologies. That perspective helped us keep the focus on building a solution to a communication challenge rather than simply optimizing a model.
No single course got us there. It was the combination of technical depth, systems thinking, and the constant practice of collaborating across time zones that prepared us to build Eye2Voice together.
Do you have any future plans for the project? Do you plan on continuing the work and if so, what are you most excited about?
Nichol: Eye2Voice is still early in its development. Looking ahead, we are building toward three goals. The first is persistent profiles, so settings, calibration, and preferences can be saved for smoother everyday use. The second is adaptive personalization, enabling the language model to learn frequently used phrases and sharpen its responses. The third is broader accessibility, reaching users with more severe physical limitations and higher frame-to-frame variability. To get there, we hope to secure funding and partner with specialty groups and non-profits to reach the people who need it most.
How could this project make an impact, or, who will it serve?
Clara: Eye2Voice is designed for people who have thoughts to share but no reliable way to share them.
This includes individuals with paralysis, traumatic brain injuries, neurodegenerative diseases, and some nonverbal autistic individuals. In the United States, an estimated 5.4 million people experience physical paralysis each year, and roughly one in four children with autism is nonverbal or minimally verbal.
But the impact extends beyond the person trying to communicate. It reaches the parent trying to understand a child, the spouse sitting beside a hospital bed, the nurse interpreting a blink, and the family member wondering if they’re missing something important. Nearly 59 million Americans serve as caregivers for adults with disabilities, many navigating the daily uncertainty that comes when communication becomes difficult or impossible.
Our hope is that Eye2Voice can make those moments a little less uncertain. By lowering the barriers to assistive communication, we aim to help more people express themselves with technology that adapts to their needs.
Now that you’ve finished your capstone, what advice would you give yourself at the start of the program? What do you wish Day One you knew?
Clara: You belong here.
Four years ago, I almost didn’t apply. I was convinced I’d be the least qualified person in every room, and honestly, that fear didn’t fully go away. It just stopped being relevant. I was accepted, told to shore up my math before starting, and proceeded to learn less calculus than anyone had hoped. I graduated with a 4.0, and our team took home one of the program’s highest honors. Self-doubt, it turns out, is not a particularly reliable predictor of performance.
On day one, I had a lot of fear about what the program would take from me. I was working full time and managing a chronic illness, and I assumed adding a rigorous graduate program would mean sacrificing most of my free time and energy. What I didn’t expect was how much it would give back.
When the thing you are doing genuinely matters to you, even the hard parts feel meaningful. The late nights building Eye2Voice were still late nights, but they never felt wasted. When you’re solving something real—something that could change how a person communicates with their family, or building an algorithm that can detect blindness before it’s too late to intervene—the effort stops feeling like a cost and starts feeling like an investment.
MIDS gave me more energy, curiosity, and connection than it ever took away.
Anything else you’d like to share?
Keta: One thing I would like to share is that this project was much more than a technical exercise for our team. While we were excited about the machine learning, computer vision, and system design challenges, what kept us motivated was the potential impact on real people.
Winning the MIDS Capstone Award was incredibly meaningful, but the most rewarding part was seeing the project evolve from an idea into a working solution that has the potential to improve people’s independence and quality of life. We hope Eye2Voice is just the beginning of a future where assistive technology is more accessible, affordable, and empowering, enabling individuals to communicate and engage with the world more independently.
