From LAWFARE
AI-Generated Voice Evidence Poses Dangers in Court
By Rebecca Wexler, Sarah Barrington, Emily Cooper, Hany Farid
Gary Schildhorn received a call that no parent wants to receive. When Schildhorn picked up the phone, the voice of his panicked son told him that he had been in a car accident and was in jail. A second call, moments later, purportedly from a lawyer, gave Schildhorn instructions on how to pay the $9,000 bond. Schildhorn was preparing payment when he received a call from his real son, who was not, in fact, in jail. Schildhorn nearly fell victim to the growing trend of artificial intelligence (AI)-powered voice scams. AI-generated voices are a problem not only for fraud but also for the legal system. Indeed, accusations of AI-generated voice clones have now made their way into the courts, and the way the courts deal with audio recording evidence needs to catch up...
Over the past few years, AI-powered voice synthesis and cloning has improved at an impressive clip, culminating this past year in dramatic breakthroughs. Perhaps most striking is the ability to convincingly clone a person’s voice from as little as 30 seconds of reference audio using easily-accessible and low-cost commercial services.
Indeed, a recent suite of perceptual studies highlights the current realism of voice cloning. In a large-scale online study, we asked 300 people to listen to pairs of audio clips of people speaking. We then asked them a simple question: Were these clips from the same person, or a different person? People were actually quite good at performing this discrimination when presented with audio clips of real human beings. When the two clips came from the same person, listeners correctly detected this fact with a median accuracy of 100 percent. At the same time, they were fooled only about 10 percent of the time into thinking two similar-sounding voices from different identities were the same...
Hany Farid is a professor in the Department of Electrical Engineering & Computer Sciences and the School of Information at UC Berkeley.
Sarah Barrington is a Ph.D. candidate at the I School, advised by Professor Hany Farid. She graduated from the MIMS degree program in 2023.
