Mom warns of hoax using AI to clone daughter's voice

An Arizona mother is warning other parents after she almost became the target of a kidnapping hoax that sounded all too real.

Jennifer DeStefano said her experience began when she received a phone call from a number she didn't recognize. She wouldn't have picked up but she knew her 15-year-old daughter Briana was on a skiing trip and she worried there could have been an emergency.

"It's my daughter’s voice crying and sobbing, saying, 'Mom.' And I'm like, 'OK, what happened?'" DeStefano recalled.

"She's like, 'Mom, these bad men have me. Help me, help me.'"

Then, DeStefano said a man demanded she pay a ransom in exchange for Briana's safe release. But he told her he didn't want a wire transfer for the ransom and wanted to go and pick her up instead.

Luckily, DeStefano was able to confirm her daughter was safe within minutes but said the scheme utilized artificial intelligence or AI to reproduce her daughter's voice, a disturbing chain of events that hit close to home for the family.

"I started to wonder, like, if these people were asking to track my mom and pick her up, they could have obviously been putting some information together to try and track me or some of my siblings to actually make this a reality," Briana said. "So it definitely scared me."

Sinéad Bovell, the founder of Waye, a technology education company, told "GMA" it may not be difficult to obtain a recording of one's voice considering the prevalence of social media.

"Most people in the modern age have some form of an online identity and have probably spoken in some way, in some aspect that's been recorded, especially if you're under the age of 25," Bovell said. "So this becomes very, very challenging as we move into a future where we do have these AI generators or synthetic audio when it comes to verification and validation."

Experts say voice cloning can happen in a matter of seconds with the right software and it's getting easier for criminals and bad actors to access and use, a troubling concern since there is currently very little oversight for AI technology.

"There's a lot of positive and exciting aspects about these technologies," Bovell said. "But then of course, they also come with a lot of risks and harms."

To minimize avoid a similar AI voice cloning scam, experts recommend keeping social media profiles private and looking for red flags, such as phone calls from unknown or international area codes and phone numbers.