The company stated that it has not yet determined how to implement the technology.
On Friday, artificial intelligence startup OpenAI showcased a preview of its new digital voice generator, Voice Engine, which it claims can create natural-sounding speech from just a 15-second audio sample.
Voice Engine is the latest development from the San Francisco-based company, known for its popular ChatGPT chatbot and DALL-E image generator.
In a blog post, OpenAI described various potential applications for Voice Engine, including assisting with reading for children, language translation, and voice restoration for cancer patients.
However, some social media users expressed concerns about possible misuse, such as unauthorized voice imitation or deepfakes.
OpenAI indicated that it is postponing the wider release of the software due to concerns about potential misuse, particularly in the context of an election year. The technology, first developed in late 2022, has been used internally in other products.
“We are taking a cautious and informed approach to a broader release due to the potential for synthetic voice misuse,” the company said in an unsigned post.
“We aim to initiate a discussion on the responsible use of synthetic voices and how society can adapt to these new technologies,” it continued. “Based on these discussions and the results of small-scale tests, we will decide whether and how to scale up the deployment of this technology.”
The 2024 election has already seen its first fake voice incident, with a January robocall in New Hampshire imitating President Joe Biden. A Democratic operative later admitted to commissioning the fake voice using AI and assistance from a New Orleans street magician.
Following this incident, the Federal Communications Commission unanimously voted to ban unsolicited AI robocalls.
OpenAI acknowledged the political risks associated with its technology in its blog post.
“We understand that creating speech that mimics people’s voices carries significant risks, particularly in an election year,” the company stated.
The company mentioned that it is collaborating with U.S. and international partners from various sectors—including government, media, entertainment, education, and civil society—to incorporate their feedback into its development process.
OpenAI’s usage policies prohibit impersonation without consent or legal authorization and emphasized that widespread deployment should include “voice authentication experiences” to ensure the original speaker has approved the use of their voice. The company also proposed a “no-go voice list” to prevent generating voices that closely resemble those of prominent figures.
However, the tech industry has found it challenging to effectively detect and label AI-generated content, with solutions like “watermarking” often being easily removed or bypassed.
Geoffrey Miller, an associate professor of psychology at the University of New Mexico, raised concerns on the platform X about potential misuse by criminals.
“When millions of older adults are defrauded out of billions of dollars by these deepfake voices, will @OpenAI be ready for the tsunami of litigation that follows?” he asked. The company had not yet responded to his inquiry.
