Today's What Could Go Wrong: A technology company that uses artificial intelligence to mimic voices is adding more "safeguards" to its technology after it was used to generate clips of celebrities reading offensive content.

Eleven Labs (opens in new tab) is a research company that specializes in AI speech software that generates realistic-sounding voices for dubbing audiobooks, games, and articles in any language One of its tools, called Voice Lab, allows users to speak a minute-long One of the tools, called Voice Lab, allows users to create a "clone" of their voice by simply uploading a one-minute clip of them speaking. The cloned voice can then be used to read up to 2,500 words using a text-to-speech function. I know what you are thinking, how could anyone abuse this system by uploading someone else's voice?

Enter 4Chan, whose contributors used the application to create sound clips of celebrities uttering racist, homophobic, and other offensive messages and spread them online. Some of the most widely circulated clips, not linked here, include Emma Watson reading excerpts from Adolf Hitler's "Mein Kampf" and Joe Biden announcing that the US will send troops to Ukraine.

ElvenLabs says it is taking steps to ensure that Voice Lab is not used for "malicious purposes," posting in a lengthy Twitter thread a plan to keep its technology out of the wrong hands. (Open in new tab)

Eleven Lab claims that it has "always had the ability to trace generated audio clips back to specific users. The company plans to release a tool next week that will allow anyone to verify and report that a clip was generated using its technology. [According to the company, the malicious content was created by "free anonymous accounts," which will add a new layer of identity verification. VoiceLab will only offer a paid version of the site, and the free version will be removed from the site immediately. Eleven Labs is currently tracking and banning accounts that create harmful content in violation of its policies.

Eleven Labs admits that putting the technology behind a paywall "does not always prevent abuse," but states that it "reduces the anonymity of voice lab users and makes them think twice before sharing inappropriate content."

But what about the issue of people generally using celebrity voices instead of their own? Eleven Labs suggests that instead of uploading any audio file, users be asked to read sample prompts in order for the AI to learn their own voice.

Free accounts can also use the text-to-speech feature, but they can only use pre-prepared voices. Eleven Labs says it will continue to monitor the situation and will reimburse affected accounts.

Just this week, OpenAI's CEO said that the misuse of AI could be "a blackout for all of us." But seriously, what did Eleven Labs think would happen if we gave the Internet the tools to make any voice say anything?