OpenAI’s Whisper AI Tool Creating Dangerous Inventions, Researchers Find
Researchers have found that OpenAI’s audio-powered transcription tool, Whisper, is inventing things that were never said with potentially dangerous consequences, according to a new report.
The AI model, Whisper, has been found to create nonsensical outputs by inventing text, commonly referred to as a ‘hallucination’. These hallucinations can include racial commentary, violence, and fantasized medical treatments, posing a risk in various contexts.
Whisper is integrated with some versions of ChatGPT and is a built-in offering in Microsoft and Oracle’s cloud computing platforms. While Microsoft has stated that the tool is not intended for high-risk use, healthcare providers are starting to adopt it to transcribe patient consultations with doctors.
Despite claims of “near human level robustness and accuracy” by its maker, Whisper has been found to make mistakes in different studies. In one study, researchers found hallucinations in eight out of every 10 audio transcriptions inspected during public meetings.
In the past month alone, Whisper was downloaded over 4.2 million times from the open-source AI platform HuggingFace, making it the most popular speech recognition model on the website. However, researchers have warned against the adoption of Whisper due to the potential harmful consequences of its hallucinations.
Calls have been made for OpenAI to address the issue, as the misinterpretation or misrepresentation of speakers could have serious consequences, especially in healthcare settings where accurate transcriptions are crucial for diagnosis and treatment.
Other AI programs have also faced criticism for similar issues, with Google’s AI Overviews suggesting using non-toxic glue to keep cheese from falling off pizza based on a sarcastic Reddit comment. Apple CEO Tim Cook has acknowledged the potential for AI hallucinations in future products, emphasizing the importance of ensuring the readiness of the technology in the areas it is used.
As companies continue to develop AI tools and programs, the prevalence of hallucinations, like those found in Whisper, remains a significant concern. OpenAI has recommended against using Whisper in decision-making contexts where flaws in accuracy could lead to serious consequences.
In conclusion, the findings regarding Whisper’s hallucinations highlight the importance of thorough testing and evaluation of AI tools to ensure their reliability and accuracy in various applications.
