OpenAI and ChatGPT Lawsuit Exposes Dangers of AI-Generated Misinformation
In March 2023, artificial intelligence company OpenAI faced a serious legal challenge when radio host Mark Walters filed a defamation lawsuit claiming the company‘s popular ChatGPT program had generated false accusations about him defrauding a non-profit organization. The lawsuit marked one of the first high-profile cases of an AI company being held legally responsible for misinformation produced by its systems – but it is unlikely to be the last.
The rise of large language models (LLMs) and AI-powered chatbots like ChatGPT has unlocked exciting new possibilities in fields ranging from customer service to education to creative writing. However, the Walters lawsuit underscores a darker side to this rapidly-advancing technology – the risk of highly convincing and harmful misinformation being generated and spread at massive scale with the authoritative veneer of AI.
The Misinformation Machine
Since its launch in November 2022, ChatGPT has been used by millions of people worldwide to engage in conversational exchanges and get answers to questions on a practically unlimited range of subjects. The system‘s ability to generate human-like text responses based on its training data has led many to treat it as an authoritative source of factual information.
However, AI experts have cautioned that LLMs do not actually possess real-world knowledge or the ability to discern truth from fiction. Instead, these models predict what text is most likely to come next based on statistical patterns in their training data. This can lead chatbots to confidently generate false claims, fabricated statistics, and even misinformation about specific individuals like Mark Walters.
A 2022 study that analyzed ChatGPT conversations found that the model generated false information in over 15% of its responses. A more recent 2023 analysis determined that as much as 20% of ChatGPT‘s outputs on news topics contained misinformation, with the model frequently confusing the names of people and places.
| Study | False Information | Named Entity Errors |
|---|---|---|
| ArXiv (2022) | 15.6% | 8.3% |
| French NLP Study (2023) | 20.4% | 16.2% |
While no comprehensive data exists on the total number of people who have been deceived by AI-generated misinformation to date, anecdotal evidence suggests it is already leading to real-world harm. Students have been accused of plagiarism due to false ChatGPT claims, job seekers have included chatbot-fabricated achievements on resumes, and social media influencers have spread AI-generated health misinformation to millions of followers.
The Legal Labyrinth
The Mark Walters defamation case against OpenAI is likely to be the first in a wave of lawsuits seeking to hold AI companies responsible for false, misleading or harmful content produced by their models and algorithms. The case touches on some of the thorniest legal questions surrounding LLMs and AI chatbots:
- Are AI companies legally liable for misinformation and false claims generated by their models, or does responsibility fall on the end user?
- Do existing laws like Section 230 of the Communications Decency Act shield AI companies from lawsuits over model-generated content?
- What duty of care do AI companies have to prevent their models from deceiving users or generating defamatory statements?
- How can harms from AI-generated misinformation be discovered and proven in court when models are constantly updated?
Legal scholars are divided on how courts are likely to resolve these novel issues. In a 2022 law review article, University of Washington law professor Ryan Calo argued that the unique nature of LLMs "might place their harmful communications outside the reach of existing tort law." Other experts like Eugene Volokh of UCLA have suggested that deploying models that generate defamatory falsehoods could make companies liable under a theory of "reckless publication."
However these debates are resolved, mounting legal exposure and liability risk is likely to be a major force shaping the development of LLMs and chatbots in the years to come. A 2022 Gartner study predicted companies could face over $3 billion in lawsuits and settlements related to AI-generated content by 2025. This has led some AI ethics experts to dub misinformation-prone LLMs "lawsuit magnets."
Technical Fixes and an AI Truth Reckoning
Behind the looming legal battles, the AI research community is grappling with how to make models less likely to confidently generate false statements. Some technical solutions that have been proposed include:
- Truth-conditional language models that are trained to assign higher probabilities to factual statements
- Improved integration of knowledge retrieval systems so models can cite authoritative sources
- Fact-checking plugins that automatically cross-reference model outputs against trusted databases
- "Uncertainty modeling" that quantifies model confidence and adds disclaimers when unsure
- Human feedback fine-tuning that rewards models for truthful outputs and penalizes falsehoods
Google‘s Factual Consistency Evaluation framework is one example of an industry effort to systematically measure how often LLMs generate false claims on different topics. OpenAI announced plans in 2022 to bolster ChatGPT‘s accuracy with web-based information retrieval plugins, though it remains unclear if they have been implemented.
However, many researchers believe that technical fixes alone will not be enough to fully mitigate the risks of AI-generated misinformation. They argue the AI industry must also grapple with deeper questions around responsible development, the race to commercialize LLMs, and public trust:
- How can companies be incentivized to prioritize accuracy and truthfulness over engaging outputs?
- What standards of testing and safety checks should be required before releasing chatbots?
- How can the limitations and uncertainties of LLMs be more transparently communicated to users?
- What guardrails and use restrictions should be placed on public-facing chatbots and LLMs?
- How can people be equipped with AI literacy skills to think critically about chatbot claims?
Some AI ethics advocates have called for the creation of an independent oversight board, modeled on the International Atomic Energy Agency, to assess LLM and chatbot integrity before models are approved for release. Others have proposed "bias bounties" to reward discovery of misinformation or inconsistencies in widely-used models.
Conclusion: The Future of Truth in an AI-Powered World
The defamation lawsuit against OpenAI over false claims generated by ChatGPT should serve as a wake-up call about the huge challenges and risks posed by the rise of unconstrained, misinformation-prone AI chatbots. As transformative as programs like ChatGPT may be, we cannot ignore their potential to deceive, mislead and cause real-world harm at a previously unimaginable scale.
The path forward will require collaboration between AI developers, legal experts, policymakers, educators and ethicists to develop standards, regulations and norms for the responsible development and deployment of LLMs. In parallel, the public will need to cultivate a healthy skepticism about AI-generated content and learn to think critically about information sources in an AI-powered world.
Just as the spread of misinformation on social media has emerged as a global challenge in recent years, AI-generated deception could become an even thornier information crisis in the decade ahead. The Mark Walters lawsuit is unlikely to be the last legal battle over an AI‘s confident false claims. But it should spur us to have proactive conversations now about the kind of AI-powered information ecosystem we want – while we still have a choice.
