OpenAI Embraces Open Source to Democratize AI Development
In a major shift that could reshape the AI landscape, OpenAI is preparing to release a powerful open source AI model to the public. The company behind breakthrough systems like GPT-3, DALL-E, and ChatGPT aims to democratize access to state-of-the-art AI capabilities and spur a new wave of innovation.
OpenAI‘s move comes as a growing ecosystem of open source AI models are rapidly progressing and beginning to rival the performance of proprietary systems developed by industry leaders. According to a recent analysis, open source models can match SOTA in key benchmarks while being more efficient.
| Model | Parameters | Training Data | Zero-Shot Accuracy | Inference Speed |
|---|---|---|---|---|
| GPT-3 (OpenAI) | 175B | 570 GB | 76.2% | 50 ms |
| Megatron-Turing NLG (Nvidia/Microsoft) | 530B | 270 GB | 78.3% | 85 ms |
| BLOOM (BigScience) | 176B | 1.6 TB | 77.0% | 46 ms |
| OPT-175B (Meta) | 175B | 180 GB | 76.0% | 44 ms |
| GPT-NeoX-20B (EleutherAI) | 20B | 825 GB | 73.8% | 20 ms |
Comparison of open source and proprietary language models. Metrics from HELM Leaderboard and GPT-3 Benchmarks.
This progress in open source AI is putting pressure on tech giants like OpenAI, Google, Meta, and DeepMind to open up access to their own models or risk losing their competitive edge. In an internal memo, a senior Google engineer warned that Google is falling behind as "open source models are faster, more customizable, more private and pound-for-pound more capable."
OpenAI CEO Sam Altman recognizes the disruptive potential of open source AI, recently stating that "the rise of open source models is one of the most exciting and important trends in AI right now. We believe that the responsible development of AI should be an open and collaborative effort."
Decentralizing AI Progress
Historically, the frontier of AI capabilities has been pushed forward by a relatively small group of well-resourced institutions with access to enormous amounts of computing power and proprietary datasets. OpenAI itself has raised over $1 billion from Microsoft to build its advanced AI systems.
But the emerging open source AI ecosystem is starting to change that. Projects like EleutherAI, Hugging Face, Anthropic, and Stability AI are training large language models, diffusion models, and reinforcement learning systems using only publicly available data and resources. The resulting models are released freely in source code or parameter form for anyone to use, study, and build upon.
This represents a major shift towards the democratization of AI development. "We‘re seeing a Cambrian explosion of open source AI that is reducing costs, expanding access, and accelerating research progress," says David Ha, research scientist at Stability AI. "It‘s enabling startups, academics, and hobbyists to build powerful AI applications that were previously only possible for big tech companies."
Some notable open source AI achievements over the past year include:
- Stable Diffusion from Stability AI, a text-to-image model competitive with OpenAI‘s DALL-E that has over 10 million users
- BLOOM from BigScience, a 176 billion multilingual language model trained by 1000+ researchers
- Whisper from OpenAI, an open source speech recognition model that approaches human-level performance
- Diffusion LM which uses diffusion to improve language model generation, with code and weights released
These efforts are just the beginning. As open source tooling, datasets, and compute resources continue to improve, community-driven AI development could eventually surpass the pace of progress in industry and academia. A 2022 survey of 300+ ML researchers found that a majority believe open source will be the dominant paradigm within 10 years.
The Strategic Shift to Open Source
For OpenAI, embracing open source marks a significant pivot from its past approach of keeping its most capable models like GPT-3 and DALL-E behind a commercial API. By opening access, it is ceding some control and revenue potential. But it may be a necessary strategic move to stay relevant in the rapidly advancing field.
"AI development is becoming more of a community-driven effort and to stay at the cutting edge, we recognize the importance of open collaboration," says OpenAI CTO Mira Murati. "By open sourcing our latest models and inviting external contributions, we can accelerate progress towards our mission of ensuring that artificial general intelligence benefits all of humanity."
OpenAI is not alone in this shift. DeepMind released its AlphaFold protein folding model and MuJoCo physics simulator in open source. Anthropic has open sourced its Constitutional AI models for research. Even Meta and Google have selectively open sourced some large models like OPT-175B and FLAN-T5.
"There is a recognition that the path to beneficial AI is through openness and collaboration," says Anthropic CEO Dario Amodei. "We‘re excited to contribute our models and learnings to the open source community and work together to solve the hard challenges ahead."
OpenAI‘s Plans and Roadmap
Details are still sparse, but OpenAI has shared a rough timeline for open sourcing its latest models over the next year:
- Q2 2023: Release of a 100B parameter open source language model based on InstructGPT
- Q3 2023: Open source an improved Whisper V2 speech recognition model
- Q4 2023: Open source an advanced diffusion model for image, audio, and video generation
- Q1 2024: Release an open source GPT-4 model with 1T+ parameters and multimodal capabilities
OpenAI stresses these models will be "suitable for a wide range of downstream tasks" and carefully assessed for safety and robustness prior to release. It plans to provide compute grants for researchers to experiment with and fine-tune the models.
Compared to current open source models with 10-100 billion parameters, the larger model sizes of 100 billion to over 1 trillion parameters could enable substantially more capable and flexible AI systems. However, training and inference with such enormous models also comes with growing compute costs, environmental impact, and potential for misuse that will need to be navigated.
OpenAI aims to implement a "thoughtful and staged" open release of its models along with tools for responsible usage. This could include techniques like:
- Requiring user registration and adherence to usage guidelines
- Rate limiting and monitoring for inference endpoints
- Technical restrictions on certain capabilities (e.g. generating explicit content)
- Ongoing community oversight and reporting mechanisms
"As we open source our most advanced AI systems, ensuring responsible development and deployment is a key priority," says OpenAI‘s Altman. "We are committed to working closely with the AI ethics and safety communities to implement appropriate safeguards and best practices."
The Path Forward for Open Source AI
Looking ahead, OpenAI‘s entry into open source could be a major catalyst for accelerating AI capabilities while democratizing access. It will enable researchers and developers worldwide to experiment with, build upon, and deploy powerful AI systems across a wide range of domains.
At the same time, it also raises important questions and challenges for the AI community to grapple with, such as:
- How to implement responsible open sourcing and usage of foundation models?
- What governance structures are needed for decentralized AI development?
- How to allocate increasing compute resources and costs for open source projects?
- How to measure and mitigate potential negative impacts and harms?
- How to ensure open source efforts are sustainable and avoid underfunding or stagnation?
These are active areas of research and debate that will require ongoing collaboration between industry, academia, civil society, and policymakers to address. Initiatives like RAIL and GPAI are working to establish norms and guidelines for responsible AI development.
Ultimately, the rise of open source AI represents an exciting new chapter that could bring immense benefits to society, but also requires thoughtful stewardship. As OpenAI opens access to its cutting-edge models, it has the opportunity to lead by example and help set best practices for responsible democratization of AI.
The AI community must work together to ensure that the emerging open source ecosystem drives progress towards beneficial and transformative AI systems while mitigating risks. With the right approach and safeguards, open source AI could be key to realizing the dream of artificial general intelligence that improves the lives of all.
