Exploring Open-Source Alternatives to OpenAI Models

The Rise of Open Source Alternatives to OpenAI Models

2022 was a breakthrough year for open source large language models (LLMs) and multi-modal models (LMMs). Thanks to the release of powerful open models from organizations like Meta, Hugging Face, and Stability AI, developers now have compelling alternatives to OpenAI‘s industry-leading but closed-source GPT-3 and GPT-4 models.

Open source AI offers several key advantages:

1. Transparency – With open models, you can inspect the model architecture, training data, and outputs. This is essential for understanding model behavior, detecting bias, and complying with regulations.

2. Customization – Open models can be fine-tuned and adapted to your specific use case, domain, or style. Closed models only allow limited prompt engineering.

3. Cost savings – Running open models on your own hardware or cloud instances can be significantly cheaper than paying per-query fees to an API. Quantization techniques can further reduce compute costs.

4. Community – Open models benefit from a thriving ecosystem of researchers, developers, and organizations collaborating to extend and improve them. Closed models rely on a single vendor.

Let‘s survey the current open source LLM and LMM landscape and explore how to put these powerful models to use.

Open Source Language Models to Rival GPT-3

While OpenAI‘s GPT-3 set a new standard for language model performance when released in 2020, a new generation of open source LLMs is quickly catching up. Here are some of the top contenders:

Llama 2 (Meta)
Meta‘s Llama 2 models, released in July 2023, span sizes from 7 billion to 70 billion parameters. In benchmarks, the largest Llama 2 model matches GPT-4‘s performance on tasks like summarization, question answering, and coding. Notably, Llama 2 models cost 30-50% less to run than GPT-3.5.

Mistral (Hugging Face)
Mistral is an open source chat model from Hugging Face. At just 7B parameters, the base Mistral model already rivals GPT-3.5 on many benchmarks. Fine-tuned versions like OpenHermes-2.5 and Zephyr-7B further improve performance while retaining a compact size. This makes Mistral a top choice for applications that need a powerful but efficient model.

Other Notable Open LLMs

• Pythia (Anthropic) – A suite of open source LLMs from 3B to 12B parameters fine-tuned for dialogs
• Falcon (Hugging Face) – A series of open source LLMs from 7B to 40B parameters
• GPT-J (Anthropic) – A 6B parameter model that matches GPT-3 on many tasks
• Dolly (Databricks) – An open source model fine-tuned on high-quality dialogs
• Bloom (BigScience) – A 176B LLM focused on multiple languages and low-resource settings

Beyond Language: Open Source Multi-Modal Models

One of GPT-4‘s most impressive capabilities is its ability to understand images in addition to text. OpenAI has not yet released this model, but several open source alternatives are making quick progress:

Llava (UCBerkeley, UW, CMU)
Llava combines an open LLM (Vicuna-13B) with an open vision encoder (CLIP). By connecting language and vision during fine-tuning, Llava can answer questions about images, visually ground its language understanding, and even imagine new images via text-to-image models. Llama performs competitively with GPT-4 on visual QA datasets.

BakLlava (Skunkworks AI)
BakLlava also uses the Llama architecture but swaps in the smaller Mistral-7B LLM. Despite its more compact size, BakLlava achieves results similar to the larger Llava-13B. This demonstrates how open source models can be remixed and optimized for different preferences.

Fuyu-8B (Adept)
Fuyu-8B takes a novel approach of training a multi-modal transformer from scratch on both images and text. Rather than combining separate vision and language models, everything is learned end-to-end. This allows for more streamlined inference and potentially richer cross-modal interactions.

Other Open Source Multi-Modal Models

• BALLE fusion models (Anthropic) combine open LLMs with CLIP for zero-shot visual QA
• MiniGPT-4 (Microsoft) distills visual-language mappings into a compact student model
• Mira (Anthropic) remixes Stable Diffusion‘s image generator with open LLMs for text-prompted editing

Tools for Running Open Models Locally

So you‘ve found an open model that suits your use case. How do you actually run it? The good news is that a vibrant ecosystem of tools has emerged to deploy open source models on consumer hardware.

The first step is often quantizing the model to a lower numeric precision such as 16-bit floating point (fp16) or 8-bit integer (int8). Libraries like GPTQ, AutoGPTQ and GGML can compress models by up to 4x with minimal loss in quality. This allows even large models like Llama-13B to run on a gaming laptop‘s GPU.

With a quantized model in hand, you have several options for local inference:

• oobabooga/text-generation-webui provides a browser-based UI for chatting with open LLMs
• llama.cpp is a popular tool for running Llama models and its many derivatives locally
• AutoGPT allows LLMs to interface with web data and a growing suite of AI tools
• VLLM (Versatile Large Language Model) supports a variety of open source LLMs
• Langchain provides modules for chaining LLM calls with other data sources and services

Thanks to these powerful open source tools, running LLMs locally is becoming increasingly accessible to developers and researchers.

Deploying Open Models in the Cloud

For production use cases, you‘ll likely want to deploy open models in the cloud rather than running them locally. A few options are available:

Hugging Face Inference API
Hugging Face hosts a number of open models that can be called via API, including Falcon-7B/40B, Dolly 2.0, and Mistral-7B. Pricing is based on compute time, with the option to call models on cheaper CPU instances. Fine-tuning is also supported.

AI Platform Hosting
New AI platforms are emerging to host open models with features like autoscaling, spot instances, and fine-tuning. These include:

• Anthropic API for deploying Pythia, GPT-J and other open models
• CoreWeave for running open models on bare metal GPUs
• Stability AI API for open source language and image models

Roll Your Own
Finally, you can always deploy open models on your own cloud instances. With the proper quantization and serving optimizations, models like GPT-J-6B can run on a single GPU on cloud platforms like AWS, GCP, and Azure. Larger models may require distributed inference across multiple GPUs or machines.

The Future of Open Source AI

The pace of progress in open source AI is astounding. In the coming months, we can expect to see:

• Even larger open language models (e.g. Llama 2 at 150B+ parameters, open source PaLM)
• Stronger multi-modal models that can reason over complex scenes and diagrams
• Open source models that incorporate retrieval, planning and algorithmic capabilities

At the same time, techniques like quantization, distillation, and optimized inference will put these ever more powerful models within reach of more developers. The era of only tech giants having access to cutting-edge AI is coming to an end.

As the open source AI ecosystem grows, it‘s an exciting time to get involved. Whether you‘re a developer looking to build a new application, a researcher exploring new model architectures, or an organization seeking more transparency and control over your AI systems, open source models offer a compelling path forward. The future of AI is open.