Structured LLM Output Storage and Parsing in Python

Introduction

In recent years, large language models (LLMs) have revolutionized the field of natural language processing with their incredible ability to understand and generate human-like text. By training on massive corpora of data, models like GPT-3, PaLM, and others have achieved state-of-the-art performance across a wide range of language tasks, from question answering to summarization to open-ended dialog.

However, while LLMs excel at producing fluent, convincing text output, the format of that output is often unstructured and difficult to work with programmatically. The generated text typically comes as one big string, without any metadata or schema to describe its contents. This makes it hard to extract specific pieces of information or seamlessly integrate the LLM output into downstream applications and databases.

Fortunately, the open source LangChain library provides a solution to this challenge. LangChain is a powerful Python framework designed to help developers build applications with LLMs. It offers a suite of tools for prompt engineering, memory management, agent creation, and more. In particular, LangChain‘s output parsers allow you to declaratively specify a desired output schema and have the LLM generate a valid data structure conforming to that schema. This unlocks the ability to reliably extract structured information from LLMs and store it for further analysis and use.

In this article, we‘ll take a deep dive into using LangChain and Pydantic to generate, parse, and store structured data from LLMs. We‘ll walk through concrete code examples of defining output schemas, crafting prompts to instruct the LLM to adhere to those schemas, and parsing the model‘s responses into usable Python objects. By the end, you‘ll be equipped with a valuable technique to incorporate into your own LLM-powered applications. Let‘s get started!

Defining Structured Data Models with Pydantic

The first step in parsing structured data from an LLM is to define the schema we want the output to conform to. For this, we‘ll leverage the excellent Pydantic library. Pydantic allows you to declaratively define data models using standard Python class syntax. You simply create a class inheriting from pydantic.BaseModel and list the fields you want, along with their types and any additional validation constraints. Pydantic will enforce the schema at runtime, providing useful error messages if the data doesn‘t match the expected structure.

Here‘s an example Pydantic model for data about planets:

from pydantic import BaseModel, Field

class PlanetData(BaseModel):
    name: str = Field(description="The name of the planet")
    type: str = Field(description="The type of planet (terrestrial, gas giant, etc.)")
    mass: float = Field(description="The mass of the planet in Earth masses")
    radius: float = Field(description="The radius of the planet in Earth radii")
    distance: float = Field(description="The average distance of the planet from its star in AU")
    num_moons: int = Field(description="The number of moons the planet has")
    description: str = Field(description="A brief description of the planet")

This PlanetData model defines the structure we want the LLM to output when given a prompt asking about planets. It has fields for the planet‘s name, type, mass, radius, distance from its star, number of moons, and a short description. The Field type from Pydantic allows us to provide a human-readable description for each field, which will come in handy later.

With our data model defined, we‘re ready to set up a parser to extract information matching this schema from the LLM‘s output.

Setting Up a Pydantic Output Parser

LangChain provides a PydanticOutputParser class that allows us to parse LLM output into a Pydantic object. We simply pass it the Pydantic model class we defined:

from langchain.output_parsers import PydanticOutputParser

planet_parser = PydanticOutputParser(pydantic_object=PlanetData)

Now planet_parser is a parser object that knows how to take a string of LLM output and extract the fields defined in the PlanetData class.

Importantly, the output parser is able to generate format instructions that tell the LLM how to structure its output to match our schema. We can access these instructions like this:

format_instructions = planet_parser.get_format_instructions()
print(format_instructions)

Your response should be a JSON object with the following fields:

name: The name of the planet
type: The type of planet (terrestrial, gas giant, etc.)
mass: The mass of the planet in Earth masses
radius: The radius of the planet in Earth radii
distance: The average distance of the planet from its star in AU
num_moons: The number of moons the planet has
description: A brief description of the planet

Your response must conform to this schema. Here is an example of a valid response:

{
  "name": "Mars",
  "type": "terrestrial",
  "mass": 0.107,
  "radius": 0.532,
  "distance": 1.52,
  "num_moons": 2,
  "description": "Mars is a cold desert planet with a thin atmosphere. It has polar ice caps and evidence of past liquid water on its surface."
}

As you can see, the format instructions provide a clear description of the JSON schema the LLM should adhere to, along with an example. By including these instructions in our prompt, we can guide the model to output data in the structured format we need.

Creating a Prompt Template

To make it easy to query the LLM for planet data, let‘s set up a prompt template that incorporates the format instructions. We‘ll use LangChain‘s PromptTemplate class:

from langchain import PromptTemplate

template = """
You are an AI assistant expert in planetary science. Given the name of a planet, you will return key information about it.

Planet: {planet}

{format_instructions}
"""

planet_prompt = PromptTemplate(
    input_variables=["planet"], 
    template=template,
    partial_variables={"format_instructions": planet_parser.get_format_instructions()}
)

Our PromptTemplate takes the name of a planet as input, inserts it into the prompt string, and appends the format instructions from our output parser. Now we can easily generate custom prompts for any planet we want to query:

prompt = planet_prompt.format(planet="Jupiter")
print(prompt)

You are an AI assistant expert in planetary science. Given the name of a planet, you will return key information about it.

Planet: Jupiter 

Your response should be a JSON object with the following fields:

name: The name of the planet
type: The type of planet (terrestrial, gas giant, etc.)
mass: The mass of the planet in Earth masses
radius: The radius of the planet in Earth radii
distance: The average distance of the planet from its star in AU
num_moons: The number of moons the planet has
description: A brief description of the planet

Your response must conform to this schema. Here is an example of a valid response:

{
  "name": "Mars",
  "type": "terrestrial", 
  "mass": 0.107,
  "radius": 0.532,
  "distance": 1.52,
  "num_moons": 2,
  "description": "Mars is a cold desert planet with a thin atmosphere. It has polar ice caps and evidence of past liquid water on its surface."
}

With our prompt ready to go, it‘s time to send it to the LLM and parse the results!

Putting It All Together: Querying the LLM

Finally, we‘ll use our prompt template and output parser to retrieve structured planet data from the LLM. For this example, we‘ll use the PaLM API from Google:

from langchain.llms import GooglePalm

llm = GooglePalm()

planet = "Neptune"
prompt = planet_prompt.format(planet=planet)

llm_output = llm(prompt)
print(llm_output)

parsed_data = planet_parser.parse(llm_output)
print(parsed_data)
print(f"Planet: {parsed_data.name}")
print(f"Type: {parsed_data.type}") 
print(f"Mass: {parsed_data.mass} Earth masses")
print(f"Radius: {parsed_data.radius} Earth radii") 
print(f"Distance: {parsed_data.distance} AU")
print(f"Moons: {parsed_data.num_moons}")
print(f"Description: {parsed_data.description}")

{
  "name": "Neptune",
  "type": "ice giant",
  "mass": 17.147,
  "radius": 3.883,
  "distance": 30.07,
  "num_moons": 14,
  "description": "Neptune is the furthest known planet from the Sun. It is an ice giant with a dynamic atmosphere and strong winds. Neptune has a faint ring system and many icy moons, including Triton which orbits in the opposite direction of the planet‘s rotation."
}

Planet: Neptune
Type: ice giant
Mass: 17.147 Earth masses
Radius: 3.883 Earth radii
Distance: 30.07 AU
Moons: 14
Description: Neptune is the furthest known planet from the Sun. It is an ice giant with a dynamic atmosphere and strong winds. Neptune has a faint ring system and many icy moons, including Triton which orbits in the opposite direction of the planet‘s rotation.

Here‘s what‘s happening:

1. We create a GooglePalm LLM instance to access the PaLM API.

2. We generate a prompt using our template and the name of the planet we want to query.

3. We send the prompt to the LLM, which returns its output as a string.

4. We print out the raw output, which shows the LLM adhered to our format instructions and generated a JSON object containing the planet data we asked for.

5. We pass the LLM output to our planet_parser, which parses the JSON string into a structured PlanetData object.

6. We access the individual fields of the parsed data and print them out.

That‘s it! With just a few lines of code, we were able to generate a schema-conformant data structure from natural language output. The possibilities for integrating this structured data into other applications and databases are endless.

Real-World Applications

Parsing unstructured LLM output into well-defined schemas opens up a huge range of potential use cases. Here are just a few examples:

• Analyzing product reviews: You could use an LLM to parse freeform user reviews into structured data containing fields like sentiment, product name, specific complaints/praise, and more. This would allow you to easily aggregate review data, identify common issues, and track changes in customer sentiment over time.

• Building informational chatbots: Imagine building a chatbot interface that uses an LLM to answer user questions on a particular topic, like a sports team or historical event. By defining a schema for the desired information and parsing the LLM‘s responses, you could store the extracted data in a knowledge base for efficient retrieval. Users could then query the chatbot for specific stats, dates, names, and other structured info rather than sifting through long passages of text.

• Enriching job postings: Job search sites could leverage LLMs to parse raw job description text into structured fields like job title, required skills, years of experience, salary range, benefits, and more. This would enable powerful faceted search and filtering to help job seekers quickly find relevant positions.

• Generating SQL queries: You could even use an LLM to translate natural language queries into structured SQL queries. By defining a Pydantic model representing a valid SQL query and providing examples, you could enable non-technical users to ask questions about data in a database and get back properly formatted queries ready to execute.

There are countless other applications in domains like finance, healthcare, e-commerce, and more. The key is to establish clear input and output schemas and leverage tools like LangChain to seamlessly translate between unstructured and structured representations.

Conclusion

In this article, we explored how to generate, parse, and store structured data from natural language model output using LangChain and Pydantic. We walked through the process of defining an output schema, setting up a PydanticOutputParser, creating a PromptTemplate to provide format instructions to the LLM, and querying the model to retrieve and parse the results.

By taking advantage of these powerful libraries, you can unlock the full potential of large language models and build applications that seamlessly integrate human-like natural language understanding with the structure and rigor of traditional databases. We encourage you to try out these techniques on your own data and see what kinds of insights and capabilities you can unlock.

It‘s worth noting that LangChain provides a variety of other output parsers beyond just the Pydantic JSON parser, including parsers for lists, comma-separated values, dates and times, and more. As you experiment with different use cases, consider which output format makes the most sense for your application and data needs.

With the right combination of natural language models, output parsing, and downstream integrations, the possibilities are truly endless. We‘re excited to see what you build!