How to Build a PDF Chatbot Without Langchain?

How to Build a PDF Chatbot Without Langchain

Introduction

Since the release of ChatGPT in late 2022, interest in building AI-powered applications with Large Language Models (LLMs) has exploded. Dozens of frameworks, libraries and platforms have emerged to help developers quickly build LLM apps without needing to be an expert in machine learning or natural language processing.

One of the most popular frameworks is Langchain, an open source library that makes it easy to build applications with LLMs through a standard, modular interface. Langchain provides modules for common LLM tasks like prompt templates, memory, and agents that can be composed together. It also integrates with various model providers, databases, and tools in the LLM ecosystem.

While frameworks like Langchain are great for enabling rapid development of LLM apps, they aren‘t always necessary. For simpler use cases, it‘s often quicker and results in cleaner code to just build what you need from scratch using basic libraries. Relying too heavily on an extensive framework can lead to unnecessary complexity and dependencies in your project.

One common use case where you may be able to get away without a heavy framework is building a chatbot interface for a PDF document. In this post, we‘ll walk through how to build a simple PDF chatbot from scratch using fundamental NLP libraries and vector databases. This will illustrate the core concepts and workflow behind these types of applications so you understand what‘s happening under the hood of abstractions like Langchain.

What is a PDF Chatbot?

A PDF chatbot is an application that allows you to "chat" with a PDF document using natural language. You provide a PDF file, and the chatbot ingests the text, breaks it up into chunks, and stores them in a way that they can be quickly searched and retrieved based on natural language queries.

When you ask the chatbot a question, it searches for the chunks of text from the PDF that are most relevant to your question. It then passes your question along with those text snippets to an LLM via an API. The LLM generates a natural language response to your question, potentially quoting or summarizing information from the supplied PDF chunks. This response is returned to you, allowing you to get answers from the PDF without needing to read or search through it manually.

PDF chatbots can be useful for quickly extracting information from long documents, generating summaries, or exploring topics covered in the PDF. Some potential real-world applications include:

• Searchable knowledge bases constructed from PDFs
• Research and study aids for long academic papers
• Extracting key facts, figures and quotes from reports
• Analyzing legal contracts and other business documents
• Creative writing aids that generate ideas based on inputted texts

While you can certainly build PDF chatbots using frameworks like Langchain, it‘s also very doable to build a basic one from scratch using just a few Python libraries. Let‘s walk through the process step-by-step.

Building a PDF Chatbot from Scratch

Here are the key components we‘ll need to build our PDF chatbot:

1. A way to extract text from a PDF file and chunk it into pieces that fit within the context length of our LLM
2. An embedding model that can generate vector representations of text snippets
3. A vector database to store and search our text snippet embeddings
4. An LLM API for generating natural language responses based on retrieved text snippets
5. A user interface for inputting questions and displaying responses (we‘ll build a simple command line interface)

Let‘s go through each of these in more detail.

Extracting and Chunking PDF Text

The first step is to extract the raw text from our input PDF file. We can use the PyPDF2 library for this. PyPDF2 allows us to open a PDF, iterate through its pages, and extract text.

However, most LLMs have a limited context window (e.g. 4096 tokens for ChatGPT), so we can‘t just dump in the entire text of the PDF. We need to break it up into smaller chunks that fit within the context length when combined with the user‘s question.

Here‘s a simple function that extracts text from a PDF and breaks it into chunks of a specified size:

import re
from typing import List

from PyPDF2 import PdfReader

def extract_chunks(file: str, chunk_size: int) -> List[str]:
    """Extract text from PDF and break into chunks that fit in LLM context."""
    reader = PdfReader(file)
    chunks = []

    for page in reader.pages:
        text = page.extract_text()
        text = re.sub(r‘\s+‘, ‘ ‘, text).strip()

        tokens = text.split()
        start_idx = 0

        while start_idx < len(tokens):
            chunk = tokens[start_idx : start_idx + chunk_size]
            chunks.append(‘ ‘.join(chunk))
            start_idx += chunk_size

    return chunks

This splits the text into chunks based on whitespace, so some chunks may end mid-sentence. You could implement more sophisticated chunking logic to keep chunks semantically meaningful. But for our purposes, this simple approach will suffice.

Generating Embeddings

Next, we need a way to generate vector embeddings of our text chunks so that we can search for chunks relevant to the user‘s question. Embeddings are dense vectors that capture the semantic meaning of text. Chunks with similar meanings will have embeddings that are close together in vector space.

There are many embedding models available, both open source and commercial. For this demo, we‘ll use the MiniLM-L6-v2 model which is open source and available through the Hugging Face Transformers library.

from typing import List

from sentence_transformers import SentenceTransformer

def get_embeddings(texts: List[str]) -> List[List[float]]:
    """Generate embeddings for a list of texts."""
    model = SentenceTransformer(‘all-MiniLM-L6-v2‘)
    embeddings = model.encode(texts)

    return embeddings.tolist()

This will return a list of embeddings, one for each text chunk. Each embedding is represented as a list of floats.

Storing Embeddings in a Vector Database

To efficiently search our text chunks, we‘ll store the embeddings and raw text in a vector database. Vector databases are optimized for similarity search on dense vectors. There are many options available, including Pinecone, Weaviate, Qdrant, and Chroma.

For this example, we‘ll use Chroma, which is open source and can store embeddings and metadata in local files or in cloud storage.

from typing import List

import chromadb

def load_data(chunks: List[str], embeddings: List[List[float]]) -> chromadb.Collection:
    """Load text chunks and embeddings into Chroma."""
    chroma_client = chromadb.Client()
    collection = chroma_client.create_collection(name="chunks")

    collection.add(
        ids=[str(i) for i in range(len(chunks))],
        documents=chunks, 
        embeddings=embeddings,
    )

    return collection

This creates a Chroma collection and adds our chunks and embeddings to it, using auto-generated integer IDs. We return the collection object so we can use it for searches later.

Retrieving Relevant Chunks

When the user asks a question, we need to find the chunks from the PDF that are most relevant to it. To do this, we‘ll generate an embedding for the question text, and then use Chroma to find the chunks with the most similar embeddings.

from typing import List, Tuple

def search_chunks(
    query: str, 
    collection: chromadb.Collection,
    n_results: int = 3,
) -> List[Tuple[str, str]]:
    """Find the chunks most similar to the query string."""
    query_embedding = get_embeddings([query])[0]

    results = collection.query(
        query_embeddings=[query_embedding],
        n_results=n_results,
    )

    chunks = []
    for id, text in zip(results[‘ids‘][0], results[‘documents‘][0]):
        chunks.append((id, text))

    return chunks  

This generates an embedding for the query text, and then uses Chroma‘s query method to find the n_results closest chunks. It returns a list of tuples containing the ID and text of each relevant chunk.

Generating a Response with the LLM

Now that we have the chunks most relevant to the user‘s question, we can generate a response using an LLM API. For this example, we‘ll use OpenAI‘s ChatCompletion API with the gpt-3.5-turbo model.

The prompt we send to the API will include the user‘s question along with the text of the retrieved chunks. We‘ll use a simple prompt template like this:

Here is some context about a question:
{context}

Question: {question}

Answer:

And here‘s the function to generate a response from the API:

import openai
from typing import List, Tuple

def get_response(
    question: str, 
    chunks: List[Tuple[str, str]],
) -> str:
    """Generate a response to a question using context chunks."""
    context = "\n\n".join([f"Chunk {i+1} ({id}): {text}" for i, (id, text) in enumerate(chunks)])
    prompt = f"Here is some context about a question:\n{context}\n\nQuestion: {question}\n\nAnswer:"

    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[
            {"role": "system", "content": "You are a helpful assistant that answers questions based on the provided context."},
            {"role": "user", "content": prompt},
        ],
        temperature=0.5,
    )

    return response.choices[0].message.content.strip()

This concatenates the text of the retrieved chunks into a context string, and then inserts it into the prompt template along with the user‘s question. It then sends the prompt to the OpenAI API and returns the generated response text.

The temperature parameter controls the creativity of the generated text. A lower temperature will produce more focused, deterministic responses, while a higher one will produce more diverse, creative responses. 0.5 is a good balanced default value.

Building a Command Line Interface

Finally, we need a way for the user to interact with our PDF chatbot. For this example, we‘ll build a simple command line interface (CLI) using Python‘s built-in argparse module.

The CLI will allow the user to specify a PDF file and ask a question. It will then go through the steps of extracting chunks, generating embeddings, retrieving relevant chunks, and generating a response from the LLM API. Here‘s the code:

import argparse

from PyPDF2 import PdfReader
import chromadb

from chunking import extract_chunks
from embeddings import get_embeddings 
from retrieval import search_chunks
from generation import get_response

parser = argparse.ArgumentParser(description=‘Ask questions about a PDF file.‘)
parser.add_argument(‘file‘, type=str, help=‘path to the input PDF file‘)
parser.add_argument(‘question‘, type=str, help=‘the question to ask about the PDF‘)
parser.add_argument(‘-c‘, ‘--chunk-size‘, type=int, default=500, help=‘size of each text chunk in tokens (default: 500)‘)
parser.add_argument(‘-n‘, ‘--num-chunks‘, type=int, default=3, help=‘number of relevant chunks to retrieve (default: 3)‘)

args = parser.parse_args()

print(f"Extracting chunks from {args.file}...")
chunks = extract_chunks(args.file, chunk_size=args.chunk_size)

print(f"Generating embeddings for {len(chunks)} chunks...")
embeddings = get_embeddings(chunks)

print("Loading data into Chroma...")
collection = load_data(chunks, embeddings)

print(f‘Asking "{args.question}"...‘)
chunks = search_chunks(args.question, collection, n_results=args.num_chunks)

print(f"Generating response...")
response = get_response(args.question, chunks)

print("\nResponse:")
print(response)

The user can run this script from the command line, passing the path to their PDF file and their question text:

$ python pdf_chatbot.py document.pdf "What are the key takeaways from this report?"

They can also optionally set the chunk_size and num_chunks parameters to control the size of each text chunk and the number of relevant chunks to retrieve for context.

This provides a simple but flexible way for users to ask questions about a PDF document and get relevant, informative responses back from the chatbot. The CLI can be extended with additional arguments and options to customize the chunk extraction, embedding generation, retrieval, and response generation steps as needed.

Conclusion

In this post, we saw how to build a basic PDF chatbot from scratch using Python libraries for text extraction, embedding generation, vector search, and LLM APIs. We implemented the core workflow of:

1. Extracting text from the PDF and chunking it into context-friendly pieces
2. Generating embeddings for each chunk
3. Storing chunks and embeddings in a vector database for efficient similarity search
4. Retrieving the most relevant chunks for a user‘s question
5. Passing the question and context chunks to an LLM to generate a response
6. Presenting the response to the user via a command line interface

By building each part of the pipeline ourselves, we gain an understanding of the fundamental concepts and moving pieces involved in this type of application. Frameworks like Langchain abstract away a lot of these details, which can be very helpful for rapid development, but can also make it harder to understand what‘s happening under the hood.

Building a basic version from scratch is a great way to learn and provides complete control and visibility into every part of the system. It also results in a simpler, lighter-weight implementation that may be easier to deploy, maintain and debug.

However, as your application grows in scope and complexity, leveraging frameworks and abstractions can help manage that complexity and keep your codebase clean. Frameworks also often provide a wider range of integrations, utilities and optimizations that would require significant work to implement on your own.

Whether you choose to use a framework or build from scratch will depend on your specific use case and constraints. But either way, understanding the core concepts and being able to implement them yourself is valuable knowledge to have. It allows you to evaluate tools and architectures more effectively and to build custom solutions when needed.

I hope this post gave you a practical example of how to build a PDF chatbot from the ground up using basic NLP components and libraries. Despite the hype around powerful abstractions like Langchain, it‘s still very doable and instructive to assemble these types of applications from first principles.

As you explore further, think about how you might extend this basic implementation with more advanced techniques like better text chunking, prompt engineering, or embedding models. Try swapping in different vector databases or LLM APIs. Consider how you could expose this functionality through other interfaces like a web app, Slack bot, or API.

Above all, have fun exploring this exciting emerging field and building informative, engaging and creative applications with language models and unstructured data! Let me know what you come up with.