What is DSPy

DSPy(Declarative Self-Improving Language Programs) is a framework for building AI applications. Unlike its counterparts i.e. Langchain, Llamaindex, it emphasizes programming the LLM model over prompt engineering. In this tutorial, you will learn how to leverage the DSPy framework to create an efficient RAG pipeline for any task of your choice.

import dspy
import os
from google.colab import userdata

api_key=userdata.get("GOOGLE_API_KEY")
lm=dspy.LM("gemini/gemini-1.5-flash",api_endpoint="https://generativelanguage.googleapis.com",api_key=api_key)
dspy.configure(lm=lm)

For this tutorial, we will use Litellm as our AI provider, however, the DSPy module offers integrations for various LLMs, including Openai and Anthropic. Select the one suitable for you and let’s begin.

Components of DSPy

Modules

Modules in DSPy are fundamental building blocks that describe the behavior of Large Language Models (LLMs) as code rather than strings. This approach allows for more structured and efficient interaction with LLMs, moving beyond traditional prompt engineering.

Some of these modules include:

dspy.Predict: This module is used to define the prediction behavior of your model. It allows you to specify how the model should generate responses based on given inputs, making it a versatile tool for various tasks.
dspy.ChainOfThought (COT): Chain of Thought encourages the model to think in steps before reaching a final conclusion or response. This module is particularly useful for complex problem-solving tasks where a step-by-step approach can enhance accuracy and clarity.
dspy.ReAct: This module combines reasoning and acting, enabling the model to perform actions based on its reasoning process. It is ideal for tasks that require dynamic decision-making and interaction with external systems.

By configuring these modules, you can tailor the behavior of your LLM to meet specific requirements, ensuring that it performs optimally for your intended applications. An example below shows the use of Chain of Thought to solve a mathematics word problem.

math = dspy.ChainOfThought("question -> answer: float")
math(question="The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have?")

Signatures

Signatures helps specify input and output behavior of your LLM. They can be defined inline for simple tasks or as class-based signatures for more complex tasks requiring additional instructions or descriptions. The framework(DSPy), then expands your signatures into prompts and parses your typed outputs. In the example above, question -> answer: float is an inline signature. You can think of them as the equivalent of prompts. The below is an example of a class-based signature.

from typing import Literal
from pprint import pprint
#An example of a class based signature for generating a blog outline
class Outline(dspy.Signature):
    """Generate a blog outline."""
    topic: str = dspy.InputField(desc="A blog topic")
    outline: str= dspy.OutputField(desc="A detailed outline based on the blog topic")

blog_topic = "An intro to AI agents" 

generated_outline= dspy.Predict(Outline)#DSPy module receives signature
prediction=generated_outline(topic=blog_topic)
pprint(prediction.outline)#Dspy parses the outline

Optimizers

Optimizers are essential tools for refining the prompts and weights of your modules. They play a crucial role in enhancing the performance of your AI applications by allowing iterative improvements and also make it easy for one to apply these across multiple modules.

To set up our optimizer, we have to provide three things:

A DSPY Module

  # Define a simple DSPy module for classification
  class Classifier(dspy.Module):
      def __init__(self):
          super().__init__()
          self.prog = dspy.ChainOfThought("text -> category")

      def forward(self, text):
          return self.prog(text=text)
  classifier = Classifier()

A Training Set

  # Define the task examples with explicit inputs
  train_data = [
      dspy.Example(text="Python is a versatile programming language.", category="technology").with_inputs("text"),
      dspy.Example(text="The stock market saw a rise in tech shares today.", category="finance").with_inputs("text"),
      dspy.Example(text="Lionel Messi won another Ballon d'Or.", category="sports").with_inputs("text"),
      dspy.Example(text="NASA successfully launched a new space telescope.", category="science").with_inputs("text"),
      dspy.Example(text="Artificial Intelligence is transforming industries.", category="technology").with_inputs("text"),
      # dspy.Example(text="Healthy eating habits can improve your lifestyle.", category="health").with_inputs("Healthy eating habits can improve your lifestyle.")
  ]

A metric/ evaluation function

  # Define an evaluation function
  def evaluation_fn(example, prediction, trace=None):
      return example.category.lower() == prediction.category.lower()

Now we can compile our optimizer using the BootstrapFewShot and test it

      # Optimize via BootstrapFewShot.
      optimizer = dspy.BootstrapFewShot(
          metric=evaluation_fn,  # Use your evaluation function
          max_labeled_demos=5,
          teacher_settings={'temperature': 0.7}
      )

      # # Compile the module with the optimizer
      compiled_module = optimizer.compile(classifier, trainset=train_data)

      # # Test the optimized module
      test_text = "DSPy enables structured learning and optimization."
      prediction = compiled_module(text=test_text)
      print(f"Text: {test_text}")
      print(f"Predicted Category: {prediction.category}")

Creating our RAG pipeline

Retrieval Augmented Generation (RAG) is a method that enhances content generation by incorporating external information retrieval. This information could be proprietary or publicly available data. For instance, consider a personal chatbot assistant. In order for it to benefit you the most, you will have to give it access to your; daily schedule, goals and personal documents i.e. propriety data. Thus, we can utilize our understanding of DSPy components to create a RAG pipeline.

For this tutorial, we will use the “app_review” dataset from Hugging Face to develop an AI-powered tester for our app product. By inputting our product specifications, we can leverage the pipeline to gain insights into user preferences and make informed modifications to improve our products.

Steps to Create the RAG Pipeline:

Install Necessary Libraries: Begin by installing the required libraries, including faiss-cpu, which is essential for efficient similarity search and retrieval.
```
 pip install faiss-cpu
```
Dataset Preparation: Load the “app_review” dataset and preprocess it to ensure compatibility with the RAG pipeline. This involves cleaning the data and structuring it for optimal retrieval.

#Import Modules
from datasets import load_dataset
# Load dataset
dataset = load_dataset("app_reviews")

# Extract JSON for retrieval
documents = [f"Product_name: {d['package_name'].split('.')[-1]}.\n Review: {d['review']}"  for d in dataset["train"]]

Set up retriever: Using the embedder module, set up a retriever which takes your embedder, documents(corpus) and number of documents to return(k).

from sentence_transformers import SentenceTransformer

# Load an embedding model (you can replace with another)
embedding_model = SentenceTransformer("all-MiniLM-L6-v2")  # Fast & lightweight
embedder=dspy.Embedder(embedding_model.encode)
search=dspy.retrievers.Embeddings(embedder=embedder,corpus=documents[:20000],k=5)

Create RAG Module: Create a class which inherits from the DSPy property. In the below, we specify inline signatures, including two input signatures;context and product_specs ,and output signatures;suggestions.

 class MultiAppReviewRAG(dspy.Module):
     def __init__(self):
         self.respond = dspy.ChainOfThought('context, product_specs -> suggestions')

     def forward(self, product_specs):
         context = search(product_specs).passages
         return self.respond(context=context, product_specs=product_specs)

Test RAG Module

The below call provides product_specs for a gaming app.

rag = MultiAppReviewRAG()
response=rag(product_specs="A multiplayer action gaming app")
response.suggestions

Conclusion

In conclusion, DSPy offers a robust framework for building AI applications by focusing on programming the LLM model rather than relying solely on prompt engineering.

Apart from creating RAG pipelines, other use cases include, AI agents, Classification tasks, finetuning and more. For further exploration and examples, visit the DSPy website.

How to Build Efficient RAG Pipelines Using DSPy

Table of contents