Bored of Prompt Engineering? Try DSPy

With the advent of LLMs in the recent few years, we stumbled upon something called Prompt Engineering - let’s speak a few words about it to set the context right and then let’s venture into DSPy

What is Prompt Engineering?

Prompt Engineering is the art of designing the instructions that we give to a language model, to get the desired result. Language models don’t really understand the intentions of the user, they just predict the next word based on the provided prompt, thus prompting (right) becomes very crucial.

Though there are different types of prompts and engineering associated with it, It usually comprises of the following steps

• Choosing the right instruction

• Providing examples

• Specifying the format

• Testing and refining

Well, If you’ve worked for some time with the prompts, you would have been quickly acquainted with its limitations and tedious process of trial and error associated with crafting the right prompt

Challenges with Prompt Engineering

• Writing the prompts, tweaking them with trial and error approaches

• Prompt can grow bigger and heavy, spanning to thousands of words (expensive)

• Heavily biased with Language models, often not LM agnostic (may work with GPT, but with LLaMa?)

What if a tool or framework can solve some or all of the above problems, that is DSPy!

Programming not Prompting, What’s DSPy?

DSPy is an open source declarative framework in Python that let’s you to define the steps of LLM workflow as modular components, compose them into pipelines, optimize prompts with feedback loops. This shifts the narrative that working with LLMs is more about prompting to think of it more like a program.

Just define the input and outputs, and the rest will be take care of by DSPy

• Declarative approach

• Optimizers

• Composable chains

• Open source and Flexible

Core components of DSPy

The core components of DSPy are

• Signature

• Modules

• Adapters

• Optimizer

We can look at these more in details, as we walk through the example

Configuring the LLM

import dspy
dspy.settings.configure(lm=dspy.LM("openai/gpt-4o-mini"))

In the above piece of code, we are just refering the language model we’ll want to use. This assumes that you have the dspy package installed on your machine and OpenAI API key is already set as an environment variable for the code to pick it up

Let’s define signature

A Signature in DSPy is like a contract that clearly describes:

✅ What inputs your module expects
✅ What outputs it should produce

Think of it as a blueprint that tells DSPy and the language model:

• Here are the fields you need to fill in.

• Here’s what the output should look like.

Signature can be class based or can be Inline signatures

# Class based Signature
class SentimentClassifier(dspy.Signature):
    """Classify the sentiment of a text."""

    text: str = dspy.InputField(desc="input text to classify sentiment")
    sentiment: int = dspy.OutputField(
        desc="sentiment, the higher the more positive", ge=0, le=10
    )

# Inline Signature
str_signature = dspy.make_signature("text -> sentiment")

As we can see in the above code snippet, we have created a Signature with Input and Output fields and what they can be, this enforces the behaviour of the LLM.

Modules

Encapsulates the logic for interacting with LMs, the simplest module that DSPy provides is dspy.Predict. Multiple modules can be chained together and can be used as a bigger module.

There are different types of modules that DSPy provides us out of the box

• dspy.Predict - Simple LLM interaction, often the building block for other complex modules

• dspy.ChainOfThought - Asks LM to add reasoning as part of the final answer, this modifies the signature

• dspy.ProgramOfThought - Asks LM to output code, whose execution results will dictate the response.

• dspy.ReAct - An agent that can use tools to implement the given signature.

predict = dspy.Predict(SentimentClassifier) 

output = predict(text="I am feeling pretty happy!")
print(f"The sentiment is: {output.sentiment}")

In the above code snippet, we have used the most fundamental module dspy.Predict. Let’s look at the prompt it has generated

dspy.inspect_history(n=1)

The above piece of snippet gives the prompt that was generated by dspy.

Let’s try the same with dspy.ChainOfThought Module and see what it does to the signature

cot = dspy.ChainOfThought(SentimentClassifier)

output = cot(text="I am feeling pretty happy!")
print(output)

Let’s look at the prompt generated

As we can see in the above prompt, the signature has been added with another output field called reasoning which was not user defined when used with dspy.ChainOfThought module. The reasoning field provides a justification on how it arrived to a specific output based on the user query.

I hope the above gives a primer to the DSPy framework, let’s discuss more about Adapters and Optimizers with a full fledged working example in the upcoming blog.

References:

DSPy: Build and Optimize Agentic Apps

DSPy: Learn