DSPy: How to Replace Manual Prompting with Prompt Compilation

Manually tuning long text prompts is a delicate process. After a model is updated, they often break and are difficult to test. The open-source DSPy framework, developed at Stanford University, takes a different approach to the problem: it treats the prompt as part of the program and allows it to be optimized on the data.

Why manual prompting has its limits

Most projects start the same way. A developer writes a long set of instructions, fine-tunes the wording, and tests examples. It works until the model, input data, or requirements change. Then everything has to be fine-tuned all over again, and no one knows exactly which text adjustment helped. That doesn’t mean manual prompting is dead. It’s sufficient for prototypes and simple tasks. However, for systems that need to be monitored and maintained, it’s starting to get in the way.

What is DSPy?

DSPy replaces writing prompts with a declarative description of the task. In the framework, you define:

• Signature: what goes into the model and what should come out, including a brief description of the task.
• Module: how the task is solved, for example, by a direct answer or step-by-step reasoning.
• Metric: a function that determines whether the answer is correct.
• Optimizer: an algorithm that adjusts the prompt and examples so that the metric yields the best possible result. It was originally called a teleprompter.

Example: sentiment analysis

Instead of text like “answer with just one word,” describe the input and output. The model name in the code is just a placeholder; replace it with the name provided by your provider.

import dspy
from typing import Literal

dspy.configure(lm=dspy.LM("supplier/model name"))

class SentimentAnalysis(dspy.Signature):
    """Determine the sentiment of a customer review."""
    review: str = dspy.InputField(desc="Customer review of the product")
    sentiment: Literal["positive", "negative", "neutral"] = dspy.OutputField()

predictor = dspy.Predict(SentimentAnalysis)
result = predictor(review="The laptop keeps overheating – I wouldn't recommend it!")
print(result.sentiment)

Note that the DSP does not completely remove the instructions. A brief description of the task remains in the signature’s documentation string. The difference is that the final prompt text is generated by the framework, which can then modify it later.

What is a “compilation”?

Compilation means running the optimizer on your data. You’ll need a few labeled examples and a metric:

from dspy.teleprompt import BootstrapFewShot

trainset = [
    dspy.Example(review="A brilliant screen – I'm very happy with it.", sentiment="positive").with_inputs
("review"),
    dspy.Example(review="After a week, it stopped working.", sentiment="negative").with_inputs("review"),
    # ... next examples
]

def metric(example, pred, trace=None):
    return example.sentiment == pred.sentiment

optimizer = BootstrapFewShot(metric=metric, max_bootstrapped_demos=4)
compiled = optimizer.compile(predictor, trainset=trainset)
compiled.save("sentiment.json")

Beware of a common misconception. BootstrapFewShot does not create new instructions. It selects and generates examples that the model handled correctly for this task and inserts them into the prompt. Instructions can be improved by another optimizer, MIPROv2, which modifies both the instructions and the examples together. It’s recommended to try it when you have more examples—about fifty or more. Running it requires more model calls.

What happens when the model changes

If you change the model, you only need to update its settings in the code and run the optimization again. The framework will then find prompts and examples suitable for the new model. This does not guarantee a perfect result, so you should measure performance again using test data.

When DSPs Are Worth It and When They Aren’t

• It’s worth it: for tasks you can measure, for multi-stage pipelines, and for systems that will be maintained for a long time.
• It’s not worth it: for one-time scripts, tasks without clear metrics, and small amounts of data.
• Costs: Optimization consumes model calls and requires labeled examples.

What to Take Away From This

DSPy shifts the focus from fine-tuning text to designing a system that can be tested and further optimized. It is not a magic substitute for prompt engineering, nor does it solve problems caused by poor data or unclear metrics. However, for teams that have been working with language models for a long time, it’s an interesting way to make results reproducible.