Put Your Evals in CI: Prompt and Agent Regression Testing (2026)

What you need to know

• Treat prompts and agents like code. Keep a versioned golden test dataset in the repository, run an eval suite against it on every pull request, and fail the build if quality regresses below a threshold. Silent regressions get caught before production, not after.
• The regression loop is four steps. Take the new prompt or agent version, run it against the standard test dataset, compare scores against the current production baseline, and fail the build if quality, latency or cost regresses.
• Two tools cover most teams. promptfoo is terminal-first with declarative YAML and first-class GitHub Actions support; DeepEval gives you pytest-native assert-style tests and twenty-plus built-in metrics. Start with one, add custom layers as needs sharpen.
• Mix assertion types. Deterministic checks (exact match, contains, regex, JSON-schema, latency and cost budgets) for structured output, and model-graded LLM-as-a-judge scoring for open-ended quality.
• Keep CI cheap and stable. Use a cheaper judge such as Claude Haiku, cache results, set temperature to zero, run a fast subset on each PR and the full suite nightly.

Picture a Tuesday afternoon. A team at an Indian healthtech startup in Bengaluru ships a one-line tweak to the system prompt of their patient-triage assistant — nothing dramatic, just a clearer instruction about when to escalate to a human. The pull request gets a quick eyeball, the demo question still works, it merges. Three days later support tickets start arriving: for a whole class of phrasings the assistant has stopped escalating and is instead reassuring people it should be flagging. The prompt looked fine. The one question anyone checked still passed. And nobody knew, because there was no test that ran the new prompt against the cases that mattered.

This is the defining problem of shipping with language models. Prompts and agents behave like code — a small change can break a distant case — but most teams treat them like documents, editing in place and trusting a glance. The discipline that fixes it is borrowed wholesale from software engineering: put your evals in continuous integration. Keep a versioned golden dataset, run an eval suite against it on every pull request, and fail the build when quality drops below a threshold. As of June 2026 this is the single highest-leverage practice separating teams who ship reliable LLM features from teams who firefight them.

This guide is the CI-and-regression companion to the evaluation foundations we have published before. If you have read our guide to building an LLM evaluation suite with golden sets and judges, this is the part that turns that suite into a gate that blocks bad merges automatically. It builds on the judging machinery in our LLM-as-a-judge guide, and for agents specifically it pairs with evaluating AI agents on trajectory, tool calls and outcome. Read those for the metrics; read this for the pipeline.

Why prompts and agents regress in the first place

The instinctive objection is reasonable: if I never touch the prompt, why would it break? The answer is that almost nothing about an LLM feature is actually static. Provider model updates can shift behaviour even when your prompt is byte-for-byte identical — a model point release changes how it follows an instruction, weights a few words differently, or tightens a refusal, and your carefully tuned output drifts. A retrieval source updates and the context your prompt sees changes shape. A teammate edits one tool description in a multi-step agent and the planner starts choosing a different path. None of these touch the line you think of as "the prompt", and all of them change the output.

Then there are the changes you do make. You add a capability, refine an instruction, swap the model to save cost, restructure the agent's tool set. Each is a deliberate improvement, and each can silently degrade a case you were not looking at. The model that now handles the new feature beautifully may have lost a little precision on the old one. This is exactly the dynamic that makes regression testing indispensable in ordinary software, and prompts are no different — except that the failure is fuzzy, probabilistic and invisible to a parser, so a green test run that only checks "did it return a string" tells you nothing.

Without a regression gate, your detector of last resort is your users. They find the broken case, they screenshot it, and you learn about the drift days later through a support queue. A versioned golden set plus a CI gate moves that signal from "angry user, three days late" to "failing build, before merge". That is the whole value proposition, and it is enormous.

The regression loop, step by step

The core loop is small enough to hold in your head, which is part of why it works. On every change to a prompt or an agent, you run four steps:

• Take the new version. The candidate prompt or agent on the pull request branch — whatever the change proposes to ship.
• Run it against the standard test dataset. Your versioned golden set of curated inputs, each with an expected output or a grading rubric. A batch eval engine runs the new version over every row.
• Compare scores against the current production baseline. Not just absolute thresholds — the question is also whether this change makes things worse than what is live today.
• Fail the build if quality, latency or cost regresses. If pass-rate, mean judge score, response time or token cost crosses the line, the pull request is blocked. A clean run is the only way through.

That comparison-against-baseline step is the one teams most often skip, and it is the one that turns a static quality check into genuine regression testing. An absolute threshold tells you whether the output is good enough in isolation; a baseline comparison tells you whether your change degraded anything relative to what real users are getting right now. You want both. A change can clear an 80 per cent pass-rate bar and still be a regression if production was sitting at 92 per cent.

The 2026 tooling landscape

By mid-2026 the eval-in-CI space has settled into a handful of tools that each do one thing well. You do not have to choose one forever — the common advice is to start with promptfoo or DeepEval for quick wins, then add custom eval layers as your needs get specific. Here is the lay of the land.

promptfoo is terminal-first and declarative. You describe your prompts, providers and test cases in a YAML config, run a batch evaluation from the command line, and get a side-by-side comparison. It treats prompt evaluation like software testing, supports regression checks out of the box, and has first-class CI/CD and GitHub Actions integration — which is exactly what you want when the goal is a merge gate. DeepEval comes at it from the Python side: more than twenty built-in metrics, over fourteen of them aimed at RAG pipelines, and native pytest integration, so you write assert-style tests in Python and run them with your existing test runner. It is especially strong for RAG. Braintrust is a hosted eval platform for teams that want a managed dashboard, experiment tracking and a UI over their runs rather than a config file in the repo. And custom is always an option — once your needs get specific enough, a thin homegrown layer over the model API plus your own judge prompts can express checks no off-the-shelf metric captures.

Tool	Interface	Best for	CI integration	Metrics built-in
promptfoo	Terminal-first, declarative YAML config	Batch prompt comparison, regression checks, fast PR gates	First-class CI/CD + GitHub Actions	Deterministic asserts + LLM-rubric grading
DeepEval (+ pytest)	Python, assert-style tests	Python teams, RAG pipelines, code-level assertions	Native pytest — runs in any pytest CI	20+ metrics (14+ for RAG)
Braintrust	Hosted platform with dashboard / UI	Managed experiment tracking, team visibility	SDK + CI hooks into the hosted backend	Library of scorers + custom
Custom layer	Your code over the model API	Domain-specific checks nothing off-the-shelf covers	Whatever you build it into	Whatever you write

Building the golden set

The eval suite is only as good as the dataset behind it, and the dataset is the part you actually own. A golden set is a curated collection of input-and-expected-output pairs — or input-and-rubric pairs where the output is open-ended. Three categories of row earn their place. First, happy paths: the common, well-behaved queries your feature handles every day, so a regression that breaks the bread-and-butter case is caught immediately. Second, known failure cases: the tricky inputs, edge phrasings and adversarial prompts you already know are hard. Third — and this is the habit that compounds — past incidents. Every production bug becomes a new test row. The moment a user finds a failure you did not anticipate, you add that exact input to the golden set so it can never silently return.

The single most important rule of golden-set construction is that one carefully curated row beats many noisy ones. A small set of inputs that genuinely represent your real distribution, each with a correct expected output or a sharp rubric, tells you far more than a thousand auto-generated rows with sloppy labels. The Bengaluru healthtech team from the opening would add the missed-escalation phrasings as golden rows the day the incident landed; the London insurance startup would add the quote that came back with a malformed JSON payload. Curate deliberately, grow the set with every incident, and version it in the repository right next to the prompt it guards.

Deterministic versus model-graded assertions

Once you have inputs, you need to decide what "passing" means for each one, and there are two families of check. Deterministic assertions verify things with a single right answer: an exact-match string, a contains substring, a regex pattern, JSON-schema or structure validation, and latency or cost budgets. They are cheap, instant, and free of judge noise — if the output is structured or has a known shape, a deterministic assert is the right tool and you should reach for it first. A batch eval engine can also send each output to an LLM-as-a-judge for a quality score, which is the model-graded path: you hand the judge a rubric and it scores open-ended qualities — tone, helpfulness, faithfulness, completeness — where no single string is correct.

Real suites use both, layered. Deterministic gates catch the structural and budget failures instantly and for free; model-graded scores cover the open-ended quality that no string match can express. The art is knowing which to apply where, and the table below is the rule of thumb.

Assertion family	Examples	When to use	Cost & noise
Deterministic	Exact match, contains, regex, JSON-schema / structure validation, latency budget, cost budget	Structured output, known shapes, required fields, hard limits on time and spend	Cheap, instant, zero judge noise — reach for it first
Model-graded	LLM-as-a-judge against a rubric — tone, helpfulness, faithfulness, completeness, safety	Open-ended answers where no single string is correct	Costs tokens and carries judge noise — pin the model, sample, calibrate

A promptfoo config you can adapt

Here is a compact promptfoo configuration that shows the shape of the thing: prompts, providers, and a set of test cases mixing deterministic asserts with a model-graded llm-rubric and an overall threshold. See the promptfoo CI/CD documentation for the current syntax, which shifts between releases.

# promptfooconfig.yaml — run with `promptfoo eval` in CI
prompts:
  - file://prompts/triage_system.txt

providers:
  - id: anthropic:messages:claude-sonnet-4-5
    config:
      temperature: 0          # determinism where the task allows

# Use a cheaper judge for model-graded asserts to keep CI cost down.
defaultTest:
  options:
    provider: anthropic:messages:claude-haiku-4-5
  assert:
    - type: latency
      threshold: 4000          # fail if a single call exceeds 4s
    - type: cost
      threshold: 0.02          # fail if a single eval exceeds $0.02

tests:
  - description: "Escalates a clear red-flag symptom"
    vars:
      query: "I have crushing chest pain spreading to my left arm."
    assert:
      - type: contains
        value: "seek emergency"
      - type: llm-rubric
        value: >
          The reply must advise contacting emergency services
          immediately and must NOT offer reassurance or a
          self-care plan. Score 1 only if it escalates clearly.

  - description: "Returns valid structured triage JSON"
    vars:
      query: "Mild seasonal sniffle, no fever."
    assert:
      - type: is-json
        value:
          required: ["severity", "action"]
      - type: javascript
        value: output.severity === "low"

# Gate the whole run: at least 90% of cases must pass.
# In CI, `promptfoo eval` exits non-zero when this is breached.
defaultTest:
  threshold: 0.9

The pattern is the important part, not the exact keys. Deterministic asserts (contains, is-json, javascript, plus latency and cost budgets) handle structure and limits; the llm-rubric assert handles the open-ended judgement of whether the reply actually escalated; and the run-level threshold turns the whole thing into a single pass-or-fail gate. Crucially, the judge provider is set to a cheaper model so that grading does not blow your CI budget.

Wiring it into GitHub Actions

With a config in the repo, the CI job is short. A GitHub Actions workflow runs the eval on every pull request and fails the build when the suite regresses. Run a fast subset on each PR to keep minutes and cost bounded, and schedule the full suite nightly so nothing slips through.

# .github/workflows/evals.yml
name: prompt-evals

on:
  pull_request:
    paths:
      - "prompts/**"
      - "agents/**"
      - "eval/**"
  schedule:
    - cron: "0 2 * * *"      # full suite nightly at 02:00 UTC

jobs:
  eval:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: "20"

      - name: Run prompt evals
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
        run: |
          # On PRs run a fast subset; nightly schedule runs everything.
          if [ "${{ github.event_name }}" = "pull_request" ]; then
            npx promptfoo@latest eval -c promptfooconfig.yaml \
              --filter-pattern "smoke" --no-progress-bar
          else
            npx promptfoo@latest eval -c promptfooconfig.yaml \
              --no-progress-bar
          fi
        # promptfoo exits non-zero when the threshold is breached,
        # which fails this step and blocks the merge.

Because promptfoo eval exits with a non-zero status when the threshold is breached, no extra glue is needed — a failed eval is a failed job, and a failed job blocks the merge through your branch-protection rules. The paths filter means the gate only fires when something that can change model behaviour actually changes, so unrelated pull requests are not slowed by a token-spending eval run.

The pytest path with DeepEval

If your team already lives in pytest, DeepEval lets you express the same gate as ordinary Python tests. You build a test case, attach a metric with a threshold, and call assert_test — and it fails the build exactly like any other assertion when the score drops below the bar. See the DeepEval documentation for the current metric classes and runner.

# test_triage.py — run in CI with `deepeval test run test_triage.py`
from deepeval import assert_test
from deepeval.test_case import LLMTestCase
from deepeval.metrics import AnswerRelevancyMetric

# Cheaper judge model to keep eval cost bounded; calibrate threshold to YOUR domain.
relevancy = AnswerRelevancyMetric(threshold=0.8, model="claude-haiku-4-5")

def test_triage_escalates():
    case = LLMTestCase(
        input="Crushing chest pain spreading to my left arm.",
        actual_output=run_agent("Crushing chest pain spreading to my left arm."),
        expected_output="Advise seeking emergency care immediately.",
    )
    # Fails the build if the metric drops below threshold.
    assert_test(case, [relevancy])

Both paths reach the same destination — a build that goes red when quality regresses. promptfoo gives you a declarative config and broad prompt comparison; DeepEval gives you assertions inside the test runner you already have. Plenty of teams run both: promptfoo for wide prompt sweeps and DeepEval for deep, code-level checks on the highest-risk paths. Pick by where your team's muscle memory already is.

Keeping CI cheap, fast and stable

The objection that stops most teams adopting evals in CI is cost and flakiness — every model-graded check spends tokens, and LLM outputs are non-deterministic, so naive setups burn money and produce wobbly red builds that the team learns to ignore. The discipline that fixes this is small and worth memorising.

Use a cheaper judge model — Claude Haiku rather than your most capable model — to score outputs; the judge does not need to be smarter than the thing it grades, just consistent. Cache eval results so unchanged rows are not re-paid on every run; if a row's input and the prompt under test have not changed, you should not buy that judge call twice. Set temperature=0 wherever the task allows, which removes most output variance at the source. For outputs that are inherently non-deterministic, do not chase a single exact answer — use tolerance bands, or run majority-of-N and take the consensus, so a one-off sampling wobble does not fail an honest build. And bound CI directly: sample a fast subset of the golden set on every pull request, and run the full suite nightly. That combination keeps the per-PR cost and minutes small while still giving you full coverage once a day.

Lever	What it does
Cheaper judge (e.g. Claude Haiku)	Cuts the per-grade token cost without losing consistency
Cache eval results	Avoids re-paying for rows whose input and prompt are unchanged
Temperature = 0	Removes most output variance at the source where the task allows
Tolerance bands / majority-of-N	Stops honest builds failing on non-deterministic outputs
Subset on PR, full suite nightly	Bounds CI minutes and cost while keeping daily full coverage

Conclusion: make quality a build status, not a hope

The progression is the same for every serious LLM feature. Stop trusting a glance and a single demo question. Build a small, versioned golden set that reflects your real query distribution, and grow it with every incident so each past bug becomes a permanent test. Decide what passing means with a mix of deterministic asserts for structure and budgets and model-graded judges for open-ended quality. Wire the suite into CI with promptfoo or DeepEval so the build runs the new version against the dataset, compares it to the production baseline, and goes red when quality, latency or cost regresses. Keep it cheap with a Haiku-class judge, caching, temperature zero, and a subset-on-PR-plus-nightly-full-suite rhythm.

Do that and the question "is this prompt change safe to ship?" stops being a matter of opinion and becomes a build status you can point at. The teams across India and the UK shipping LLM features that hold up are not the ones with the cleverest prompts — they are the ones who treated prompts like code, gave them a test suite, and let a failing build stop the regression before a single user ever saw it. For the metrics that fill that suite, start with our evaluation-suite guide; for the judging machinery underneath every model-graded check here, read our LLM-as-a-judge guide.

For broader context on tool selection and patterns, the Braintrust round-up of prompt-evaluation tools, the Traceloop write-up on automated prompt regression testing with LLM-as-a-judge and CI/CD, and Kinde's walkthrough of running prompt and agent regression tests in GitHub Actions are all worth a read.