Skip to main content

How to use Pipelines

Dan Cleary avatar
Written by Dan Cleary
Updated over 2 weeks ago

Pipelines are PromptHub’s CI/CD layer for prompts.

A Pipeline bundles one or more evaluators with an optional dataset and then auto‑runs on the events you choose—commits, merge requests, API calls, etc.

Pipelines help you:

Automate Guardrails – set up safety evals to stop secret leaks, profanity, or regression failures before they hit production.

Stay Data‑Driven – benchmark every new prompt version against the same dataset for apples‑to‑apples scoring.

Move Faster With Confidence– see pass/fail right inside the merge request plus a full history in the Pipelines dashboard.

Quick Walk-Through

This is based on the video above. For more detail, check out the video!

Goal: Ensure a Product‑Feedback Classifier prompt keeps passing for all test cases in a dataset before any new prompt version is deployed.

  1. Create a pipeline

    • Event: Merge Requests

    • Evaluator: LLM‑as‑Judge

    • Dataset: product‑feedback‑classifier (5 rows)

    • Sampling: 100 %

  2. Open a Merge Request – PromptHub automatically runs the 5 tests and shows a green check when all pass (or blocks the merge if any fail).

Review results – Go to the history tab and click the open merge request to view the pipeline run details. You can also view all pipeline data on the pipeline dashboard page.





Additional information

  • Availability – Pipelines are available on Team and Enterprise plans.

  • Roadmap – Slack alerts!


Did this answer your question?