Pipelines are PromptHub’s CI/CD layer for prompts.
A Pipeline bundles one or more evaluators with an optional dataset and then auto‑runs on the events you choose—commits, merge requests, API calls, etc.
Pipelines help you:
Automate Guardrails – set up safety evals to stop secret leaks, profanity, or regression failures before they hit production.
Stay Data‑Driven – benchmark every new prompt version against the same dataset for apples‑to‑apples scoring.
Move Faster With Confidence– see pass/fail right inside the merge request plus a full history in the Pipelines dashboard.
Quick Walk-Through
This is based on the video above. For more detail, check out the video!
Goal: Ensure a Product‑Feedback Classifier prompt keeps passing for all test cases in a dataset before any new prompt version is deployed.
Create a pipeline
Event: Merge Requests
Evaluator: LLM‑as‑Judge
Dataset: product‑feedback‑classifier (5 rows)
Sampling: 100 %
Open a Merge Request – PromptHub automatically runs the 5 tests and shows a green check when all pass (or blocks the merge if any fail).
Review results – Go to the history tab and click the open merge request to view the pipeline run details. You can also view all pipeline data on the pipeline dashboard page.
Additional information
Availability – Pipelines are available on Team and Enterprise plans.
Roadmap – Slack alerts!