Skip to main content
All CollectionsPrompt Creation & Basics
How to run your first prompt A/B test
How to run your first prompt A/B test
Dan Cleary avatar
Written by Dan Cleary
Updated over a week ago

PromptHub makes it easy to test different versions of prompts, allowing you to compare outputs side-by-side.

You can test prompts and outputs side-by-side with just a few steps:

  1. Write your prompt and click "Run Test"

  2. Click the "Commit Changes" button at the bottom right of the output

  3. Make any edits to the prompt, model, or parameters and click the "Run Test" button

  4. You'll now see your last saved prompt left, and your latest version on the right!

  5. If you like the new output, click the commit changes button and continue with your iterative development!

Common examples for judging prompts side-by-side:

  • Testing format changes in the prompt and/or the system message

  • Testing different prompt engineering templates against each other.

  • Testing changes in models or parameters.

Here is a quick demo video going through the process of saving a prompt and comparing it against another prompt:

Did this answer your question?