All Collections
Advanced Features
Importing Dataset Test-Cases
Importing Dataset Test-Cases

How to format test-cases for import into a dataset

Dalton Pierce avatar
Written by Dalton Pierce
Updated over a week ago

Importing Dataset Test-Cases

PromptHub now supports mass importing of Dataset Test-Cases from a CSV file. Once imported, your Dataset can be re-used with all Prompts that share the same variables.

Note: We currently only support import files up to 20MB


Supported Formats

We recommend using CSV or TSV format for imports because it's simple and well-documented.


Importing From CSV

In order for us to import your test-cases, your CSV file must adhere to a prescribed format.

Header Row

The first row must contain column headers for any of the test-case properties such as name, description, target, as well as the names of any variables you want to import.

Static Columns (Optional)

These are optional columns used to pre-fill each test-case with helpful context. If excluded, the name will be auto-generated and the description and target fields will be left empty. You can always edit these after you finish importing.

Column

Details

name

The name of this test-case

description

The description of this test-case

target

The target output for this test-case

Variable Columns

Any other columns defined in your CSV file will be considered variable names that should be attached to each test-case.

Header Row Example

name,description,target,type_var,target_var,tone_var,...

Value Rows

All subsequent rows in the CSV each represent a new "test-case" and the data that should be injected into it.

Since CSV files use commas as a delimiter, any column values that also include commas must be wrapped in quotation marks: "Here's my value, with commas"

Value Row Example

Test 1,My first test,A song about the ocean,poem,ocean,from the perspective of a whale

Putting the Pieces Together

Here's a full CSV file example that defines the test-case static columns, and defines 3 dynamic variables for each test-case. You can include as many rows (i.e. test-cases) as you like, as long as the total file size is less than 20MB.

name,description,target,type_var,subject_var,tone_var
Test 1,"Should return a song about hotdogs, from a dog's perspective.",A short song about hotdogs,song,hotdogs,as a dog
Test 2,"Short poem about the ocean from whale's perspective",A poem about the ocean,poem,ocean,as a whale
Test 3,"Haiku about being a beat cop",Haiku about being a beat cop,haiku,daily life,as a beat cop
Test 4,"Poem about being a software engineer",Poem about being software engineer,poem,daily life,as a software engineer
Test 5,"Song about being a CEO",Song about being CEO,song,running a SaaS company,as CEO of that company

Importing this CSV file would produce a Dataset that looks like this:

As we can see, "Test 4" has our 3 dynamic variables attached with the appropriate values for that row:

Here's another template with 2 variables.

Here's a video demo:

Did this answer your question?