How to Use Rule Based Eval
Open the Evaluations Hub by clicking the “Edit Evaluations” button on the prompt header.
In the Evaluations Hub, click the “Create New Custom Rule Based Eval” button.
Give your Eval a meaningful name, and optionally provide a description.
Define one or more rules for your eval.
Click the Create Validation button to store your eval in the Evaluation Hub.
Navigate to a Prompt Block you would like to experiment with the new Eval.
From the Evaluations Hub, choose the relevant evaluation and select "Apply Eval".
Run your experiment and view aggregated scores and detailed breakdowns.
How to Use Dynamic Variables as a Rule Condition Value
When defining a value for a Rule, you can use the Arato Variables markdown. Values for such variables will be populated at runtime from the relevant Prompt DataSet.
For example:
How to Use Regular Expression for Rule Condition
Select the Regular Expression condition from the dropdown list and enter the regular expression value you want to match against.
For example:
^.\b(Paris|London)\b.$
^asserts the start of the line..*allows any character (zero or more times) after the start of the line.\\b(Paris|London)\\bmatches either "Paris" or "London" as whole words..*allows any character (zero or more times) after the match.$asserts the end of the line.