Human in the Loop Scoring lets you decide when a QA Team member must check or finish an AI-scored interaction before the result reaches the Agent. It is a set of configurable scorecard rules that change the workflow during AI scoring, so AI scoring fits the way your team already works.
When to use it
You want a QA Team member involved under certain conditions, such as a very high or very low overall score, or specific questions failed, before results are shared with Agents.
You want AI to score only some questions, with a QA Team member completing the rest and releasing the result.
The three rules
Rule | What triggers it | Where you set it | Status on the list page |
Overall Score Rule | The AI's total score is above or below a percentage you set | Main Scorecard builder page, in the AI Review Rules panel | Needs Review |
Answers that Flag for Review | AI picks one of the answers you chose on a question you chose (for example Fail or N/A) | AI Rules tab of the question details sidebar | Needs Review |
Exclude from AI Scoring | Always, for the questions you exclude: AI skips them | AI Rules tab of the question details sidebar | Needs Completion |
You can combine rules on the same scorecard. Answers that Flag for Review can be applied to as many questions as you need.
What happens to a triggered score
A triggered score is held back from the Agent. It appears on the list page as Needs Review (check the AI's answers) or Needs Completion (answer the excluded questions). A QA Team member opens it in the scoring panel, reviews or completes it, then releases it to the Agent.
To get started, see How to set up Human in the Loop Scoring.