Skip to main content

AI Knowledge to Improve Autoscoring

Use AI Knowledge to improve ScorebuddyCX Autoscoring by using your knowledge base as a source for knowledge-based questions.

Written by Jordan McGovern

AI Knowledge Overview

AI Knowledge improves ScorebuddyCX Autoscoring by querying your knowledge base and feeding relevant information into the AI model when scoring knowledge-based questions.

It makes evaluations more accurate and grounded in your documentation, so you can verify whether agents demonstrate good product understanding and align with your guidance.

Benefits

  • Smarter, more context-aware autoscoring

  • Improved scoring accuracy

  • Increased efficiency for QA teams

How it works

For each agent customer interaction, AI Knowledge retrieves relevant knowledge base articles. Those retrieved articles are then used as context to evaluate the agent’s knowledge in the interaction.

Setting up AI Knowledge

1. Upload Articles

  1. Go to the Admin tab

  2. Open the AI Knowledge tab

  3. Click Upload files

  4. Upload your knowledge base PDFs

Once uploaded, these files can be used to retrieve relevant information for an interaction.

Upload restrictions

  • Accepted format: PDF

  • Max file size: 50 MB

    AI Knowledge Feature Overview

Further information on uploading and managing your AI Knowledge articles can be found here.

2. Configure AI Knowledge on scorecard questions

  1. Open the relevant Scorecard > Edit

  2. Navigate to the question you want to configure

  3. Click Update Tip (or open the Tip editor for that question)

  4. Find the AI Knowledge toggle

  5. Switch AI Knowledge = On

AI Knowledge can be enabled on any scorecard question, and is controlled inside the question “Tip” settings via an AI Knowledge toggle.

3. Update the Tip text after enabling AI Knowledge

  1. With AI Knowledge = On, stay in the Tip editor

  2. Update the Tip text to explicitly instruct the AI to use the KB context it receives e.g Use the AI Knowledge context provided to evaluate this question.

  3. Save the Tip Sheet

Once AI Knowledge is turned on, you should also update the Tip prompt so the AI knows it should use the returned KB context when scoring.

4. Test AI Knowledge

  1. Open the Scorecard > Edit

  2. Click Preview AI Auto Scoring

  3. Select from previously built Interaction lists to select a relevant test interaction to test against.

  4. Select Generate AI Score

  5. Review results for questions that have AI Knowledge enabled

AI Knowledge runs during automatic scoring as long as the scorecard has AI Knowledge enabled for the relevant questions. You can also preview it using Preview AI Auto Scoring and then selecting Generate AI Score.

If you find results aren't giving correct outcomes it's recommended to first check your scorecard question tip prompt. Tune and refine the prompt to be more in-line with your outcome. The below article explains best practices:
​AI Scorecard Guidelines | Scorebuddy Help Center
​
When you are happy with your outcome in the preview make sure to lock your scorecard before running a scoring list.

Tips for best results

  • Enable AI Knowledge only on questions that genuinely require documentation context (product knowledge, policy, approved messaging).

  • Keep procedural or timing based checks as standard autoscoring questions without AI Knowledge.

  • Make your Tip instructions specify that the AI must use the Knowledge Base context.

Please note: AI Knowledge is part of the ScorebuddyCX Elite package and can only be activated by your Account Manager or Customer Success Manager

Did this answer your question?