Please note: Ai Knowledge is part of the Scorebuddy Elite package and can only be activated by your Account Manager or Customer Success Manager
What is AI Knowledge
AI Knowledge improves Scorebuddy Autoscoring by querying your knowledge base and feeding relevant information into the AI model when scoring knowledge-based questions.
What problem does it solve
It makes evaluations more accurate and grounded in your documentation, so you can verify whether agents demonstrate good product understanding and align with your guidance.
AI Knowledge Feature Overview
Benefits
Smarter, more context-aware autoscoring
Improved scoring accuracy
Increased efficiency for QA teams
AI Knowledge Feature Overview
How it works
For each agent customer interaction, AI Knowledge retrieves relevant knowledge base articles. Those retrieved articles are then used as context to evaluate the agent’s knowledge in the interaction.
Configure AI Knowledge on a scorecard question
AI Knowledge can be enabled on any scorecard question, and is controlled inside the question “Tip” settings via an AI Knowledge toggle.
Step by step
Open the relevant Scorecard > Edit
Navigate to the question you want to configure
Click Update Tip (or open the Tip editor for that question)
Find the AI Knowledge toggle
Switch AI Knowledge = On
Next: Update the Tip text after enabling AI Knowledge
Once AI Knowledge is turned on, you should also update the Tip prompt so the AI knows it should use the returned KB context when scoring.
Step by step
With AI Knowledge = On, stay in the Tip editor
Update the Tip text to explicitly instruct the AI to use the KB context it receives e.g Use the AI Knowledge context provided to evaluate this question.
Save the Tip Sheet
Upload knowledge base articles (PDFs)
Step by step
Go to the Admin tab
Open the AI Knowledge tab
Click Upload files
Upload your knowledge base PDFs
Once uploaded, these files can be used to retrieve relevant information for an interaction.
Upload restrictions
Accepted format: PDF
Max file size: 50 MB
AI Knowledge Feature Overview
Where to test AI Knowledge
AI Knowledge runs during automatic scoring as long as the scorecard has AI Knowledge enabled for the relevant questions. You can also preview it using Preview AI Auto Scoring and then selecting Generate AI Score.
This will allow you to test your results
Step by step (preview)
Open the Scorecard > Edit
Click Preview AI Auto Scoring
Select from previously built Interaction lists to select a relevant test interaction to test against.
Select Generate AI Score
Review results for questions that have AI Knowledge enabled
If you find results aren't giving correct outcomes it's recommended to first check your scorecard question tip prompt. Tune and refine the prompt to be more in-line with your outcome. The below article explains best practices:
AI Scorecard Guidelines | Scorebuddy Help Center
When you are happy with your outcome in the preview make sure to lock your scorecard before running a scoring list.
Tips for best results
Enable AI Knowledge only on questions that genuinely require documentation context (product knowledge, policy, approved messaging).
Keep procedural or timing based checks as standard autoscoring questions without AI Knowledge.
Make your Tip instructions explicit that the AI must use the KB context.