Maintaining high standards in Acuity Insights' rating processes is essential for delivering an accurate and fair assessment. To achieve this, we have established a comprehensive quality assurance framework that begins the moment Raters are onboarded. New Raters participate in a self-paced, paid online training program designed to equip them with the essential knowledge and skills needed for their roles. This program includes interactive knowledge checks, engaging activities, and practical exercises tailored to enhance their understanding of the rating criteria.

Before being certified to assess live responses, Raters undergo calibration training, which is critical for demonstrating their ability to align with benchmark ratings. This training promotes uniformity and fairness in the assessment process, ensuring all Raters apply the criteria consistently. We conduct annual recertification training to keep our Raters updated on best practices and any updates to the rating criteria. This refresher course reinforces their knowledge and introduces new developments that may impact their responsibilities, ensuring they remain well-equipped to perform effectively.

Our rating platform includes built-in checks that actively monitor Rater performance, flagging any irregularities, such as moving through responses too quickly. If a Rater is rating too rapidly, they are blocked from continuing, and we conduct a thorough review of all previously scored responses from that session to determine if re-rating is necessary. Additionally, we emphasize ongoing performance monitoring, particularly inter-Rater reliability, which allows us to assess whether Raters score consistently across different tests and in relation to one another.