The Effectiveness section allows you to analyze how successfully learners are completing quizzes, identify patterns in their results, and determine who needs additional attention.
The core of the report is the effectiveness matrix (heatmap), which shows the relationship between average quiz scores and the average number of attempts. This makes it possible to determine each learner's learning effectiveness level and automatically sort them into individual categories.
To access this section, go to Statistics&Reports, Effectiveness: https://app.academyocean.com/account/reports/quizzes-effectiveness
You'll also be able to customize the report using various filters. This lets you narrow the analytics down to the specific data slice you need – for example, to see the effectiveness for just one team or a specific quiz. To add the filter you need, click Add filter:
You can export the finished effectiveness matrix with your selected filters to PDF by clicking the three dots and selecting the corresponding option:
📌 Access to generating the effectiveness matrix is available to admins with the following roles: Owner, Admin, Analyst and Instructor.
Insights
Right under the filters is the Insights block with six cards.
📌 Insights are brief summaries that show your top-performing learners as well as at-risk learners who need immediate intervention or motivation.
Each card shows the number of learners in the corresponding category and the percentage change compared to the previous selected period:
The categories are broken down as follows:
📎Total – the overall number of learners;
📎Stars – learners with an average score of 71–100% and 1–2 attempts per quiz;
📎Driven – average score of 71–100% with 3–6 attempts;
📎Not Engaged – average score of 0–70% with 1–2 attempts;
📎Need Help – average score of 0–70% with 3–6 attempts;
📎Outliers – all learners with 6 or more attempts, regardless of average score.
Clicking on any category will open a modal window with a table of learners who fall into that category:
You can also export the data by clicking the three dots and selecting the format you need:
In this window, you can also use Recommendations – this reveals a hint with recommendations on what to do with learners in that specific category:
Average Score vs Average Attempts Matrix
The main element of this section is a chart built on two axes:
The Y-axis shows Average Score – the average percentage of points a learner received across all completed tests, on a scale from 0% to 100% in increments of 10%.
The X-axis shows Average Attempts – the average number of attempts a learner spent on completed quizzes, on a scale from 1 to 6, with a 6+ marker.
The chart is divided into color-coded zones that correspond to the same categories as the cards in Insights.
Below the matrix, there's an explanation of exactly how each status is calculated, so it's clear why a learner ended up in a particular zone:
Each cell in the matrix shows the number of learners who match that combination of score and number of attempts, and just like the Insights cards, it's clickable:
You can also export the data by clicking the three dots and selecting the format you need:
