Normalized Report Overview
The Normalized Report is an advanced reporting option available only to select Reviewr clients upon request. It provides a standardized view of evaluation scores to help ensure fairness across evaluators, particularly in cases where scoring tendencies may vary significantly. This report is generated on a per-event basis.
How to Access the Report
From the Admin dashboard, click the Reports icon.
Navigate to the Reports page.
Scroll to the bottom of the page to find the Normalized Report.
Generating the Report
To generate your Normalized Report, you must select:
Evaluation Form – The specific evaluation form you want to report on.
Group – The reviewer group used to evaluate submissions.
Division – The division that contains the submissions.
⚠️ Important: These three selections—evaluation form, group, and division—must all correlate.
Ensure that:
The group you select was actually assigned to the evaluation form.
The group contains reviewers who submitted evaluations using that form.
The submissions evaluated by that group exist within the selected division.
If these do not align, the report may be blank.
Once all selections are made, click Email Results to Me. The report will be sent to your inbox. Large evaluation forms may take up to 30 minutes to process. Be sure to check your spam folder before reaching out to the Reviewr support team.
Understanding the Report Format
Once you receive and open the file, here’s how the data is structured:
1. Initial Columns
Group – Displays the name of the group you selected.
Submission – Lists the names of each submission within that group.
2. Evaluation Form Breakdown
For each question on the evaluation form:
The column header is the question name.
Below the question header, each column represents a reviewer.
The cells show the scores each reviewer gave per submission.
If a cell is empty, that reviewer did not score the submission.
Text/comment fields follow the same format, but contain written responses.
This pattern continues for all questions on the form.
3. Display Scores
After the form data, a column break separates the next section:
Display Scores – Shows each reviewer’s total score for every submission.
The bottom row contains the average score each reviewer gave across all submissions they scored.
4. Display Score (Centered)
This section adjusts for reviewer scoring bias:
A score of 1.0 represents the reviewer’s average score.
Scores above 1.0 (e.g., 1.25) are above the reviewer’s average.
Scores below 1.0 (e.g., 0.75) are below the reviewer’s average.
This helps normalize tough vs. lenient scorers.
5. Average Center Score
Displays the average centered score per submission.
Helps balance out unusually high or low individual scores.
6. Rank
Ranks submissions from highest to lowest average centered score.
Provides a normalized order of performance across all evaluated submissions.
Summary
The Normalized Report is a powerful tool for evaluating submissions with increased fairness and clarity. By adjusting scores relative to each reviewer's tendencies, it helps eliminate scoring bias and surfaces a clearer picture of submission performance.
If you’d like access to the Normalized Report for your event, reach out to your Reviewr account representative.