Using the Judging Queue Tool
The Judging Queue is a powerful tool designed to streamline the evaluation process — especially for events with a high volume of submissions. It allows you to set limits on how many submissions each judge can evaluate, as well as how many times each submission should be reviewed. Paired with randomized judge search results, this tool helps ensure an even, efficient distribution of evaluations across your judging panel.
What It Does
Sets a maximum number of submissions a judge can review.
Sets a maximum number of evaluations per submission.
Optionally randomizes the order in which submissions appear to judges each time they access the portal.
Works across multiple groups within a single event.
Helps maintain fairness and consistency in evaluation distribution.
How It Works
When enabled, the Judging Queue dynamically assigns submissions to judges based on the limits you define. For example, if a submission should be reviewed no more than three times, the system will stop offering it to judges once it’s hit that limit.
To improve fairness and avoid bias toward submissions listed first, we recommend also enabling Randomized Judge Search Results — this ensures each judge sees submissions in a different order every time they access the judging portal.
Note: In rare cases where multiple judges open the same submission at the same time and begin reviewing it, the submission may end up with one additional evaluation above the set maximum. The system cannot override simultaneous activity, so it will allow both evaluations to be submitted. Admins can then decide how to treat the extra review.
How to Set It Up
Follow these steps to enable the Judging Queue and its related settings:
1. Prepare Your Submissions and Groups
Confirm all submissions are fully submitted.
Add submissions to their appropriate group(s).
Assign your reviewers to the same group(s) that the submissions belong to.
2. Enable the Judging Queue
Go to the Configuration page in your admin dashboard.
Navigate to the Judge Settings tab and click Edit.
Scroll to find Enable Evaluation Limit.
Enter:
The maximum number of submissions each judge can evaluate (e.g., 100).
The maximum number of times each submission can be reviewed (e.g., 3).
(Optional but recommended) Enable Randomize Judge Search Results to shuffle the submission order for each judge.
Click Save.
These settings apply across all groups and will override any evaluation settings previously configured at the group level.
3. Finalize the Assignments
Go to each Group.
Click Manage Assignments.
Select Assign All to activate your Judging Queue settings.
4. (Optional) Adjust Group-Specific Settings
If you prefer to control the Maximum Evaluations Per Submission at the group level:
Go to Groups and Divisions.
Click Edit on the desired group.
Set the Maximum Evaluations Per Submission value.
Note: If a different value is also set in the Configuration page, the Configuration value will take precedence.