This guide will help you understand more about why demographic quotas fill the way they do, and when you should consider using the ‘Remove quotas’ functionality. Removing demographic quotas can speed up completion of your survey, especially when the remaining target audience has become more difficult to achieve over several days.
Whilst this guide should help you understand when to remove quotas, remember you can always contact your Customer Success Manager (CSM) or the Customer Research Manager (CRM) team for further advice and support.
How are my demographic quotas filling?
When you set the Audience for your survey, you are likely to have selected specific demographics and applied quotas (a % required) to your Audience targeting.
As your survey progresses, you may begin to see that there are some segments which fill more quickly than others. For example, female might fill a lot faster than male.
The two main reasons why quotas might fill at different speeds are:
Certain demographic groups are quicker to respond to the survey: the nature of online surveys means some demographic groups are easier to reach than others e.g. younger age groups tend to respond more quickly.
The quotas you have set are not representative of your target audience: you may be looking to get a ‘Nationally Representative’ audience for people who go to nightclubs, but you will struggle to reach older age groups as they are less likely to go. This means you will see far fewer respondents of this age being eligible for your survey.
In both these scenarios, survey progress will slow down and it may be harder to fill the final completes you need to achieve your full sample size.
You may also find that as different elements of quotas fill, the remaining respondents required end up needing to meet a very specific profile e.g. males aged 60+ in Wales.
How to remove demographic quotas on a live survey
Go to the manage page of your survey
Click on the arrow next to the specific date and country you want to manage
Click on "remove quotas" next to the demographic quota you wish to remove
As your survey progresses, you may begin to see that there are some segments which fill more quickly than others. For example, female might fill a lot faster than male.
‘Removing Quotas’ can help your survey complete more quickly as it removes the need for respondents to meet specific criteria.
Consider the following before removing demographic quotas:
Will removing quotas mean your data skews differently and therefore you lose some robustness in the data? Consider whether your business is comfortable making decisions on half the sample of 50 + year olds that you had initially set out to achieve.
Do you have enough people to be able to do analysis on the subgroups of interest? Typically we would recommend a minimum of n=100 to conduct analysis on (you can use fewer respondents but beware of statistical robustness).
Is there a quota that is more important than the others e.g. age vs. region? Perhaps you can remove one set of quotas e.g. region, before any other to see the impact it has on your survey progress.
When do you need the survey to complete by? If you have more time, and your survey is continuing to fill, you can just leave it to continue as it is.
How many completes are you getting a day? If you see over a period of a few days you only get a handful of completes but you are still well below 80% of your required total sample, we recommend you talk to your CSM or CE who can offer more advice as there may be other options available to you.
The ideal scenario to ‘Remove quotas’
If your survey is at around 90% of your required total sample, and the survey is starting to slow down – this would signal the idea scenario in which to remove your quotas. You are likely to have enough robustness in your data that you will not see a huge bias towards any specific group.
Click the ‘Remove quotas’ icon and click ‘confirm’ to continue with the action. You will now no longer have quotas (%s) for this demographic on your survey, but you will retain any targeting you set e.g. targeting 18-70 year olds now with no quota requirement.
If you are in doubt, please do contact your CSM or CE rep who will be able to advise you further.