In User Experience (UX) research, studies may take on either a qualitative, quantitative, or mixed methods approach. During the planning phase of a study, among other considerations, decisions must be made regarding the implementation of randomization.
What is Randomization?
Randomization is a technique used to control for potential biases and confounding variables that might influence the results of a study or experiment. Randomization involves randomly assigning participants to different experimental conditions/products/variations or the random presentation of tasks, questions, sets of questions, and response options.
Why Randomize?
Randomization is a good practice in both qualitative and quantitative studies and helps to prevent patterns, biases, fatigue, or predictability/learning that might affect user behavior or perception and distort your data, thus making the results more reliable, valid, and generalizable, while at the same time providing a more realistic user experience to participants during a study.
How to Randomize with Useberry
The “Group & Randomize” block that Useberry introduces in BETA, allows you to create "Stages" and "Sets", apply Randomization, and define how many "Sets" or "Blocks" you would like participants to see.
A Group & Randomize block can have multiple Stages and each Stage can contain multiple Sets.
To create a new Stage you must have more than one Set in your current Stage
To create a new Stage click the "+Add Stage" button
A "Set" allows you to bunch (group) Blocks (i.e. single task, tree test, open analytics, etc) or Sets together.
To create a new "Set" on a Stage, click on the "+ Add Set" button on that Stage
In Stage 1 you can only create Sets of blocks
For any other Stage than Stage 1, you can create Sets of the sets you created in the previous Stage.
Randomization allows you to show participants blocks or sets in a random order so that participants do not see the stimuli in the same order.
On Stage 1, you can randomize the blocks within each Set
For any other Stage than Stage 1, you can Randomzie the sets of the previous Stage
Note: Logic Jumps might not work as intended if you apply Randomization (edited)
You can also define the number of sets or blocks presented to participants.
The “Assign” option appears when you enable randomization for one set
The “Assign” option allows you to select how many blocks or sets will be shown to participants.
In Stage 1 you can select how many blocks within a set will appear to participants
In any other Stage than Stage 1, you can select how many sets from the previous Stage will appear to participants
Watch a How-To Tutorial
This step-by-step tutorial showcases how to set up your UX research studies with the Randomization feature.
Examples of Randomization with Useberry
Example 1:
Imagine that you would like to test one prototype with two tasks, each task followed by two questions. You would also like participants to see both tasks in a randomized order.
Add 4 sets to Stage 1
Set 1.1 contains Task 1
Set 1.2 contains the 2 Questions for Task 1
Set 1.3 contains Task 2
Set 1.4 contains the 2 Questions for Task 2
Create a new Stage (”Stage 2”)
Add a new Set (”Set 2.2”) to “Stage 2”
Move set 1.3 (Task 2) and set 1.4 (Task 2 Questions) from Set 2.1 to Set 2.2
Add a new Stage (”Stage 3”)
Enable the “Randomize Sets” options located on set 3.1
If you would like participants to conduct only one task and answer the questions for that task then on Set 3.1 change the “Assign” option from “All” to “1”
If you would like participants to conduct only one task and see both questions for that task in a randomized order, then enable the “Randomize blocks” option on Set 1.2 and Set 1.4.
If you would like participants to conduct only one task and see one of the questions for the presented task in a random order, then after enabling the randomization on Set 1.2 and Set 1.4, change the option “Assign” in both Sets from “All” to “1”.
Example 2:
Imagine that you would like to test 2 prototypes. Prototype A with 3 tasks and Prototype B with 3 tasks.
Set 1.1 contains 3 tasks with prototype A
Set 1.2 contains 3 tasks with prototype B
Let’s continue with the same example where you have Prototype A (3 tasks) and Prototype B (3 tasks). You would like participants to always see Prototype A tasks first followed by Prototype B tasks. In addition, you would like the tasks for each prototype to be shown in random order to participants.
Enable the randomization options located in Set 1.1 and Set 1.2 to randomize the order in which tasks are presented to participants for each Prototype.
Now imagine that you would also like participants to see both Prototype A and Prototype B and randomize the order of the prototype they see first (Within Subjects study)
Create a new stage "Stage 2"
Set 2.1 contains Set 1.1 and Set 1.2
Enable the randomization options located in Set 2.1 to randomize the order in which participants see the prototypes.
Let’s continue with the same example where you have Prototype A (3 tasks) and Prototype B (3 tasks), you want participants to see only Prototype A tasks or only Prototype B tasks (Between Subjects study).
In Set 2.1, change the randomization option “Assign” from “All” to “1” to show participants randomly one prototype
Now imagine, that you would like participants to only conduct 2 tasks with the prototype they are presented with.
Change the randomization option “Assign” in Set 1.1 and Set 1.2 from “All” to “2” to show randomly to participants two tasks for the prototype they are presented with.
