The width of the confidence interval equals two margins of error, thus the width of the CI is affected by the margin of error. The three variables affecting the margin of error and thus determine the width of the confidence interval are:
Variability of data
Confidence level
Sample Size
The standard deviation is used to measure the variability of the data in the sample. Given the same confidence level, the more variable the sample or population is, the wider the CI will be. For binary data, the standard deviation is equal to the square root of the proportion (p) multiplied by one minus the proportion (1-p). While for Continuous data (i.e. Time, SUS score, Likert scale responses) the standard deviation is equal to the square root of the variance, where variance is the sum of the squared distance each data point has from the mean, divided by the total number of the data points minus one.
The confidence level (CL) indicates how certain we would like to be in our assessment. Some common values for the CL are 95% and 90% but lower and higher values are also used. Given that the sample size and variability remain the same, the higher the confidence level is, the wider the CI will be.
For example, Table 2 shows the Confidence Intervals around the task success proportion for the same sample of users’ data with 95% and 90% Confidence Levels.
The 95% Confidence Level produces wider Confidence Intervals. |
The sample size has an inverse square root relationship with the CI. This tells us that to reduce the CI by half, we almost have to quadruple our sample size. Given that the confidence level remains the same, smaller samples generate wider CI while larger samples tend to generate narrower CI, thus a better estimate of the population parameter.
For example, Table 3 shows the 95% CI around the proportion of users that successfully completed a task.
On the left graph of Table 3, 8 out of 10 users (80%) completed the task successfully, while on the right graph of Table 3, 32 out of 40 users (80%) completed the task successfully.
Despite both samples having a task success proportion of 80%, we can see that the diagram on the right provides a narrower confidence interval, thus more precision for our estimation of the population parameter. |