Userhub

Introduction to Quantitative and Qualitative Usability Testing

August 19, 2018

In usability testing, the user performs some tasks on one or multiple designs. By performing this testing, two types of data can be collected: Quantitative and Qualitative. Qualitative testing consists of findings that refers to design and its features; if they are easy to use or not. On the other hand, Quantitative testing results focus on one or several metrics that indicate if a task was easy to perform or not.

Qualitative testing data directly evaluates the usability of the system. The testers will directly see what are the UI (User Interface) and Design elements that causes problem for the user to understand and operate, or that are user friendly. They will ask questions to the users to find out the elements that are creating difficulties for the users. After getting all the data from the testing phase, the testers will take decisions on the overall user experience.

Quantitative testing data does not directly assess the usability of the system. Rather, its result can be based on a user’s performance to complete a certain task; such as, time to complete a task, success rates, error counts. The test reflects the users perception of usability of the given system. Quantitative metrics can be simple numbers; for example, it observes if 60% of the users were able to complete a task. But it does not imply that the system was useful or not. It’s hard to absolutely deduce a solution in Quantitative testing. This is why these tests aim not to take absolute decision on the system usability. Rather, to compare it with another similar system, or the previous version of the same system; that can be seen as A/B Testing methods. These quant data may indicate if the design is useful compared to a certain reference but it may not tell us exactly what are the problems that the users faced. It may tell us if 40% of the users were able to perform a task. But, it does not tell us why the users had the problems, and how to make the task more usable. Hence, the usability experts need to use qualitative testing methods to make sense of the quantitative data derive the results properly.

Qualitative and quantitative testing require different study setups and analysis methods. Both qualitative and quantitative testing are essential in the iterative design cycle.  Based on the test results, a design is modified and the usability is ensured.

Qualitative studies are performed to identify the main problems in a design. We can run a qualitative study to see what prevents users from performing a task easily. Qualitative data generally consists of a set of findings, which let us identify the strengths and weaknesses of a design. Most quantitative studies are performed on a complete version of the system, with an aim to evaluate the overall usability of the system.