Usability Testing

Usability Testing also known as User Experience (UX) Testing, is a testing method for measuring
how easy and user-friendly a software application is. A small set of target end-users, use
software application to expose usability defects. Usability testing mainly focuses on user’s ease
of using application, flexibility of application to handle controls and ability of application to
meet its objectives.
This testing is recommended during the initial design phase of SDLC, which gives more visibility
on the expectations of the users.

Aesthetics and design are important. How well a product looks usually determines how well it
works.

There are many software applications/websites, which miserably fail, once launched, due to
following reasons –

 Where do I click next?

 Which page needs to be navigated?
 Which Icon or Jargon represents what?
 Error messages are not consistent or effectively displayed
 Session time not sufficient.

Software Engineering, Usability Testing identifies usability errors in the system early in the
development cycle and can save a product from failure.

Example Usability Testing Test Cases

The goal of this testing is to satisfy users and it mainly concentrates on the following
parameters of a system:

The effectiveness of the system

 Is the system is easy to learn?

 Is the system useful and adds value to the target audience?
 Are Content, Color, Icons, Images used are aesthetically pleasing?

Efficiency

 Little navigation should be required to reach the desired screen or webpage, and
scrollbars should be used infrequently.
 Uniformity in the format of screen/pages in your application/website.
 Option to search within your software application or website.

Accuracy

 No outdated or incorrect data like contact information/address should be present.

 No broken links should be present.

User Friendliness

 Controls used should be self-explanatory and must not require training to operate
 Help should be provided for the users to understand the application/website
 Alignment with the above goals helps in effective usability testing

How to do Usability Testing: Complete Process

Usability testing process consists of the following phases

Phase 1) Planning: During this phase the goals of usability test are determined. Having
volunteers sit in front of your application and recording their actions is not a goal. You need to
determine critical functionalities and objectives of the system. You need to assign tasks to your
testers, which exercise these critical functionalities. During this phase, the usability testing
method, number & demographics of usability testers, test report formats are also determined
Phase 2) Recruiting: During this phase, you recruit the desired number of testers as per your
usability test plan. Finding testers who match your demographic (age, sex etc.) and professional
( education, job etc.) profile can take time.

Phase 3) Usability Testing: During this phase, usability tests are actually executed.

Phase 4) Data Analysis: Data from usability tests is thoroughly analyzed to derive meaningful
inferences and give actionable recommendations to improve the overall usability of your
product.

Phase 5) Reporting: Findings of the usability test is shared with all concerned stakeholders
which can include designer, developer, client, and CEO

Methods of Usability Testing: 2 Techniques

There are two methods available to do usability testing –

1. Laboratory Usability Testing

2. Remote Usability Testing

Laboratory Usability Testing:. This testing is conducted in a separate lab room in presence of
the observers. The testers are assigned tasks to execute. The role of the observer is to monitor
the behavior of the testers and report the outcome of testing. The observer remains silent
during the course of testing. In this testing, both observers and testers are present in a same
physical location.

Remote Usability Testing: Under this testing observers and testers are remotely located.
Testers access the System Under Test, remotely and perform assigned tasks. Tester’s voice ,
screen activity , testers facial expressions are recorded by an automated software. Observers
analyze this data and report findings of the test.

How many users do you need ?

Research (Virzi, 1992 and Neilsen Landauer, 1993) indicates that 5 users are enough to uncover
80% of usability problems. Some researchers suggest other numbers.

The truth is , the actual number of the user required depends on the complexity of the given
application and your usability goals. Increase in usability participants results into increased
cost , planning , participant management and data analysis.

But as a general guideline, if you on a small budget and interested in DIY usability testing 5 is a
good number to start with. If budget is not a constraint its best consult experienced
professionals to determine the number of users.
UX Testing Checklist
The primary goal of this testing is to find crucial usability problems before the product is
launched. Following things have to be considered to make a testing success:

 Start the UX testing during the early stage of design and development
 It’s a good practice to conduct usability testing on your competitor’s product before you
begin development. This will help you determine usability standards for your target
audience
 Select the appropriate users to test the system(Can be experts/non-experts users/50-50
of Experts and Non-Experts users)
 Use a bandwidth shaper . For instance , your target audience has poor network
connectivity , limit network bandwidth to say 56 Kbps for your usability testers.
 Testers need to concentrate on critical & frequently used functionalities of the system.
 Assign a single observer to each tester. This helps observer to accurately note tester’s
behavior. If an observer is assigned to multiple testers, results may be compromised
 Educate Designers and Developers that this testing outcomes is not a sign of failure but
it’s a sign of Improvement

Usability Testing Advantages

As with anything in life, usability testing has its merits and de-merits. Let’s look at them

 It helps uncover usability issues before the product is marketed.

 It helps improve end-user satisfaction
 It makes your system highly effective and efficient
 It helps gather true feedback from your target audience who actually use your system
during a usability test. You do not need to rely on “opinions” from random people.

Usability Testing Disadvantages

 Cost is a major consideration in usability testing. It takes lots of resources to set up a

Usability Test Lab. Recruiting and management of usability testers can also be expensive

However, these costs pay themselves up in form of higher customer satisfaction, retention and
repeat business. Usability testing is therefore highly recommended.

Source: What is Usability Testing? Software UX

By ByThomas Hamilton. Retrieved at, https://www.guru99.com/usability-testing-tutorial.html.
February 2023
System Usability Scale (SUS)
The System Usability Scale (SUS) provides a “quick and dirty”, reliable tool for measuring the
usability.   It consists of a 10 item questionnaire with five response options for respondents;
from Strongly agree to Strongly disagree.  Originally created by John Brooke in 1986, it allows
you to evaluate a wide variety of products and services, including hardware, software, mobile
devices, websites and applications.  

Benefits of using a SUS

SUS has become an industry standard, with references in over 1300 articles and publications. 
The noted benefits of using SUS include that it:

 Is a very easy scale to administer to participants

 Can be used on small sample sizes with reliable results
 Is valid – it can effectively differentiate between usable and unusable systems

Considerations when using a SUS

If you are considering using a SUS, keep the following in mind:

 The scoring system is somewhat complex

 There is a temptation, when you look at the scores, since they are on a scale of 0-100, to
interpret them as percentages, they are not
 The best way to interpret your results involves “normalizing” the scores to produce a
percentile ranking
 SUS is not diagnostic - its use is in classifying the ease of use of the site, application or
environment being tested

The System Usability Scale

When a SUS is used, participants are asked to score the following 10 items with one of five
responses that range from Strongly Agree to Strongly disagree:

1. I think that I would like to use this system frequently.

2. I found the system unnecessarily complex.
3. I thought the system was easy to use.
4. I think that I would need the support of a technical person to be able to use this system.
5. I found the various functions in this system were well integrated.
6. I thought there was too much inconsistency in this system.
7. I would imagine that most people would learn to use this system very quickly.
8. I found the system very cumbersome to use.
9. I felt very confident using the system.
10. I needed to learn a lot of things before I could get going with this system.
The questionnaire and scoring are outlined in the System Usability Scale (SUS) Template.

Interpreting Scores
Interpreting scoring can be complex. The participant’s scores for each question are converted
to a new number, added together and then multiplied by 2.5 to convert the original scores of 0-
40 to 0-100.  Though the scores are 0-100, these are not percentages and should be considered
only in terms of their percentile ranking.

Based on research, a SUS score above a 68 would be considered above average and anything
below 68 is below average, however the best way to interpret your results involves
“normalizing” the scores to produce a percentile ranking.

The references at the end of this page and the template provide more information in context
about the process.

References

 SUS: A Quick and Dirty Usability Scale   by John Brooke (read full text or download as a
PDF, and download the proper citation)

 Measuring Usability with the System Usability Scale (SUS)   by Jeff Sauro, PhD

 SUS: A Retrospective   by John Brooke

 Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale   by
Aaron Bangor, PhD, CHFP, Philip Kortum, PhD, and James Miller, PhD