USABILITY TESTING PLAN 1

Usability Testing Plan

Jocelyn Tolbert, Hannah Othmer, & Rebekah Goodrich

University of North Florida

ENC6721: Usability Testing

Professor Sindelar

March 24th, 2023

USABILITY TESTING PLAN 2

Table of Contents

Table of Contents.........................................................................................................................................................
Executive Summary.....................................................................................................................................................
Methodology..................................................................................................................................................................
Participants..................................................................................................................................................................................
Procedure......................................................................................................................................................................................
Usability Tasks.............................................................................................................................................................
Usability Metrics..........................................................................................................................................................
Reporting Results........................................................................................................................................................
USABILITY TESTING PLAN 3

Executive Summary
Within the study we have developed a series of surveys and tasks for our
participants to use on the UNF English website. We have decided to focus on
English graduate students, due to some early testing we found that they might be
the most beneficial to fully understand the website. After finding our participants,
we will conduct the tasks and tests either in-person or on Zoom, after the
completion of the tests, we will have the participants fill out a survey on how they
felt about the website and some of the tasks we asked them to do.

We will be evaluating how easily participants can navigate the website, trying to
find options that relate to; Class Descriptions, Faculty Bios, as well as club and
event information. We will be doing this by using effective tools like the three-
click-rule, eye tracking, and having the participants speak aloud their thoughts.

Upon review of this usability test plan, including the draft task scenarios and
usability goals for https://www.unf.edu/coas/english/index.html, documented
acceptance of the plan is expected.
USABILITY TESTING PLAN 4

Methodology

Participants
The study will consist of no more than five participants who will be graduate level
students at UNF in the English program. Preliminary research suggested that
students were most likely to use the UNF English Department website, but the
graduate students’ opinions might be underrepresented. Participants will include
students outside of our Usability Testing course and will not be previously aware
of the project. Fellow students will be asked to participate through email
recruitment. Basic technological skills and ability to navigate a website are
required skills. Participants may have experience with the UNF English website,
but experience with the specific tasks is not expected or required.

Procedure
Participants will take part in the usability test at a remote location on the UNF
campus, or through Zoom. A desktop or laptop with the website and supporting
software will be used in whichever environment the participant is able to work in.
The participant’s interaction with the website will be monitored by the facilitators
seated in the same area, or on the same Zoom call. Note takers and a data logger
will monitor the sessions in the environment, or connected by video camera feed
if needed on Zoom. The test sessions will be recorded and documented.

Participants will sign an informed consent that acknowledges: the participation is

voluntary, that participation can end at any time, and that the session will be
recorded but their privacy of identification will be kept anonymous. Participants
will complete a background questionnaire and a final survey. At the start of each
task, the participant will read aloud the task description from the printed copy and
begin the task.

The facilitator will instruct the participant to ‘think aloud’ so that a verbal record
exists of their interaction with the website. The facilitator will observe and enter
user behavior, user comments, and system actions in the data logging application
will be documented on paper and later put into a series of graphs for comparison.

The facilitator will have the participant use an application that uses the camera of
the computer to track the eye movements of the participant. Along with any notes
the facilitator takes down.

For participants that will be doing the test remotely, the participant will be seated
at their workstation in their work environment. Verbal communication will be
supported via Zoom.
USABILITY TESTING PLAN 5

The facilitator will instruct the participant to read aloud the task description from
the printed copy and begin the task. The facilitator will encourage the participants
to ‘think aloud’. The facilitator will observe and enter user behavior and
comments, and system interaction.

After all tasks have been completed, the participant will complete a post-test
satisfaction survey.

Usability Tasks
The tasks described below are representative of real use cases and will evaluate
the usability of the Department of English website for common tasks that a
graduate student might need to accomplish.

We will ask 4-5 participants to complete the tasks below. Each task should take no
more than five minutes to complete. Tasks will be identical for all participants.

Task 1: Find the phone number or email address of Dr. Kailan Sindelar
- This task will test the usability of the “Faculty & Administration” tab
of the site.

Task 2: Locate the class meeting times of AML4242 20th Century American
Literature
- This task will examine whether the current layout of the “Course
Descriptions” tab is useful and functional.

Task 3: Find out when the first English Graduate Organization conference was.
- This task will test the speed at which a user can find information that
is not prominently displayed on the site. This is important because
participants usually do not want to spend time looking for a specific
goal.

Task 4: Choose an upcoming event that you would like to attend, that is not
connected to the English Graduate Organization, and find out when and where it
will be held.
- This task tests the “Events” tab’s usability.

The testing will be completed via Zoom meetings. The participants will be
asked to share their computer screens so that the test proctor can observe
the ways in which they go about the tasks. Recordings will be made of the
sessions for further analysis post-test.
USABILITY TESTING PLAN 6

Usability Metrics
Usability metrics refers to user performance measured against specific
performance goals necessary to satisfy usability requirements. We will measure
the outcome of each task on four main factors:

- The time which it takes the participants to complete each task

- How many errors they make
- How often they recover from errors
- How many participants can complete the task successfully

Additionally, our subjective evaluation survey outlined above will provide more
insight into the participants’ attitudes toward the site and its usability.

Task Completion
Each task will require that the participant obtains data that would be used in the
course of a typical graduate student’s semester. The task is completed when the
task's goal has been obtained, the participant shares that they cannot complete
the task, or the participant requests to end the test.

Subjective Evaluations
A subjective evaluation survey regarding ease of use and satisfaction will be
collected via a questionnaire at the conclusion of the session. The questionnaires
will utilize Likert rating scales and will include the following questions:

- I thought the website was easy to use.

- I think I would need the support of a technical person to be able to
use this system.
- I found the tasks easy to complete.
- I would imagine that most people would learn to use this system
very quickly.

Participants will also be provided a portion of time in which to write or share

verbally any additional open-ended observations or feelings. Verbal responses
will be recorded.

Scenario Completion Time (time on task)

The time to complete each task, not including subjective evaluation survey time,
will be recorded. Recording times will help evaluate what is more difficult to find
on the website to help with recommendations in the final product of the study. We
predict that each task should take about 30-45 seconds.
USABILITY TESTING PLAN 7

Reporting Results
The UNF Department of English Website Usability Test Report will be prepared
after the conclusion of the usability testing phase. It will consist of a report and a
presentation of the results in the form of graphs, quotes from participants, and
specific usability problems and recommendations for resolution. The report is
anticipated to be delivered by April 10.