INSTRUCTIONS FOR COMPLETING ASSIGNMENT #2: Usability Evaluation Test
Plan
Associated Learning Outcomes with Assignment #2:
Inspect and contextualize major considerations of user experience (UX), including defining the
user, user experience levels, and various usability attributes (e.g., ease of use, user satisfaction,
other).
Explain the major aspects of usability engineering and user testing.
Specify established methods for identifying users’ requirements and analyzing users’ tasks in
developing usability test plans.
About the Structure of the Paper:
It is important that you read the entire document and follow its contents to prepare the paper. In
general,
Sections 1 through 3 will involve a detailed proposal
for a usability test plan of an interface of your choice that details what would include three
potential participants but without running the actual test.
Focus on planning!
General Directives: (1) Review and integrate into the paper recommended textbook or
articles for background on usability evaluation and test planning, (2) Review professor's notes on
usability evaluation, and (3) Locate some of your own usability articles or sources to integrate
into the paper.
Overall Task: You will PROPOSE a Usability Evaluation/Test Plan -- how you would
plan a usability evaluation, commonly known as a usability test or a user test or a user study. The
planning process of the usability evaluation can lead to valuable recommendations for improving
the quality of the interface. This exercise will also give you, the novice researcher, experience in
proposing a test plan for assessing user interface design. Note: these instructions are a general
guide to proposing a usability evaluation. Your usability evaluation test plan will contain details
based on this general method. I recommend Rubin and Chisnell’s text for details on conducting
usability testing to help you plan your particular usability test. Remember, you are only focusing
on the test plan parts.
Choose an interface (e.g., anything -- some specific technology device, hardware or
software or website, or smart device app, other) that is manageable, an interface that you can
propose to test on three participants to work through independently the major features of the user
interface you have chosen to evaluate. This assignment is focused on planning, rather than
actually performing the user test with three participants. In following this approach, you need to
address the details involved with test planning.
The goal of usability evaluation is to locate usability problems and to recommend
improvements to the interface. To do this you will need a test plan with test goals and other
elements. You need to devise a usability task list that potential participants would work through
independently to help you discover usability problems (and/or good design achievements) with
the interface. You will prepare a list of tasks to be performed. The task list includes a brief
description of each task to be performed with the interface as the focus of the evaluation. You
may (if appropriate) prepare a flowchart showing the order tasks would be performed and other
12 important events and sequences. Your job, as the student researcher, is to describe the process
in the form of a test plan you would follow to give potential participants appropriate written
and/or verbal directions on how to complete the task list; this includes how you would observe
the participants working through each task and how you would record (on paper) the sequence of
events and data observed.
A reminder to student researchers in HCI: It is the product/interface
that is being evaluated, not the potential participants! Your test plan should also cover the
essentials of ethical considerations in working with participants, if you were to run the usability
test with them!
Important Note: The assignment requires extensive discussion of detail about process. You need
to detail an understanding about process and how you describe the user experience of
participants doing a planned walkthrough of the chosen interface, if you were to conduct the
walkthrough. Synthesize the literature to support your notions or decisions regarding process.
Do NOT include website citations unless they are suitable UX sources like ACM SIGCHI,
UXPA, etc. Throughout the entire paper, you should provide a substantial synthesis of current
HCI literature sources that support or contradict conceptual aspects that relate to your user study.
The paper should be about 15-20 pages, but this might vary, depending on items that are
included in the Appendices.
General Guide to the User Study Test Plan and General Structure of the Paper
Please refer to all of the presentation/lecture notes in Canvas for details on different phases of a
user study and how you might propose a user study. Consult the course text on information
about usability test planning. Focus only on planning. The general outline is as follows:
Front Matter
• Title Page (include an actual title, like “A Proposal of a User Study Test Plan of The
Canvas Learning Management System” – whatever “technology interface” you would
test). Make sure you add your name, course and semester.
• Table of Contents
• Abstract
Section 1
Introduction (with subheadings as is below) (Focus only on test planning as if
you were proposing to do the test)
Background Description of the technology interface being tested.
State the problem or perceived problem being addressed.
Purpose Test Goals (include usability attributes)
Potential participants and user profile
Test environment
Role(s) of the Researcher
Section 2
Review of the Literature (deep literature synthesis about usability concepts, usability
testing methods, issues in observing users, user experience design, evaluation paradigms, etc.)
Section 3
Methods and Procedures (Focus only on test planning as if you were proposing
to do the test)
Restatement of test goals
Usability metrics (match to test goals)
Task List and Task Scenarios (match to test goals and metrics)
Specific usability evaluation methods used for test planning and test procedures
Detailed description of the test procedures to be done (pre-session, during, post-session).
Section 4. Discussion
Conclusions (including your own reflections about proposing to conduct a user study and the
advantages and disadvantages of usability evaluation and testing)
References (APA 6th or 7th)
Appendices (as needed)
Required: Informed Consent Forms (a draft/proposed form that would be used)
Potential materials given to participants (that would be used)
Questionnaires or Surveys (that would be used)
