DR.

FARAH ZAHID
HUMAN FACTORS IN MEDICAL
REHABILITATION EQUIPMENT:
PRODUCT DEVELOPMENT AND
USABILITY TESTING
User-centered design.
The process by which a product is
designed so that the user is given the
most important influence.
Prototype testing. The evaluation of
a newly developed trial product by
the end-users who represent the target
market
Efficacy testing.
A more formal process of performance
testing in a controlled setting to determine
the effectiveness of the product.
Magnitude estimation.
An experimental technique used in
psychophysical experiments that involves
having a subject compare his or her current
sensation with a reference sensation.
WHAT IS USABILITY TESTING?

 Usability measures the quality of a user’s experience when interacting

with a product or system— whether a website, a software application,
mobile technology, or any user-operated device.
 In general, usability refers to how well users can learn and use a product
to achieve their goals and how satisfied they are with that process.
 Usability may also consider such factors as cost effectiveness and
usefulness.
 Two international standards further define usability and human-centered
design:
 [Usability refers to] the extent to which a product can be used by
specified users to achieve specified goals with effectiveness, efficiency
and satisfaction in a specified context of user (ISO 9241-11).
 Human-centered design is characterized by the active involvement of
users and a clear understanding of user and task requirements; an
appropriate allocation of function between users and technology; the
iteration of design solutions; multidisciplinary design (ISO 13407).
 User-centered design (UCD) is the
structured process for product
development that includes users
throughout each phase of the design
process.
 In addition, a macroergonomic approach is
often used that includes the overall
business mission, goals, and culture, as
well as the target audiences’ preferences,
abilities, and requirements.
 Testing should resemble the actual situation in which the item will
be used, as closely as possible.
 A. That is, participants should complete a task simulation that
closely resembles their normal activity or activities.
 It is important to know whether the item being tested is easy and
helpful to use during the tasks for which it was designed.
 After this is known, adding additional tasks can be useful, such as
more complicated tasks or alternate scenarios. Some researchers
will test using a more diffi cult task or scenario, with the idea that if
the worker can do the more diffi cult task, then surely they can
accomplish the easier tasks.
 Although this may be true, if the target audience participants
cannot do the additional task, the essential question will remain:
Can this person use this device or process in the way it was
intended to be used, with the intended consequences, easily?
 Care should be taken when adding additional tasks, as this could
prolong a test session beyond the length of time the target
population would normally engage in an activity. This is
especially true in situations in which the individual is able to
work at his or her own pace.
 This is another area in which therapists can provide valuable
information. For example, certain diseases and disabling
conditions are especially likely to cause fatigue, such as multiple
sclerosis, amyotrophic lateral sclerosis, and even recent stroke.
 Therapists can help researchers design tasks and scenarios that
are realistic and will not unduly challenge the participants.
 This will reduce frustration for participants and should result in
more accurate test feedback in terms of the number and pattern
of errors, as well as subjective responses.
2. Worst case scenario testing reveals worst case information.
A. As mentioned in the preceding section, at times it is benefi cial to
use additional tasks during usability testing, sometimes even using
a “worst case scenario.” There are good reasons for doing this.
 First, with the introduction of diffi cult tasks the maximum

capabilities of the participants can be defi ned.

 This type of testing is often done when it is imperative to design a

task within a person’s capabilities in order to reduce human error.

 An example would be testing airplane pilots on dual task
performance; as their primary task becomes more diffi cult, they
spend less time on secondary tasks.
 By carefully annotating where and when this happens, designers

gain knowledge about designing the equipment and tasks in a

cockpit so the pilot is not overtaxed.
 A second reason for this type of testing is to identify the maximum number and
diversity of problems associated with a product or procedure.
 This is important so designers can use the information to redesign the product
or procedure to address the identifi ed problems.
 The diffi culty can be in the interpretation of this information. Although a
carefully designed study that slowly introduces more and more diffi culty can
tell you about a participant’s basic capabilities, a study that simply has a
participant do very diffi cult tasks does not answer the same question.
 For example, if a person can lift and carry 50 pounds (i.e., accomplish the most
diffi cult task scenario), he or she can probably lift and carry 30 pounds (i.e.,
accomplish the less
 diffi cult “basic” task). However, if testing shows the participant cannot lift and
carry 50 pounds (i.e., accomplish the most diffi cult task scenario), the tester
has noidea if the participant can lift and carry 30, 25, or even 20 pounds (i.e.,
accomplish the less diffi cult “basic” task).
 In other words, using a worst case scenario does not answer the question of
whether the participant can do a particular job or task other than the one
tested.
Usability testing should be unobtrusive.
A. Although participants will be aware they are taking part
in a study and will typically sign a consent form to
participate, it helps if task accomplishment is
paramount during testing and the test process is
invisible to the user.
For example, camera setup should be done before arrival
and tested, so that participants can perform the task as
normally as possible during testing.
 Minimal adjustments should be made after the

participant arrives, such as raising or lowering the

camera to capture the full individual or pertinent actions
on camera.
 During data collection, extraneous variables that
could infl uence the outcome need to be
controlled as much as possible.
 This means that the individuals conducting
testing should offer no coaching, no additional
instructions during the task (unless those
instructions are part of the normal process), and
no feedback to the participant.
 In addition, no additional distractions should be
present, other than those that are normally part
of the task or situation.
 During testing, all instructions need to be precise and
exactly the same for each participant.
A. This is a basic tenet of all research and data collection, as
to do otherwise can bias the results (as mentioned in
3B,above).
 “Instructions” include all information (verbal or written)
on how to use a product or do a procedure. It also includes
any verbal feedback to participants.
 Positive feedback, negative feedback, and coaching
during testing have been shown to infl uence the test
results.
 This means that all feedback to participants should be

exactly the same.

 influence of the individual testers include
following a specifi c protocol, including all
verbalizations and/or having each tester
brief and evaluate an equal number of
participants from each disability group.
 That is, if a tester gives instructions to and
evaluates an equal number of persons in
each testingsituation, then this potential
infl uenceon outcome can be controlled.
5. Usability testing should be free of bias.
.
A. All instructions and comments byreviewers
must be free of bias.
This can be more diffi cult than it seems, and it
is always benefi cial to have individuals
experienced in writing surveys and
questionnaires to help in their design.
 Even the wording of a survey question, if

different from wording typically used by the

target group, could bias the results
 All recording of data must be precise and free
from bias. That is, when recording subjective
data from participants, the individuals
conducting the evaluation should not record the
information in their own words.
 Instead, participants can select their responses
from a given list or a Likert Scale or their
comments should be recorded verbatim.
 If data collection is done in a focus group,
having one or two recorders, as well as a tape
recorder, can help.
 Measures must reflect the target
audience, the product, and the actual
situation in which the person would act.
A. Usability testing should simulate the
actual situation in which the item will
be used, use the appropriate target
audience(s), and demonstrate product
use in the way the product is intended
to be used.
 When conducting usability testing with health
care products, the target audience can
include the client, client’s family members or
caregivers, and health care professionals.
 If the client is elderly, the caregiver may be a
spouse who is also elderly and could have
associated difficulties, such as reading small
print (presbyopia) or having difficulty with
precision tasks. All of these considerations
must be taken into account.
 If the product is likely to be used in numerous
situations and environments, some of those may
need to be added to the testing schedule.
 For example, people with diabetes do not test their
blood sugar only at home and under excellent
lighting conditions.
 They may also test their blood sugar just before
eating at a restaurant or while on a picnic at the
beach with their family.
 Therefore, the ability to read the digital signal must
be evaluated in differing lighting and environmental
conditions.
Ease-of-use is partially determined by user feedback.
A. Acceptance testing must be accomplished, in part, by
having participants report on how easy a product was
to use and what problems they had.
However, if they have no reference point of comparison,
their feedback cannot be taken in context. This means
that a member of a target audience who has never
previously used the product or attempted to do the task
in question may have diffi culty providing useful
feedback.
In these situations, participants may be asked to
accomplish a task with and without a particular device,
thus providing contextual information.
 If participants have performed the task
previously, it is helpful to understand the
conditions in which they completed the
task, whether assistance was provided, or
whether an additional, but somewhat
different, tool was used to assist with task
completion.
 This information is essential to determine
whether the present product or procedure
offers advantages or disadvantages.
 Concomitant verbalization is a good technique during
usability testing but must be accomplished with care.
A. Concomitant verbalization means that the participant
verbalizes aloud what he or she is doing, and why, while
taking action. The purpose of this technique is to have
the participant “think out loud,” so the tester can
understand why a process or device is a problem or one
is better than another.
Without this information, an evaluation may discover that a
mistake has been made (an error), but not why— that is,
the evaluators may not understand whether the product
design or an errant thought process might have
contributed to the error.
 There are diffi culties with this process, however. Verbalizing what
you are doing, while you are doing it, requires additional cognitive
effort. Therefore the participant must be permitted to do the task
and verbalize what he or she is doing and why, without interruptions.
 Additional instructions, coaching, or feedback will disrupt the
process. If the participants have to listen to and process additional
feedback while verbalizing what they are doing, they are likely to
lose track of what they are doing as they seek to listen to,
remember, and act on the new instructions.
 Therefore, as previously stated, this technique requires the monitor
to quietly observe (or fi lm) without disrupting the process or
distracting the participant.
 In addition, this technique introduces additional time, as the
respondent will take longer to verbalize what he or she is doing than
if he or she were to simply perform the actions.
 Each process should clearly be evaluated with regard
to the impact on the system.
 If the process or product being tested includes

reading, understanding, and following instructions,

these portions may need to be evaluated separately
from the task itself.
 The participant’s ability to remember the instructions

and the need for repeated exposure such as looking

back or asking questions are also important.
B. In the same way, if a task has several subtasks, they
may also need to be incorporated into testing.
 The target audience needs to be well-defi ned and
appropriately represented.
A. This is necessary for accuracy of representation and
generalizability of results.
As mentioned previously, the target audience for
health care and rehabilitation equipment may
include the client, family member caregivers, or
medical professionals.
Members of each group may have a very different
experience with the equipment based on their own
abilities and needs, and each experience is equally
valid.
 Without suffi cient representation, the end users may be
misrepresented. For example, having fi ve members of a
target group may not provide suffi cient information to
generalize the results:
 statistical assessments cannot reach an appropriate level of
signifi cance, and designers may be left to draw conclusions
from insuffi cient descriptive data.
 Although it is possible to conduct assessments with a low
number of participants, and indeed, the target audience may
be so small that fi nding a large enough representation is diffi
cult, care should be taken to ensure a welldefi ned target
audience and suffi cient representation.
 Regardless, the population should be thoroughly described in
any consequent reports.