Usability

Where weve been

reactive
oops I forgot the manual

Where are we going

proactive.
Understanding with minimal use of manual

Copyright 2015 BSI. All rights reserved. Usability: BSI Roadshow 2015 Richard Stein
What is Usability?

The ability for a human to interact easily and relatively error-free with a system, product
or procedure

What is Human Factors? A synonym for usability engineering

Assume the system or product is a medical device for this presentation

Copyright 2015 BSI. All rights reserved. 08/12/2015 3

Users

Who* Where*

Patients On the go Labs

Caregiver Home Clinics

Clinicians (doc, nurse, tech) Emergency transport In the field

EMTS ORs ERs

Hospital room Nursing home

*consider age, dexterity, education/training, context
This presentation doesnt focus on the function

Copyright 2015 BSI. All rights reserved. 08/12/2015 4

What types of devices may apply: implantable devices

Copyright 2015 BSI. All rights reserved. 08/12/2015 Devices pictured are examples obtained from the Web, not an endorsement, nor assessed by BSI. 6
What types of devices may apply: implantable devices

small

Copyright 2015 BSI. All rights reserved. 08/12/2015 Devices pictured are examples obtained from the Web, not an endorsement, nor assessed by BSI. 7
What types of devices may apply: Robotic Surgery

Users?

Copyright 2015 BSI. All rights reserved. 08/12/2015 Devices pictured are examples obtained from the Web, not an endorsement, nor assessed by BSI. 8
What types of devices may apply: CPAP

Copyright 2015 BSI. All rights reserved. 08/12/2015 Devices pictured are examples obtained from the Web, not an endorsement, nor assessed by BSI. 9
What types of devices may apply: Ventilator

Copyright 2015 BSI. All rights reserved. 08/12/2015 Devices pictured are examples obtained from the Web, not an endorsement, nor assessed by BSI. 12
What types of devices may apply: Medical Apps with Sensors

Copyright 2015 BSI. All rights reserved. 08/12/2015 Devices pictured are examples obtained from the Web, not an endorsement, nor assessed by BSI. 15
What types of devices may apply
CPAP
nebulizers
compression therapy
EKGs
Blood Pressure monitors
Infusion pumps
Implantable devices
Catheters
LVADs
Surgical equipment (manual and powered)
Endoscopes
Apps, some with sensors
Robotic surgery
Ultrasound
Ventilator
Defibrillator
Infant incubator
Combination devices
Software
Expect sensors to facilitate the human OS
More.

Copyright 2015 BSI. All rights reserved. 15/09/2015 18

Why implement usability in Your medical device design?

Understandable
Useful device
Successful
Safer
Sales
Directives (EU)
Standards (IEC)
Guidance (FDA)
Serious Adverse Events, serious injury or death

Copyright 2015 BSI. All rights reserved. 08/12/2015 19

Why the concern?
In the past:
Often, there has not been enough time to correct the usability shortcomings in the
medical device that was found in validation, so the manual was re-written.
If manual was not enough to address shortcomings, then training was provided.
If training did not address the remainder of the shortcomings, then complaints may
have occurred which were addressed via instructions on the phone, etc.
Now:
It is expected to provide a safe product that the user understands, errors minimized
Still:
SAEs have occurred, see MAUDE, some due to Use Errors
Complaints to the manufacturers are still occurring

Copyright 2015 BSI. All rights reserved. 15/09/2015 20

Some history
Human Factors (Usability) first addressed by Peter Carstensen of FDA
Initiatives to make anesthesia equipment safer through application of human factors principles
Motivated the creation of an international committee to create standards
ANSI/AAMI HE-48* Human Factors Engineering Guidelines and Preferred Practices for the Design of Medical Devices
o *Much of which adopted from MIL-STD-1472 Human Engineering
ANSI/AAMI HE-74 Human factors design process for medical devices
IEC 60601-1-6 Medical electrical equipment Part 1-6: General requirements for basic safety and essential performance
Collateral standard: Usability
IEC 62366:2007 Application of usability engineering to medical devices
ANSI/AAMI HE-75 Human Factors Engineering Design of Medical Devices (a Tutorial to HE-74)
Today
IEC 62366-1:2015 Part 1: Application of usability engineering to medical devices
Soon
IEC/TR 62366-2:2 Part 2: Guidance on the application of usability engineering to medical devices

Copyright 2015 BSI. All rights reserved. 08/12/2015 21

Injuries associated with anesthesia*
The anaesthetized patient is at risk of complications resulting from the actions, or inaction,
of the anesthetist, from the actions of the surgeon, and from failure or malfunction of
anesthetic equipment. One of many of the root causes attributable to usability:

Failure to check equipment, lack of vigilance, distraction, and carelessness were the
most frequently associated human factors

More see FDA CDRH, MAUDE Adverse Events Report

Why?

*British Journal of Anesthesia Volume 95, Issue 1, pages 95-109

Copyright 2015 BSI. All rights reserved. 08/12/2015 22

Injuries associated with Infusion Pumps*
From 2005 through 2009, FDA received approximately 56,000 reports of Adverse Events associated
with the use of infusion pumps, including numerous injuries and deaths
These adverse event reports and device recalls have not been isolated to a specific manufacturer, type
of infusion pump, or use environment; rather, they have occurred across the board. Some examples:
Infusion pump screen confuses the user
Infusion pump screen.. Which units of measurement to enter.. Decimal point error
pump labelsor...become damaged under routine uselabels worn
IFU or cues for mechanical set-up are not specific or clear enough
Infusion pump screen design is clunky or confusing to users, cause delays
IFU are confusing, inadequate, outdated, or unavailable.

In 2010 the FDA announced three steps it would take to improve infusion pump safety. These steps
were to:
(1) increase user awareness,
(2) proactively facilitate device improvements, and
(3) publish new guidance for industry
*See FDA CDRH Infusion Pumps

Copyright 2015 BSI. All rights reserved. 08/12/2015 23

Examples of Different screens

Copyright 2015 BSI. All rights reserved. 08/12/2015 Devices pictured are examples obtained from the Web, not an endorsement, nor assessed by BSI 24
Why IEC 62366-1:2015 we already have 62366:2007?
Created to be more usable, easier to understand than original 62366
Original 62366 required an expert to understand
Implements practices by developers
Attempts to harmonize methods to meet FDA and international regulators
Contains the what requirements, the how is in 62366-2
Closer ties to risk management, EN ISO 14971
Hazard related use scenarios
No reference to usability goals, however goals may still have a need
Errors and/or shortcomings in the design to be addressed through risk evaluation
Reduced emphasis on functional analysis
Closer to FDA guidance
Legacy still addressed in Annex C (UOUP)

*Annexes are very helpful in implementation

Copyright 2015 BSI. All rights reserved. 08/12/2015 25

Major activities (shalls*) in IEC 62366-1:2015
Usability Engineering activities shall be planned
To reduce risk do: safe design, protective measures, and/or information on safety
Establish a usability engineering process
Documents Usability activities to a usability file
Use specification
ID user interface** characteristics
ID hazards & situations (foreseeable, the unforeseeable is gone)
ID hazard use scenarios for summative evaluation
User interface spec
Prepare user interface evaluation plan (summative & formative)
Perform the design & summative evaluations

* Most of the requirements

**what the user looks at, uses

Copyright 2015 BSI. All rights reserved. 08/12/2015 26

Whats in the new EN 62366-2:2016?
This is the how document or tutorial
How safety relates to usability
Reasons to invest in usability
How to implement a usability program
Overall usability process
Prepare the use specification
Identify characteristics related to safety and potential use errors
Identify hazard-related use scenarios
Select hazard-related use scenarios for summative evaluation
Establish user interface specification
Establish user interface evaluation plan
Design/implement user interface
Perform formative evaluations
Perform summative evaluations

*note: the above is paraphrased by the author and the final version is not yet released

Copyright 2015 BSI. All rights reserved. 08/12/2015 27

Isnt the usability process an addition to my development?

When you think about it, the activities are part of the development process

Design process Usability process

Concept development Preliminary analysis

Design User interface design
Prototype development Formative evaluation
Verification & Summative evaluation
validation

Copyright 2015 BSI. All rights reserved. 08/12/2015 28

Challenges

Process to include usability

Conduct adequate formative evaluation
Conduct adequate summative evaluation
Address and reduce use risks
Just an IFU to address usability may no longer be enough

Copyright 2015 BSI. All rights reserved. 08/12/2015 29

Notified Body expectations of manufacturers
Address usability in the Essential Requirements, in the Directives
Asking a couple of clinicians if they like the product is not enoughbe careful of KOLs
Follow a process, ideally the harmonised standard or rationale, to meet ERs
Specifications should include the intended user, environment and procedure (as applicable)
Include usability in the requirements of the device/system and risk management
Verify the usability requirements of the device/system
Reduce usability risks as low as possible
Legacy products will need to be addressed, (See Annex 3)

Expectations are rising in Europe.

For example: MHRA has task force on HF that is starting to look deeper into usability and how it can be improved to
reduce use errors. A presumption is that the needs will require the NBs to audit for usability.

FDAs expectations have also increased.

Copyright 2015 BSI. All rights reserved. 8/12/2015 30

When does BSI apply EN62366-1:2015?

Now? Auditors may use EN62366-1:2015 as a benchmark, BSI will use the Directives.
Auditors need training
Auditors will look for evidence of compliance
Legacy is addressed
If I do not have high risk, must I apply it? Remember the shalls and risk based in
standard
If EN 62366-2 (TR) is guidance, then must I apply it? What is in it for me?

Copyright 2015 BSI. All rights reserved. 08/12/2015 32

Case study: Abbott Symbiq IV Infusion System

Abbotts Process:
Literature Review
User Analysis
Use Scenarios
paper prototypes use screens (GUI)
Formative evaluation
Simulations & models
Design
Summative evaluation (some simulated)

Copyright 2015 BSI. All rights reserved. 08/12/2015 Case study supplied by usability consultant, not assessed by BSI 33
Case study: Intelliject Epicard

Consultants Process
Development motivated by needs, mistakes, complaints
Conduct preliminary evaluations to understand mistakes
Observed on how used
Learned from use errors
Create the mock-up design
Conduct formative evaluation with mock-ups

Copyright 2015 BSI. All rights reserved. 08/12/2015 Case study supplied by usability consultant, not assessed by BSI 34
Case study: Intelliject epicard

Multiple concepts were evaluated in formative, resulted in refine design

Copyright 2015 BSI. All rights reserved. 08/12/2015 Case study supplied by usability consultant, not assessed by BSI 35
Case study: Intelliject epicard
Final design then evaluated in summative evaluations (simulated, many users)

Copyright 2015 BSI. All rights reserved. 08/12/2015 Case study supplied by usability consultant, not assessed by BSI 36
Case study: Drug injection for Rheumatoid Arthritis

Previous systems
Difficult to use with hands
Time
Multiple steps
Mistakes
Know the user
Understand the context

Copyright 2015 BSI. All rights reserved. 08/12/2015 Case study supplied by usability consultant, not assessed by BSI 37
Case study: Drug injection for Rheumatoid Arthritis
Consultants Process
Created the mock-up design
Conducted formative evaluation with mock-ups, simulated, many users
Created design based on learning from formative evaluation

Copyright 2015 BSI. All rights reserved. 08/12/2015 Case study supplied by usability consultant, not assessed by BSI 38
Case study: Drug injection for Rheumatoid Arthritis
Summative design, many users

Copyright 2015 BSI. All rights reserved. 08/12/2015 Case study supplied by usability consultant, not assessed by BSI 39
Case Study: Mobile Apps
Mobile apps may be largest increasing segment in medical devices

Consultants Process
Formative evaluation, simulated, many users
What was learned
Simplify
Minimize
Dont repeat unnecessarily
Use regular English
Visually appealing
Show progress
Provide actionable choices
Encourage action

Copyright 2015 BSI. All rights reserved. 08/12/2015 40

Case Study: Mobile Apps

Copyright 2015 BSI. All rights reserved. 08/12/2015 Case study supplied by usability consultant, not assessed by BSI 41
Case Study: Mobile Apps

Summative evaluation, not simulated, many users,

Copyright 2015 BSI. All rights reserved. 08/12/2015 Case study supplied by usability consultant, not assessed by BSI 42
 Where can I learn more?
AAMI HF Committee:
AAMI Website: for webinars and training
ANSI/AAMI HE-74
ANSI/AAMI HE-75
IEC 62366-1:2015
IEC/TR 62366-2
Handbook of Human Factors Medical Device Design, Matthew Weinger
et.al. ISBN 978-0-8058-5627-9
FDA Guidance, http://www.fda.gov/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm259748.htm
The psychology of everyday things, Donald A. Norman ISBN 0-465-06709-3
Set Phasers on stun, Steven Casey, ISBN 0-936178-8-5
Usability testing of medical devices, Michael Wicklund et. al. ISBN 978-1-4398-1183-2
The growing role of human factors and usability engineering for medical device,
http://medicaldevices.bsigroup.com/en-GB/resources/Whitepapers-and-articles/

Copyright 2015 BSI. All rights reserved. 08/12/2015 43

Conclusion

Usability has become a vital part of the medical device development process
Usability activities should be conducted throughout all phases of the development
process, include it not a separate process
Usability should be part of the overall risk management process

Copyright 2015 BSI. All rights reserved. 08/12/2015 44

Questions?

Copyright 2015 BSI. All rights reserved. 08/12/2015 45