Clinical Decision Support Lecture 1b

Brief History and State of the Art of Clinical

Decision Support

www.cs.man.ac.uk/ai/modules/cds 1
 The Hype of the Time

• Guidelines
• Evidence Based Medicine
• Clinical Errors (reducing)
– Improving prescribing practice
– Reducing adverse drug reactions

• Protocols
• Knowledge Management
• ...
www.cs.man.ac.uk/ai/modules/cds 2
 Clinical Judgement and Clinical Errors

• To Err is Human
http://www.nap.edu/books/0309068371/html/
• Supporting a Humanly Impossible Task
• Johnson Articles - see resources

www.cs.man.ac.uk/ai/modules/cds 3
 Computer Aided Decision Support Works
(sometimes)

• Evidence of effectiveness growing

– 25 years since Clem McDonald’s
Protocol-based computer reminders, the quality of care and
the non-perfectability of man
• Use still limited

• Meta studies and reviews a decade old

• Elson R E and Connelly D P (1995). Computerized patient records in primary care: Their
role in mediating guideline-driven physician behaviour change. Archives of Family
Medicine 4: 698-705.
• Grimshaw J and Russell I (1993). Effect of clinical guidelines on medical practice: a
systematic review of rigorous evaluations. Lancet 342: 1317-1322.
• Johnston M, Langton K, Haynes R and Mathieu A (1994). Effects of computer-based
clinical decision support systems on clinical performance and patient outcome. A critical
appraisal of research. Archives of Internal Medicine 120: 135-142.

www.cs.man.ac.uk/ai/modules/cds 4
 Examples of Protocols –
See handouts

www.cs.man.ac.uk/ai/modules/cds 5
 Types of Decision Support: Information Tasks
• Informative
– Guidelines e.g. eBNF, BMJ Clinical Evidence,...
– Literature search - DxPlain

• Information structuring
– intelligent records (EPRs)
• PEN&PAD, Medcin vocabulary, ...

• Triggers and warnings

– MLMs, McDonald’s original work, HELP, ...

• Critiquing - Perry Miller

• Advising

www.cs.man.ac.uk/ai/modules/cds 6
 Types of Decision Support: Clinical Tasks

• Management Protocols
(often effective, Johnston et. al1994)
– Prescribing
– Protocol based care
• Oncocin, T-Helper, etc.
– Referral

• Diagnostics
(rarely effective, Johnston et. al1994)
• Mycin
• Internist I
• Knowledge Couplers

www.cs.man.ac.uk/ai/modules/cds 7
 Reasons for success and failure(1)

• Understanding of problem
– Meeting real and recognised needs
• Forsythe D E (1992). Using ethnography to build a working system:
rethinking basic design assumptions. Sixteenth Annual Symposium on
Computer Applications in Medical Care (SCAMC-92), Baltimore,
MD, Baltimore, MD: 505-509.

• Meeting them effectively

– “The user is always right…
but the user is usually wrong”
– The technology is still crude at best

• Implementing it successfully
www.cs.man.ac.uk/ai/modules/cds 8
 Reasons for success and failure(2)

• Most projects fail at implementation!

• The technology only works if people want it and use it
– Requires emphasis on participation, ownership, training, respect for
practicalities

• ‘Implementation’ begins with design

• Evaluation begins with design
– Formative evaluation essential

www.cs.man.ac.uk/ai/modules/cds 9
 Potted History (1)

• Bayesian stream
– 1968 Ledley and Lusted: Diagnosis using ‘Idiot Bayes’
discriminant
• Followed by Pauker Decision Support using utility theory
– 1970-1985 - de Dombal: ‘Idiot Bayes’ abdominal pain and other
surgical diagnostic problems
• Meanwhile RCP Computer Workshop refined discriminants and then
stimulated Spiegelhalter to come up with practical algorithms for
belief nets in early 1990s
– 1980s Society for Medical Decision Making formed and statistical
work largely separated from rule based work

www.cs.man.ac.uk/ai/modules/cds 10
 Bayes Methods

• Probablistic
– If you think the patient has one of a set of diseases:
• e.g. for Acute Abdominal Pain: Appendicitis, Obstruction, Perforating
Ulcer, Pancreatitis, Gallbladder Inflammation, Tubal preganancy (if
female), or ‘Other’
– If you know
• a) That a patient has an indicant
• a) The prior probability of each disease
• b) The probability that a patient with each disease has the indicant
– You can calculate
• c) the posterior probability that the patient has each disease given that
they have the indicant.
– And you can do so for all of a set of indicants.

www.cs.man.ac.uk/ai/modules/cds 11
 Bayes: simple example

• Prior probabilities: appendicitis 70% obstruction 10% other 20%

• Probability of indicant given diagnosis: (“vomiting without nausea”):
– 1% of patients with Appendicitis have indicant
– 25% of patients with obstruction have indicant Easy to collect
– 5% of patients with ‘other’ have indicant

• Posterior probabilities of appendicitis given indicant

– appendicitis70*1/correction  17% of pts with indicant have appe’is
– obstruction 1*25/correction  60% of pts with indicant have obstr
– other  2*5/correction  23% of pts with indicant have other

www.cs.man.ac.uk/ai/modules/cds 12
 Potted History (2)
• Rule based stream
– 1972 - Shortliffe Mycin: First rule based system
– 1970s US AIM Workshop produced “Big 4”
• Mycin/Oncocin/Puff - Backwards chaining ‘shells’
• Interist I - NEJM CPCs from a large network
– Became QMR as a general reference
• Casnet - Multilayer causal reasoning (glaucoma)
• Abel - Complex causal networks (acid-base metabolism)
– 1990s Protocol based reasoning
• Protégé/Eon successors to Mycin/Oncocin at Stanford
– Musen MA. Domain ontologies in software engineering use of Protégé with the EON
architecture. SMI Technical Report 97-0657. Methods of Information in Medicine
37:540-550, 1998.
• ProForma at ICRF
• ASBRU
• PRODIGY III

www.cs.man.ac.uk/ai/modules/cds 13
 Rules: Example from MYCIN

• IF the site of culture is blood

AND the method of culture is sterile
AND the aerobicity of the organism is anaerobic

THEN there is good (.65) evidence that the Diagnosis of

the Organism is Enterobacter

www.cs.man.ac.uk/ai/modules/cds 14
 Potted History (3)

• Reminders
– 1970 - Homer Warner, HELP, LDS
• 1980s - Arden Syntax
• 1990s - MLMs - standardised Arden
– 1970s - Clem McDonald - ‘…reminders and the nonperfectability
of man”
• Regenstrief laboratory systems
– Many variations
• PRODIGY II
• Systematic Review: Johnston M, Langton K, Haynes R and Mathieu A (1994).

www.cs.man.ac.uk/ai/modules/cds 15
 Examples

• CONDITION: the serum potassium is over 4.8 & the

patient is on any digitalis derivative
• ACTION: issue alarm

www.cs.man.ac.uk/ai/modules/cds 16
 Potted History (4)

• Offshoots and Idiosyncratics

– Critiquing - Perry Miller
• Also Johan van der Lei
– Quick Medical Reference - Chip Masari
– Intelligent Records - Alan Rector and Anthony Nowlan
– Knowledge Couplers - Larry Weed

www.cs.man.ac.uk/ai/modules/cds 17
 Potted History (5)
• Knowledge Management and the Web
– 1980s Grateful Med (PubMed) and DxPlain
• Quick access to Medline abstracts and related
– 1990s “The Web with everything”
• Rise of Evidence Based Medicine
– Cochrane, NICE, NELH, Health on the Web (HoN),…
• Indexing and ‘meta data’ & the Semantic Web
– How do you find it
• Portals and certification
– How do you know if it is any good
• Information for Public and Patients
– Its an open world out there
• Type “Diabetes Support” at Google 776,000 hits, AllTheWeb 295,000
Yahoo 26, Netscape 2000
• Classic Information Retrieval and Librarianship
– Digital Libraries
• Different fields with little contact
www.cs.man.ac.uk/ai/modules/cds 18
 Potted History 7:
Cuing and Intelligent Medical Records

• PEN&PAD, MedCin, …
• Almost took off
• Where we came from…

www.cs.man.ac.uk/ai/modules/cds 19
 Where we come from

Clinical Data Clinical Decision Best

Terminology Entry Record Support Practice

GALEN Data Electronic Decision

Clinical Best
Terminology Entry Health Support & Practice
Records Aggregated
Data

www.cs.man.ac.uk/ai/modules/cds 20
 Potted History 6:
Re-use, Terminologies and Ontologies

• Transferring the logic is easy

• Transferring the access rules in curly brackets is hard
– And it takes your most skilled people
• Subtle dependencies and system indiosyncracies

• The need for a common vocabulary

www.cs.man.ac.uk/ai/modules/cds 21
 Why isn’t decision support in routine use?
• Hypothesis one: “Pearls before swine”
– Doctors are ‘resistant’

• Hypothesis two: “The Emperor’s new clothes”

– Systems are not clinically worthwhile
• Not clinically useful
• Too time consuming - too hard to learn
• Too expensive
• Too inaccessible
• Too sparse
– How many diabetic patients does a GP see per week?
• Easier ways to get help
– The technology is still primitive
• Developers misunderstand medicine
– They think it is rational!

www.cs.man.ac.uk/ai/modules/cds 22
 Why isn’t decision support in routine use?

• Hypothesis 3: “The invisible computer”

– When it works, no one notices
• ECG interpretation
• Alerts and reminders
• NHS Direct
– Simple but effective?
– Junior doctors’ PDAs
• Convergence of communication and computing
• Upmarket PDAs have 10-100 times the power of the machine that
first ran Mycin!
– Why Web technology and XML are critical to this course
• divorce content and presentation

www.cs.man.ac.uk/ai/modules/cds 23
 A modern View
The Tripod

• Electronic Patient Records

• Decision Support
• Terminology and Ontologies

Plus
• Knowledge Management/Information Retrieval

www.cs.man.ac.uk/ai/modules/cds 24
 Patient Specific Records

Information Model
(Patient Data Model)

Inference Model Concept Model

(Guideline Model) (Ontology)

Dynamic Guideline Static Domain

Knowledge Knowledge (2a)
 A Protocol

www.cs.man.ac.uk/ai/modules/cds 26
Who Should Be Evaluated for UTI?
Under the assumptions of the analysis, all febrile children between the ages of 2 months and 24 months with no
obvious cause of infection should be evaluated for UTI, with the exception of circumcised males older than 12
months.

Minimal Test Characteristics of Diagnosis of UTI

To be as cost-effective as a culture of a urine specimen obtained by transurethral catheter or suprapubic tap, a
test must have a sensitivity of at least 92% and a specificity of at least 99%. With the possible exception of a
complete UA performed within 1 hour of urine collection by an on-site laboratory technician, no other test meets
these criteria.

Performing a dipstick UA and obtaining a urine specimen by catheterization or tap for culture from patients with a
positive LE or nitrite test result is nearly as effective and slightly less costly than culturing specimens from all
febrile children.

Treatment of UTI
The data suggest that short-term treatment of UTI should not be for <7 days. The data do not support treatment
for >14 days if an appropriate clinical response is observed. There are no data comparing intravenous with oral
administration of medications.

Evaluation of the Urinary Tract

Available data support the imaging evaluation of the urinary tracts of all 2- to 24-month-olds with their first
documented UTI. Imaging should include VCUG and renal ultrasonography. The method for documenting the UTI
must yield a positive predictive value of at least 49% to justify the evaluation. Culture of a urine specimen obtained
by bag does not meet this criterion unless the previous probability of a UTI is >22%.

FOOTNOTES
The recommendations in this statement do not indicate an exclusive course of treatment or serve as a standard of
medical care. Variations, taking into account individual circumstances, may be appropriate.
www.cs.man.ac.uk/ai/modules/cds 27
 The Technologies

• Semi Structured text and XML – GEM/GEMCUTTER

– Systematising what is needed
– XML is now the generic syntax for everything
• Find out about it from any handy book or web tutorial

• Knowledge representation, Ontologies, terminologies, –

Protégé, OilEd and OWL
• Rule based systems - Tallis/ProForma
• Bayesian inference MicroHugin

www.cs.man.ac.uk/ai/modules/cds 28
 Exercises/Lab
• Follow links or your own knowledge to find a range of
“guidelines” on the Web.
– Compare what is on offer
• Who are they for? What are they for?
• Who has supplied them?
• How might you use them as a clinician treating patients?
• How might you author one?
• How might you provide computer support to follow one?

• Pick a disease
– Look it up in Google with “protocol”, “guideline”, “systematic review”
• Answer the questions above.

• Find some protocols in your institution, if you work in the NHS

– answer the questions above.

www.cs.man.ac.uk/ai/modules/cds 29
 Apology
…We hope this is fixed…

• Illogical order due to technical difficulties

– Due to the wrong kind of .dll in the MS automatic upgrade, we will
discuss & demonstrate GEM later when the software is has been
upgraded to fit the new ‘standard’. Microsoft is unlikely to
apologise for the inconvenience this will cause to your module, but
we apologise and will do our best to cope.

www.cs.man.ac.uk/ai/modules/cds 30