SCS 412 Knowledge Based

Systems
Chapter 6 Knowledge Acquisition
 Sources of Knowledge
• documented
– books, journals, procedures
– films, databases
• undocumented
– people’s knowledge and expertise
– people’s minds, other senses
 Knowledge Acquisition
• Knowledge acquisition is the process by which
knowledge available in the world is transformed
and transferred into a representation that can be
used to create a KBS. World knowledge can come
from many sources and be represented in many
forms.
• Knowledge acquisition is a multifaceted problem
that encompasses many of the technical problems
of knowledge engineering, the enterprise of
building knowledge base systems.
 Knowledge Acquisition

• This has proved to be the most difficult

component of the knowledge engineering
process. It's become known as the 'knowledge
acquisition bottleneck', and expert system
projects are more likely to fail at this stage than
any other.
• This is the principle reason why expert systems
have not become more widespread.
 Knowledge Acquisition
Five stages:
• Identification Phase
– scope of problem
• Conceptualization Phase
– key concepts are operationalized and paper prototype built
• Formulation Phase
– paper prototype mapped onto some formal representation
and AI tools selected
• Implementation Phase
– formal representation rewritten for AI tools
 KA Phases

• Testing Phase
– check both "classic" test cases and "hard"
boundary” cases
– most likely problems
• I/O failures (user interface problems)
• Logic errors (e.g. bad rules)
• Control strategy problems
• Prototype Revision
 Organizing the Knowledge
• Knowledge Engineer
– Interacts between expert and Knowledge Base
– Needs to be skilled in extracting knowledge
– Uses a variety of techniques
 Knowledge Acquisition
• The basic model of knowledge acquisition
requires that the knowledge engineer mediate
between the expert and the knowledge base. The
knowledge engineer elicits knowledge from the
expert, refines it in conjunction with the expert
and represents the knowledge in the knowledge
base using a suitable knowledge structure.
• Elicitation of knowledge done either manually or
with a computer.
 Knowledge Elicitation

¨ Expert knowledge includes:

¨ domain-related facts & principles;
¨ problem-solving strategies;
¨ meta-knowledge - for instance, knowledge about
when to use a particular piece of knowledge;
¨ explanations and justifications.
 Knowledge Elicitation

¨ The knowledge elicitation/analysis task involves

¨ finding at least one expert in the domain who:
¨ is willing to provide his/her knowledge;
¨ has the time to provide his/her knowledge;
¨ is able to provide his/her knowledge.
- any or all of these are liable to prove difficult.
 Knowledge Elicitation

¨ The knowledge elicitation/analysis task involves

¨ repeated interviews with the expert(s), probably
combined with other, non-interview, techniques.
 Knowledge Elicitation - the
compiled knowledge problem
¨ One major obstacle to knowledge elicitation:
experts cannot easily describe all they know
about their subject.
¨ They do not necessarily have much insight into
the methods they use to solve problems.
¨ Their knowledge is "compiled" (like a compiled
computer program - fast & efficient, but
unreadable).
 Knowledge Acquisition
• Manual:
– interview with experts.
– structured, semi structured, unstructured interviews.
– track reasoning process and observing.
• Semi Automatic:
– Use a computerised system to support and help
experts and knowledge engineers.
• Automatic:
– minimise the need for a knowledge engineer or expert.
Knowledge Elicitation - interview
techniques
¨ Some of the interview techniques used in
knowledge elicitation:
¨ Unstructured interview. A general discussion of the
domain, designed to provide a list of topics and
concepts.
¨ Structured interview. Concerned with a particular
concept within the domain - a particular problem-
solving skill or small group of skills.
Knowledge Elicitation - interview
techniques
¨ interview techniques :
¨ Problem-solving interview. The DE is provided with a
real-life problem, of a kind that they deal with
during their working life, and asked to solve it. As
they do so, they are required to describe each step,
and their reasons for doing what they do. The
transcript of their verbal account is called a
protocol.
Knowledge Elicitation - interview
techniques
¨ interview techniques :
¨ Think-aloud interview. As above, but the DE merely
imagines that they are solving the problem
presented to them, rather than actually doing it.
Once again, they describe the steps involved in
solving the problem.
Knowledge Elicitation - interview
techniques
¨ interview techniques :
¨ Critical incident analysis. The DE is asked to provide
details of cases which were particularly difficult, or
of special interest for some other reason. He/she
describes how they were solved, and the lessons
that were learnt.
Knowledge Elicitation - interview
techniques
¨ interview techniques :
¨ Dialogue. The DE interacts with a client, in the way
that they would normally do during their normal
work routine.
Knowledge Elicitation - interview
techniques
¨ interview techniques :
¨ Review. The KE and DE examine the record of an
interview session together.
 Knowledge Elicitation -
non-interview techniques
¨ Some of the non-interview techniques used in
knowledge elicitation:
– Sample lecture preparation. The DE prepares a
lecture, and the KE analyses its content.
 Knowledge Elicitation -
non-interview techniques
¨ non-interview techniques:
¨ Concept sorting ("card sort"). The DE is presented
with a series of cards, with the names of domain
concepts written on them, spread out on a table top,
and asked to arrange them into clusters, in such a
way that the cards in each cluster have something
important in common. Then the DE is asked to name
the principles that he/she has used to form these
clusters. This process can be repeated to produce a
hierarchy of concepts.
 Knowledge Elicitation -
non-interview techniques
¨ non-interview techniques:
¨ Repertory grid (particularly the "laddered grid"
technique).
¨ Questionnaires. Especially useful when the
knowledge is to be elicited from several different
experts.
 Interviewing Techniques – Do's
and Don'ts
• It is essential to record and transcribe all the (video/audio-taped)
interviews.
• Transcripts should be clearly cross-referenced to (video/audio-tape)
recorder counter numbers.
• Include all the sketches, photocopies or reproductions of diagrams,
tables or the like, that were referred to during the interview(s).
• Once completed a copy should be sent to the interviewee for
comments, corrections and criticism. There is always the possibility of
misunderstanding by the knowledge engineer when interpreting a
statement or explanation.
• By involving the expert in validating his or her own transcript it
reduces the chance of erroneous information appearing in the
prototype's knowledge base.
 Computer-assisted knowledge
elicitation
'knowledge elicitation workbenches', or
'knowledge engineering environments', are
commercially available
e.g. KEE, KnAcqTools, ETS, KRITON, AQUINAS;
their principle use is to simplify the task of
converting a protocol into frames, rules, etc.,
and inserting these structures into an expert
system shell as soon as they are formulated.
 Fully computerised knowledge
acquisition
• It might be thought that one could avoid using a
domain expert altogether, by building a system
that could extract knowledge, given facts about
the domain.
¨ This is the approach taken by machine learning
systems:
¨ "classic" machine learning systems such as ID3
(Quinlan, 1979) & AQ11 (Michalski & Chilauski,
1980);
Fully computerised knowledge
acquisition
– systems designed to provide knowledge for a
particular system's knowledge base, e.g. META-
DENDRAL, designed to discover rules for the rule-
base in DENDRAL;
– data mining systems; these do a similar job to classic
machine learning systems, but work on a very large
database of information.
– sub-symbolic systems, i.e. neural nets and genetic
algorithms.
 Knowledge discovery

• e.g.(1) META-DENDRAL produced rules

about the behaviour of molecules in a mass
spectroscope that were published in a chemistry
journal as original contributions to the field;
• e.g.(2) AQ11 produced rules about how to
diagnose diseases in Soya bean plants. They
were correct 97% of the time. The domain
expert's rules were correct 83% of the time; he
abandoned his rules, and adopted AQ11's rules
instead.
 Knowledge Acquisition Difficulties

• Knowledge is not easy to acquire or maintain

• More efficient and faster ways needed to acquire
knowledge.
• System's performance dependant on level and quality
of knowledge "in knowledge lies power.”
• Transferring knowledge from one person to another is
difficult. Even more difficult in AI.
 Other Problems
• Other Reasons
– experts busy or unwilling to part with
knowledge.
– methods for eliciting knowledge not refined.
– collection should involve several sources not
just one.
– it is often difficult to recognise the relevant
parts of the expert's knowledge.
– experts change
 KA Techniques

• Description
– expert lectures or writes about solving the task
• Observation
– KE watches domain expert solve the task unobtrusively
• Introspection
– KE interviews expert after the fact
– goal-directed KE tries to find out which goal is being
accomplished at each step