MIW - Methods of Knowledge Engineering, Introduction - lecture 1 1

Methods of Knowledge Engineering

Introduction - lecture 1
Adam Szmigielski
[email protected]
materials: f tp(public) : //aszmigie/M IW English
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 2

Goal of lectures
• Review of knowledge engineering techniques
– construction and representation of knowledge,
– ways to use knowledge,
– automation of inference.
• Review of methods of knowledge representation and inference in
conditions of uncertainty,
• Preservation of selected AI issues in the light of applications in
specializations D, E, F.
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 3

How to pass MIW ?

• There are two marks from the subject of the MIW - the grade
from the exercises and the mark from the exam,
• The subject ends with an exam. At the end of the semester there
will be a colloquium. The grade from the colloquium is the grade
from the zero exam,
• No exemptions from the exam are expected.
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 4

Detailed conditions for passing the exercises

• To get 100 points = 70 p. Exercises + 30 p. Colloquium
– Exercises 70 p. - During the classes, students will get 7
mini-projects to be completed on the exercises - each after 10
points. The project without losing points can be completed for the
next week. An extension for another week results in a loss of 5
points.
– Colloquium 30 p. - colloquium on 14-th exercise,
– In case of a justified absence on the colloquium (14 exercises) it is
possible to write a retake test on the 15th exercise.
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 5

• Assessment of the exercises is based on the following table:

grade points - exercises + colloquium
2 0- 50
3 50,5 - 60
3,5 60,5 - 70
4 70,5 - 80
4,5 80,5 - 90
5 90,5 - 100
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 6

Colloquium - zero exam

• On 14 exercises there is a colloquium for 30 points,

• Colloquium can be treated as a zero exam,

• The mark from the colloquium is transcribed for the exam based on
the following table:
grade points from the colloquium
2 0- 15
3 15,5 - 18
3,5 18,5 - 21
4 21,5 - 24
4,5 24,5 - 27
5 27,5 - 30

• The necessary condition for passing the grade from the test to the zero
exam is to pass the exercises.
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 7

Semester lecture schedule

1. Introduction, Knowledge Engineering,

2. Information, data and knowledge - parametric and nonparametric models,

3. Probabilistic modeling - Bayesian networks, Markov chains,

4. Classification - KNN algorithms, logistic regression,

5. Combining different models for group learning,

6. Linear regression,

7. Learning systems - classic approach,

8. Convolutionary networks (CNN),

9. Recursive networks (SRU, GRU, LSTM),

10. Unsupervised and partially supervised learning,

11. Reasoning systems, inference automation,

12. Modeling and fuzzy reasoning,

13. Data mining, association rules,

14. Approximate inference, rules, reductions,

15. Deep learning.

MIW - Methods of Knowledge Engineering, Introduction - lecture 1 8

Semester exercise plan

1. Introduction to Python,

2. Getting to know the libraries Pyplot, Numpy, Scipy, Pandas etc.

3. Determination of the stationary distribution for the markova chain using Numpy (project 1),

4. Implementation of the KNN algorithm, the Numpy logistic regression model (project 2),

5. Implementation of a decision tree and random forests (project 3)

6. Performing linear regression with Numpy (project 4),

7. Implementation of a simple one-way network - back propagation (project 5),

8. Classification of a data set using a convolution network using Keras (project 6),

9. Prediction using recursive networks using Keras (project 7),

10. Implementation of an autoencoder or Boltzman machine (project 8)

11. Introduction to the prolog (project 9)

12. Fuzzy logic,

13. Apriori algorithm,

14. Test,

15. Finding rules, reducts.

MIW - Methods of Knowledge Engineering, Introduction - lecture 1 9

Knowledge Engineering
• Knowledge engineering - the engineering and science grew on the
basis of artificial intelligence, dealing with methods of
exploration, representation and modeling of knowledge and
methods of inference based on them.
• Is an interdisciplinary science from the borderline of
mathematics, logic, computer science, automation, psychology,
cognitive science.
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 10

Knowledge Engineering
• Knowledge Engineering initially dealt with expert systems, i.e.
systems in which the knowledge of experts in a given field was
usually represented in a rule-based knowledge base, and
processing was limited to logical inference,
• Knowledge bases (including models) are currently being created
and various technologies are used to process knowledge by
computer, autonomous and interactive systems,
• Creating knowledge base systems requires cooperation not only
with IT specialists, but also psychologists, cognitive scientists,
linguists, mathematicians, etc.
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 11

Topics of knowledge engineering

• Machine learning (automatic learning),
• Extracting knowledge from data,
• Text processing engineering (extracting knowledge expressed in
word form).
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 12

Fields of knowledge engineering

• Machine learning,
• Pattern recognition,
• Data analysis
• Database techniques (including data warehouses),
• Software engineering,
• Algorithmics,
• etc.
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 13

Hierarchical structure of knowledge

• Data
Syntax
Relationship between characters
Character relationships with each other, apart from character
relationships with their counterparts in the world. The
relationship between characters is important.
• Information
Semantics
Relationship between characters and their meanings
Meaning of characters - the content that characters (data) carry
in the message. The relationship between a set of characters with
a specific structure and marked reality (content of the message)
is important
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 14

• Knowledge
Pragmatics
Relationship between characters and their users and their use in
a specific context
Users who set the objectives of the required actions in a given
situation (context of the actions taken) are included. The third
dimension of character analysis is taken into account - the
purposefulness of their use and the dependence of their sense on
the context and users of the characters
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 15

Tarski’s semantic definition of truth

Sentence “a” is true if a is true:

sentence
logic 6
truth: relation fact - sentence

world ?
fact
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 16

Inference and knowledge

Knowledge

Representation Sentence X - Sentence A

Inference 6 6

semantic
? ?
World Fact X - Fact A

• Inference is the relationship between sentences,

• The sentence A infers from the set of sentences X when it
reflects the sequence of facts corresponding to these symbols in
the modeled world.
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 17

Main areas of interest in Knowledge Engineering,

• Knowledge modeling - creating models from data,
• Data mining - finding dependencies in your data,
• Discovery of knowledge - this is the process of discovering
knowledge in data or consisting in searching for regularity,
repeatability between data.
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 18

Data mining

It is a multi-stage process associated with

• initial data processing (cleaning, standardization, standardization
or other type of transformation),
• comparison,
• integration and grouping,
• selection of data of their characteristics, groups, similarities,
differences and dependencies,
• data visualization.
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 19

Data mining techniques

• Transformation,
• Reduction,
• Selection,
• Classification,
• Clustering,
• Prediction,
• Association,
• Aggregation and consolidation,
• Visualization and presentation.
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 20

Data mining process

It is multi-stage, you can roughly distinguish the following stages of
the process:
1. Determining the purpose of data analysis - defining the purpose
of practical exploration,
2. Preparing the database for analysis - defining the records related
to the purpose of the analysis,
3. Cleaning and initial transformation - through their
standardization, standardization, deletion of outliers, removal or
completion of incomplete patterns,
4. Transformation of data from symbolic form to numeric form,
5. Data dimension reduction - selection of the most significant
attributes,
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 21

6. Choosing a data mining method

7. Choosing how to implement the data mining technique,
8. Implementation of the process and tuning the algorithm of data
mining,
9. System operation - inference based on the created system,
10. Training the system on new data.
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 22

Classification
• The classification consists in finding a method of mapping data
into a set of predefined classes.
• Based on the data, classifier is built (eg, decision tree, logistic
regression model), which is used to classify new objects,
• The created classifier assigns new objects to one of the classes.
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 23

Regression

A method that allows testing the relationship between data volumes.

Allows you to predict unknown values of one value based on the
known values of others.

The use of regression in practice comes down to two phases:

• Constructing the model - construction of the so-called the
regression model, a function that describes how the expected
value of the explained variable depends on the explanatory
variables.
• Applying the model - using the computed model to the data in
which we only know explanatory variables to determine the
expected value of the explained variable.
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 24

Grouping - cluster analysis

Cluster analysis (grouping) is a task of data mining, which consists in

dividing (usually multidimensional) data sets into groups in such a
way that the elements in the same group are similar to each other,
and at the same time as different from the other groups.

The cluster analysis task includes two related subtasks:

• Segmentation - dividing the presented examples into groups,
• Characteristics of groups - generating descriptions of relevant
terms for each of the groups to be distinguished, and then using
these descriptions to classify new examples.
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 25

Discrimination
• Discrimination is about finding traits that differentiate the
indicated class of objects (target class) from other classes
(contrasting classes).
• For example, a set of discriminatory rules can describe those
symptomatic features that distinguish a given disease from
others.
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 26

Discovering associations

is to find relationships between the occurrence of groups of elements

in given data sets.
• The most popular example of discovering associations is the
so-called basket analysis - processing supermarket and warehouse
databases to find groups of goods that are most common bought
together.
• Associations found can, for example, indicate that when a
customer buys salty chips, he also buys beer.
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 27

Data visualization

is a method of creating, analyzing and transmitting information.

• With visual means, people exchange both abstract ideas and
messages that are directly based on reality,
• It is routinely used in technical disciplines to better present the
problem,
• In many cases, it can capture some hard-to-describe
dependencies, e.g. the Anscombe quartet
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 28

Visualization - Anscombe quartet

it is a set of four sets of data with identical statistical features, such as

arithmetic mean, variance, correlation coefficient or linear regression
equation, simultaneously looking quite different in the graphic
representation.
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 29

Learning systems

Systems that have the ability to improve the quality of their

activities by gaining new experiences that are then used during
subsequent interactions with the environment
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 30

Deep learning

class of machine learning methods, where the model has a

hierarchical structure composed of many non-linear layers.
• In 2006, Hinton showed that deep networks can be taught in a
different way,
• The proposed method assumed teaching the network layer by
layer, and then supervised the training,
• Multi-layer networks began to be called deep networks, and the
methods of learning such networks and the general issues related
to deep learning.
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 31

Automatic inference
• Forward inference,
• Backward inference,
• Algorithm of resolution,
• Programming in logic.
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 32

Expert systems
• These systems try to replace a specialist in one particular area of
knowledge,
• They are modeled on deductive processes, similar to those used
by everyone every day, without even realizing it.
MIW - Methods of Knowledge Engineering, Introduction - lecture 1 33

Literature - e-books in Polish

MIW - Methods of Knowledge Engineering, Introduction - lecture 1 34

Literature
1. Stuart Russell, Peter Norvig: Artificial Intelligence A Modern
Approach, Prentice-Hall, 1995

2. Richard O. Duda, Peter E. Hart, David G. Stork: Pattern

Classification,2ed, Wiley, 2001

3. Christopher M. Bishop: Pattern Recognition and Machine Learning,

Springer, 2006

4. Ian Goodfellow, Yoshua Bengio, Aaron Courville: Deep Learning,

PWN, 2018