An Introduction to Artificial Intelligence

[Fall 2023 – Module Main Contents]

Lecture 1: An Introduction to Artificial Intelligence

1.1 What is Intelligence

 Some Foundations of AI – What is Intelligence? – What is Artificial Intelligence?

1.2 Systems that Act Like Humans

 Systems that Act Like Humans – Turing Test? – The Chinese Room Argument – Strong Vs. Weak
AI – Where are we?

1.3 AI as the Study & Design of Intelligent Agents

 Systems that Think like Humans – Systems that Think Rationally – Main Research Problems /
Challenges – Systems that Act Rationally – AI as the Study & Design of Intelligent Agents –
Intelligent Agents in the World

Lecture 2: Intelligent Agents, & AI Related Disciplines

2.1 Intelligent “Rational” Agents?

 Recap: AI as the Study & Design of Intelligent Agents – Intelligent Agents in the World –
Specifying the Task Environment [PEAS] – Goad-based vs. Cost-based Agents

2.2 Specifying the Task Environment

 Environment Types

2.3 AI: Related Disciplines

 Learning Agents – AI vs. Machine Learning vs. Deep Learning – Machine Learning? –
Datamining? – AI vs. Data Science – Why Data Science? – Why Big Data?

Lecture 3: Problem Solving as Search & State Space Search

3.1 Solving Problems by Searching

 Types of Agents (Reflex vs. Planning) - State Space Search - Example: Tic-Tac-Toe Game -
Example: Mechanical Fault Diagnosing - How human beings think? - Heuristic Search

3.2 State Space Search Graph & Strategies

 Search Problem Components - State Space & State Space Graph - Examples: Vacuum World, The
8-Puzzle, Romania, etc. - State Space Search Strategies - Selecting Search Strategy

3.3 Basic Idea of Search & the Backtracking Search Algorithm

  Search: Basic idea - Search Tree - Tree Search Algorithm-Outline & Example - Handling Repeated
States - Backtracking Search Algorithm & Data Structures

Lecture 4: Knowledge Representation via Propositional & Predicate Calculi

 Propositional Calculus
o Propositional Calculus Symbols
o Propositional Calculus Sentences (WFFs)
 Well Formed Formulas (WFFs)
o Propositional Calculus Semantics
 Truth Tables
 Truth Tables Example
 Common Identities
o Limitations of propositional Calculus
 Predicate Calculus?
o Predicate Calculus Symbols and Terms
o Predicates and Atomic Sentences
 Predicate Calculus Sentences
 Predicate Calculus Sentences Examples
o Predicate Calculus Semantics
 Predicate Calculus Expressions Truth Value
 English Sentences Represented in PC
 Inference Rules
 Common Inference Rules
o Unification
 Unification Rules
 Unification Examples

Lecture 5: Problem Solving as Search | Blind vs. Heuristic Search

5.1 Blind / Uninformed Search [ Part 1 ]

 Blind vs. Heuristic Strategies – Blind / Exhaustive Strategies - Depth-First Strategy (DFS) –
Breadth-First Strategy (BFS)

5.2 Blind / Uninformed Search [ Part 2 ]

 Depth-Limited & Iterative-Deepening Strategies – Complexity / Comparison of Blind Search

Strategies – Bidirectional Search – Avoiding Repeated States – Uniform-Cost Strategy

5.3 Heuristic Search & Functions [ Part 1 ]

 Best-First Search – Examples of Evaluation function

5.4 Heuristic Search & Functions [ Part 2 ]

 Symmetry / Heuristic Reductions – Hill Climbing Strategy

Lecture 6: Problem Solving as Search – Adversarial Search

6.1 Adversarial Search [ Heuristics in Two-Person Games ]

 Minimax Procedure

 Exhaustive Minimax for Nim [ Splitting Stacks ]

 Minimax to a Fixed Ply Depth

6.2 Alpha-Beta Pruning

 Alpha-Beta Procedure

 Alpha-Beta Example

6.3 Search … What’s the Issue?

 Other Search Techniques

 When to Use Search Techniques?

Lecture 7: Beyond Classical Search: Evolutionary Computation [ Genetic Algorithms ]

7.1 Principal Heuristic Algorithms & Genetic Algorithms Overview

 Principal Heuristic Algorithms - Genetic Algorithms (GA): An Overview - Premise of GAs: Natural
Selection - Premise of GAs: Inheritance of Characteristics - General Scheme of Genetic
Algorithms - Stochastic Operators - The Metaphor

7.2 General Scheme of Genetic Algorithms

 A Basic Example: The 8 Queens Problem ( Representation, Fitness Function, Crossover &
Mutation, Selection for Replacement ) - Typical Behaviour of an EA - Pseudocode for a Typical
GA - Termination Condition - Progression of Fitness - GA Convergence - Exploration versus
Exploitation - Are long runs beneficial?

7.3 GA Terminology

 Chromosome, Gene, Alleles, Population, Genotype, & Phenotype – Encoding versus Decoding –
Example: the MAXONE problem

7.4 More Details on the Components of a GAs

 Representation (Encoding) – Initialization – Selection (Selection by Ranking, Roulette Wheel

Selection, Tournament Selection) – Recombination (Crossover) – Mutation – Elitism –
Evaluation (Fitness Function) – Termination condition – Example: The Traveling Salesman
Problem (TSP)

Lecture 8: Supervised Machine Learning: Decision Trees via ID3 Iterative Dichotomiser 3 Algorithm

8.1 Classical Supervised Machine Learning.

8.2 Decision Trees (the ID3 algorithm – based on information gain).

8.3 An Application to Business Intelligence using Weka.

Lecture 9 [Part A]: the Learning Problem, the Perceptron, & the Perceptron Learning Algorithm [ PLA ]

9.1 The Learning Problem:

• Learning / Recognition versus Defining – Examples: The Netflix Prize; & The Credit Risk
Assessment Problem; – Empirical versus Deductive Solutions – The Essence of Machine Learning

9.2 Learning:

• Learning Hypotheses / Model – Learning Algorithm

9.3 The Perceptron [ Linear Separator ]

• Geometry of the Perceptron – Perceptron Learning Algorithm (PLA) – Linearly Separable?

9.4 The three main types of Learning Problems in ML

Lecture 9 [Part B]: the Multi-Layer Perceptron, the Artificial Neuron, Neural Networks & the
Backpropagation Algorithm

• What can be achieved using a Single Perceptron? [ Linear Patterns ]

• Linear Classification; an Example: Perceptrons for logical or, and, & xor

• Non-Linear Classification? Multilayer Perceptron for the xor operation

• Multilayer Perceptron [ a network of perceptrons ]

• Introduction: What are Neural Networks?

• Background & How the Human Brain learns

• A Neuron Model: A Simple Neuron

• A Complicated Perceptron

• Neural Networks: Architectures – A Bit of History – Network Layers – Network Structure –

Weights – Training: Back-Propagation Algorithm

Lecture 10: Ensemble Learning

[ Part 1 ] Basic Concepts:

 Wisdom of Crowds & Condorcet’s Jury Theorem

 Strong Learners versus Weak Learners
 Primary Types of Ensembles: Data-Centered versus Model-Centered

[ Part 2 ] Constructing Ensembles: A Generic Approach

 Conditions to Create an Ensemble?

 Diverse Classifiers, and How to produce them.
 Example: Randomization of Decision Trees
 Questions examined to construct an ensemble?
[ Part 3 ] Aggregation Methods (Combining Classifiers) [ PART A – Classifier Fusion ]

 Algebraic Combiners
 Simple Majority (Plurality) Voting
 Weighted Majority Voting
 Track–Record Voting

[ Part 4 ] Aggregation Methods (Combining Classifiers) [ PART B – Classifier Selection ]

 Meta Learning Methods (Meta Classifiers) .. ?

 MoE: Mixture of Experts (Community of Experts)
 Bucket of Models
 Stacked Generalization (Stacking)

[ Part 5 ] Specific Instantiations of Ensemble Learning (Part A):

 Bagging (based on Bootstrap Resampling)

 Pasting (based on Resampling w/o Replacement)
 Random Forests (based on Decision Trees) - Variants of Random Forests

[ Part 6 ] Specific Instantiations of Ensemble Learning (Part B):

 Boosting
 Boosting by Sampling vs. Boosting by Weighting
 AdaBoost (Adaptive Boosting)
 Bagging versus Boosting: A Summary

Lecture 11: Evolutionary Computation: Differential Evolution as an Optimization Method