Opportunities of eXplainable AI for

trustworthiness and knowledge

discovery
Sara Narteni
PhD Student @ CNR-IEIIT & Politecnico di Torino
Email: [email protected]
First International Forum on Computing
and Information Technology Trends 2022,
Online Event, 28-29 August 2022

IFCITT-2022, 28/08/2022
 Outline
• Introduction: XAI motivations

• A XAI model: the Logic Learning Machine (LLM)

• Usage of LLM for:

• Reliable AI  «safety regions» design

• GAN data augmentation assessment

• Rules comparison/pruning via rule similarity

• Some results on physical fatigue prediction domain

IFCITT-2022, 28/08/2022
 Why eXplainable AI
• Increasing AI legislation
• Demand for transparency
• Demand for technical robustness
• Demand for human agency

Black-box opening:
eXplainable AI
Figure 1. 7 principles from European Commission, Directorate-
General for Communications Networks, Content and algorithms
Technology, Ethics guidelines for trustworthy AI, Publications Office,
2019, https://data.europa.eu/doi/10.2759/177365

IFCITT-2022, 28/08/2022
 How eXplainable AI

Picture from:
S. Narteni, V. Orani, E. Cambiaso, M. Rucco and M. Mongelli, "On the Intersection of Explainable and
Reliable AI for Physical Fatigue Prediction," in IEEE Access, vol. 10, pp. 76243-76260, 2022, doi:
10.1109/ACCESS.2022.3191907.

IFCITT-2022, 28/08/2022
 The Adopted XAI model
Logic Learning Machine (LLM)
Discretization & • Global rule-based model
Latticization
• Developed by Rulex (CNR-IEIIT ex spin-off) and
available in Rulex Analytics platform
• Three-step design
Shadow
• For each rule : covering C() and error E()
Clustering

Rule Generation
(IF-THEN)

IFCITT-2022, 28/08/2022
 LLM Feature and Value Ranking
• Relevance of rules conditions:
relevance of condition c

acts as weight for condition c

• Relevance of values for a given feature:

Relevance of a value
assumed by feature j

• Feature ranking: aggregates values relevances and finds the

most influent attributes

• Value ranking: same of feature ranking, but on a subset of

feature values; finds the most relevant intervals

IFCITT-2022, 28/08/2022
Feature Ranking example

IFCITT-2022, 28/08/2022
Value Ranking example

IFCITT-2022, 28/08/2022
 Reliable AI from LLM
The goal: designing safety regions, i.e. the largest
regions in the feature space where predictions are
made with FNR=0

From Feature and Value Ranking:

1. Reliability from Outside
2. Reliability from Inside

IFCITT-2022, 28/08/2022
 Reliable AI from LLM
• Common structure

Value ranking
thresholds
tuning

IFCITT-2022, 28/08/2022
 Reliability Reliability
from Outside from Inside

• Start with the unsafe class (y = 1) • Start with the safe class (y = 0)
feature/value ranking feature/value ranking

• Find as: • Find as:

• Get the complementary to P as

safety region

IFCITT-2022, 28/08/2022
Inside and Outside
Application
Physical Fatigue Prediction case

IFCITT-2022, 28/08/2022
 Outside

Original intervals from value ranking of

fatigued class:
• Back rotation position in sagittal plane > 0.03
• Wrist jerk coefficient of variation > 0.03

Safety regions:
AGE > 40:
-1.86 < Back rotation position in sagittal plane < 0.42 AND
-2.0 < Wrist jerk coefficient of variation < -0.81
Metrics:
FNR = 0
AGE ≤ 40: TNR = 0.20
-2.52 < Back rotation position in sagittal plane <0.42 AND
-1.78 < Wrist jerk coefficient of variation < -0.81

IFCITT-2022, 28/08/2022
 Inside

Original intervals from value ranking of

not fatigued class:
• Back rotation position in sagittal plane < 0.03
• Chest Acceleration Mean > -0.47

Safety regions:
AGE > 40:
-1.86 < Back rotation position in sagittal plane < -1.68 OR
3.65 < Chest Acceleration Mean < 3.99 Metrics:
FNR=0
AGE ≤ 40: TNR=0.06
-2.52 < Back rotation position in sagittal plane < -1.68 OR
3.65 < Chest Acceleration Mean < 3.99

IFCITT-2022, 28/08/2022
LLM for GAN
evaluation

IFCITT-2022, 28/08/2022
The Idea

Adopted XAI:
LLM

Performance
evaluation in
brand new
scenarios

IFCITT-2022, 28/08/2022
Which is the most suitable way to
use GANs?

Best scenario selection

(largest performance metrics)

IFCITT-2022, 28/08/2022
What are the ‘best’ fake datasets
generated through the GAN runs?
• New rule similarity with a syntactic approach

• Given two rules and

1. Similarity between two generic conditions in and in

1 if conditions cover the Overlap between

same features, 0 otherwise conditions domains

IFCITT-2022, 28/08/2022
 2. Similarity between rules and :
Similarity between conditions

Weights of the conditions

• Average of within-class rule similarities averages

gives a ruleset similarity to compare different
rulesets
mean rule similarity between rules in ruleset
and rules in ruleset for class 0

mean rule similarity between in ruleset

and rules in ruleset for class 1

IFCITT-2022, 28/08/2022
GAN & Rule Similarity
Application
Physical Fatigue Prediction case

IFCITT-2022, 28/08/2022
 Experimental Setup
• Age groups definition:
- 161 samples for age 40  under 40 group;
- 108 samples for age > 40  over 40 group;

• Baseline performance on real data:

- Under 40: accuracy = 0.63; F1-score= 0.57
- Over 40: accuracy = 0.84; F1-score= 0.80

• 10 runs of GAN training on both under 40 and over 40:

# Generator neurons per layer: 128, 64, 32, 1
# Discriminator neurons per layer: 32, 64, 128
Batch Size: 64 adopted GAN parameters
Epochs: 5000
Learning Rate: 5e-5
IFCITT-2022, 28/08/2022
 Scenario selection
Under 40 group

Over 40 group

• For both groups, retraining the

LLM on combined real and fake
data reveals as the best scenario
(Scenario 2)

• Low performance on Scenario 1

• FID seems not indicative

IFCITT-2022, 28/08/2022
 Rule similarity results

• Rule similarity between real ruleset and “real + fake” ruleset

at each run
• Interest in minimum and maximum rule similarity values
IFCITT-2022, 28/08/2022
 Rules example (over 40, non-fatigued)
Real over 40 non-fatigued rules

Real + fake over 40 non-fatigued rules

IFCITT-2022, 28/08/2022
Esempio minima?

IFCITT-2022, 28/08/2022
Reliable AI without vs with data
augmentation

IFCITT-2022, 28/08/2022
 Outside on
Outside on
Over 40 –
Over 40 - real
real+fake

IFCITT-2022, 28/08/2022
 Conclusion
• Reliable AI methods for error (FNR) control based on XAI

• New framework of GANs evaluation driven by XAI instruments:

• LLM performance in three scenarios:
1. Fake LLM on real data
2. New LLM trained on real+fake
3. Fake LLM on real+fake
• Rule similarity

• Using GANs for data stratification (age groups)

• Understanding how GANs work to discover NEW KNOWLEDGE

• Using fake data to enhance reliable AI performance

IFCITT-2022, 28/08/2022
Thank You!

IFCITT-2022, 28/08/2022