Graphical Models For Data Mining: NLP-AI Seminar
Graphical Models For Data Mining: NLP-AI Seminar
NLP-AI Seminar
Title Page
Contents
JJ
II
Page 1 of 39
Go Back
Full Screen
Close
Quit
Motivation
Bayesian Networks
JJ
II
Page 2 of 39
Go Back
Full Screen
Close
Expressive Power
Example Applications
Gene Expression Analysis
Web Page Classification
Quit
Summary
Contents
JJ
II
Page 3 of 39
Go Back
Full Screen
Close
Quit
Title Page
Sprinkler
Contents
JJ
II
Rain
Wet Grass
Page 4 of 39
Go Back
Full Screen
Close
Quit
A B : A causes B
Title Page
Contents
JJ
II
Page 5 of 39
Go Back
Full Screen
Close
Quit
JJ
II
Page 6 of 39
Go Back
Full Screen
Close
Quit
Examples : DNA Sequences, Social Networks, Hyperlink structure of Web, Phylogeny Trees
Contents
JJ
II
Page 7 of 39
Go Back
P (X1 , . . . , Xn ) =
n
Y
Pi (Xi | P a(Xi ))
i=1
Full Screen
Close
Quit
Figure
from [RN95]
adapted
in Vision
Title Page
Contents
JJ
II
into
X1
P (X1 , . . . , Xn ) =
X3
X2
1 Y
c (Xc )
Z
cC
Page 8 of 39
Go Back
X4
Full Screen
Close
X5
Quit
Z=
XY
~
x cC
c (Xc )
Expressive Power
Directed vs Undirected Models
Dependencies which can be modeled - Not exactly
similar
Title Page
Example :
Contents
JJ
II
Page 9 of 39
Go Back
Full Screen
Close
Quit
JJ
II
Page 10 of 39
Go Back
Full Screen
Close
Quit
Inference
Title Page
Contents
II
Page 11 of 39
Go Back
Full Screen
Close
Quit
exponential terms
Complexity handled by exploiting
mate
Some Examples : Variable Elimina-
Learning
Title Page
Contents
JJ
II
Page 12 of 39
Go Back
Full Screen
Close
Quit
Title Page
Contents
JJ
II
Page 13 of 39
Go Back
Full Screen
Close
Quit
Applications
Bio-informatics
Gene Expression Analysis
Title Page
Contents
JJ
II
Page 14 of 39
Go Back
Full Screen
Close
Quit
Contents
JJ
II
Page 15 of 39
Go Back
Full Screen
Close
Quit
JJ
II
Page 16 of 39
Go Back
Full Screen
Close
Quit
Gene Expression
Each cell has same copy of DNA still different cells
Title Page
Contents
JJ
II
Page 17 of 39
Go Back
Full Screen
Close
Quit
Genes expressed vary - Based on time, location, environmental and biological conditions
JJ
II
Page 18 of 39
Go Back
Full Screen
Close
Quit
Expression Level
Estimated based on amount of mRNA for that gene currently present in that cell
Ratio of expression level under experiment condition to expression under normal condition taken instead
Title Page
Contents
JJ
II
Page 19 of 39
Go Back
Full Screen
Some Examples
EBI
Micro-array
data
repository
(http://www.ebi.ac.uk/arrayexpress/)
Stanford Micro-array Database (http://genomewww5.stanford.edu/) etc.
Close
Quit
JJ
II
Page 20 of 39
Go Back
Full Screen
Close
Quit
ing
JJ
II
Page 21 of 39
Go Back
Approaches
Clustering
Bayesian Networks
Probabilistic Relational Models (PRMs)
Full Screen
Close
Quit
Clustering
Title Page
Contents
JJ
II
Two-Side Clustering
Genes and Experiments partitioned into clusters G1 , . . . , Gk
and E1 , . . . , El simultaneously
Summarizes data into groups of k l
Assumption - Expression governed by a distribution specific
to each combination of Gene/Experiment clusters
Page 22 of 39
Go Back
Full Screen
Close
Quit
Bayesian Networks
Bayes Net - DAG encoding the con-
JJ
II
Page 23 of 39
Go Back
n
Y
P (Xi | P a(Xi ))
i=1
Full Screen
JJ
II
Page 24 of 39
Go Back
Full Screen
Close
Quit
Contents
JJ
II
Page 25 of 39
Go Back
Full Screen
Close
Quit
PRMs (Contd. . . )
Title Page
Contents
JJ
II
Page 26 of 39
Go Back
Full Screen
Close
Quit
A Sample PRM
Title Page
Contents
JJ
II
Page 27 of 39
Go Back
Full Screen
Close
Quit
a
a
Gene
Contents
JJ
II
Array
GCluster
Phase
AAM
ACluster
Page 28 of 39
Go Back
Level
Expression
Full Screen
Close
Quit
Inferencing in PRMs
A Relational Skeleton is an instantiation of this
Title Page
Contents
JJ
II
Page 29 of 39
Go Back
Full Screen
Close
Quit
schema
Relational skeleton completely specifies the values for the reference slots
Objective
Given , with observed evidence regarding some
variables, update the probabilistic distribution over
the rest of the variables
Title Page
Contents
JJ
II
Page 30 of 39
Go Back
Full Screen
Close
Quit
JJ
II
Page 31 of 39
Go Back
Full Screen
Close
Quit
Title Page
S1
S2
S3
Array
Contents
g.R(t1)
JJ
II
g.R(t2)
Phase
ACluster
Page 32 of 39
Go Back
Level
Full Screen
Expression
Close
a
Quit
Contents
JJ
II
Page 33 of 39
Go Back
Full Screen
Close
Quit
ative to data
Search Algorithm - finding the structure with highest score
Bayesian Score as scoring function- Posterior of structure
given data P (S | D)
Greedy local structure search used for search algorithm
JJ
II
Page 34 of 39
Go Back
Full Screen
Close
Quit
Capable of learning unified models integrating sequence information, expression data and annotation
data
Web Mining
Collective Web Page Classification [CDI98]
Title Page
Contents
JJ
II
Page 35 of 39
Go Back
Full Screen
Close
Quit
Summary
Title Page
Contents
JJ
II
Page 36 of 39
Go Back
Full Screen
Close
Quit
Graphical Models - A natural formalism for modeling multiple correlated random variables
Title Page
Contents
JJ
II
Page 37 of 39
Go Back
Full Screen
Close
Quit
Thanks!
References
[NLD99] Nir Friedman, Lise Getoor, Daphne Koller and Avi Pfeffer, Learning Probabilistic Relational Models, In Proceedings of IJCAI 1999, pages 1300-1309, 1999.
[CDI98] Soumen Chakrabarti, Byron E. Dom and Piotr Indyk , Enhanced hypertext categorization using hyperlinks , In Proceedings of SIGMOD-98, ACM International
Conference on Management of Data , pages 307318, 1998.
Title Page
[Chi02] David Maxwell Chickering, The WinMine Toolkit, Microsoft, MSR-TR-2002103, 2002, Redmond, WA.
Contents
JJ
II
Page 38 of 39
Go Back
[Col02] Michael Collins, Discriminative Training Methods for Hidden Markov Models:
Theory and Experiments with Perceptron Algorithms, In the proceedings of EMNLP
2002, pages 18, 2002.
[Fri00] Friedman N., Linial, Nachman I. and Peer D., Using Bayesian Networks to Analyze Expression Data, Journal of Computational Biology, vol 7, pages 601-620,
2000.
[GS04] Shantanu Godbole and Sunita Sarawagi, Discriminative Methods for MultiLabeled Classification, In Proceedings of PAKDD 2004, 2004.
Full Screen
Close
Quit
JJ
II
Page 39 of 39
Go Back
Full Screen
Close
Quit
[MWJ99] Kevin P. Murphy, Yair Weiss and Michael I. Jordan, Loopy belief propagation
for approximate inference : An emperical Study. In Proceedings of UAI 99, Pages
467-475, 1999.
[JP98] Pearl, J., Probabilistic Reasoning in Intelligent Systems: Networks of Plausible
Inference, Morgan Kaufmann Publishers, 1988.
27 6, 35
6 27 28 32 4, 7 9