a ee Ecole Nationale d'Electronique et des ns Télécommunications de Sfax (ENET’COM) ener‘com Bache es as Graph Machine Learning 3eme année IDSD-ID Tarak BEN SAID [email protected] Graph Machine Learning Prerequisites Machine Learning Algorithms and graph theory Probability and statistics Python programming References Graph Representation Learning by William L. Hamilton Networks, Crowds, and Markets: Reasoning About a Highly Connected World by David Easley and Jon Kleinberg Network Science by Albert-Laszlé Barabdsi https://www.stanford.edu/Graph Machine Learning Course Outline Introduction Traditional Feature-based Methods Node Embeddings Label Propagation for Node Classification Graph Neural Networks Applications of GNN IntroductionIntroduction What is a Graph 2? Mathematical structure for pairwise relations Nodes Edges Why Graphs Graph is a general method for describing and modeling complex systems Introduction Many Data are Graphs at Aa at gt Aa at Pe DR ae ___Pationts aye eRe 2 & Aa a ats \ (oo ® aae ye a fag a3 1 ae ae $A x Ae £ Sg dog ae eS 1k ang é. yy : af Aa FOR a Sgt wet Aye Bipartite Graph Social Networks Health Records Nodes: Person/Account Nodes: Patient, Medical Service Edges: Friendship/Follows Edges: Treatment by DoctorsIntroduction Many Data are Graphs Directed Graph Heterogenous Graph Financial Transactions E-Commerce Data Nodes: Customer, Merchant Nodes: Person, Product, Credit Cards, ... Edges: Transaction/Payment Edges: Has Phone, Has Address, Orders, .. Introduction Graph ML tasks Node level Graph-level oo 4a Community rediction, Graph (subgraph) generation level Edge-levelIntroduction Graph ML tasks Node Prediction PQ WEE — meer “ee fe Input: Social Network Input: Transactional Graph Task: Identify fake user with Task: Find user that use stolen influence power credit card in the transactions Introduction Graph ML tasks Edge Prediction Patients Medical Services Input: Health Records Graph Task: Predict if a patient need to see a doctor for medical treatmentIntroduction = a Classical ML Algorithms Tabular Data —~ seh) Logistic Regression (Boosted) Decision Tree a Introduction = 2 Classical ML Algorithms Information Loss Graph TabularIntroduction Node Embedding Map nodes in a graph to numerical features (embedding) similar nodes > similar embeddings . _—* * a oe | y : . = : Classifier Graph Embedding (Mit models) = Similarity? = Limitations 2? Introduction Neural Networks and Deep Learning Input aye Maple hidden tayo output ayerIntroduction Neural Networks and Deep Learning Grids Images Introduction Neural Networks and Deep Learning , Fee oA fA LALLA ? 64.446 Recurrent Networks (RNN) Text J Pa] Ca Gee] Come ae Cee} Sequences ® ; 6Introduction Neural Networks and Deep Learning Graphs ? = Arbitrary size and complex topological structure (no spatial locality like grids) eevee Text Networks. Images = No fixed node ordering or reference point = Often dynamic and have multimodal features > Graphs are the new frontier of deep learning Introduction Graph Neural Networks End-to-end learning for graph data Convolutional Neural Networks (CNN) Hiden ayer Hiden ayer Graph Convolutional Networks (GCN)Introduction Graph Neural Networks Convolution Operator e \ CNW layer with 33 iter oO Oo computation flow SOO * wen GCN computation — i —_} flow ° 6 - / ” oe € yy 2 layers GCN computation flow Introduction Graph Neural Networks Weight Sharing INPUT GRAPH Compute graph fornodeA © Compute graph for node B Inductive CapabilityIntroduction Graph Neural Networks Graph Convolutional Networks (GCN) GraphSAGE [Sample and Aggregate] Graph Attention Networks (GAT) Introduction Tools for Graph Neural Networks Deep Learning Frameworks © PyTorch ‘F TensorFlow Jom /™ G@ Py DG — MaSpektra Graph Neural Network Frameworks