Machine Learning for Complex Data

Networks and Sequences

Summer semester 2023

Prof. Dr. Martin Atzmüller

Osnabrück University & DFKI

Semantic Information Systems Group
https://sis.cs.uos.de/
2 – Graphs/Networks: Basic Models, Generation
1 Overview
2 Graph Generation
3 Basic Network/Graph Properties
4 Network/graph properties (Real Networks)
5 Traditional Graph Generators (Generative Models)
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Graph Generation - Overview

¡ We want to generate realistic graphs, using
graphWe
• Goal: generative models realistic graphs
want to generate
which is
Graph Generate similar to
Generative
Model
≈
Synthetic graph Real graph
[Leskovec, 2021]
¡Applications:
• Exemplary applications:
§ Drug discovery, material Drug
designdiscovery,
material design, modeling
§ Social network complex network modeling
11/11/21 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, cs224w.stanford.edu 3

1
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Why Graph Generation?

• Insights:
– understand the formulation of graphs
• Predictions:
– predict how the graph will further evolve
• Simulations:
– generate general novel graph instances
• Anomaly detection:
– decide if a graph is normal / abnormal

2
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Prerequisites – Understanding ...

• Properties of real-world graphs
è successful graph generative model should
fit these properties
• Traditional graph generative models
è Each with different assumptions on the
graph formulation process
• Deep graph generative models
è Learn graph formation process from the
data (later)

3
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Networks & graph

... more (basic) properties

4
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Reprise: Scale-invariant Networks

• Some nodes exhibit a lot of connections (hubs)
compared to others (few)
• For example: Robust against random attacks, but
not targeted ones
• Prominent nodes can have millions of links
è Networks seems to have no scale
• Two prerequisites:
– Growth
– Preferential Attachment

• Problem
– Scale-invariant networks only “really scale-invariant“ if
network is infinitely large ...
– Cut off
5
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Scale-invariant Networks
• Cut off

6
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Scale-invariant Networks
• Cut off

7
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Examples [Newman 2003]

Vorlesung
Web
Science

8
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Random Graphs • Random graph: A

network of nodes,
which are
connected via edges
completely at
random

9
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

10
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Background [Newman 2003]

• First example of a scale-free network:

– Citation network of scientific articles
– Power-law for in-/out-degree
• How do power-low distributions emerge?
• "The rich getting richer"
• "Preferential Attachment": High probability,
that a new node connects to an existing
node having a high degree
• "Matthew Effect": "For to every one who
has will more be given ..."

11
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Giant-Components
• How do Giant components emerge?
http://ccl.northwestern.edu/netlogo/models/GiantComponent

12
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

13
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Giant Component – Random Graph (1)

14
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Giant Component – Random Graph(2)

15
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Giant Component – Random Graph (3)

16
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Network properties
... of real networks

17
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Assumptions [Leskovec 2006]

• Assumed to be “well known", that networks

which are growing, are characterized by:
– Constant average nodal degree
è Edges grow linearly with nodes
– Slowly increasing diameter
è Increasing diameter via new nodes
• However: Empirical analysis shows that
– Networks get denser and denser
è densification power laws
– Effective diameters decreases in many cases when
networks are growing (decreasing diameter)

18
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Empirical Observation: Densification

[Leskovec 2006]

19
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Empirical Observation: Densification

[Leskovec 2006]

20
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Empirical Observation: Effective Diameter

[Leskovec 2006]

21
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Motivation [Leskovec 2006]

• Which processes cause graphs to

– systematically densify?
– lose diameter with increasing size?

• Study processes, modeling options

• Modeling è Generators
• ... i.e., Graph generators

22
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Graph Generators [Leskovec 2006]

• Potential of algorithms which can create

networks, that show similar characteristics as
empirical ones (real-world networks)
– Extrapolation: Predicting future
state/characteristics/properties
– Sampling: Random sample + generalization
– Anomaly detection: Identify deviations from the
“normal“ behavior of the network
– Simulation: Analysis of "what if" scenarios, e.g.,
removal of hubs, network resilience, etc.

23
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Simple Graph Generators

[Leskovec 2006]
• Simple Algorithm:
– Choose a set of nodes and connect every pair with
probability p
è Erdos-Renyi/Poisson random graph
• Erdos-Renyi/Poisson Modell:
– G(n,m): Set of all graphs with n nodes and m edges;
every graph occurs with the same probability
– Example: G(3,2) set of all graphs with 3 nodes and 2
edges; every graph – probability 1/3
è but: does not reflect reality

24
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

25
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Random Graphs
[Faloutsos/Leskovec ECML/PKDD 2007]
• Advantages:
– Simple model
– Phase transitions
(Giant component, average degree > 1)
– Giant component
• Disadvantages:
– Degree distribution
– No community structure (clusters)
– No node correlation (structure)
• Extensions:
– Configuration model
• Random graph with arbitrary sequence of node degrees

26
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Configuration Model

27
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Configuration Model

28
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Configuration Model - Example

29
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Configuration Model – Example 2

30
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Configuration Model
• Able to generate networks with power-law
degree distribution
– n.b.: Input is arbitrary degree sequence
• Thus, does not explain the "natural" evolution
of power-law networks
• Does not explain network growth/evolution

31
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Reprise: Generating Scale-free Networks

[Barabasi & Albert 1999]
"Preferential Attachment": High probability, that a new node
connects to an existing node having a high degree

32
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Edge copying model

[Faloutsos/Leskovec ECML/PKDD 2007]

33
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Forest Fire Model

[Faloutsos/Leskovec ECML/PKDD 2007]

34
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Fore fire - igraph description

https://igraph.org/r/doc/sample_forestfire.html

35
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Forest-Fire Modell
[Faloutsos/Leskovec ECML/PKDD 2007]

36
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Forest-Fire Modell
[Faloutsos/Leskovec ECML/PKDD 2007]

37
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Overview - Network Generators

38
Machine Learning for Complex Data, Prof. Dr. M. Atzmüller, Osnabrück University

Summary

What Did We Learn?

• Basics of graph generation

• Important network/graph properties

• ... of real networks

• Traditional graph generators (Generative Models)

• Random graphs ... forest fire: here, different capabilities w.r.t.

the possible structures to generate

39