SGL: Stata graph library for network analysis

Hirotaka Miura
Federal Reserve Bank of San Francisco

Stata Conference Chicago 2011

The views presented here are my own and do not necessarily represent the views of the Federal Reserve System or any person aliated with it.

Stata network analysis Introduction

What is network analysis?

Network analysis is an application of network theory, which is a subeld of graph theory, and is concerned with analyzing relational data. Some questions network analysis tries to address is how important, or how central are the actors in the network and how concentrated is the network. Example usages of network analysis include:
Determining the importance of a web page using Googles PageRank. Examining communication networks in intelligence and computer security. Solving transportation problems that involve ow of trac or commodities. Addressing the too-connected-to-fail problem in nancial networks. Analyzing social relationships between individuals in social network analysis.
2 / 41

Stata network analysis Introduction Outline

Outline

Modeling relational data Matrix representations Centrality measures Clustering coecient Stata implementation Conclusion

3 / 41

Stata network analysis Modeling relational data Denitions

Graph model
A graph model representing a network G = (V , E ) consists of a set of vertices V and a set of edges E .
|V | equals the number of vertices. |E | equals the number of edges.

An edge is dened as a link between two vertices i and j , not necessarily distinct, that has vertex i on one end and vertex j on the other.
An edge may be directed or undirected and may also be weighted with diering edge values or have all equal edge values of one in which case the network is said to be unweighted.

4 / 41

Stata network analysis Modeling relational data Denitions

Special cases

Special types of vertices and edges exist for which standard graph algorithms are not designed to handle or there simply does not exist routines to accommodate such types. Thus the following types of vertices and edges are currently excluded from analysis:
Isolated vertex - a vertex that is not attached to any edges. Parallel edges - two or more edges that connect the same pair of vertices. Self-loop - an edge connecting vertex i to itself. Zero or negative weighted edge.

5 / 41

Stata network analysis Modeling relational data Storing data

Storing relational data

A variety of storage types are available for capturing relational data:

Adjacency matrix Adjacency list
Core SGL algorithms use this structure.

Edge list
Most suited for storing relational data in Stata, as it allows the use of options such as if exp and in range.

Plus others such as the Compressed Sparse Row format for ecient storage and access.

6 / 41

Stata network analysis Modeling relational data Storing data

Example storage types

4 2 1

Undirected unweighted network. Drawn using NETPLOT (Corten, 2011).

Adjacency matrix 0 1 0 0 1 0 1 1 0 1 0 0 0 1 0 0

Adjacency list
Vertex 1 2 3 4 Neighbor(s) 2 1 3 4 2 2

Edge list 1 2 2 3 4 2
7 / 41

Stata network analysis Modeling relational data Using joinby

Creating an edge list using joinby (1)

We illustrate a method of creating an edge list using the joinby command and example datasets used in [D] Data-Management Reference Manual, child.dta and parent.dta.
Directed edges can represent parent vertices providing care to child vertices.
. use child, clear (Data on Children) . list family~d 1. 2. 3. 4. 5. 1025 1025 1025 1026 1027 child_id 3 1 4 2 5 x1 11 12 10 13 15 x2 320 300 275 280 210 . use parent, clear (Data on Parents) . list family~d 1. 2. 3. 4. 5. 6. 1030 1025 1025 1026 1026 1030 parent~d 10 11 12 13 14 15 x1 39 20 27 30 26 32 x3 600 643 721 760 668 684
8 / 41

Stata network analysis Modeling relational data Using joinby

Creating an edge list using joinby (2)

1 1
. . . sort family_id joinby family_id using child list parent_id child_id parent~d 1. 2. 3. 4. 5. 6. 7. 8. 12 12 12 11 11 11 14 13 child_id 4 1 3 4 3 1 2 2

3 3 12 12 4 4 11 11 2 2

13 13

14 14

Drawn using NETPLOT (Corten, 2011).

9 / 41

Stata network analysis Matrix representations

Matrix representations

10 / 41

Stata network analysis Matrix representations Adjacency matrix

Adjacency matrix
Adjacency matrix A for unweighted networks is dened as a |V | |V | matrix with Aij entries being equal to one if an edge connects vertices i and j and zero otherwise. Aii entries are set to zero. Matrix A is symmetric if the network is undirected. For directed networks, rows of matrix A represent outgoing edges and columns represent incoming edges.
The convention of denoting Xij entries as an edge from i to j is adopted for all matrices.

For weighted networks, Aij entries are equal to the weight of the edge connecting vertices i and j .

11 / 41

Stata network analysis Matrix representations Distance matrix

Distance matrix
Distance matrix D is dened as a |V | |V | matrix with Dij entries being equal to the length of the shortest path between vertices i and j.
A path is dened as a way of reaching vertex j starting from vertex i using a combination of edges that do not go through a particular vertex more than once.

If no path connects vertices i and j , Dij is set to missing.

Signies what is sometimes referred to as an innite path.

Dii is set to zero. For undirected networks, matrix D is symmetric.

12 / 41

Stata network analysis Matrix representations Path matrix

Path matrix

Path matrix P is dened as a |V | |V | matrix with Pij entries being equal to the number of shortest paths between vertices i and j . If no paths exist between vertices i and j , Pij is set to zero. Pii is set to one. P matrix is symmetric for undirected networks.

13 / 41

Stata network analysis Matrix representations Example

Example matrices
4 2 1

Undirected unweighted network. Drawn using NETPLOT (Corten, 2011).

Adjacency matrix 0 1 0 0 1 0 1 1 0 1 0 0 0 1 0 0

Distance matrix 0 1 2 2 1 0 1 1 2 1 0 2 2 1 2 0

Path matrix 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
14 / 41

Stata network analysis Centrality measures

Centrality measures

15 / 41

Stata network analysis Centrality measures Degree centrality

Degree centrality (1)

Undirected network

Degree centrality measures the importance of a vertex by the number of connection the vertex has if the network is unweighted, and by the aggregate of the weights of edges connected to the vertex if the network is weighted (Freeman, 1978). For an undirected network, degree centrality for vertex i is dened as 1 |V | 1 Aij
j (=i )

(1)

where the leading divisor is adjusted for the exclusion of the j = i term.
16 / 41

Stata network analysis Centrality measures Degree centrality

Degree centrality (2)

Directed network

Directed networks may entail vertices having dierent number of incoming and outgoing edges, and thus we have out-degree and in-degree centrality. Out-degree centrality for vertex i is dened similarly to equation (1). For in-degree, we simply transpose the adjacency matrix: 1 |V | 1 Aij .
j (=i )

(2)

17 / 41

Stata network analysis Centrality measures Degree centrality

Example
Undirected unweighted network
G (0.33) A (0.33)

Centrality comparison
Centrality A B F G Degree Closeness Betweenness Eigenvector Katz-Bonacich 0.33 Vertex

E (0.50)

D (0.33)

C (0.50)

C E 0.50

D 0.33

F (0.33)

B (0.33)

Degree centrality in parentheses. Figure from Jackson (2008). Drawn using NETPLOT (Corten, 2011).

18 / 41

Stata network analysis Centrality measures Closeness centrality

Closeness centrality
Closeness centrality provides higher centrality scores to vertices that are situated closer to members of their component, or the set of reachable vertices, by taking the inverse of the average shortest paths as a measure of proximity (Freeman, 1978). That is, closeness centrality for vertex i is dened as (|V | 1) , j (=i ) Dij (3)

which reects how vertices with smaller average shortest path lengths receive higher centrality scores than those that are situated farther away from members of their component.
19 / 41

Stata network analysis Centrality measures Closeness centrality

Example
Undirected unweighted network
G (0.40) A (0.40)

Centrality comparison
Centrality A B F G Degree Closeness Betweenness Eigenvector Katz-Bonacich 0.33 0.40 Vertex

E (0.55)

D (0.60)

C (0.55)

C E 0.50 0.55

D 0.33 0.60

F (0.40)

B (0.40)

Closeness centrality in parentheses. Figure from Jackson (2008). Drawn using NETPLOT (Corten, 2011). 20 / 41

Stata network analysis Centrality measures Betweenness centrality

Betweenness centrality
Betweenness centrality bestows larger centrality scores on vertices that lie on a higher proportion of shortest paths linking vertices other than itself. Let Pij denote the number of shortest paths from vertex i to j . Let Pij (k ) denote the number of shortest paths from vertex i to j that vertex k lies on. Then following Anthonisse (1971) and Freeman (1977), betweenness centrality measure for vertex k is dened as Pij (k ) . Pij (4)

ij :i =j ,k ij

To normalize (4), divide by (|V | 1)(|V | 2), the maximum number of paths a given vertex could lie on between pairs of other vertices.
21 / 41

Stata network analysis Centrality measures Betweenness centrality

Example
Undirected unweighted network
G (0.00) A (0.00)

Centrality comparison
Centrality A B F G Degree Closeness Betweenness Eigenvector Katz-Bonacich 0.33 0.40 0.00 Vertex

E (0.53)

D (0.60)

C (0.53)

C E 0.50 0.55 0.53

D 0.33 0.60 0.60

F (0.00)

B (0.00)

Normalized betweenness centrality in parentheses. Figure from Jackson (2008). Drawn using NETPLOT (Corten, 2011). 22 / 41

Stata network analysis Centrality measures Eigenvector centrality

Eigenvector centrality (1)

Eigenvector centrality can provide an indication on how important a vertex is by having the property of being large if a vertex has many neighbors, important neighbors, or both (Bonacich, 1972). For an undirected network with adjacency matrix A, centrality of vertex i , xi , can be expressed as xi = 1
j

Aij xj

(5)

which can be rewritten as x = Ax. The convention is to use the eigenvector corresponding to the dominant eigenvalue of A.
23 / 41

(6)

Stata network analysis Centrality measures Eigenvector centrality

Eigenvector centrality (2)

For directed networks, the general concern is in obtaining a centrality measure based on how often a vertex is being pointed to and the importance of neighbors associated with the incoming edges. Thus with a slight modication to equation (6), eigenvector centrality is redened as x = A x (7) where A is the transposed adjacency matrix. There are several shortcomings to the eigenvector centrality:
A vertex with no incoming edges will always have centrality of zero. Vertices with neighbors that all have zero incoming edges will also have zero centrality since the sum in equation (5), j Aij xj , will not have any terms.

The Katz-Bonacich centrality, a variation of the eigenvector centrality, seeks to address these issues.
24 / 41

Stata network analysis Centrality measures Eigenvector centrality

Example
Undirected unweighted network
G (0.33) A (0.33)

Centrality comparison
Centrality A B F G Degree Closeness Betweenness Eigenvector Katz-Bonacich 0.33 0.40 0.00 0.33 Vertex

E (0.45)

D (0.38)

C (0.45)

C E 0.50 0.55 0.53 0.45

D 0.33 0.60 0.60 0.38

F (0.33)

B (0.33)

Eigenvector centrality in parentheses. Figure from Jackson (2008). Drawn using NETPLOT (Corten, 2011). 25 / 41

Stata network analysis Centrality measures Katz-Bonacich centrality

Katz-Bonacich centrality
The additional inclusion of a free parameter (also referred to as a decay factor) and a vector of exogenous factors into equation (7):
Avoids the exclusion of vertices with zero incoming edges. Allows connection values to decay over distance.

Attributed to Katz (1953), Bonacich (1987), and Bonacich and Lloyd (2001). Centrality measure is dened as a solution to the equation x = A x + (8) where is the free parameter and is the vector of exogenous factors which can vary or be constant across vertices. For the centrality measure to converge properly, absolute value of must be less than the absolute value of the inverse of the dominant eigenvalue of A.
A positive allows vertices with important neighbors to have higher status while a negative value reduces the status.
26 / 41

Stata network analysis Centrality measures Katz-Bonacich centrality

Example
Undirected unweighted network
G (3.99) A (3.99)

Centrality comparison
Centrality A B F G Vertex

C E 0.50 0.55 0.53 0.45 5.06

D 0.33 0.60 0.60 0.38 4.34

E (5.06)

D (4.34)

C (5.06)

Degree Closeness Betweenness Eigenvector Katz-Bonacich*

B (3.99)

0.33 0.40 0.00 0.33 3.99

F (3.99)

* Maximum = 0.43 (0.33 used). Exogenous factors set to one for all vertices. Katz-Bonacich centrality in parentheses. Figure from Jackson (2008). Drawn using NETPLOT (Corten, 2011).
27 / 41

Stata network analysis Clustering coecient

Clustering coecient

28 / 41

Stata network analysis Clustering coecient Introduction

Clustering coecient
Clustering coecient is one way of gauging how tightly connected a network is. The general idea is to consider transitive relations:
If vertex j is connected to vertex i , and i is connected to k , then j is also connected to k .

Global clustering coecients provide indication on the degree of concentration of the entire network and consists of overall and average clustering coecients.
Overall clustering coecient is equal to all observed transitive relations divided by all possible transitive relations in the network. Average clustering coecient involves applying the denition of overall clustering coecient at the vertex level, then averaging across all the vertices.
29 / 41

Stata network analysis Clustering coecient Overall

Overall clustering coecient

For an undirected unweighted adjacency matrix A, overall clustering coecient is dened as
Aji Aik Ajk c (A) =
o i ;j =i ;k =j ;k =i

Aji Aik
i ;j =i ;k =j ;k =i

(9)

where the numerator represents the sum over i of all closed triplets in which transitivity holds, and the denominator represents the sum over i of all possible triplets.

30 / 41

Stata network analysis Clustering coecient Local and average

Local and average clustering coecient

With a slight modication in notation, local clustering coecient for vertex i is dened as
Aji Aik Ajk ci (A) =
j =i ;k =j ;k =i

Aji Aik
j =i ;k =j ;k =i

(10)

which leads to the average clustering coecient:

c a (A) = 1 |V | ci (A).
i

(11)

By convention, ci (A) = 0 if vertex i has zero or only one link.

31 / 41

Stata network analysis Clustering coecient Generalized methods

Generalized clustering coecient

Building upon the works of Barrat et al. (2004), Opsahl and Panzarasa (2009) propose generalized methods. Clustering coecients for vertex i based on weighted adjacency matrix W and corresponding unweighted adjacency matrix A are calculated as
Ajk ci (W) =
j =i ;k =j ;k =i

j =i ;k =j ;k =i

(12)

where equals (Wji + Wik )/2 for arithmetic mean, Wji Wik for geometric mean, max(Wji , Wik ) for maximum, and min(Wji , Wik ) for minimum. For unweighted networks, W = A and the four types of clustering coecients are all equal.
32 / 41

Stata network analysis Clustering coecient Generalized methods

Example
Undirected unweighted network
G (1.00) A (1.00)

E (0.33)

D (0.00)

C (0.33)

Average clustering coecient: 0.67 Overall clustering coecient: 0.55

F (1.00)

B (1.00)

Local clustering coecients in parentheses. Figure from Jackson (2008). Drawn using NETPLOT (Corten, 2011).

33 / 41

Stata network analysis Stata implementation

Stata implementation

34 / 41

Stata network analysis Stata implementation Using network and netsummarize

network and netsummarize commands

. use florentine_marriages, clear

We demonstrate the use of network and netsummarize commands on a dataset of 15th-century Florentine marriages from Padgett and Ansell (1993) to compute betweenness and eigenvector centrality measures.
network generates vectors of betweenness and eigenvector centralities. netsummarize merges vectors to Stata dataset.

(15th century Florentine marriages 15th-century Florentine > (Padgett and Ansell 1993)) marriages Edge list stored as Stata dataset . list v1 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. Peruzzi Peruzzi Peruzzi Castellan Castellan Strozzi Strozzi Bischeri Barbadori Ridolfi Ridolfi Medici Medici Medici Medici Tornabuon Guadagni Guadagni Albizzi Salviati v2 Castellan Strozzi Bischeri Strozzi Barbadori Ridolfi Bischeri Guadagni Medici Medici Tornabuon Tornabuon Albizzi Salviati Acciaiuol Guadagni Lambertes Albizzi Ginori Pazzi

35 / 41

Stata network analysis Stata implementation Using network and netsummarize

Computing network centrality measures

. // Generate betweenness centrality. . network v1 v2, measure(betweenness) name(b,replace) Breadth-first search algorithm (15 vertices) ----+--- 1 ---+--- 2 ---+--- 3 ---+--- 4 ---+--- 5 ............... Breadth-first search algorithm completed Betweenness centrality calculation (15 vertices) ----+--- 1 ---+--- 2 ---+--- 3 ---+--- 4 ---+--- 5 ............... Betweenness centrality calculation completed matrix b saved in Mata . netsummarize b/((rows(b)-1)*(rows(b)-2)), > generate(betweenness) statistic(rowsum) . // Generate eigenvector centrality. . network v1 v2, measure(eigenvector) name(e,replace) matrix e saved in Mata . netsummarize e, generate(eigenvector) statistic(rowsum)
36 / 41

Stata network analysis Stata implementation Using network and netsummarize

Data description (1) . describe, full

Contains data from florentine_marriages.dta obs: 20 > (Padgett and Ansell 1993) vars: 10 size: 1,160 (99.9% of memory free) variable name storage type display format %9s %9s %9.0g %9.0g %9.0g %9.0g rowsum of Mata matrix b/((rows(b)-1)*(rows(b)-2)) rowsum of Mata matrix b/((rows(b)-1)*(rows(b)-2)) rowsum of Mata matrix e rowsum of Mata matrix e value label 15th century Florentine marriages 10 Dec 2010 09:45

variable label

v1 str9 v2 str9 betweenness_source float betweenness_target float eigenvector_source float eigenvector_target float Sorted by: Note:

dataset has changed since last saved

37 / 41

Stata network analysis Stata implementation Using network and netsummarize

Data description (2)

. list v1 v2 betweenness_source betweenness_target eigenvector_source eigenve > ctor_target v1 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. Peruzzi Peruzzi Peruzzi Castellan Castellan Strozzi Strozzi Bischeri Barbadori Ridolfi Ridolfi Medici Medici Medici Medici Tornabuon Guadagni Guadagni Albizzi Salviati v2 Castellan Strozzi Bischeri Strozzi Barbadori Ridolfi Bischeri Guadagni Medici Medici Tornabuon Tornabuon Albizzi Salviati Acciaiuol Guadagni Lambertes Albizzi Ginori Pazzi betwee~e .021978 .021978 .021978 .0549451 .0549451 .1025641 .1025641 .1043956 .0934066 .1135531 .1135531 .521978 .521978 .521978 .521978 .0915751 .2545788 .2545788 .2124542 .1428571 betwee~t .0549451 .1025641 .1043956 .1025641 .0934066 .1135531 .1043956 .2545788 .521978 .521978 .0915751 .0915751 .2124542 .1428571 0 .2545788 0 .2124542 0 0 eigenv~e .2757304 .2757304 .2757304 .2590262 .2590262 .3559805 .3559805 .2828001 .2117053 .3415526 .3415526 .4303081 .4303081 .4303081 .4303081 .3258423 .2891156 .2891156 .2439561 .1459172 eigenv~t .2590262 .3559805 .2828001 .3559805 .2117053 .3415526 .2828001 .2891156 .4303081 .4303081 .3258423 .3258423 .2439561 .1459172 .1321543 .2891156 .0887919 .2439561 .0749227 .0448134

38 / 41

Stata network analysis Stata implementation Using network and netsummarize

Network visualization
Betweenness centrality
Ginori (0.00)

Eigenvector centrality
Ginori (0.07)

Lambertes (0.00) Albizzi (0.21) Guadagni (0.25) Acciaiuol (0.00)

Lambertes (0.09) Albizzi (0.24) Guadagni (0.29) Acciaiuol (0.13)

Tornabuon (0.09) Medici (0.52) Bischeri (0.10) Ridolfi (0.11) Salviati (0.14) Pazzi (0.00) Bischeri (0.28)

Tornabuon (0.33) Medici (0.43) Ridolfi (0.34) Salviati (0.15) Pazzi (0.04)

Strozzi (0.10) Barbadori (0.09) Peruzzi (0.02) Castellan (0.05)

Strozzi (0.36) Barbadori (0.21) Peruzzi (0.28) Castellan (0.26)

Centrality scores in parentheses. Drawn using NETPLOT (Corten, 2011).

39 / 41

Stata network analysis Conclusion

Conclusion
Dierent types of matrices, centrality measures, and clustering coecients can be generated from information retrieved from relational data. The command network provides access to SGL functions that generate network measures based on edge list stored in Stata. The postcomputation command netsummarize allows the user to generate standard and customized network measures which are merged into Stata dataset. Future developments include:
More ecient algorithms. Designing functions for additional network measures. Optimizing SGL in Mata.
40 / 41

Stata network analysis References

References I
Anthonisse, J. M. (1971): The rush in a directed graph, Discussion paper, Stichting Mathematisch Centrum, Amsterdam, Technical Report BN 9/71. Barrat, A., M. Barth elemy, R. Pastor-Satorras, A. Vespignani (2004): The architecture of complex weighted networks, Proceedings of the National Academy of Sciences, 101(11), 37473752. Bonacich, P. (1972): Factoring and weighting approaches to status scores and clique identication, Journal of Mathematical Sociology 2, pp. 113120. (1987): Power and Centrality: A Family of Measures, The American Journal of Sociology, 92(5), 11701182. Bonacich, P., P. Lloyd (2001): Eigenvector-like measures of centrality for asymmetric relations, Social Networks, 23(3), 191201. Corten, R. (2011): Visualization of social networks in Stata using multidimensional scaling, Stata Journal, 11(1), 5263. Freeman, L. C. (1977): A set of measures of centrality based on betweenness, Sociometry, 40(1), 3541. (1978): Centrality in social networks: Conceptual clarication, Social Networks, (1), 215239. Jackson, M. O. (2008): Social and Economic Networks. Princeton University Press. Katz, L. (1953): A New Status Index Derived from Sociometric Analysis, Psychometrika, 18, 3943. Opsahl, T., Padgett, J. F., C. K. Ansell (1993): Robust action and the rise of the Medici, 1400-1434, The American Journal of Sociology, 98(6), 12591319.

and

and

and P. Panzarasa (2009): Clustering in weighted networks, Social Networks, 31(2), 155163. and

41 / 41