Community Structure

Modularity Maximiztion, Spectral Modularity

MA 653: Network Science

Instructor: Ashok Singh Sairam
[email protected]
Applications
• So far, we have focused on measurements and models of network
• provide a solid foundation for understanding
• Apply what we have learned to gain insight into function of
networked systems
• Community detection
• Studies of network failure and resilience
• Epidemic and other spreading process

MA 653: Network Science 2

Ex: Network of coauthorships in a department

MA 653: Network Science 3

Figure 11.1: Network of coauthorships in a university department. The vertices in this
network represent scientists in a university department, and edges links pairs of scientists who
Dividing network into groups
• Graph partitioning and community detection refer to the division of
the vertices in a graph based on the connection patterns of the edges
• The most common approach is to divide vertices in groups, such that the
majority of edges connect members from the same group
• Discovering communities in a network can be useful for revealing
structure and organization in a network beyond the scale of a single
vertex
• Allows us to study large networks and break them up into smaller subsets that
can be studied separately, for example for visulaizaton
• Simplified representation of large networks

MA 653: Network Science 4

Number of groups
• Depending on whether we have specified the number of groups we
want to divide our network in we have two different problems:
• Graph partitioning: Minimum interaction between partitions
• Community detection: Discover densely connected regions

• Even though the general problem is the same, there are different
algorithms that deal with each one of the problems

5
Graph partitioning
• In graph partitioning the number of groups k is pre-defined
• In particular, we want to divide the network vertices into k non-
overlapping groups of given sizes such that the number of edges
across groups are minimized
• Sizes can also be provided in approximation only
• E.g., within specific range
• E.g., divide the network nodes into two groups of equal size, such that the
number of edges between them is minimized
• Graph partitioning is useful in parallel processing of numerical
solutions of network processes

6
Community detection
• In community detection neither the number of groups nor their sizes
are pre-defined
• The goal is to find the natural fault lines along which the network
separates
• Few edges between groups and many edges within groups
• Community detection is not as well-posed of a problem as graph
partitioning
• What do we mean “many” and “few”?
• Many different objectives  different algorithms

7
Problem formulation
• Formulate the problem of community detection as a maximization
prob.
• Consider a simple, undirected network
• Divide the network into every possible division
• Assign high score if division is “good” (division has many edges within communities),
• Assign low score if division is “bad”.
• Search through the divisions to find the one with the highest score

MA 653: Network Science 8

A brute force approach
• Consider a network with n nodes and the problem of dividing the
network into 2 groups
• There are 2n ways to divide n nodes into two groups
• The number of possible divisions will quickly grow with n
• For n=100, 2100 ≈ 1030
• The success of the approach relies on how to satisfactorily define
score
• We can use the measure of assortative mixing known as modularity
• Look for the division that has the highest modularity score

MA 653: Network Science 9

Assortative Mixing and Community detection
• Nodes with similar characteristics tend to be connected by edges
• Modularity is a measure of assortative mixing
• It has high value when connections are b/w nodes of same type
• We can think of good divisions as ones with high modularity score
• Thus one way to detect community is to find divisions with highest
modularity score

MA 653: Network Science 10

Form of the Modularity function
• The modularity of a division of network is
1 𝑘𝑖 𝑘𝑗 1
𝑄= ෍ 𝐴𝑖𝑗 − 𝛿𝑔𝑖 𝑔𝑗 = ෍ 𝐵𝑖𝑗 𝛿𝑔𝑖𝑔𝑗
2𝑚 2𝑚 2𝑚
𝑖𝑗 𝑖𝑗
• Note that Bij has the property that its sum wrt either of its indices is 0
𝑘𝑗 𝑘𝑗
෍ 𝐵𝑖𝑗 = ෍ 𝐴𝑖𝑗 − ෍ 𝑘𝑖 = 𝑘𝑗 − 2𝑚 = 0
2𝑚 𝑖 2𝑚
𝑖 𝑖
𝑘𝑖 𝑘𝑖
෍ 𝐵𝑖𝑗 = ෍ 𝐴𝑖𝑗 − ෍ 𝑘𝑗 = 𝑘𝑖 − 2𝑚 = 0
2𝑚 𝑗 2𝑚
𝑗 𝑗

MA 653: Network Science 11

• The definition of modularity is general in the sense that it allows the division of a
network into any number of groups, but let us consider the problem of just two
groups
+1, if node 𝑖 belongs to group 1
𝑠𝑖 = ቊ
−1, if node 𝑖 belongs to group 2
1
• The quantity (𝑠𝑖 𝑠𝑗 + 1) is 1 if i and j are in the same group and 0 otherwise
2
1
𝛿𝑔𝑖 𝑔𝑗 = (𝑠𝑖 𝑠𝑗 + 1)
2
1 1 1
𝑄= σ𝑖𝑗 𝐵𝑖𝑗 𝛿𝑔𝑖 𝑔𝑗 = σ𝑖𝑗 𝐵𝑖𝑗 (𝑠𝑖 𝑠𝑗 + 1) = σ𝑖𝑗 𝐵𝑖𝑗 𝑠𝑖 𝑠𝑗
2𝑚 4𝑚 4𝑚
• In the second term, we have used the property that sum of 𝐵𝑖𝑗 with either indices is equal to
0

MA 653: Network Science 12

Problem formulation: Modularity maximization
1
𝑄= ෍ 𝐵𝑖𝑗 𝑠𝑖 𝑠𝑗
4𝑚
𝑖𝑗
• The leading constant factor 1/4m is not important, it has no effect on the
maximization problem
• The quantity Bij is fixed by the structure of the network
• The quantity si denote a particular division of the network into two parts
• So the modularity maximization problem can be rephrased as follows
• Given a particular set of values, Bij , i.e. a particular network, find the
quantities 𝑠𝑖 = ±1 that maximizes the above equation

MA 653: Network Science 13

Approaches to Modularity Maximization
• This problem falls in the generic class of discrete optimization problems, we are
maximizing a known function of a set of discrete-valued variables
• There are general computational methods available for finding solution to such
problems
• This solutions give approximate results. The approximations are acceptable as
long as the methods basically finds the right division
• However, if the modularity value is high but the division is not optimal, the
solution is not acceptable.
• Both the solutions are possible
• The approximate detection method works well where the community
structure leaps out
MA 653: Network Science 14
Simple Modularity Maximization
• We start with a random division of the network into equally sized groups
• At each step, go through all vertices and compute the change in the
modularity if this vertex was placed in the other group (i.e. momentarily
change the sign of si)
• Among all vertices examined, swap groups to the one that has the best effect
on modularity – i.e., largest increase or smallest decrease
• We repeat the above process by considering only the vertices that we have
not already swapped
• A round ends when all the nodes have been moved exactly once.
1
𝑄= ෍ 𝐵𝑖𝑗 𝑠𝑖 𝑠𝑗
4𝑚
𝑖𝑗

MA 653: Network Science 15

• After we go through all vertices, we go through all the states of the
network and keep the one with the higher modularity
• We repeat the whole process starting with the best network that we
identified in the previous round until the modularity no longer improves

• What is the time complexity of this algorithm ?

MA 653: Network Science 16

Simple Modularity Maximization: Time complexity
1
• Consider the equation 𝑄 = σ𝑖𝑗 𝐵𝑖𝑗 𝑠𝑖 𝑠𝑗
4𝑚
• Suppose we are moving node v, so only the variable sv changes
• Only the terms in the sum that contain sv changes
𝐵𝑣𝑣 𝑠𝑣2 + 𝑠𝑣 ෍ 𝐵𝑖𝑣 𝑠𝑖 + 𝑠𝑣 ෍ 𝐵𝑣𝑗 𝑠𝑗
𝑖(≠𝑣) 𝑗(≠𝑣)

• The first does not change when we change the sign of sv and we can ignore it
• Since 𝐵𝑖𝑗 = 𝐵𝑗𝑖 , the other two terms are equal and their sum is 2𝑠𝑣 σ𝑖(≠𝑣) 𝐵𝑖𝑣 𝑠𝑖
• Change in modularity, when we flip 𝑠𝑣 to − 𝑠𝑣
1 𝑠𝑣
• Δ𝑄 = −2𝑠𝑣 σ𝑖(≠𝑣) 𝐵𝑖𝑣 𝑠𝑖 − 2𝑠𝑣 σ𝑖(≠𝑣) 𝐵𝑖𝑣 𝑠𝑖 = − σ𝑖(≠𝑣) 𝐵𝑖𝑣 𝑠𝑖
4𝑚 𝑚

MA 653: Network Science 17

 𝑠𝑣
• In the equation Δ𝑄 = − σ 𝐵 𝑠, time consuming part of the evaluation is the sum
𝑚 𝑖(≠𝑣) 𝑖𝑣 𝑖
𝑘𝑖 𝑘𝑗
• Note that 𝐵𝑖𝑗 = 𝐴𝑖𝑗 − , thus
2𝑚
𝑘𝑣 𝑘𝑣 𝑘𝑣2 𝑠𝑣
• σ𝑖(≠𝑣) 𝐵𝑖𝑣 𝑠𝑖 = σ𝑖(≠𝑣) 𝐴𝑖𝑣 𝑠𝑖 − σ 𝑘𝑠 = σ𝑖(≠𝑣) 𝐴𝑖𝑣 𝑠𝑖 − σ 𝑘𝑠 +
2𝑚 𝑖(≠𝑣) 𝑖 𝑖 2𝑚 𝑖 𝑖 𝑖 2𝑚
• The first term is the sum of the 𝑠𝑖 for all nodes that are neighbours of sv : O(m/n) i.e O(m)
• The second term involves summing over all the nodes: O(n)
• It does not depend on v
• The third term will take constant time
• Hence the total running time is: O(n(m+n)) for one round
• The number of rounds required to converge is not known, but it is generally less (5 to 10) for
small to moderate networks

MA 653: Network Science 18

Spectral Modularity Maximization
• The simple modularity maximization works well but it is not the
fastest approach
• We will look at a more advanced “spectral” algorithm
• Spectral graph theory uses the eigenvalues and eigenvectors of graph
matrices, such as the adjacency matrix
• Spectral here means the connection between the graph and spectral
properties of the associated matrices

MA 653: Network Science 19

Spectral Modularity Maximization
• Let us again consider the division of a network into just two parts
+1, if node 𝑖 belongs to group 1
𝑠𝑖 = ቊ
−1, if node 𝑖 belongs to group 2
1
• Then the modularity 𝑄 = σ 𝐵 𝑠𝑠 can be written in vector form as
4𝑚 𝑖𝑗 𝑖𝑗 𝑖 𝑗
1 𝑇
𝑄= 𝐬 𝐁𝐬
4𝑚
• where s is the n-dimensional vector with elements si
• B is the n x n matrix with elements Bij, called the modularity matrix
• We want to find the value of s that maximizes the above equation for a given B
• The reason why it is hard to solve is the fact that the elements si cannot take any real
value but only specific integer values (+/- 1)
• Integer optimization problems can be solved approximately using the relaxation method

MA 653: Network Science 20

• In the original integer problem the vector s would point to one of the
2n corners of an n-dimensional hypercube centered at the origins
• The length of this vector is n

• We relax the above constraint by allowing the vector s to point to any

direction on a hypersphere circumscribing the original hypercube as
long as its length is still n

å i =n
s 2

MA 653: Network Science 21

• We solve the problem using the relaxation method

• Or in matrix notation 𝐁𝐬 = 𝛽𝐬
• The optimal s is one of the eigenvectors of the modularity matrix and 𝛽 is the
corresponding eigenvalue
1 𝑇
n
𝑄= 𝛽𝐬 𝐬 = 𝛽 , where s 𝑇 𝑠 = 𝑛
4𝑚 4𝑚
• Since our goal is to make the modularity as large as possible, we want the
eigenvalue 𝛽 to be as large as possible.
MA 653: Network Science 22
• For removing the relaxation (i.e. s should take values ±1)
• We want to minimize the angle between s and the leading eigenvector
(denoted by u)
• Equivalently, we want to maximize the inner product : 𝑠 𝑇 𝑢 = σ𝑖 𝑠𝑖 𝑢𝑖
• The maximum is achieved when 𝑠𝑖 𝑢𝑖 is positive for all i
• This occurs when si has the same sign as ui for all i
• Thus we approximate the vector s as follows
+1, if 𝑢𝑖 > 0
• 𝑠𝑖 = ቊ
−1, if 𝑢𝑖 < 0

MA 653: Network Science 23

Spectral Modularity Max: conclusion
• So finally we follow the simple algorithm
• Calculate the eigenvector of the modularity matrix corresponding to the
largest (most +ve) eigenvalue
• Assign nodes to communities according to the signs of the eigenvector
• The leading eigenvector can be computed using the power method or
the Lanczos algorithm
• Requires repeated multiplication: O(m+n)
• Time required for entire algorithm: O((m+n)log n) or O(n log n) for a
sparse matrix m∝ n

MA 653: Network Science 24