Chapter 7

Interconnection Networks (MIMD)

Dr. Fatemah Al- Assfor

Interconnection Networks (IN) for Parallel Computer
Interconnection Networks (IN) is a type of MIMD. Interconnection networks carry data between
processors and to memory that are called nodes. They are made of switches and links (wires, fiber,…).

- IN can be classified to static or dynamic networks

- Connections in a static network are fixed links, while connections in a dynamic network
are established on the fly as needed.
1- Static networks (direct networks)
Consists of point-to-point communication links among processing nodes. ex.
a. Linear array
b. Ring,
c. Tree,
d. Star,
e. Mesh,
f. Torus,
g. Hyper-cube (n-cube), ….
1- Static networks (direct networks)
Static (direct) interconnection networks are characterized by having fixed paths,
unidirectional or bidirectional, between processors.
Two types of static networks can be identified:
a) completely connected networks (CCNs)
b) limited connection networks (LCNs).

a) Completely Connected Networks (CCNs)

- In a completely connected network (CCN) each node is connected to all other nodes in
the network.
- Completely connected networks guarantee fast delivery of messages from any source
node to any destination node (only one link has to be traversed).
- Expensive in terms of the number of links needed for their construction
a) Completely Connected Networks (CCNs)

b- Limited Connection Networks (LCN)

- does not provide a direct link from every node to every other node in the network. Instead,
communications between some nodes have to be routed through other nodes in the network.

- The length of the path between nodes, measured in terms of the number of links that have to
be traversed, is expected to be longer compared to the case of CCNs
Interconnection Networks (IN) for Parallel Computer
2- Dynamic networks
–Built using switches (switching element) and links
–Communication links are connected to one another dynamically by the switches in addition to
links to establish paths among processing nodes and memory banks.
–They are also referred to as indirect networks.

e.g.
a. Digital bus,
b. Multistage net (Omega net, Baseline net and Crossbar net)
Network topologies
The topology of an interconnection network refers to the arrangement or layout of the
nodes (e.g., computers, processors, switches) and the communication links between them
in a network. It defines how different components are connected and how data is
transmitted across the network.
In simpler terms, it describes the physical or logical structure of the network.
The topology affects the efficiency, performance, fault tolerance, scalability, and overall
design of the network.
-Physical topology the actual physical layout or arrangement of the network's hardware
components (such as cables, routers, switches, and computers) and how they are
interconnected. It defines how the devices are physically connected.
- This includes the arrangement and location of network nodes and the way in which
they are connected (e.g. bus, star, ring, extended star, hierarchical, mesh,…).

-Logical topologies describes the flow of data or how devices communicate.

- Every network has both a logical and a physical topology
- Physical Topologies

6 nodes Ring
Topology

Mesh
Topology

- Logical Topology
A logical topology describes the paths taken by data packets as they travel over the network. The
logical topology of a network is governed by the protocols used by the data moving over it.
 Networks Performance
Generally, all IN performance are affected by the following five factors:
1- Functionality: how does the network function. *Best is preferred.
2- Network latency: worst case delay of the message (data) in the network. * Least is better.
3- Bandwidth: maximum data transfer rate. *Higher is better.
4- Hardware Complexity: Cost of net implementation (wires, switches,..) *lower is better.
5- Scalability: Ability to maintain good performance as it grows bigger. *higher is better.
Networks Performance
To estimate the above factors, the following parameters are defined:
a) Node Degree (d): the number of edges on a node (in & out). They must be kept constant and as
small as possible in order to reduce cost. [It affects scalability].

Example,
for linear array with and without wrap around node degree is d = 2

b) Network Diameter (D): The network diameter of an interconnection network refers to the
maximum number of communication steps required to travel between any two nodes (devices) in the
network. It is a key metric used to measure the "longest" possible path between two nodes in terms of
the number of intermediate nodes or links involved in communication. It is calculated by identifying
the The maximum of shortest path between any two nodes. [It affects latency].

Example,
for the following network, D = 3.
Networks Performance

The diameter gives you an idea of the "worst-case" scenario for how far apart two nodes
might be in terms of the number of hops. A smaller diameter indicates that data can be
transmitted between any two nodes with fewer hops, making the network more efficient.

c) The bisection width (or bandwidth) of an interconnection network is a key metric that
measures the capacity or the number of communication links that must be cut to divide the
network into two equal halves. It essentially represents the minimum number of
connections that, if removed, would cause the network to be split into two disjoint parts,
where each part cannot communicate with the other directly. [It provides good indicator of
the maximum communication allowed between two equal halves of the network and it
should be as big as possible].
- The higher the bisection width, the more robust the network is in terms of handling large
amounts of data transfer between these parts.
Networks Performance (Cont.)
Example:
For the following net, d = 4, D = 6 and B = 4

d) Total no. of Links (L): Number of links in the net. [It affects cost].
e) Network size (N): Number of nodes in the net. [It affects scalability].
Example: For the above net, L = 24 and N = 16.

f) Symmetry (S): If the network has the same looking from any node it is called symmetric. [it affects
scalability]…. S = Yes or No
Example: In static topology of interconnection networks (IN), draw the block diagram and
Calculate the network performance parameters (a- Degree, b- Diameter, c- Bandwidth, d-
Number of links, e- Network size, and f- Symmetry)

1) Ring network: The devices in ring network are connected in a circular fashion, with each
device having exactly two neighbors.
•Data travels in one direction (or sometimes both directions in a dual ring) around the
ring.
•It's efficient but can be disrupted if a single device or connection fails.
Example: Network Size [means no. of nodes ] = N = 8
–degree, d = 2
–Diameter: Uni-Directional D = N -1= 7,
–No. of links, L = N = 8
–Bisection Bandwidth, B = 2
–Symmetric, S = Yes
2) 2D Mesh network
•Every device is connected directly to every other device.
• It can be expensive and complex to set up due to the large number of connections
required
- Size [no. of nodes] = N = r*c
–degree, d = 4
–Diameter: D = (r-1)+(c-1) = (r+c-2) ,
–No. of links, L = 2*N-r-c
–Bisection Width = minimum of [r , c]
–Symmetric, S = No
(4X5) mesh network
for a (4x5) mesh network: the parameters are:
N = 20, d = 4, D = 7, L =31, B = min[4, 5 ] = 4 and S = No
3) 2D Torus:
- Size [no. of nodes] = N = r*r
–degree, d = 4
–Diameter: D = 2 𝒓/𝟐 ,
–No. of links, L = 2N
–Bisection Width, B = 2r
–Symmetric, S = Yes

Example: In static topology of interconnection networks, draw the diagram and Calculate
the network performance parameters (a- Degree, b- Diameter, c- Bandwidth, d- Number of
links, e- Network size, and f- Symmetry) for (5X5) torus network.
N = 25, d = 4, D = 2 𝟓/𝟐 = 4 , L = 2 N = 50, B = 2*5 = 10
S = Yes symmetric
Construction of Hypercubes from hypercubes of lower dimension.
4) Hypercubes (n-cube) networks

n: dimension of the network

– degree, d = n
- Size [no. of nodes] = N =𝟐𝒏
– Diameter: D = n ,
– No. of links, L = n*(N/2)
– Bisection Width = B = N/2
– Symmetric, S = Yes

Notes:
- Only 1- bit changes for any adjacent neighbors.
- Number of bits gives the number of edges.
Example: draw the 4-cube network and evaluate the six parameters:

d = n=4,
N =2^n = 2^4= 16 or 2*2^3 =2*[3-cube]
D = n= 4,
L = n *(N/2) =4*(16/2)=32,
B = N/2 = 8
S = Yes symmetric
5) Static tree networks:
A static tree network is an interconnection network that uses a tree topology to connect its nodes.
The nodes are arranged in a hierarchical structure where one node (typically called the root) acts
as the central hub, and other nodes branch out from it in a tree-like fashion. In this case, the
network is static, meaning its structure and connections do not change dynamically over time. Once
the tree is constructed, the connections remain fixed, and there is no reconfiguration based on
traffic or conditions.

Characteristics of Tree interconnection networks:

• In a tree network there is only one path between any two nodes.
•In tree network, a node has almost three links .
(i.e. a processor can interact with three processors point-to-point)
•The degree of the tree network is three d=3. Root node

•It has low diameter = 2 (k-1); where k is the depth of the tree.
•It has a poor bisection width of B=1.
5) Static tree networks (cont.)
Key Characteristics of Static Tree Networks:
• Tree Topology: The network is structured as a tree with nodes (processors, switches,
or computers) connected by links. Each node (except the root) has a parent node and
potentially several child nodes.
• Root and Branches: The central node is the root, and other nodes are its branches.
Each node has exactly one parent, and it can have multiple child nodes. The leaves of
the tree (nodes with no children) are typically the end points of communication in the
network.

To route a message in a tree network:

• The source node/processor sends the message up the tree network until it reaches
the node at the root of the smallest subtree containing both the source and
destination nodes.
• Then the message is routed down the tree towards the destination node
Example: draw the diagram of a 4- depth static tree network and determine its parameters:

a) The degree of the tree network is three d=3

b) Size N=2𝑘 − 1 = 24 − 1 = 15 node.
c) Diameter D= 2 (k-1)= 2(4-1)= 6.
d) Bisection width of B=1.
e) No. of Links : L= N-1=15-1=14
f) Symmetry S: No
Dynamic (indirect) Networking
-To build efficient multiprocessor networks (Systems), we need the dynamic facilities.
They can be obtained through using of buses, switches and multistage networks.
-These networks are called dynamic networks because they are under program control.
- In dynamic networks, we deal with cost versus performance
* Cost; includes wires, switches, connectors, etc.
* Performance; includes bandwidth, latency, data transfer rate, many communication
patterns supported.

Switches in dynamic IN
- Each input can be connected to an output in different ways.
- The size of switches can be
* (2X2)
* (4X4)
……
a) Single-Stage Networks
- A single-stage of switching elements (SEs) exists between the inputs and the outputs of
the network.
- The simplest switching element that can be used is (2X2) switching element (SE).
Figure 2.1 illustrates the four possible settings that a SE can assume.
1) Straight; the upper input is transferred to the upper output and the lower input is
transferred to the lower output.
2) Exchange (or crossbar); the upper input is transferred to the lower output and the
lower input is transferred to the upper output.
3) Upper-broadcast; the upper input is broadcast to both the upper and the lower
outputs.
4) Lower-broadcast; the lower input is broadcast to both the upper and the lower
outputs.
a) Single-Stage Networks
CONCEPT OF PERMUTATION NETWORK
In permutation interconnection networks the information exchange requires
data transfer from input set of nodes to output set of nodes and possible
connections between edges are established by applying various
permutations in available links.
1)Shuffle-Exchange

2) Butterfly permutation
a) Single-Stage Networks
To establish communication between a given input (source) to a given output (destination), data should
be circulated a number of times around the network.
•A well-known connection pattern for interconnecting the inputs and the outputs of a single-stage
network is the Shuffle-Exchange.

In Shuffle-Exchange, two operations are used:

1.Shuffle:

• Node i connected to the node 2i for i = 0, … , (p/2)-1

• Node i connects to Node 2i+1-p for i = p/2, … , p-1

2.Exchange:
With shuffle (S) and exchange (E) operations, data is circulated from input to output
until it reaches its destination.
•If the number of inputs, for example, processors, in a single-stage IN is N and the
number of outputs, for example, memories, is N, the number of SEs in each stage is N/2.
•The maximum length of a path from an input to an output in the network, measured
by the number of SEs along the path: is 𝒍𝒐𝒈𝟐 𝑵

•The network complexity of the single-stage interconnection network is O(N)

 Fig. 8- Shuffle-exchange single stage network

Example: In an 8-input single stage Shuffle-Exchange (Fig. 8), if the source is 0(000) and the
destination is 6(110), then the following path is the required sequence of Shuffle/ Exchange
operations and circulation of data: