Link

aggregation/bonding
and load balancing
with SAF products

© SAF Tehnika JSC 2016

 Link aggregation/bonding and load
balancing with SAF products

CONTENTS
1. About LAG in Ethernet terminology .............................................................. 2
2. Link aggregation types with SAF products ................................................... 3
3. Table of aggregation types available with SAF products .............................. 4
4. Options and considerations for link aggregation .......................................... 5
5. Built-in configuration examples ................................................................... 8
5.1 CFIP Lumina ................................................................................................................................ 8
5.2 CFIP PhoeniX .............................................................................................................................. 9
5.3 Sample configuration with CFIP Phoenix M ........................................................................... 11
5.4 Integra, Integra S ....................................................................................................................... 11

6. Link aggregation using external equipment ............................................... 16

6.1 Link aggregation using external switches via LACP (or PAgP) protocol with MAC or IP
address hashing on CISCO switches. ........................................................................................... 16
6.2 Link aggregation using external switches via LACP protocol with MSTP enabled on
external switch. ............................................................................................................................... 20
6.3 Load balancing based on per packet or per destination using routers ................................ 21
6.4 Load balancing based on VLANs with protection .................................................................. 23
6.5 Link aggregation using CISCO switches and LACP (or PgAP) aggregation method with
Integra S radios. .............................................................................................................................. 25
6.6 Link aggregation using CISCO switches and LACP (or PgAP) aggregation method with
Integra-W radios. ............................................................................................................................. 30

Link aggregation/bonding and load balancing with SAF products 1

 1. About LAG in Ethernet terminology

Link aggregation, load balancing, and link bonding are computer networking umbrella terms.
They describe various methods of aggregating multiple network connections in parallel links to
increase throughput beyond what a single connection could sustain, as well as provide
redundancy in case one of the links fails.

From the network point of view the, link bonding is a single physical channel. Link bonding is
Layer 1 aggregation of frames, which is delivered as a single physical link to Layer 2, whereas
Ethernet aggregation (mostly Layer 2) and load balancing (Layer 3) are methods for combining
two or more Ethernet paths.

Link bonding – Layer 1 aggregation based on frame or bit aggregation.

Link aggregation – Layer 2 aggregation is based on IEEE 802.3ad with MAC address hashing.
Some implementations also allow Layer 3 aggregation based on hashing of IP address/ports.

Load balancing – Traffic segmentation based on pre-defined traffic path configuration, usually
implemented in routers or Layer 3 switches. Traffic segmentation can be defined by VLANs, IP
addresses, ports, etc, depending on options in the device. Load balancing can be applied on
Layer 2 switches based on pre-defined paths per VLAN.

CFIP Lumina, PhoeniX and Marathon all support built-in Ethernet link aggregation. SAF Tehnika
has designed a proprietary microwave Ethernet link aggregation mechanism. It was designed
along the guidelines of the 802.3ad protocol and complies with its provisions. While the main
enhancement specifically for microwave radio link aggregation is a proprietary control protocol,
it takes into account specific properties of the microwave link such as: link capacity, received
signal level, radial MSE, LDPC stress etc.

The Integra and Integra S radios support link bonding which works on Layer 1 and aggregates
two parallel links capacity utilizing a single Ethernet connection e.g. single MAC to MAC or IP to
IP connection may be used.

Link aggregation/bonding and load balancing with SAF products 2

 2. Link aggregation types within different SAF products

 Built-in link aggregation:

o CFIP Phoenix M 2+0 utilizes link bonding, the method which can perform the
link aggregation with one pair of MAC addresses.
 This type of aggregation is based at the modem level which uses frame
aggregation (link bonding).
o With the Integra or Integra S radios you can setup 2+0 link bonding using
various interconnection schemes. Please refer to Installation Manual.
o With the CFIP Lumina - you can setup a 2+0 configuration using at least one
pair of four port external switches and these switches are not involved in link
aggregation.
o With the CFIP PhoeniX/Marathon radios, you can setup 2+0, 3+0 and 4+0
configuration. Basic external switches are required for interconnecting
management traffic.

 Link aggregation/balancing using external equipment:

o With the CFIP Lumina radio, you can setup n+0 configuration using external
switches to achieve link aggregation or routers to achieve load balancing. *
o With the Integra, Integra S, Integra W and Integra WS radios you can setup n+0
configuration using external switches to achieve link aggregation or routers to
achieve load balancing. *
o With the CFIP PhoeniX/Marathon radios you can setup n+0 configuration using
external switches to achieve link aggregation or routers to achieve load
balancing. *
o With the CFIP-106/108 FODU and SAF Freemile radios you can setup n+0 load
balancing with external switches. Note: this will be load balancing, not link
aggregation.

* The number of ports depend on desired LAG method

Link aggregation/bonding and load balancing with SAF products 3

 3. Table of aggregation types available with SAF products

Aggregation type
Product name Layer 1 Layer 2 Layer 3
Internal External Internal External Internal External
Integra and Integra S X X* - X - X
Integra W and Intergra WS - X* - X - X
CFIP Lumina 2 LAN ports - X* X X - X
CFIP Lumina 1 LAN port - X* - X - X
CFIP Phoenix M X X* X X - X
CFIP PhoeniX - X* X X - X
CFIP Marathon - X* X X - X
SAF Freemile 17/24 - X* - X - X
SAF Freemile 5.8GE - X* - X - X

Layer 1 - Link bonding, any devices (router, switch, PC) can be connected;
Layer 2 - Link aggregation like LACP or load balancing by VLANs. Available with both Layer 2
and Layer 3 switches
Layer 3 - Load balancing - IP based. Available with routers and some Layer 3 switches.

* - In most cases it is not used in Layer 2, 3 switches or routers however it is possible.

There are other types of load balancing or aggregation based on higher layers of OSI.

Link aggregation/bonding and load balancing with SAF products 4

 4. Options and considerations for link aggregation

a. Splitters, couplers, OMT, branching system

Couplers and splitters are passive devices for dividing a microwave signal, which allows
combining the signal of one radio with another radio into a single path and provides a single
polarized coupling. The microwave signal can be divided into various proportions, for example
1:6 (asymmetrical) or 3:3 (symmetrical).

An OMT or Orthomode transducer is a passive device for filtering polarization from/to circular
waveguide paths.

A standard SAF radio adapted coupler and OMT can be combined to connect 4 radios to a single
antenna.

Standard OMT + coupler Compact hybrid combiner OMT + coupler

Outdoor Branching Unit

Multiple radios can be connected to an antenna by using circulators and filters, which can be
used either indoor (for all-indoor application) or outdoor. Frequency channel branching systems
can allow connecting multiple radios to single antenna with minimal attenuation.

Link aggregation/bonding and load balancing with SAF products 5

Comparison for various attenuations:

Solution Attenuation per port

OMT ~ 0.5 dB
Symmetrical 3dB coupler ~ 3.5 dB
Asymmetrical 6dB coupler Port1: ~1.8 dB, Port2: ~7.2 dB
OBU ~ 1.5 dB

Configuration Total attenuation with OBU, Total attenuation with

per radio coupler, per radio
4+0 in single polarization ~ 1.5 dB per radio ~ 7 dB
4+0 with dual pol antenna ~ 1 dB per radio ~ 3.5 dB

Connection via flexible waveguide

Alternatively, a 4+0 system can be attached to any dual-polarization antenna with standard
flanges by using external stand-alone couplers and waveguides.

Link aggregation/bonding and load balancing with SAF products 6

 b. Capacity and link options

In many cases the target capacity is 1Gbps, which is built from multiple parallel links. Below is a
table for different options of achieving close to 1Gbps capacity with CFIP product series
equipment.

Configuration Total capacity Total BW Comment

3+0 at 56MHz 256QAM 3x 334 = 1002 Mbps 168 MHz
4+0 at 56MHz 64QAM 4x 241 = 964 Mbps 224 MHz Best system gain for
longer distances or
smaller antennas
4+0 at 40MHz 256QAM 4x 248 = 992 Mbps 160 MHz
4+0 at 50MHz 128QAM 4x 249 = 996 Mbps 200 MHz
4+0 at 56MHZ 256QAM 4x 360 = 1440 Mbps 224 MHz

With the CFIP PhoeniX M system there is possibility to build radio links with mixed payload.

Example capacities with Integra

Configuration Total capacity Total BW Comment

2+0 at 60MHz 1024QAM Integra/Integra-S 2x 474 = 948 120MHz
Mbps
2+0 at 80MHz 256QAM Integra-W/Integra-WS 2x 514 = 1028 160MHz Very high system gain
Mbps for link budget at
1Gbps *

* With external aggregation only

Please contact SAF representative to get more details on capacity and link options.

Link aggregation/bonding and load balancing with SAF products 7

 5. Built-in configuration examples

5.1 CFIP Lumina

With the CFIP Lumina, the link aggregation in 2+0 mode allows utilizing up to 732 Mbps
Ethernet Layer 2 throughput (256QAM @ 56MHz) by using independent frequency pair for each
link. Link aggregation of 2+0 traffic distribution between two links is based upon source and
destination MAC addresses of Ethernet packets. Link aggregation (2+0) requires multiple MAC
to MAC address pair connections as a path since each connection is chosen based upon
Ethernet frame’s source and destination MAC addresses. A sufficient diversity of MAC
addresses is required to achieve maximum aggregate throughput. The CFIP Lumina 2+0 link
aggregation features link and power redundancy. If link loses synchronization or any unit fails
traffic will be rerouted to the active link.

Necessary equipment:

 Two CFIP Lumina links. Each CFIP Lumina FODU should have two Ethernet ports
(optical or electrical)
 Two Gigabit Ethernet switches.
 In order to connect radios to single antenna you can use coupler or OMT. Alternatively,
you can use two separate antennas (please see Chapter 4).

Interconnection scheme for Lumina 2+0

FODU1 and FODU2 are in the master link. FODU3 and FODU4 are in the slave link.

Link aggregation/bonding and load balancing with SAF products 8

Features:

 Up to 732 Mbps
 Physical layer protection:
o Protection of hardware failure
o Radio protection
o Modem protection
o Ethernet protection

 Automatic reconfiguration to 1+0 (traffic rerouting) in case of:

o Synchronization loss of master link
o Synchronization loss of slave link
o Hardware failure of any CFIP Lumina FODU
o Power supply failure

 Average switchover time 100ms

If the modulation of one link is downshifted, modulation on the other link will be aligned to the
same configuration.

5.2 CFIP PhoeniX

The CFIP Phoenix has built-in link aggregation of 2+0, 3+0 and 4+0 modes. The modes allow the
utilization of up to 1Gbps Ethernet Layer 2 throughput (256QAM @ 56MHz) by using
independent frequency pair for each link.
Optional CFIP PhoeniX IDU (P/N S0GIP*11) is equipped with the power protection port. It
provides redundancy for the internal power board, external power supply and power cables.

Necessary equipment:

 From four up to eight CFIP Phoenix IDU/ODU pairs.

 Two Gigabit Ethernet switches with at least 4 ports.
 OMT and/or couplers (please see Chapter 4).

Link aggregation/bonding and load balancing with SAF products 9

 Interconnection scheme for CFIP PhoeniX 2+0
IDU1 and IDU2 are in the master link. IDU3 and IDU4 are in the slave link.

Interconnection scheme for CFIP PhoeniX 4+0

IDU1 and IDU2 are in the master link. All other links are slave links.

Link aggregation/bonding and load balancing with SAF products 10

5.3 Sample configuration with CFIP Phoenix M

CFIP ODU can utilize adapted mounting and hybrid combiner. A hybrid combiner consists of a
built-in OMT with couplers attached to each polarization.

5.4 Integra, Integra S

The Integra/Integra-S’s 2+0 aggregation (link bonding) provides ACM-aware bonding. The ACM-
aware bonding of the user available capacities take two parallel links, and each will use an
individual frequency pair. Traffic is split per-frame over two links on the modem level.

Aggregation is not based on MAC-MAC connections. Single MAC address (e.g.

router) can be used.

If any (master or slave) link synchronization goes down, switching to a working link is hitless.
Two Integra or Integra-S FODUs are required. In the case of the Integra-S OMT, a dual-polarized
antenna or coupler can be used.

Necessary equipment for Integra 2+0

1. 4 Integra/Integra-S FODUs – 2 low side, 2 high side
2. 2 or 4 SFP modules and appropriate FO cables (multi-mode or single mode) for Integra
interconnection (depending on chosen interconnection scheme)
3. Electrical or optical Ethernet cables for user traffic (depending on chosen
interconnection scheme)
4. In case of Integra-S – an additional 2 or 4 antennas and OMT/couplers

Link aggregation/bonding and load balancing with SAF products 11

Interconnection schemes

There are 4 possible interconnection schemes:

Scheme 1

1. Optical cable between the LAN2 (optical) ports

on both units. Mandatory in all 4 schemes.
2. Electrical Ethernet cable (1000Base-T) between
PoE injector’s (#1) DATA+PWR port and LAN1
(electrical) port of Slave Integra/Integra-S FODU.
Both data and power are carried, therefore total
length of cables #2, #3 and #4 combined should
not exceed 100m.
3. Electrical Ethernet cable (1000Base-T) between
PoE injector’s (#2) DATA+PWR port and LAN1
(electrical) port of Master Integra/Integra-S
FODU. Both data and power are carried, therefore
total length of cables #2, #3 and #4 combined
should not exceed 100m.
4. Electrical Ethernet cable (1000Base-T) between
PoE injectors’ (#1 and #2) DATA ports. Provides
management access to Slave Integra/Integra-S
FODU. Total length of cables #2, #3 and #4
combined should not exceed 100m.
5. Optical fiber cable between LAN3 (optical) port of
Master or Slave Integra/Integra-S FODU and CPE
for both traffic and management traffic.

Advantages: An external switch is not required; length of optical cable for

traffic/management up to 10km.

Link aggregation/bonding and load balancing with SAF products 12

 Scheme 2

1. Optical cable between the LAN2 (optical) ports on

both units. Mandatory in all 4 schemes.
2. Cable for powering Slave Integra/Integra-S FODU.
You can use 2-wire power cable with DC power
adapter (P/N D0ACPW01) or standard Ethernet
cable with PoE injector. Depending on power
consumption cable length can be extended up to
700m. Refer to chapter RJ-45 port for details.
3. Electrical Ethernet cable (1000Base-T) between
PoE injector’s (#2) DATA+PWR port and LAN1
(electrical) port of Master Integra/Integra-S
FODU. Both data and power are carried, therefore
total length of cables #3 and #5 combined
should not exceed 100m.
4. Optical fiber cable between LAN3 (optical) ports
on both units. Provides management access to
Slave Integra/Integra-S FODU.
5. Electrical Ethernet cable (1000Base-T) between
PoE injector’s (#1 or #2) DATA port and CPE or
both traffic and management traffic. Total length
of cables #3 and #5 combined should not exceed
100m.

Advantages: An external switch is not required; optical cables used only for
interconnection between both Integra/Integra-S FODUs; only two cables
installed between Integra/Integra-S FODUs and indoor facility.

Link aggregation/bonding and load balancing with SAF products 13

 Scheme 3

1. Optical cable between the LAN2 (optical) ports on

both units. Mandatory in all 4 schemes.
2. Cable for powering Slave Integra/Integra-S FODU.
You can use 2-wire power cable with DC power
adapter (P/N D0ACPW01) or standard Ethernet
cable with PoE injector. Depending on power
consumption cable length can be extended up to
700m. Refer to chapter RJ-45 port for details.
3. Cable for powering Master Integra/Integra-S
FODU. You can use 2-wire power cable with DC
power adapter (P/N D0ACPW01) or standard
Ethernet cable with PoE injector. Depending on
power consumption cable length can be extended
up to 700m. Refer to chapter RJ-45 port for
details.
4. Optical cable between the LAN3 (optical) port of
the Slave Integra/Integra-S FODU and external
switch. Provides management access to Slave
Integra/Integra-S FODU.
5. Electrical Ethernet cable (1000Base-T) between
the external switch and CPE for both traffic and
management traffic.
6. Optical cable between the LAN3 (optical) port of
Master Integra/Integra-S FODU and external
switch for both traffic and management traffic.

Advantages: Solution provides greatest cable length for powering

Integra/Integra-S and length of optical cable for traffic/management can be
up to 10km. For details on power cable length refer to chapter RJ-45 port.

Link aggregation/bonding and load balancing with SAF products 14

 Scheme 4

1. Optical cable between the LAN2 (optical) ports

on both units. Mandatory in all 4 schemes.
2. Electrical Ethernet cable (1000Base-T) between
PoE injector’s (#1) DATA+PWR port and LAN1
(electrical) port of the Slave Integra/Integra-S
FODU. Both data and power are carried, therefore
total length of cables #2 and #6 combined
should not exceed 100m.
3. Electrical Ethernet cable (1000Base-T) between
PoE injector’s (#2) DATA+PWR port and LAN1
(electrical) port of the Master Integra/Integra-S
FODU. Both data and power are carried, therefore
total length of cables #3 and #4 combined
should not exceed 100m.
4. Electrical Ethernet cable (1000Base-T) between
the PoE injector’s (#1) DATA port and external
switch. The total length of cables #3 and #4
combined should not exceed 100m.
5. Electrical Ethernet cable (1000Base-T) between
the external switch and CPE or both traffic and
management traffic.
6. Electrical Ethernet cable (1000Base-T) between
the PoE injector’s (#2) DATA port and external
switch. The total length of cables #2 and #6
combined should not exceed 100m.

Advantages: Only a single optical cable required; only two cables installed
between Integra/Integra-S FODUs and indoor facility.

Link aggregation/bonding and load balancing with SAF products 15

 6. Link aggregation using external equipment
Some network operators would like to use their own link aggregation methods based on LACP
as well other methods. The most frequently ignored issue is related to putting CFIP
management inside one of two balanced Ethernet paths. This is a violation because balancing
logic does not expect any Ethernet device in-between two aggregated switches. As a result,
traffic will go through, but management access to each CFIP Lumina will be available only from
50% of MAC addresses – might be available from one PC but not available from another, which
is not predictable.

Following are some examples showing how to implement link aggregation, or load balance,
using external switches and routers and ensure stable management access to CFIP products.

6.1 Link aggregation using external switches via LACP (or PAgP) protocol with
MAC or IP address hashing on CISCO switches.

After the switch CRC hash calculation, based on its chosen method SA-DA, SA or DA, the switch
will send packets using a chosen path. Along with traffic, the CFIP management packets can be
routed along paths where the requested CFIP unit doesn’t exist. This may cause inaccessibility
to CFIP management. To avoid this problem, it is recommend that a two a port CFIP Lumina or
CFIP PhoeniX, where you can separate data traffic from management traffic using VLANs, is
used.

Below is an interconnection example for 2+0 Lumina with 2 Ethernet ports and external
managed switches.

Interconnection scheme for Lumina 2+0 external aggregation with switches

Link aggregation/bonding and load balancing with SAF products 16

In this setup 2-port (electrical) Luminas and CISCO 3750 switches are used. Each CFIP Lumina
is configured to trunk traffic on LAN port number 4 and Lumina radios management is on LAN
port number 3.

VLAN configuration on Lumina

Note: Traffic is tagged on external switches. VLAN100 is management and VLAN200 is data
traffic. VLAN1 will pass untagged traffic which is required for communication between external
switches. You also can use Lumina radios built-in switch to tag Ethernet traffic (data and
management traffic).

In such configuration it is important to configure CISCO switches properly. As configuration is

the same on all four units and management VLAN is trunked via CFIP Lumina WAN port and
external switches, a network loop on the management VLAN will be created. Utilizing the
spanning tree protocol (PVST – CISCO proprietary protocol) will block one of four ports to avoid
network loop, and management still will be accessible on all four units. Other switches are using
MSTP protocol to avoid Ethernet loops with Multiple VLANs (configuration for MSTP is
described below).

The link between two CISCO switches is considered working from the switch perspective when
the port condition is “UP”. In case of synchronization loss in the Lumina link which connects
two switches, Ethernet ports will be still in the “UP” state and switches will consider that the link
is working, thereby continuing to send the traffic which will be dropped on the Lumina device.
LACP protocol can detect broken the link even if Ethernet port is “UP” and redirect all traffic to
another path, however, this procedure might take more than 1 minute with slow LACP.
Aggregation restoration is more or less dependent on external switches in the case of radio
sync loss.

Link aggregation/bonding and load balancing with SAF products 17

The CFIP Lumina, Marathon and PhoeniX products have a “Link State Propagation” feature
which allows shutting down specified LAN ports if synchronization loss events occur. In this
case, CISCO switches will immediately detect that the port switched to the “DOWN” state and
the LACP protocol will redirect traffic to another path. This scenario is much faster than the
broken link detection by LACP – less than 1sec, compared to around 60 to 90 seconds by slow
LACP.

The link state propagation configuration is available in Configuration -> Ethernet configuration
Web GUI page.

Link state propagation configuration

Explanation:

LAN auto recovery* (0..600) sec – synchronization loss timeout after which port is reenabled even
if link synchronization is still lost, otherwise timeout is ignored.

SyncLoss keepalive timeout (0..10) sec – LAN port shutdown timeout after synchronization loss
and synchronization recovery events.

LSP startup timeout (0..3600) sec – LSP activity timeout after management CPU start up and
configuration script execution. During this period synchronization events are ignored.
SNMP traps – SNMP trap will be sent if enabled.

Configuration of CISCO switches:

!
spanning-tree mode pvst
no spanning-tree optimize bpdu transmission
spanning-tree extend system-id
!
interface Port-channel1
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 200
switchport mode trunk
!

Link aggregation/bonding and load balancing with SAF products 18

interface GigabitEthernet2/0/1
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 200
switchport mode trunk
channel-group 1 mode active
!
interface GigabitEthernet2/0/2
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 200
switchport mode trunk
channel-group 1 mode active
!
interface GigabitEthernet2/0/3
switchport access vlan 200
switchport mode access
!
interface GigabitEthernet2/0/4
switchport access vlan 100
switchport mode access
!
interface GigabitEthernet2/0/5
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 100
switchport mode trunk
!
interface GigabitEthernet2/0/6
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 100
switchport mode trunk
!

Explanation:

interface Port-channel1 – Specifies EtherChannel number

Interfaces ge0/1 - 0/2 are used to trunk aggregated traffic to other peer thru Lumina
radios.
Interface ge0/3 – is network cloud (lots of SA-DA MACs)
Interface ge0/4 – is for management PC
Interfaces ge0/5 – 0/6 are used to trunk management data to Lumina.

There is no limitation to the number of aggregated parallel links, it depends on the number of
external switches.

The same configuration of Lumina radios and switches should be used in case of link
aggregation, which is based on IP hashing. You will need to add “port-channel load-balance src-
dst-ip” in to CISCO global configuration.

Link aggregation/bonding and load balancing with SAF products 19

6.2 Link aggregation using external switches via LACP protocol with MSTP
enabled on external switch.

By default, CISCO switches are using a proprietary PVST protocol. This protocol works very well
without special “fine tuning” of STP (Spanning-Tree Protocol). Other brands have MSTP
(Multiple Spanning-Tree protocol) which requires “fine tuning” for correct STP behavior.
Important properties are: setup region name and revision, instances per VLAN, BPDU filter must
be applied on aggregated port (not physical interface), correct port priority and path costs for
aggregated ports.

Configuration of MSTP:

spanning-tree mode mst <- Spanning-tree mode MSTP

no spanning-tree optimize bpdu transmission
spanning-tree extend system-id

spanning-tree mst configuration

name REGION <- name for MST region, revision is 0 by default
instance 1 vlan 100 <- Instance for management VLAN
instance 2 vlan 200 <- Instance for traffic VLAN

Instance 0 (MST00) is enabled by default. This instance contains all untagged service packets
for correct communication between external switches.

Configuration of aggregation interface:

interface Port-channel1 <- Etherchannel interface (LAG)

switchport trunk allowed vlan 200 <- Command to allow only traffic VLAN
switchport mode trunk
flowcontrol send off
spanning-tree bpdufilter enable <- Command to enable BPDU filter
spanning-tree port-priority 0 <- Command to set port priority for Etherchannel
interface (0 - highest)

Please check the status of STP after successful configuration, the aggregation port must be in
“forwarding” state.

You need to check STP behavior on the external switches after correct configuration. The
aggregation port must be in “forwarding” state.

Link aggregation/bonding and load balancing with SAF products 20

6.3 Load balancing based on per packet or per destination using routers

Another method is to use routers and load balancing between nodes. On CISCO routers there
are two ways:
per-packet – load balancing allows the router to send data packets over successive
equal-cost paths without regard to individual destination hosts or user sessions. Path
utilization is good, but packets destined for a given destination host might take different
paths and might arrive out of order.
per-destination - load balancing allows the router to use multiple, equal-cost paths to
achieve load sharing. Packets for a given source-destination host pair are guaranteed to
take the same path, even if multiple, equal-cost paths are available. Traffic for different
source-destination host pairs tend to take different paths.

The “per-packet” method which can be used to load balance with one source IP and destination
IP will be explained further on.

This is interconnection example for 2+0 Lumina with one or two Ethernet ports and CISCO routers.

Link aggregation/bonding and load balancing with SAF products 21

Configuration of CISCO routers:

Router Nr.1:

!
ip cef
ip cef load-sharing algorithm original
!
interface FastEthernet0/0
ip address 192.168.20.2 255.255.255.0
ip load-sharing per-packet
duplex auto
speed auto
!
interface FastEthernet0/1
ip address 192.168.30.2 255.255.255.0
ip load-sharing per-packet
duplex auto
speed auto
!
interface FastEthernet0/1/0
switchport access vlan 20
!
interface Vlan20
ip address 192.168.205.1 255.255.255.0
!
!
ip route 0.0.0.0 0.0.0.0 FastEthernet0/0 192.168.20.1
ip route 0.0.0.0 0.0.0.0 FastEthernet0/1 192.168.30.1
ip route 192.168.205.0 255.255.255.0 FastEthernet0/0 192.168.20.1
ip route 192.168.205.0 255.255.255.0 FastEthernet0/1 192.168.30.1
ip route 192.168.205.0 255.255.255.0 Vlan20
!

Router Nr.2:

!
ip cef
ip cef load-sharing algorithm original
!
!
interface FastEthernet0/0
ip address 192.168.20.1 255.255.255.0
ip load-sharing per-packet
duplex auto
speed auto
!
interface FastEthernet0/1
ip address 192.168.30.1 255.255.255.0
ip load-sharing per-packet
duplex auto
speed auto
!

Link aggregation/bonding and load balancing with SAF products 22

interface FastEthernet0/0/1
switchport access vlan 20
!
!
interface Vlan20
ip address 192.168.206.1 255.255.255.0
!
ip route 0.0.0.0 0.0.0.0 FastEthernet0/0 192.168.20.2
ip route 0.0.0.0 0.0.0.0 FastEthernet0/1 192.168.30.2
ip route 192.168.206.0 255.255.255.0 FastEthernet0/0 192.168.20.2
ip route 192.168.206.0 255.255.255.0 FastEthernet0/1 192.168.30.2
ip route 192.168.206.0 255.255.255.0 Vlan20
!

CEF switching (Layer3) requires higher performance of equipment than Layer2 link aggregation
methods. Please check the CISCO routers CEF performance by this URL:
http://www.cisco.com/web/partners/downloads/765/tools/quickreference/routerperformance.
pdf

6.4 Load balancing based on VLANs with protection

Load balancing with VLANs can be configured on any CFIP device or Integra S/W with one
Ethernet port and any switch which has VLANs and Spanning-tree per VLAN (PVST, MSTP).
Load balance also provides protection in case of one link fail.

spanning-tree mode mst <- PVST can be used instead of MST

no spanning-tree optimize bpdu transmission
spanning-tree extend system-id
!
spanning-tree mst configuration
name REGION
instance 1 vlan 100
instance 2 vlan 200
instance 3 vlan 300
!
!vlan 100,200,300
!
interface FastEthernet0/1
switchport access vlan 100
switchport mode access
!
interface FastEthernet0/13
switchport trunk allowed vlan 200,300
switchport mode trunk
!
interface FastEthernet0/17
switchport trunk allowed vlan 100,200,300
switchport mode trunk
spanning-tree vlan 300 port-priority 0 <- port priority for VLAN 300
!

Link aggregation/bonding and load balancing with SAF products 23

interface FastEthernet0/18
switchport trunk allowed vlan 100,200,300
spanning-tree vlan 200 port-priority 0 <- port priority for VLAN 200

Interface Fa0/1 - port for management PC

Interface Fa0/13 – port where traffic is already tagged with VLANs 200,300
Interface Fa0/17 – port to CFIP device with management and traffic VLANs
Interface Fa0/18 – port to CFIP device with management and traffic VLANs

VLAN 200 will be blocked on interface Fa0/17 and VLAN will be blocked on interface Fa0/18 by
STP in such configuration. VLAN 300 will pass through link “1” and VLAN 200 will pass through
link “2”. Also management VLAN will be blocked to avoid loop and all CFIP devices will be
accessible.

Link aggregation/bonding and load balancing with SAF products 24

6.5 Link aggregation using CISCO switches and LACP (or PgAP) aggregation
method with Integra S radios.

After the CISCO switch CRC hash calculation, based on chosen method SA-DA, SA or DA (or IPs),
the switch will send packets using one or another physical port, which is included in the Port-
Channel interface (virtual interface which contains bunch on physical interfaces). Along with
traffic, the Integra S management packet can be routed to one or another path where the
requested radio unit doesn’t exist, this may cause the inaccessibility of radio’s management. To
avoid this problem, we recommend using two ports of radios where you can separate data
traffic from management traffic using VLANs.

For example: PoE ports (electrical) at each radio will be used for device management with power
only and LAN2 (fiber ports) will be used as LACP ports.

Physical interconnection example

As all management ports will be connected to the same broadcast domain, Ethernet loop will
occur. To avoid loop, STP will block one of the management ports in CISCO switch, if STP is
enabled. Which port will be disabled depends on STP configuration. Of course it is not
necessary to connect all management ports to CISCO, but it will additional protection to the
management ports in case if one (or more) unit failure.
Note: Ethernet loop will not occur on LACP (Port-Channel) ports.

Link aggregation/bonding and load balancing with SAF products 25

 Traffic MNG MNG Traffic

LAN1 LAN1
Gi0/1 Trunk Trunk Gi0/1

LAN2
RADIO 1 WAN RADIO 2 LAN2
Gi0/5 Access Gi0/5
Access

Switch B
Switch A
Port-Channel 1

Port-Channel 1
Gi0/3 Gi0/3
Channel-Group 1 Channel-Group 1
LACP Virtual Port LACP Virtual Port
Gi0/4 Gi0/4

LAN2 LAN2
Gi0/6 Access Gi0/6
Access
LAN1
RADIO 3 WAN RADIO 4 LAN1
Gi0/2 Gi0/2
Trunk Trunk
Disabled
by STP
LACP path 1 MNG
MNG MNG MNG (Optional)
LACP path 2

Logical scheme and CISCO (ME3400) configuration example

Swtich A configuration and status output:

interface Port-channel1
port-type nni
!
!
interface GigabitEthernet0/1
port-type nni
switchport trunk allowed vlan 200
switchport mode trunk
media-type rj45
!
interface GigabitEthernet0/2
port-type nni
switchport trunk allowed vlan 200
switchport mode trunk
media-type rj45
!
interface GigabitEthernet0/3
port-type nni
channel-group 1 mode active
!
interface GigabitEthernet0/4
port-type nni
channel-group 1 mode active
!

Link aggregation/bonding and load balancing with SAF products 26

tw01-Main#show spanning-tree

VLAN0001
Spanning tree enabled protocol rstp
Root ID Priority 32769
Address 108c.cf8d.7800
Cost 3
Port 56 (Port-channel1)
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 32769 (priority 32768 sys-id-ext 1)

Address 108c.cf8d.8700
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Aging Time 300 sec

Interface Role Sts Cost Prio.Nbr Type

------------------- ---- --- --------- -------- -------------------------------
-
Po1 Root FWD 3 128.56 P2p

VLAN0200
Spanning tree enabled protocol rstp
Root ID Priority 32968
Address 108c.cf8d.7800
Cost 4
Port 1 (GigabitEthernet0/1)
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 32968 (priority 32768 sys-id-ext 200)

Address 108c.cf8d.8700
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Aging Time 300 sec

Interface Role Sts Cost Prio.Nbr Type

------------------- ---- --- --------- -------- -------------------------------
-
Gi0/1 Root FWD 4 128.1 P2p
Gi0/2 Altn BLK 4 128.2 P2p

tw01-Main#show lacp neighbor

Flags: S - Device is requesting Slow LACPDUs
F - Device is requesting Fast LACPDUs
A - Device is in Active mode P - Device is in Passive mode

Channel group 1 neighbors

Partner's information:

LACP port Admin Oper Port Port

Port Flags Priority Dev ID Age key Key Number State
Gi0/3 SA 32768 108c.cf8d.7800 12s 0x0 0x1 0x104 0x3D
Gi0/4 SA 32768 108c.cf8d.7800 20s 0x0 0x1 0x105 0x3D

Link aggregation/bonding and load balancing with SAF products 27

Swtich B configuration and status output:

interface Port-channel1
port-type nni
!
interface GigabitEthernet0/1
port-type nni
switchport trunk allowed vlan 200
switchport mode trunk
media-type rj45
!
interface GigabitEthernet0/2
port-type nni
switchport trunk allowed vlan 200
switchport mode trunk
media-type rj45
!
interface GigabitEthernet0/3
port-type nni
channel-group 1 mode active
!
interface GigabitEthernet0/4
port-type nni
channel-group 1 mode active
!

tw3-main#show lacp neighbor

Flags: S - Device is requesting Slow LACPDUs
F - Device is requesting Fast LACPDUs
A - Device is in Active mode P - Device is in Passive mode

Channel group 1 neighbors

Partner's information:

LACP port Admin Oper Port Port

Port Flags Priority Dev ID Age key Key Number State
Gi0/3 SA 32768 108c.cf8d.8700 10s 0x0 0x1 0x104 0x3D
Gi0/4 SA 32768 108c.cf8d.8700 7s 0x0 0x1 0x105 0x3D

tw3-main#show spanning-tree

VLAN0001
Spanning tree enabled protocol rstp
Root ID Priority 32769
Address 108c.cf8d.7800
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 32769 (priority 32768 sys-id-ext 1)

Address 108c.cf8d.7800
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Aging Time 300 sec

Interface Role Sts Cost Prio.Nbr Type

------------------- ---- --- --------- -------- -------------------------------
-
Po1 Desg FWD 3 128.56 P2p

Link aggregation/bonding and load balancing with SAF products 28

VLAN0200
Spanning tree enabled protocol rstp
Root ID Priority 32968
Address 108c.cf8d.7800
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 32968 (priority 32768 sys-id-ext 200)

Address 108c.cf8d.7800
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Aging Time 300 sec

Interface Role Sts Cost Prio.Nbr Type

------------------- ---- --- --------- -------- -------------------------------
-
Gi0/1 Desg FWD 4 128.1 P2p
Gi0/2 Desg FWD 4 128.2 P2p

Configuration of radios is the same in all units:

LAN1 – The management port to be used as the trunk for management VLAN200.
LAN2 – LACP port as access for LACP BPDUs with traffic.

Note: D – VLAN disabled, T – port is in trunk mode (tagged), U – port is in access mode
(untagged).

Link aggregation/bonding and load balancing with SAF products 29

6.6 Link aggregation using CISCO switches and LACP (or PgAP) aggregation
method with Integra-W radios.

There are two scenarios:

1. User traffic is not separated from management traffic with VLANs in the network.
2. User traffic is separated from management traffic with VLANs in the network.

Since the Integra-W internal switch is not configurable, it is not possible to separate
management traffic with VLANs within the radio link. It is recommend using Out of Band
management software where two ports of radios are used – MNG port for management, LAN
port for traffic. In the following scenario both Integra-W MNG and LAN ports are connected to
the CISCO 3750g switches.

Physical interconnection example

The Integra-W link with out-band management implements two Ethernet connections – one for
management traffic and the second for user traffic. Since user traffic and management circuits
are parallel, an Ethernet loop will occur. To avoid the loop, management on WAN interfaces has
to be disabled by entering this command in CLI “modem management 0”. Thereby management

Link aggregation/bonding and load balancing with SAF products 30

traffic will only pass through the wireless link via the user traffic connection egressing the MNG
port.

Disable management on WAN interfaces

Traffic MNG MNG Traffic

Gi0/3 MNG MNG Gi0/3

RADIO 1 RADIO 2
Access Access
WAN WAN
LAN LAN
Gi0/5 Gi0/5
Trunk Trunk Switch B
Switch A
Port-Channel 1

Gi0/1
Port-Channel 1

Gi0/1
Trunk Trunk
Channel-Group 1 Channel-Group 1
LACP Virtual Port LACP Virtual Port
Gi0/2 Gi0/2
Trunk Trunk
Gi0/6 LAN LAN Gi0/6

RADIO 3 RADIO 4
Access Access
WAN WAN
Gi0/4 MNG MNG Gi0/4
Access Access

LACP path 1 MNG

MNG MNG MNG (Optional)
LACP path 2

Logical scheme and CISCO (Catalyst 3750G) configuration example with management on WAN interfaces disabled and
VLANs enabled

Link aggregation/bonding and load balancing with SAF products 31

After the CISCO switch CRC hash calculation, based on the chosen method SA-DA, SA or DA (or
IPs), the switch will send packets using one or another physical port which is included in Port-
Channel interface (virtual interface which contains bunch on physical interfaces). The following
load-balance hashing methods are available:
 Source or Destination MAC addresses (Load distribution is based on the source-host or
destination-host MAC address of the incoming packet)
 Source XOR Destination MAC addresses (Load distribution is based on the source-and-
destination host-MAC address)
 Source or Destination IP addresses (Load distribution is based only on the source-host
or destination-host IP address)
 Source XOR Destination IP addresses (Load distribution is based on the source-and-
estination host-IP address)

In the following example, the Source XOR Destination MAC addresses method is used.
Note: Ethernet loop will not occur on LACP (Port-Channel) ports.

Another important thing to be taken into account is the monitoring status of the Ethernet link.
The link between two CISCO switches is considered working from the switch perspective when
the port condition is “UP”. In case of synchronization loss in the Integra-W link which connects
two switches, Ethernet ports will be still in “UP” state and switches will consider that the link is
working, thereby continuing to send the traffic which will be dropped on the Integra-W device.
LACP protocol can detect broken link even if Ethernet port is “UP” and redirect all traffic to
another path. However, this procedure might take more than 1 minute with slow LACP.
Aggregation restoring is more or less depended on external switches in case of radio sync loss.

Configuration of CISCO switches without VLAN configuration:

!
port-channel load-balance src-dst-mac
!
spanning-tree mode pvst
spanning-tree extend system-id
!
!
interface Port-channel1
description Integra Link
!
interface GigabitEthernet2/0/1
channel-group 1 mode active
!
interface GigabitEthernet2/0/2
channel-group 1 mode active
!
interface GigabitEthernet2/0/3
description Management Radio
!
interface GigabitEthernet2/0/4
description Management Radio
!
interface GigabitEthernet2/0/5
description Traffic + Mng

Link aggregation/bonding and load balancing with SAF products 32

!
interface GigabitEthernet2/0/6
description Management Port
!
!

Note: This scenario is meant for untagged traffic. VLANs are not configured on the CISCO
switch interfaces. All incoming traffic passes the link and management traffic is not isolated in
separate VLANs within the network.

Configuration of CISCO switches with VLAN configuration:

!
port-channel load-balance src-dst-mac
!
spanning-tree mode pvst
spanning-tree extend system-id
!
!
interface Port-channel1
description Integra Link
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1,101-200,990
switchport mode trunk
!
interface GigabitEthernet2/0/1
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1,101-200,990
switchport mode trunk
channel-group 1 mode active
!
interface GigabitEthernet2/0/2
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1,101-200,990
switchport mode trunk
channel-group 1 mode active
!
interface GigabitEthernet2/0/3
description Management Radio
switchport access vlan 990
switchport mode access
!
interface GigabitEthernet2/0/4
description Management Radio
switchport access vlan 990
switchport mode access
!
interface GigabitEthernet2/0/5
description Traffic + Mng
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1,101-200,990
switchport mode trunk
!

Link aggregation/bonding and load balancing with SAF products 33

interface GigabitEthernet2/0/6
description Management PC Port
switchport access vlan 990
switchport mode access
!
!

switch#show lacp neighbor

Flags: S - Device is requesting Slow LACPDUs
F - Device is requesting Fast LACPDUs
A - Device is in Active mode P - Device is in Passive mode

Channel group 1 neighbors

Partner's information:

LACP port Admin Oper Port Port

Port Flags Priority Dev ID Age key Key Number State
Gi2/0/1 SA 32768 0013.c4df.4600 14s 0x0 0x1 0x43 0x3D
Gi2/0/2 SA 32768 0013.c4df.4600 21s 0x0 0x1 0x44 0x3D

Note: Traffic is tagged on external switches. VLAN990 is management and VLANs 101-200 are
data traffic. VLAN1 will pass untagged traffic which is required for communication between
external switches. Since Integra-W does not support VLANs, management traffic has to arrive
untagged to the Integra-W management port. Therefore, the Integra-W management port is
connected to CISCO Access port with management VLAN configured on it. Also you can use
CISCO external switch to tag Ethernet traffic (data and management traffic) on specific
interfaces.

Explanation:

interface Port-channel1 – specifies EtherChannel number.

channel-group 1 mode active – ties the physical interface to the Port-Channel 1 logical
interface.

Interfaces ge2/0/1 – 2/0/2 are used to trunk aggregated traffic to other peer thru
Integra-W radios.
Interface ge2/0/3 – is network cloud (lots of SA-DA MACs).
Interface ge2/0/4 – is for management PC.
Interfaces ge2/0/5 – 2/0/6 are used for management data to Integra-W.

SAF Tehnika, JSC has no liability for typing errors in this document or damages of any kind that result from the use of
this document.
To get up to date information about features explained in this document, please contact SAF representative.

Link aggregation/bonding and load balancing with SAF products 34