Flexibilis Redundant Switch (FRS) : Manual
Flexibilis Redundant Switch (FRS) : Manual
Manual
This document could contain technical inaccuracies or typographical errors. Flexibilis Oy may
make changes in the product described in this document at any time.
Trademarks
Contents
1 About this Document ............................................................................................................7
2 General Overview ..................................................................................................................9
2.1 FRS Features ................................................................................................................9
2.1.1 Interface between MAC and PHY ................................................................ 10
2.1.2 Host System Interface .................................................................................. 10
3 Introduction to Ethernet and Switching ........................................................................... 11
3.1 Ethernet Media Access Control (MAC) ...................................................................... 11
3.1.1 Preamble and Start Frame Delimiter (SFD) ................................................. 11
3.1.2 Header .......................................................................................................... 12
3.1.3 CRC .............................................................................................................. 12
3.1.4 Media Independent Interface (MII/GMII) ...................................................... 12
3.2 Media Access Control (MAC) Bridge ......................................................................... 12
3.2.1 MAC Address Table ..................................................................................... 13
3.2.2 Shared Media versus Dedicated Media ....................................................... 13
3.2.3 Cut-through versus Store-and-Forward Operation ...................................... 13
3.2.4 Spanning Tree Protocol................................................................................ 14
3.3 Precision Time Protocol ............................................................................................. 14
3.3.1 Transparent Clock ........................................................................................ 14
3.4 HSR (High-availability Seamless Redundancy) ........................................................ 15
3.4.1 HSR Node Types ......................................................................................... 17
3.5 Parallel Redundancy Protocol (PRP)......................................................................... 18
3.5.1 PRP Node Types .......................................................................................... 19
4 Functional Description ....................................................................................................... 20
4.1 Inbound Processing ................................................................................................... 23
4.1.1 RX MII........................................................................................................... 23
4.1.2 Timestamp .................................................................................................... 24
4.1.3 RX Pre-process ............................................................................................ 24
4.1.4 Forwarding Decision ..................................................................................... 24
4.1.5 Inbound Policy .............................................................................................. 24
4.1.5.1 Priority-setting ............................................................................. 25
4.1.6 Precision Time Protocol ............................................................................... 25
4.1.6.1 End-to-end Transparent Clock Functionality .............................. 25
4.1.6.2 Peer-to-Peer Transparent Clock Support ................................... 25
4.1.7 Management Trailer ..................................................................................... 25
4.1.8 MAC Address Table ..................................................................................... 26
4.1.8.1 Address Table Entry ................................................................... 27
4.1.8.2 Address Learning ........................................................................ 27
4.1.8.3 Address Aging ............................................................................. 29
4.1.8.4 MAC Address and Forward Decision .......................................... 30
4.1.8.5 Port-based VLAN ........................................................................ 31
4.1.8.6 Forward Portmask ....................................................................... 32
4.2 Outbound Processing ................................................................................................ 33
4.2.1 TX Post-process ........................................................................................... 34
4.2.2 TX MII ........................................................................................................... 34
4.2.3 Timestamp .................................................................................................... 34
4.2.4 PTP Overwrite .............................................................................................. 34
4.3 Forwarding Core ........................................................................................................ 34
4.3.1 Memory Controller ........................................................................................ 34
4.3.2 Priority Queues ............................................................................................. 35
4.3.3 Core RX ........................................................................................................ 35
4.3.4 Core TX ........................................................................................................ 36
4.3.5 Frame Early Drop ......................................................................................... 36
4.4 HSR (High-availability Seamless Redundancy) ........................................................ 38
4.4.1 Forwarding of HSR Frames ......................................................................... 38
Figures
Figure 1. Bit and Byte Order ...................................................................................................... 7
Figure 2. FRS Overview ............................................................................................................ 9
Figure 3. Ethernet Frame Formats .......................................................................................... 11
Figure 4. HSR Ring Example with Unicast Message .............................................................. 15
Figure 5. HSR Ring Example with Multicast Message ............................................................ 16
Figure 6. HSR Tagged frame .................................................................................................. 17
Figure 7. QuadBox .................................................................................................................. 17
Figure 8. PRP Double LAN ...................................................................................................... 18
Figure 9. PRP Frame ............................................................................................................... 19
Figure 10. FRS Block Diagram ................................................................................................ 20
Figure 11. FRS Forwarding Path ............................................................................................. 21
Figure 12. FRS Inbound and Outbound Processing Blocks .................................................... 22
Figure 13. Ethernet Frame Timestamp Point .......................................................................... 23
Figure 14. Ethernet Frame with Management Trailer.............................................................. 26
Figure 15. MAC Address Entry ................................................................................................ 27
Figure 16. Address Learning ................................................................................................... 28
Figure 17. Address Aging ........................................................................................................ 29
Figure 18. Address Search (+ Port Based VLAN) ................................................................... 31
Figure 19. Address Search (+ Forward Portmask) .................................................................. 33
Figure 20. Priority Queues (only three ports shown) ............................................................... 35
Figure 21. Frame Early Drop Algorithm ................................................................................... 37
Tables
Table 1. FRS MDIO Registers ................................................................................................. 46
Table 2. FRS Switch Configuration Register Groups .............................................................. 47
Table 3. General Switch Configuration Registers ................................................................... 49
Table 4. FRS Port Configuration Register Groups .................................................................. 51
Table 5. General Configuration and State Registers ............................................................... 52
Table 6. HSR Registers ........................................................................................................... 53
Table 7. PTP Registers ........................................................................................................... 53
Table 8. Inbound Policy Registers ........................................................................................... 55
Table 9. FRS Generics ............................................................................................................ 57
Table 10. General Signals ....................................................................................................... 58
Table 11. Time Interface Signals ............................................................................................. 59
Table 12. MII/GMII Signals ...................................................................................................... 61
Table 13. MDIO Station Management Signals ........................................................................ 62
Table 14. MDIO Manageable Device Signals ......................................................................... 62
Table 15. MII/GMII Clock Mux Configuration .......................................................................... 66
Revision History
Rev Date Description
1.0 13.6.2011 First release.
1.1 14.10.2011 Minor improvements.
1.2 14.3.2012 - HSR configuration register modified
- Configuration pins defined to give system clock frequency
- PRP added
- HSR updates
2.0 23.3.2012 Approved
2.1 1.6.2012 HSR_CFG register clarifications (Table 6)
2.2 11.6.2012 Added IEEE 1588 Layer 2 functionality and Peer-to-peer transparent clock
2.3 13.6.2012 Support for link local traffic without HSR tag or PRP trailer
HSR_CFG register updates (Table 6)
2.4 28.6.2012 HSR_CFG register correction (Table 6)
2.5 29.6.2012 Changed PTP mode bits
2.6 15.8.2012 Minor changes
2.7 25.9.2012 Clock interface changed.
2.8 14.12.2012 Inbound Policy (IPO) registers changed.
2.9 29.1.2013 FES-HSR renamed to FRS
2.10 7.2.2013 IP License Authentication added
Bit 15 8 7 0
16bit Bus
or Byte 1 Byte 0
Register:
MSB LSB
Bit 31 24 23 16 15 8 7 0
32bit Bus
or Byte 3 Byte 2 Byte 1 Byte 0
Register:
MSB LSB
Bit 63 56 55 48 47 40 39 32 31 24 23 16 15 8 7 0
64bit Bus
or Byte 7 Byte 6 Byte 5 Byte 4 Byte 3 Byte 2 Byte 1 Byte 0
Register:
MSB LSB
commands are written in CommandLine style. References are marked in the document in
[brackets].
2 General Overview
FRS is an Ethernet switch IP block designed to be used in programmable System on Chip
(SoC) environments. FRS includes multiple Ethernet Media Access Controller (MAC)
functional entities and provides MII/GMII interfaces for Ethernet PHY devices and optionally
for a host system CPU (see Figure 2). FRS also supports MDIO interfaces for the host CPU
to be able access the registers of FRS and for FRS to poll the status of the connected PHY
devices.
MII/GMII
PHY MAC Registers
MII/GMII
MII/GMII
PHY MAC MAC CPU
FRS
Interlink/
maintenance
port
10/100/1000 Mbps
Ethernet Medium
MII/GMII
PHY MAC
S
destination source
802.3 preamble F type data payload CRC
address address
D
64-1518 bytes
14 bytes
1
7 bytes 6 bytes 6 bytes 2B 2B 46-1500 bytes 4B
B
S
destination source VLAN
802.1Q preamble F TCI data payload CRC
address address TPID
D
68-1522 bytes
C
priority F VLAN identifier
I
1
3 bits 12 bits
b
3.1.2 Header
MAC header consists of three parts:
Destination Address field (6 bytes)
The Destination Address specifies whether the recipient is a single node (unicast), a
group of nodes (multicast), or all the nodes at the medium (broadcast). If the first bit is
zero, then the recipient is a single node and this field contains the physical address of
the receiver‟s interface.
Source address field (6 bytes)
The physical address of the interface that originated the frame.
Type/Length field (2 bytes)
The type field indicates the protocol being carried. In the case of IEEE 802.3 LLC, this
field can also be used to indicate the length of the data. If the value of the field is less
than or equal to 1500, then it is indicating the length of the frame.
Ethernet nodes are usually capable of choosing whether they want to receive only frames
sent to their own Ethernet address and the broadcast address, or also frames sent to some,
or all of the multicast addresses. The nodes may also choose to receive all the frames,
including those destined for other nodes. This is called promiscuous mode.
3.1.3 CRC
The 32-bit cyclic redundancy check (CRC) checksum at the end of the frame provides an
error detection mechanism. The CRC checksum is calculated and added to the frame when
the frame is sent, and it is checked when the frame is received at the opposite end. Frames
received with an invalid CRC checksum contain an error and should be discarded by the
receiver.
destination addresses are forwarded by switches to every other port that the source port as
they are supposed to reach every station in the network.
CRC of the received frames and comparing it to the CRC checksum at the end of the frame.
Frames with erroneous CRC checksums should be silently discarded.
The so called cut-through switches that start to forward the frame before it is fully received, do
not conform with the standard in this respect, because they are not able to discard the
received frames if they have an erroneous CRC checksum. The opposite of cut-through
operation is store-and-forward operation, in which case frames are fully received and their
CRC checksums are checked before they are forwarded. This is standard compliant method
of operation.
Interlink
End-node End-node
Redbox
A frame in a ring is always HSR tagged. HSR tag in an Ethernet frame is presented in Figure
6. Tags are added / removed by the nodes connected to ring and ring exterior. Such nodes
are called Redundancy Boxes (RedBox). Source nodes send always two copies (red and
green arrows in Figure 5 and Figure 6) of the original frame (blue arrow) to the ring. The
intermediate nodes in the ring forward the frames and the destination node discards the
duplicate (the frame that arrives later). The duplicate frames are identified by having the same
source MAC address and sequence number. In case the frame travels full round (in case of
unicast this happens when destination is not found, in case of multicast this happens always),
the source node takes care of removing the frame from ring (X-marking in Figure 5).
14 bytes
1
7 bytes 6 bytes 6 bytes 2B 4B 2B 4B
B
HSR S
destination source HSR
frame preamble F HSR PT data payload CRC
address address PT
D
Figure 7. QuadBox
FRS can be used as a part of HSR RedBox and HSR QuadBox design as well as HSR End-
node.
VDAN
SAN
DANP DANP
Redbox
SAN
SAN
LAN_A LAN_B
switch switch
DANP DANP
Each PRP frame contains a PRP trailer as depicted in Figure 9. Trailers are added / removed
by the PRP nodes connected to a PRP network. Normal non-PRP Ethernet switches can be
used to form PRP LAN_A and LAN_B network segments. Redundancy Box (RedBox) can be
used to connect non-PRP (SAN) devices to a PRP network. In PRP protocol the source node
sends always two copies of the original frame, one for both network segments (LAN_A and
LAN_B), and the destination node(s) discards the duplicate (the frame that arrives later). The
duplicate frames are identified by having the same source MAC address and sequence
number. Non-PRP (SAN) devices are also allowed to connect to LAN_A and LAN_B directly
without a RedBox. In that case SANs do not benefit from the redundancy.
14 bytes
1
7 bytes 6 bytes 6 bytes 2B 6B 4B
B
PRP S
destination source PRP
frame preamble F PT data payload CRC
address address trailer
D
4 Functional Description
The functional blocks of FRS are presented in Figure 10. The functionality of FRS can be
controlled via configuration registers accessed using MDIO interface. The configuration
registers are defined in Chapter 5.
FRS
MDIO
Forwarding core
STA
queues
queues
queues
queues
Priority
Priority
Priority
Priority
Outbound
Outbound
Outbound
Outbound
Inbound
Inbound
Inbound
Inbound
MDIO
...
MAC MAC MAC MAC
FRS consists of four main blocks: Forwarding Core, Inbound and Outbound processing and
MDIO controller. The inbound and outbound processing include Ethernet Media Access
Control (MAC).
The Forwarding core is responsible for managing the frames inside the switch. The
forwarding core is common for all the ports and it does the actual forwarding of frames
between ports. As frames may need to spend time inside the switch, they are stored in into a
buffer memory. The forwarding core does the memory management for the buffer memory
and it is also responsible for queuing of the frames.
Every port has its own MAC and inbound and outbound processing. The inbound and
outbound processing of a port is independent from the other ports. The only exception to this
is that the inbound processing entities share the same MAC address table.
MDIO Controller block is used for accessing registers of FRS and the connected external
PHY devices. The PHY devices are connected to STA port of the MDIO controller. A
management CPU can be connected to the MMD port of the MDIO Controller. The CPU can
then access all the configuration registers of FRS and also the registers of the attached PHY
devices through FRS. FRS itself uses the MDIO controller to poll the status of the connected
PHY devices to determine their auto negotiation results.
Inbound processing
Forwarding core
Memory
Inbound RX control Buffer TX Outbound
Buffer
Buffer
processing processing Buffer processing processing
CTRL info
CTRL info
Inbound RX CTRL info TX Outbound
Buffer
Buffer
processing processing Buffer
Buffer processing processing
Buffer
Buffer
Buffer
Buffer
Inbound RX TX Outbound
processing processing processing processing
Outbound processing
HW time
stamping
Figure 11 presents the processing path of a forwarded frame inside FRS. The path contains
Inbound processing, Outbound processing and Forwarding core functions.
Inbound processing contains Ethernet MAC and data processing blocks that are able to
analyze and modify the frame. Every port has its own Inbound processing functionality, that
uses MAC address table common to all ports.
The Forwarding core contains RX and TX processing blocks for controlling Inbound and
Outbound processing blocks. Memory controller block manages the memory used for storing
frames and their control information. Storing of frames is needed, because there can be more
frames forwarded to an output port that what its capacity is. Frame data is stored into a buffer
memory, and the state of the frame is managed via a CTRL info data structure. Every frame is
associated with one CTRL info structure. CTRL info data structure contains all the needed
information about the state of the frame, including the destination port and the whereabouts of
the memory buffers where the frame data is stored. Frames are stored into the buffer memory
in chunks of 512 Bytes. This means that every stored frame consumes N * 512 B of buffer
memory, where N=1…3. When frames are stored into the buffer memory waiting to be
transmitted from an output port, they are in an output priority queue of the output port. The
output priority queues contain pointers to the frames; no actual frame data is moved from
place to another when queuing the frames.
The main function of Outbound processing is to send frames from buffer memory to Ethernet.
It contains functionality for retrieving data from the buffer memory, time stamping functions for
PTP use and Ethernet MAC functionality for sending data to the medium. Every port has its
own individual Outbound processing entity.
Master CLK
FRS
Inbound Outbound
Control
Data
Construct Port mask &
priority info
PTP
Modify Forwarding Control
(port 0 only)
decision Data
Extract
PTP
Detect
HSR Tag
Add
HSR&PRP&V
LAN&Manag MAC
ement Address MGMT
header/trailer
Remove Table Trailer
Add
Inbound
HSR, VLAN,
MAC Address
Policy PTP
info Extract Overwrite
RX pre- TX post-
process MAC process
Clock domain
Time interface
Time interface
RX MII TX MII
RX MII TX MII
CLK CLK
GMII/MII
GMII/MII
FRS Inbound and Outbound processing paths are depicted in Figure 12. The Inbound
processing and Outbound processing functionality are completely independent of each other.
The Inbound processing and Outbound processing data paths are designed so that it is
relatively easy to add more processing blocks to them. Such blocks could collect information
from the frames, modify them and/or affect their forwarding decision. It is also possible to
forward the data path outside of FES, to allow adding of customer‟s own Inbound and
Outbound processing blocks.
4.1.1 RX MII
The RX MII receives frames from the Ethernet PHY. When RX MII is operational and frames
arrive from the network, it writes the frames received from the network to the RX pre-process
block. While the frame is being received, RX MII calculates CRC over the frame. After the
reception of the frame is completed it indicates the status of the CRC calculation. Frames with
invalid CRC are discarded by the forwarding core and MAC RX error counter is incremented.
RX MII block indicates the start of the frame to Timestamp block via Time interface. The
timestamp point for a frame is defined in Figure 13.
S
preamble F
D
dst address ...
timestamp
point
There are four kinds of errors that can occur while RX MII is receiving a frame. These are:
Size Error, CRC Error, Octet Error and Line Error.
Size Error indicates that the received frame is over 1532 bytes long (without
preamble, SFD and CRC). In that case the frame is truncated to 1532 bytes, and Size
Error is generated incrementing MAC RX Error Counter.
CRC Error signals that the CRC checksum in the received frame was not the same
as the one that was calculated while receiving the frame. This is a result of an error in
the data of the received frame and an indication that the frame should be discarded.
Size Error, Octet Error and Line Error usually cause also a CRC error to the received
frame.
Octet Error occurs when the received frame contains an uneven number of half bytes
(nibbles). This kind of a frame is not valid.
Line Error indicates that while receiving the Frame the PHY reported RX MII of an
error.
All the frames received with an error are dropped by the Forwarding Core and the MAC RX
error counter is incremented (see Table 5). Frames whose size is less than 64 bytes are
silently discarded.
4.1.2 Timestamp
The Timestamp block uses the start-of-frame indication from RX MII block to determine the
exact value of the reception time. The reception time of the frame is then given to the RX pre-
process block.
The Timestamp block is external to FRS. FRS communicates with the Timestamp block using
Time interface. Time interface uses time presentation that has accuracy of 2^-16
nanoseconds and timestamp length of 96 bits. The actual accuracy depends on the system
clock frequency used, and the type of clock. Time interface is presented in Chapter 6.3.
Timestamp block is external to FRS because it is quite hardware dependent. The
implementation of the Timestamp block depends on the FPGA family used, available FPGA
resources (PLLs for example) and the timing related hardware external to FPGA; different
kinds of boards have different kinds of clocks, with different accuracies and different
frequencies, fixed and adjustable.
4.1.3 RX Pre-process
The RX pre-process block provides the received frames to the rest of the Inbound processing
blocks. The Inbound processing blocks are chained in a row between the RX pre-process and
the Forwarding core. The inbound processing blocks are connected to each other with a
standard interface called EIF (Extended Interface). The RX pre-process is the source of the
EIF bus and it generates EIF-cycles for each frame it receives from the RX MII.
Regardless of a frame being dropped or not, it is always received to the buffer memory of the
forwarding core, which means that it goes through the whole inbound processing chain. If the
frame is to be dropped the memory resources allocated by the frame are freed right after the
reception.
4.1.5.1 Priority-setting
In inbound policy block the received frames get their priority according to the destination MAC
address.
Inbound policy sets the priorities for the frames according to the priority set in Inbound Policy
Configuration Register (Table 8). The priority is used by the Forwarding core to place the
frames into correct transmit priority queues. Default priority for frames whose priority is not
modified by the Inbound Policy block is 0, which is the lowest priority.
Inbound Policy is able to force the forwarding of the selected frames from any other port
to the management port (see Table 8).
From the management port, it is possible to send Ethernet frames to any other port
independent from other configurations (MAC table etc.).
Management mode configuration includes three steps:
1. Configure the Management port to management mode (see Table 5).
2. Configure the Management port HW mode and speed (see Table 5).
3. Configure which frames to forward to Management port from other ports (see Table
8).
When in management mode, every frame sent and received via a management port is
equipped with a management trailer. FRS inserts a management trailer to a frame when it
sends the frame to a management port and FRS expects every frame received from a
management port to contain one. The management trailer contains information about the
input/output port: from which port has the frame been received or to which port(s) is it to be
sent. An Ethernet frame containing the management trailer is depicted in Figure 14.
Management
Ethernet header Data CRC
Trailer
Bit 7 2 1 0
The length of the Management Trailer is one byte. Every bit in the trailer corresponds to one
port, starting from port number 0 in the least significant bit. Unused trailer bits are reserved
and they are always set to 0 by FRS and they should be written 0 by the user of the
Management Port.
When the host CPU sends a frame via management port to port number 1, it adds a
management trailer to the frame with the bit corresponding to „Port 1‟ set to one and the other
bits in the management trailer set to zero. The host CPU can send a frame to multiple FRS
ports by setting multiple bits in the management trailer. By setting all management trailer bits
to zero, the host CPU lets FRS make the forwarding decision.
When FRS sends a frame to host CPU to the management port (and the management port is
in management mode), it adds a management trailer to the frame always with one of the bits
set. The bit corresponds to the port from which the frame was received.
The MAC address entry contains a USED bit, the actual MAC Address, Expiration time and a
Port number. USED bit set to „1‟ indicates that the entry is currently in use. The port where the
frame was received from is stored into the Port field.
The MAC address aging time is set to Expiration time field. There is an internal counter that
keeps track of the current time, and when the expiration time is updated a value of current
time + address lifetime is written to the Expiration time field. The address lifetime is user
configurable in the register map (see Table 3). The Expiration time value is stored in multiples
of 16 seconds.
Source
address is NO
unicast?
YES
Search address
from table
Update
Address
YES Timestamp and
found?
Port field
NO
YES
Copy MAC
address, set
Timestamp and
Port field
Set USED=1
5. If unused entry is not found, select one of the already used entries.
6. Copy MAC address and Port fields to the selected entry. Set Expiration time to
expiration time field and set USED bit to „1‟.
Get next
Whole table
Wait 16s YES
processed?
NO
If USED = 1 NO
YES
Expiration Time
NO
= current time?
YES
Set USED = 0
The Address Aging process goes through all the MAC Address Table entries every 16
seconds. All the expired entries are removed, in other words, their USED bit is set to „0‟.
Expiration is detected from the Expiration time field of the MAC Address entry: If the
Expiration time is the same as the current time, the entry is to be removed. The address entry
expiration time is updated by the Address Learning process, so only the entries that have not
been refreshed for a certain time (Address lifetime) are removed by the Address Aging
process.
Check destination
address
Unicast? NO
YES
Search address
from table
Address
NO
found?
YES
dst port =
YES receive port
NO
Forward frame to
dst port
every other port
valid in VLAN NO
valid in VLAN
conf.
configuration
YES
Check destination
address
Unicast? NO
YES
Search address
from table
Address
NO
found?
YES
dst port =
YES receive port
NO
Forward frame to
Allowed by
every other port
Forward NO
Forward portmask
Portmask
allows
YES
4.2.1 TX Post-process
The Outbound processing blocks are chained in a row between the Forwarding core and the
TX post-process. The Outbound processing blocks are connected to each other with a
standard interface called EIF (Extended Interface). TX MII controls the rate at which the data
flows through the Outbound processing path and the Forwarding core feeds the data at that
rate. TX post-process transfers data to TX MII block for sending.
4.2.2 TX MII
TX MII block automatically calculates and inserts correct CRC checksums to transmitted
frames. Also the preamble and the Start Frame Delimiter (SFD) are automatically inserted. TX
MII block indicates the start of the frame to Timestamp block via Time interface (see Figure
12). The timestamp point of a frame is defined in Figure 13.
4.2.3 Timestamp
The Timestamp block uses the start-of-frame indication from TX MII block to determine the
exact value of the transmit time. The transmit time is then given to the PTP overwrite block.
The Time-stamp block is external to FRS. FRS communicates with the Time-stamp block
using Time interface. The timestamp block and Time interface at Outbound processing are
similar to the Timestamp block and Time interface at Inbound processing. See more
information at Chapter 4.1.2.
Data_memory
... Fragment Fragment Fragment Fragment ...
Control_memory
... CTRL_info CTRL_info CTRL_info CTRL_info ...
Data
Data
Priority 0
Priority 1
Inbound Outbound
Core RX Core TX
processing processing
Priority 2
Control
Memory
Pointers Priority 3
Priority 0
Priority 1
Inbound Outbound
Core RX Core TX
processing processing
Priority 2
Priority 3
Control
Memory TX queues for Port 2:
Pointers
Priority 0
Priority 1
Inbound Outbound
Core RX Core TX
processing processing
Priority 2
Priority 3
4.3.3 Core RX
Core RX blocks terminate the Inbound processing paths (EIF buses, see Chapter 4.1.3). The
RX Core blocks reserve a CTLR_info structures for the frames from the Control memory and
handle the writing of the frame data to the Data memory. A CTLR_info structure contains the
information on where the data fragments are located in the Data memory. The Core RX block
places a pointer to the CTRL_info struct into correct priority queue of the correct output port(s)
based on the information from Inbound processing. If the queue is full the frame is not
forwarded to that port at all. Note that it is not allowed to place the frame into a priority queue
with lower or higher priority, because the frames belonging to the same data stream in
Ethernet have to preserve their order.
If a frame cannot be forwarded to any destination port because their TX queues are full, the
frame is dropped. The frame is dropped also in case Control memory is full (out of CTRL_info
structures) or Data memory is full. In case Inbound processing indicates that there was an
error in the frame, Core RX block drops the frame and increments the corresponding error
counter. When frames are dropped by Core RX, both the CTRL_info stucture and the
reserved Data_memory are deallocated.
4.3.4 Core TX
Core TX blocks source the Outbound processing paths (EIF buses). A Core TX block gets
information of which frame to transfer to the Outbound processing path from the TX queues of
the corresponding port. The Core TX block then transfers the data from the Data_memory to
the Outbound processing path at the rate the Outbound processing path is able to handle.
This rate depends on the speed setting of the output interface (10/100/1000 Mbit/s).
After transferring the frame to the Outbound processing, it is checked if the frame has to be
transferred to Outbound processing of other ports as well. If this was the last port to forward
to, both the CTRL_info stucture and the reserved Data memory are freed.
Memory low
Select lowest
priority queue P = queue priority (0 is lowest)
(P = 0)
Check number of
frames in the
queue
YES YES
Memory cannot be
Free oldest frame low (this should
from selected never happen)
queue
The Frame Early Drop algorithm is presented in Figure 21. The algorithm is run when buffer
memory resources are running low. The algorithm selects one frame to be dropped and drops
it.
The Frame Early Drop algorithm selects first the output port that has the most frames queued.
Then it selects the lowest priority queue of that port that has queued frames it, and removes
the oldest frame from that queue. After dropping the frame, the appropriate error counter (see
Table 5) is incremented.
SRC found
from MAC tbl?
YES
SRC is
in Interlink YES DROP
port?
NO
Duplicate
detection
NO
Already
Add/update entry
DROP YES relayed by
and store seq
same port?
NO
Already
relayed by the
YES
other ring port
?
NO
Forward to DST,
according to the
Unicast FWD chain
DST is in decision
remove non-HSR port?
from ring
YES NO
FWD
FWD
For HSR frames received from a HSR port, it is first checked if the source MAC address
exists in the MAC address table and if the source node is located in non-redundant (interlink)
port. The duplicate detection is then done by looking at the MAC address entry‟s stored HSR
sequence numbers for the other HSR redundant port: if it matches with the incoming frame‟s
HSR Tag‟s sequence number, we have a duplicate. Additionally, it is checked whether a
frame with this same sequence number and source MAC address, coming in from this same
port has already been forwarded, in which case the frame is circulating in the ring/network
and has to be deleted. If the frame is neither duplicate nor circulating, it is forwarded towards
its destination(s) and the corresponding entry is updated.
Multicast and broadcast frames (see Figure 5) always circulate the whole ring. The duplicate
detection for multicast and broadcast frames is made in two phases: by first looking if the
frame has already been forwarded into this direction. If the answer is yes, the frame already
circulated the whole ring and it is dropped (note that checking whether the source address is
behind a non-HSR port probably drops the frame earlier). The next step is to see whether the
frame was already received from the other redundant port and has therefore already been
forwarded to the interlink ports. If not, the frame is forwarded to all the other ports (except the
input port). Otherwise the frame is forwarded only to the other redundant port.
SRC found
from MAC tbl?
YES
Duplicate
detection
NO
Already
Add/update entry
DROP YES relayed by
and store seq
same port?
NO
Already
relayed by the
YES
other PRP port
?
NO
Add/update entry
and store seq
Forward to DST,
according to the
FWD chain
decision
FWD
For PRP frames received from a PRP port, it is first checked if the source MAC address exists
in the MAC address table. The duplicate detection is then done by looking at the MAC
address entry‟s stored PRP sequence numbers for the other PRP port: if it matches with the
incoming frame‟s PRP trailer‟s sequence number, we have a duplicate. Additionally, it is
checked whether a frame with this same sequence number and source MAC address, coming
in from this same port has already been forwarded. If the frame is not duplicate it is forwarded
towards its destination(s) and the corresponding entry is updated.
The MDIO Controller has two ports: a station management entity (STA) port and an MDIO
Manageable Device (MMD) port. All the Ethernet PHYs are connected to the STA port and
the optional host CPU can be connected to the MMD port (see Figure 24).
CPU
MDIO MDC
MMD
MDIO Controller
STA
MDC
MDIO
By default, MDIO Controller block polls the registers of the PHYs periodically and configures
its interfaces according to their current status. The polled configuration includes Link State,
Speed Selection and Duplex Mode.
If the host CPU wants to access the PHY registers, it has to temporarily disable the polling of
the PHY registers (see Table 3). After disabling the polling feature, the CPU has to check that
the polling is disabled before accessing the PHYs, as it may take a while to disable it. After
accessing the PHY registers, the CPU must enable polling again to allow FRS to access to
the PHY registers. MDIO PHY access from the CPU through FRS is depicted in Figure 25.
Wait until
check MDIO
polling state
polling state
is disabled
read/write MDIO
4.8 Reset
4.8.1 Software Reset
Software reset is made by writing value “1” to the Software reset bit (see Table 3) in General
Register. After reset command FRS cancels all of its current operations and waits until all of
its state machines have returned to their reset states. After the reset has completed, FRS
clears the Software reset bit in the register. After software reset FRS is in the same state as
after hardware reset.
5 Configuration Registers
Functionality of FRS is controlled through registers. The configuration registers of FRS are
accessed using MDIO. Each MDIO access has two addresses: PHY address and Register
address. Both the PHY address and the Register address are 5 bits in length.
When accessing registers of external PHY devices, the PHY address selects the PHY device
to be accessed and the Register address selects the Address inside the selected PHY device.
When accessing registers of FRS, the PHY address selects whether the access goes to FRS
Switch configuration registers or to FRS Port configuration registers (see Figure 26). If
accessing FRS port configuration registers, the PHY Address also defines which port‟s port
configuration registers are accessed.
FRS Configuration registers (FRS Port configuration registers and FRS Switch configuration
registers) are accessed indirectly through FRS MDIO registers as shown in Figure 26. The
FRS MDIO registers include registers PRA (Port Register Address) and PRD (Port Register
Data) into which address and data of FRS Configuration registers (FRS Port configuration
registers and FRS Switch configuration registers) are written when using the indirect access.
PHY Addresses of external PHY devices and FRS ports are configured using external signals
of FRS, see Table 10. Note that the same PHY addresses are used:
when CPU accesses external PHY devices
when CPU accesses FRS Port configuration registers
when FRS accesses external PHY devices
In addition to this there is a separate PHY address the CPU uses when accessing FRS
Switch configuration registers (see Figure 26 and Table 10).
FRS
MDIO
PHY Addr = PHY Addr = PHY Addr =
PORT 0 PHY PORT 1 PHY PORT n PHY
If PHY ADDR ADDR ADDR
MMD polling
enabled
...
STA configuration configuration configuration Configuration
configuration
registers registers for registers for registers for registers
Port 0 Port 1 Port n
MDIO
Write PRA
(address & read enable)
Read register
(address)
(data)
Read PRD
(data)
Indirect write access to FRS Configuration register (FRS Switch configuration register or FRS
Port configuration register) is presented in Figure 28. In indirect write access the address of
the FRS Configuration register is written to FRS MDIO register named PRA and the value for
the FRS Configuration register is written to the FRS MDIO register named PRD. FRS then
immediately writes the value to the FRS Configuration register.
FRS MDIO FRS Configuration
CPU registers register
Write PRD
(data)
Write PRA
(address & write enable)
Note that accessing FRS Port configuration registers is possible only when PHY polling
feature of FRS is enabled. When PHY polling feature is disabled, MDIO access goes to
external PHY devices as presented in Figure 26.
FRS PHY Address (configured with signal cfg_fes_addr(4:0), see Table 10)
Port PHY Addresses (configured with signal cfg_port_addr(n:0)(4:0), see Table 10)
FRS does not respond to accesses with any other PHY addresses. To Port PHY Addresses it
responds only when PHY polling feature is enabled. When doing indirect access using FRS
MDIO registers PRA and PRD, the PHY Address of the MDIO access defines whether the
access goes to FRS Switch configuration register or one of the FRS Port configuration
registers.
PHYIDR1 and PHYIDR2 registers are used to identify FRS. For more information about using
these registers, see IEEE std. 802.3 [8].
The exact formula for the minimum and maximum of the duplicate aging time is the following:
Duplicate aging time (min) = 2 * (AGING_BASE_TIME * Tclk) * (2 ^
HSR_ENTRY_FORGET_TIME);
Duplicate aging time (max) = 3 * (AGING_BASE_TIME * Tclk) * (2 ^
HSR_ENTRY_FORGET_TIME);
, where AGING_BASE_TIME is the Aging Base Time in registers AGING_BASE_TIME_LO
and AGIGNG_BASE_TIME_HI, HSR_ENTRY_FORGET_TIME is the value in HSR
EntryForgetTime bits in ADDRESS_AGING register and Tclk is the period of the Main Clock
(clk).
The exact formula for the minimum and maximum of the MAC entry aging time is the
following:
MAC entry aging time (min) = 6 * (AGING_BASE_TIME * Tclk) * 600 *
ADDRESS_LIFETIME;
6 External Signals
The external signals of FRS are presented in Figure 29. The external signals consist of
MII/GMII signals, MDIO signals, Time interface signals and General signals.
Symbols alingment
tx_ alignment
The tri-state buffers for MDIO signals are external to FRS (see Figure 29), because tri-state
signals are normally not used inside an FPGA. The tri-state buffer is used only when the
MDIO signal is wired outside of the FPGA. It is not used if the signal is wired to another block
inside the FPGA.
6.1 Generics
The compile time configuration is presented in Table 9.
Generic name Default value Description
Active edge of reset signal. The allowed values are 0 (active
RST_ACTIVE 1
low) and 1 (active high).
PHY id of FES. Readable from registers PHYIDR1 and
FES_PHY_ID 0x 0000 0000
PHYIDR2 (see Table 1).
Clock divider length for sta_mdc clock. This is used for
sta_mdc clock when FRS is polling the PHYs.
MDC_DIV_MSB 5
Typical value: 5 => Fsta_mdc = Fclk/ 64
Allowed values: >2
Width of external time data bus. Common setting for Rx and
TIME_WORD_MSB 95
Tx. The only supported value is 95
Number of Ethernet ports minus one.
Allowed values: 2 to 7
3-port FRS: use 2
FES_PORT_HIGH 3
4-port FRS: use 3
5-port FRS: use 4
…
HSR_PORT_OPT 0 „0‟: All the ports have HSR/PRP capability
„1‟: Only two ports have HSR/PRP capability
When HSR_PORT_OPT=‟1‟, this defines which port is
HSR_PORT_A 1 HSR/PRP redundant port A. The value of HSR_PORT_A
cannot be the same as the value of HSR_PORT_B
When HSR_PORT_OPT=‟1‟, this defines which port is
HSR_PORT_B 2 HSR/PRP redundant port B. The value of HSR_PORT_B
cannot be the same as the value of HSR_PORT_A.
FPGA device family, for example “ECP3”, “Stratix II GX” or
DEVICE_FAMILY “”
“Spartan 6”
rx_clk
rx_dv
rxd(3:0)
x x
rxd(7:4)
rx_er
FRS creates tx_en, txd and tx_er on rising edge of tx_clk and
registers crs and col on rising edge of tx_clk
(NOTE: crs and col effective only in half-duplex mode) T tx_clk
tx_rst
tx_clk
tx_en
txd(3:0) x
txd(7:4)
tx_er
crs
col
Manual
mmd_mdio write: FRS registers mmd_mdio_in signal on rising edge of mmd_mdc*
mmd_mdc
Preamble
Idle SFD OP code PHY address Register address Turn around Data Idle
(32 ‟1's)
* exact registering moment of mmd_mdio_in is 2-4 clk_1x cycles after rising edge of mmd_mdc due to synchronization to clk_1x domain
* mmd_mdio_ena stays logic ’0'
mmd_mdio read operation: FRS registers mmd_mdio_in signal on rising edge of mmd_mdc* and
generates mmd_mdio_out with rising edge of mmd_mdc
63 (68)
mmd_mdc
mmd_mdio_in 0 1 1 0 A4 A3 A0 R4 R3 R0 Z
Preamble Turn
Idle SFD OP code PHY address Register address Data Idle
(32 ‟1's) around
mmd_mdio_ena
* exact registering moment of mmd_mdio_in is 2-4 clk_1x cycles after rising edge of mmd_mdc due to synchronization to clk_1x domain
Version 2.10
FRS
Manual
sta_mdio write operation: FRS generates sta_mdc and sta_mdio_out signal on rising edge of sta_mdc
sta_mdc
Preamble
Idle SFD OP code PHY address Register address Turn around Data Idle
(32 ‟1's)
64 (68)
sta_mdc
sta_mdio_out 0 1 1 0 A4 A3 A0 R4 R3 R0 Z
Preamble Turn
Idle SFD OP code PHY address Register address Data Idle
(32 ‟1's) around
sta_mdio_ena
Version 2.10
FRS
125MHz clk
generation
gtx_clk (GMII)
FRS MAC
tx_clk clk_
mux
tx_clk (MII)
PHY
gtx_clk_sel
rx_clk
clk_mux
rx_clk gtx_clk_sel
tx_clk rx_clk
gtx_clk tx_clk
Table 15 presents functionality of clk_mux for MII/GMII clocks in MII and GMII modes.
7 Glossary
8 References
[1] “Flexibilis Ethernet Switch/Router Specification”, Flexibilis Oy, 2011
[2] IEEE Std 802.3x-1997, “Specification for 802.3 Full Duplex Operation and Physical
Layer Specification for 100 Mb/s Operation on Two Pairs of Category 3 Or Better
Balanced Twisted Pair Cable (100BASE-T2)”, 1997
[3] IEEE Std 802.1D, “Media Access Control (MAC) Bridges”, 2004.
[4] IEEE Std 1588-2008: IEEE Standard for a Precision Clock Synchronization Protocol
for Networked Measurement and Control Systems, 2008.
[5] Rich Seifert, “The Switch Book, The complete guide to LAN switching technology”,
John Wiley & Sons inc, 2000, page 299
[6] IEEE Std 802.1w-2001, “Media Access Control (MAC) Bridges: Amendment 2 –
Rapid Reconfiguration“, 2001
[7] IEC62439-3 , “Industrial communication networks - High availability automation
networks - Part 3: Parallel Redundancy Protocol (PRP) and High-availability
Seamless Redundancy (HSR)”, 2010, Clause 5
[8] IEEE Std 802.3-2008, “Carrier Sense Multiple Access with Collision Detection
(CSMA/CD) Access Method and Physical Layer Specifications”, 2008