x

Context Aware Offload

NOV 2019
RONI BAR YANAI
MELLANOX TECHNOLOGIES
Agenda

• RTE_FLOW Match:Action API

• Multi flow table
• TAG a new way of context aware offload

2
RTE_FLOW Match:Action
Using RTE_FLOW API user can define HW rule
HW is consisted of list of matches and a list of actions

• Match packet header fields

Match Eth: Match Ipv4: Match UDP:
Fields/mask Fields/mask Fields/mask
• Can include few layers (tunneling)
• Partial match with mask
• Support priority
• Port based

• Can have a list of actions

Count Decap tunnel Output
• Action should have packet fate
• Support NAT/DECAP/RSS and more
• Can jump to another table

3
RTE_FLOW UDP Match Code Example

struct rte_flow_item_udp {
struct udp_hdr hdr;
};

4
RTE_FLOW Output Action Example

• API receive an array of actions. Action has a type and pointer to data
• In presented example, there must be additional action to steer packet

5
Offload HW Table

HW Table dpdk port X • RX: searching for a match

• On a match execute actions
• On a miss packet goes to software
Packet

• Rule can be applied on VF or PF

• In switch mode (transfer), packet can be
steered between VF’s
• TX has also a table
• In switch mode, there is option for table rules
and switch rules

6
Context Aware MULTI TABLE OFFLOAD USE
CASE
Offload
Multi-Table Logic

• Some control plane use multi-table architecture

• Each table is consisted of match->action rules
• Jump action is used to define the next table
• Next table is not restricted to phase (tree)

recirc_id(0),in_port(5),ct_state(-trk),eth_type(0x0800),ipv4(frag=no), packets:4, bytes:300,

used:2.230s, flags:P., actions:ct,recirc(0x9)

recirc_id is logical table

8
Use of Multiple Table

• Flexibility, running different actions on matching different fields

• Logical separation, multi stage processing:
• Forwarding VS connection tracking
• Classification
• Rules can be more generic
• All use cases can be handled by a single rule:
• Add complexity to control
• Total number of rules might increase significantly

9
Multi-Stage processing might use context

Classifier Forwarding QoS

I’m a Video
packet
1. Packet is classified
2. Forwarding can be NAT,MAC…etc.
3. QoS will use the classification to enqueue

Classification ID is not part of the packet

Classification should be stored on classifier phase and will be used on QoS

10
Multi-Stage processing

• Table 1 can use data that was set by Table 0

Table 0 Table 1
A1: Set classification 1

M(1,0) A1,A2,JMP 2 M(1,1): classification == 1

M(1,1) A1,A2,Output 1
M(2,0) A1,A2,JMP 3 M(2,1) drop
M(3,1) A1,A2,Output 2
M(4,1) A1,A2,JMP 2

11
Multi-Stages Compile to a single rule
Solution for offload where control is multi-table and hardware can’t support
• We might run logic that compiles multiple rules into a single one.
• Compilation is packet driven

Table 0 Table 1

M(1,0) A1,A2,JMP 2 M(1,1) A1’,A2’,Output 1

M(2,0) A1,A2,JMP 3 M(2,1) drop
M(3,1) A1’,A2’,Output 2
M(4,1) A1’,A2’,JMP 2

M1(1,0)+M(1,1):A1,A2,A1’,A2’

12
Multi-Stages Compile to a Single Rule

• Not following control plane

• Requires complex logic of knowing when to add or remove flow
• Counting is different
• Combined flow correctness is not always trivial. M1,A1 -> M2,A2 !=
M1+M2,A1+A2
• In most cases requites exact match (outer+inner 5-tuple)
• Some use cases combine different logic, for example CT and forwarding
• Probably need per packet processing

13
Multi-Stage VS Single-Stage

• It is a tradeoff
• Right approach is use case dependent
• Generalization
• Memory
• PPS (Multi-Table usually has more processing per packet)
• Number of rules and update rate
• Open Flow (OVS) use multi-stage processing

14
Context Aware RTE_FLOW TAG
Offload
RTE_FLOW TAG

• Generic API action for setting meta data on packet in HW

• Generic API matching for meta data on packet in HW

TAG is consisted of:

• Data (32 bit) struct rte_flow_item_tag {
• Index (0-X, defined by HW) uint32_t data;
uint8_t index;
TAG matching is maskable };
• Can be used with bit granularity
struct rte_flow_action_set_tag {
uint32_t data;
TAG0 TAG1 TAG N uint32_t mask;
uint8_t index;
};

32bits
16
 TAG is generic

Classifier Forwarding QoS

Index = 1
data = VIDEO
match_cls.data = VIDEO
cls_tag.data = cls;
match_cls.index = CLS_INDEX
cls_tag.mask = CLS_MASK;
cls_tag.index = CLS_INDEX;
m_match_cls.data = CLS_MASK
m_match_cls.index =CLS_INDEX

Using TAG we can implement the full pipeline

17
TAG limitations
• Limited length
• Limited resource and HW dependent
• Can have impact on performance
• Software logic can be hard to adjust in some cases

Example:
• OVS Tunnel:
• OVS saves the outer ip and tunneling information in packet meta data
• For IPv6 we will need 4 such actions per IP and 1 for VNI, total of 9 actions

• OVS Tunnel offload using TAG

• Mapping between (ip,ip,vni) to 3-tuple id
• Underlay number of nodes is limited to few thousands and not on the same time
• Single TAG can replace 9

18
OVS CT Meta Data

match 5-tuple + zone MD: zone, state, mark, label

• CT entry holds meta data

• On a hit, packet meta data is written.
• Meta data fields can be matched on following rules
• The meta data cannot be stored in the packet (another header for example)
• Limits offloading
State and zone are matched
Example of matching on meta data:
recirc_id(0),in_port=2,ct_state(+est),ct_zone(1),ip,action=1

recirc_id(0x7),in_port(1),ct_state(+est+rpl+trk),ct_label(0/0x1),……

State and label are matched

19
OVS CT hardware offload using TAG

match 5-tuple + zone MD: zone, state, mark, label

• Part of the HW actions on CT table rules is to set the meta data

• Information is encoded into TAGs (TAG is generic)
• Few fields can be encoded into a single TAG, for example zone (16 bit) and zone (8 bit)
• Fields can be reduced, for example 128 bit can be reduced to 32 (mapping or trimming)
• The common use cases can be supported
• Offloading is following OVS DP flows

20
 OVS CT Meta Data

Match 5-tuple + zone MD: zone, state, mark, label

Set TAG, index:CT_STATE_INDEX, VAL:ct_state,
CT_STATE_MASK

Set TAG index:CT_ZONE_INDEX, VAL:zone, MASK ZONE_MASK

recirc_id(0),in_port(5),ct_state(-trk), eth_type(0x0800),
ipv4(frag=no), actions:ct(zone=1),recirc(0x7) JMP 0x7

Match Actions:OUTPUT:1

Match TAG, index:CT_STATE_INDEX,VAL:+est,mask:CT_STATE_MASK

recirc_id(0x7),in_port=2,ct_state(+est),ct_zone(1),ip,action=1
Match TAG index:CT_ZONE_INDEX, VAL:1, mask:CT_ZONE_MASK

21
TAG Summary
• Application dependent
• 5G UPF can have QFI
• OVS can use for CT state/zone…etc
• Allow multie-stage processing
• Match and action on different field
• Multi-Stage logic
• Will be supported on 19.11

22
Q&A

Q&A

23
THANKS!