P4 Tutorial
P4 Tutorial
Slides: https://bit.ly/p4d2-2018-spring
Software Setup
◦ Download VM or copy from USB stick
◦ Import VM into VirtualBox or VMware Fusion
◦ Boot VM and login as user "p4" with passwd "p4"
◦ Open Terminal
◦ cd ~/tutorials; git pull
Copyright © 2018 – P4.org
Goals
• Learn P4 Language
◦ Traditional applications
◦ Novel applications
• Learn P4 software tools
◦ P4 Compiler
◦ BMv2
◦ P4Runtime
• Learn about future technology trends
◦ Keynote by Arvind Krishnamurthy (Washington)
◦ Panel with representatives from ON.Lab, Kaloom, and Keysight
• Networking (the other kind)
• Have fun!
Copyright © 2018 – P4.org
Introduction to Data Plane Programming
Language Basics
Break
Software Tools & P4 Runtime
Lunch
Keynote (Arvind Krishnamurthy)
Monitoring & Debugging
Break
Advanced Data Structures
Panel Discussion
Reception
Copyright © 2018 – P4.org
Instructors
• Stanford
• Sedef Ozcana
• Instructors
• Arvind Krishnamurthy
• Instructors
• TAs
◦ Sean Choi
◦ Sarah Tollman
◦ Santiago Bautista
• You!
• Issues Southbound
◦ Data-plane protocol evolution requires changes to I/F
standards (12 → 40 OpenFlow match fields)
◦ Limited interoperability between vendors
(OpenFlow / netconf / JSON / XML variants) xn
◦ Limited programmability
Switch OS
Network Demands
?
“This is how I know to
Run-time API
Driver
process packets”
(i.e. the ASIC datasheet
makes the rules)
Fixed-function ASIC
Copyright © 2018 – P4.org 9
A Better Approach: Top-down design
Switch OS
Network Demands
Feedback
Run-time API
Driver
P4
[1] Miao, Rui, et al. "SilkRoad: Making Stateful Layer-4 Load Balancing Fast and Cheap Using Switching ASICs." SIGCOMM, 2017.
[2] Handley, Mark, et al. "Re-architecting datacenter networks and stacks for low latency and high performance.” SIGCOMM, 2017.
[3] Kim, Changhoon, et al. "In-band network telemetry via programmable dataplanes.” SIGCOMM. 2015.
[4] Xin Jin et al. “NetCache: Balancing Key-Value Stores with Fast In-Network Caching.” To appear at SOSP 2017
[5] Jin, Xin, et al. "NetChain: Scale-Free Sub-RTT Coordination." NSDI, 2018.
[6] Dang, Huynh Tu, et al. "NetPaxos: Consensus at network speed.” SIGCOMM, 2015.
[7] Sapio, Amedeo, et al. "In-Network Computation is a Dumb Idea Whose Time Has Come." Hot Topics in Networks. ACM, 2017.
State machine,
Parsers bitfield extraction
Tables, Actions,
Controls control flow
statements
Basic operations
Expressions and operators
Bistrings, headers,
Data Types structures, arrays
Term Explanation
Community-Developed Vendor-supplied
P416
Language
P416 Core
Library
Extern Architecture
Libraries Definition
TM
SimpleSumeSwitch
TM
TM Anything
Control Plane
RUNTIME
P4 Program P4 Compiler Add/remove Extern Packet-in/out
table entries control
CPU port
P4 Architecture Target-specific Extern
configuration Load Tables Data Plane
Model objects
binary
Target
Vendor supplied
Traffic
Manager
h1 h2
(10.0.1.1) (10.0.2.2)
s1 s2
s3
h3
(10.0.3.3)
Copyright © 2018 – P4.org 30
P416 Types (Basic and Header Types)
typedef bit<48> macAddr_t; Basic Types
typedef bit<32> ip4Addr_t;
header ethernet_t { • bit<n>: Unsigned integer (bitstring) of size n
macAddr_t dstAddr; • bit is the same as bit<1>
macAddr_t srcAddr;
• int<n>: Signed integer of size n (>=2)
bit<16> etherType;
} • varbit<n>: Variable-length bitstring
header ipv4_t {
bit<4> version;
Header Types: Ordered collection of members
bit<4> ihl;
bit<8> diffserv; • Can contain bit<n>, int<n>, and varbit<n>
bit<16> totalLen; • Byte-aligned
bit<16> identification;
bit<3> flags;
• Can be valid or invalid
bit<13> fragOffset; • Provides several operations to test and set validity bit:
bit<8> ttl; isValid(), setValid(), and setInvalid()
bit<8> protocol;
bit<16> hdrChecksum;
ip4Addr_t srcAddr; Typedef: Alternative name for a type
ip4Addr_t dstAddr;
}
/* User program */
struct metadata {
...
}
struct headers {
ethernet_t ethernet;
ipv4_t ipv4;
}
(DataPlane)
Parameters
Directional
Headers and Metadata
Lookup Key
Hit
Key Action ID Action Data
Action
ID
Action ID
Code
Selector
Hit/Miss
(Action Data)
Directionless
Parameters
Data
(DataPlane)
Parameters
Directional
action NoAction() { types of parameters
} ◦ Directional (from the Data Plane)
◦ Directionless (from the Control
/* basic.p4 */ Plane)
action drop() { • Actions that are called directly:
mark_to_drop(); ◦ Only use directional parameters Action
} • Actions used in tables: Code
(Action Data)
Directionless
Parameters
action ipv4_forward(macAddr_t dstAddr, ◦ May sometimes use directional
bit<9> port) { parameters too
...
}
Action
Execution
h1 h2
1 1
(10.0.1.1) 2 2 (10.0.2.2)
(dst_id: 1) s1 3 3 s2 (dst_id: 2)
2 3
1
s3
h3
(10.0.3.3) (dst_id: 3)
Copyright © 2018 – P4.org 51
Coding Break
Program-independent
Control Server
p4c-bm2-ss test.json
L
o
TM g
simple_switch (BMv2)
Egress
Ingress
P4
D Debugger
test.json e
b
u
g
Parser Deparser
Packet Packet
generator Port Interface sniffer
veth0..n
Linux Kernel
Copyright © 2018 – P4.org 56
Step 1: P4 Program Compilation
p4c-bm2-ss
test.json
test.jso
test.json veth
n 0 2 4 2n
Linux
Kernel
2n
veth 1 3 5 +1
Program-independent
Control Server
L
test.json
TM o
g
g
i
n
Egress
BMv2
Ingress
g
test.jso
test.json
n
Parser Deparser
veth0..n
Linux Kernel
Copyright © 2018 – P4.org 59
Step 4: Starting the CLI
$ simple_switch_CLI BMv2 CLI
Program-independent
CLI and Client
test.p4 Program-independent
Control Server
L
test.json
TM o
g
g
i
n
Egress
BMv2
Ingress
g
test.jso
test.json
n Parser Deparser
Port Interface
veth0..n
Linux Kernel
Copyright © 2018 – P4.org 60
Working with Tables in simple_switch_CLI
RuntimeCmd: show_tables
m_filter [meta.meter_tag(exact, 32)]
m_table [ethernet.srcAddr(ternary, 48)]
Program-independent
Control Server
L
test.json
TM o
g
g
•scapy i
p = Ethernet()/IP()/UDP()/”Payload” n
•scapy
Egress
BMv2
Ingress
g
sendp(p, iface=“veth0”) sniff(iface=“veth9”, prn=lambda x: x.show())
•Ethereal, etc.. •Wirehark, tshark, tcpdump
Packet Packet
Generator Sniffer
Parser Deparser
Port Interface
veth 0 2 4 2n
Linux
Kernel 2n
veth 1 3 5 +1
64
Copyright © 2018 – P4.org, ONF
P4 Software Tools
65
Copyright © 2018 – P4.org, ONF
Makefile: under the hood
P4Runtime static controller
runtime.json
P4Runtime client
test.p4
P4Runtime server
p4c-bm2-ss L
simple_switch_grpc (BMv2)
o
TM g
Egress
Ingress
P4
D Debugger
test.json test.p4info e
b
u
g
Parser Deparser
Packet Packet
generator Port Interface sniffer
veth0..n
Linux Kernel
66
Copyright © 2018 – P4.org, ONF
Step 1: P4 Program compilation
test.json test.p4info
67
Copyright © 2018 – P4.org, ONF
Step 2: Preparing veth Interfaces
Linux
Kernel
2n
veth 1 3 5 +1
68
Copyright © 2018 – P4.org, ONF
Step 3: Starting BMv2
$ sudo simple_switch_grpc --log-console --dump-packet-data 64 \
–i 0@veth0 -i 1@veth2 … [--pcap] \
test.p4 -- --grpc-server-addr 0.0.0.0:50051 --cpu-port 255 \
test.json
p4c-bm2-ss L
simple_switch_grpc (BMv2)
o
TM g
Egress
Ingress
D
test.json test.p4info e
b
u
g
Parser Deparser
Port Interface
veth0..n
Linux Kernel
69
Copyright © 2018 – P4.org, ONF
Step 4: Starting P4Runtime static controller
$ python $(RUN_SCRIPT) -t $(TOPO) P4Runtime static controller
runtime.json
$(run_args) P4Runtime client
simple_switch_grpc (BMv2)
o
TM g
p4c-bm2-ss
Egress
Ingress
D
e
b
u
g
Port Interface
veth0..n
Linux Kernel
70
Copyright © 2018 – P4.org, ONF
P4Runtime
• API overview
• Workflow
• Exercise - Tunneling
• Example - ONOS
71
Copyright © 2018 – P4.org, ONF
Runtime control of P4 data planes
User supplied
Control Plane
P4 Program P4 Compiler Add/remove Extern Packet-in/out
table entries control
CPU port
P4 Architecture Target-specific Extern
configuration Load Tables Data Plane
Model objects
binary
• BMv2 CLI
◦ Program-independent, but target-specific -- control plane not portable!
• OpenFlow
◦ Target-independent, but protocol-dependent -- protocol headers and
actions baked in the specification!
74
Copyright © 2018 – P4.org, ONF
Properties of a runtime control API
P4 compiler
auto-generated ✔ ✘
BMv2 CLI ✘ ✔
OpenFlow ✔ ✘
SAI ✔ ✘
P4Runtime ✔ ✔
75
Copyright © 2018 – P4.org, ONF
What is P4Runtime?
• Framework for runtime control of P4 targets
◦ Open-source API + server implementation
■ https://github.com/p4lang/PI
◦ Initial contribution by Google and Barefoot Control plane
77
Copyright © 2018 – P4.org, ONF
gRPC Basics
● Use Protocol Buffers to define service API and messages
● Automatically generate native stubs in:
○ C / C++
○ C#
○ Dart
○ Go
○ Java
○ Node.js
○ PHP
○ Python
○ Ruby
● Transport over HTTP/2.0 and TLS
○ Efficient single TCP connection implementation that supports
bidirectional streaming
78
Copyright © 2018 – P4.org, ONF
gRPC Service Example
// The greeter service definition.
service Greeter {
// Sends a greeting
rpc SayHello (HelloRequest) returns (HelloReply) {}
}
80
Copyright © 2018 – P4.org, ONF
P4Runtime Service
Protobuf Definition:
https://github.com/p4lang/PI/blob/master/proto/p4/v1/p4runtime.proto
Service Specification:
Working draft of version 1.0 is available now
https://p4.org/p4-spec/docs/P4Runtime-v1.0.0.pdf
81
Copyright © 2018 – P4.org, ONF
P4Runtime Write Request
message WriteRequest { message Entity {
uint64 device_id = 1; oneof entity {
uint64 role_id = 2; ExternEntry extern_entry = 1;
Uint128 election_id = 3; TableEntry table_entry = 2;
repeated Update updates = 4; ActionProfileMember
} action_profile_member = 3;
ActionProfileGroup
action_profile_group = 4;
message Update { MeterEntry meter_entry = 5;
enum Type { DirectMeterEntry direct_meter_entry = 6;
UNSPECIFIED = 0; CounterEntry counter_entry = 7;
INSERT = 1; DirectCounterEntry direct_counter_entry = 8;
MODIFY = 2; PacketReplicationEngineEntry
DELETE = 3; packet_replication_engine_entry = 9;
} ValueSetEntry value_set_entry = 10;
Type type = 1; RegisterEntry register_entry = 11;
Entity entity = 2; }
} }
82
Copyright © 2018 – P4.org, ONF
P4Runtime Table Entry
p4runtime.proto simplified excerpts:
message TableEntry { message FieldMatch {
To add a table entry, the control
uint32 table_id; uint32 field_id; plane needs to know:
repeated FieldMatch match; message Exact {
Action action; bytes value;
int32 priority; } • IDs of P4 entities
... message Ternary { ◦ Tables, field matches, actions,
} bytes value; params, etc.
bytes mask;
}
message Action { ... • Field matches for the
uint32 action_id; oneof field_match_type { particular table
message Param { Exact exact;
◦ Match type, bitwidth, etc.
uint32 param_id; Ternary ternary;
bytes value; ...
}
} • Parameters for the particular
}
repeated Param params; action
}
83
Copyright © 2018 – P4.org, ONF
P4Runtime workflow
P4Info test.p4
test.p4info
p4runtime.proto
• Protobuf-based format
P4Runtime server
• Target-independent compiler output test.json
BMv2 driver
◦ Same P4Info for BMv2, ASIC, etc.
simple_switch
86
Copyright © 2018 – P4.org, ONF
P4Runtime SetPipelineConfig
test.
message SetForwardingPipelineConfigRequest {
p4
enum Action {
UNSPECIFIED = 0; Pipeline config
VERIFY = 1; p4info json Control plane
VERIFY_AND_SAVE = 2;
VERIFY_AND_COMMIT = 3; p4c-bm2-ss SetPipelineConfig()
COMMIT = 4; (compiler)
Pipeline config bits
RECONCILE_AND_COMMIT = 5;
} P4Runtime server
uint64 device_id = 1;
BMv2 driver
uint64 role_id = 2;
Uint128 election_id = 3; BMv2 simple_switch_grpc
Action action = 4;
ForwardingPipelineConfig config = 5;
} message ForwardingPipelineConfig {
config.P4Info p4info = 1;
// Target-specific P4 configuration.
bytes p4_device_config = 2;
}
87
Copyright © 2018 – P4.org, ONF
P4Runtime StreamChannel
// Packet sent from the controller to the switch.
message StreamMessageRequest { message PacketOut {
oneof update { bytes payload = 1;
MasterArbitrationUpdate // This will be based on P4 header annotated as
arbitration = 1; // @controller_header("packet_out").
PacketOut packet = 2; // At most one P4 header can have this annotation.
} repeated PacketMetadata metadata = 2;
} }
88
Copyright © 2018 – P4.org, ONF
P4Runtime Common Parameters
• device_id
◦ Specifies the specific forwarding chip or software bridge
◦ Set to 0 for single chip platforms
• role_id
◦ Corresponds to a role with specific capabilities (i.e. what operations, P4
entities, behaviors, etc. are in the scope of a given role)
◦ Role definition is currently agreed upon between control and data planes
offline
◦ Default role_id (0) has full pipeline access
• election_id
◦ P4Runtime supports mastership on a per-role basis
◦ Client with the highest election ID is referred to as the "master", while all
other clients are referred to as "slaves"
◦ Set to 0 for single instance controllers
89
Copyright © 2018 – P4.org, ONF
Mastership Arbitration
• Upon connecting to the device, the client (e.g. controller)
needs to open a StreamChannel
• The client must advertise its role_id and election_id using a
MasterArbitrationUpdate message
◦ If role_id is not set, it implies the default role and will be granted full
pipeline access
◦ The election_id is opaque to the server (e.g. Stratum) and determined
by the control plane (can be omitted for single-instance control plane)
• The switch marks the client for each role with the highest
election_id as master
• Master can:
◦ Perform Write requests
◦ Receive PacketIn messages
◦ Send PacketOut messages 90
Copyright © 2018 – P4.org, ONF
Remote control
table_entry {
table_id: 33581985
Target-independent P4-defined
match { protobuf format OSPF BGP etc.
field_id: 1
lpm {
custom protocol
value: "\f\000\...
prefix_len: 8
p4info
}
}
action {
Remote control plane
action_id: 16786453
params {
param_id: 1
value: "\000\0...
}
params {
param_id: 2
value: 7
}
}
}
91
Copyright © 2018 – P4.org, ONF
Local control
table_entry {
Same table_id: 33581985
match {
P4-defined
etc.
target-independent field_id: 1
OSPF BGP lpm {
protocol protobuf format value: "\f\000\...
prefix_len: 8
p4info }
Local control plane }
action {
action_id: 16786453
params {
param_id: 1
p4info value: "\000\0...
P4Runtime }
params {
control server param_id: 2
value: 7
}
}
Target driver }
P4 target
94
Copyright © 2018 – P4.org, ONF
Exercise Overview
Controller’s responsibilities:
1. Establish a gRPC connection to the
switches for the P4Runtime service mycontroller.py
2. Push the P4 program to each switch
3. Write the tunnel forwarding rules:
a. myTunnel_ingress rule to
encapsulate packets on the ingress
switch
b. myTunnel_forward rule to
forward packets on the ingress
switch
c. myTunnel_egress rule to
decapsulate and forward packets
on the egress switch
4. Read the tunnel ingress and egress
counters every 2 seconds
95
Copyright © 2018 – P4.org, ONF
Getting started
The source code has already been downloaded on your VM:
~/tutorials/exercises/p4runtime
96
Copyright © 2018 – P4.org, ONF
Lab 3: Monitoring & Debugging
Congestion!
Fine … h2
h1
(10.0.1.1) (10.0.2.2)
h11 h22
(10.0.1.11) s1 s2 (10.0.2.22)
h3
(10.0.3.3)
s1
Copyright © 2018 – P4.org 99
Explicit Congestion Notification
TOS
Traffic
Manager
Ingress Egress
s1 s2
h22
h11 (10.0.2.22)
(10.0.1.11)
h1 2 2 1 h2
(10.0.1.1) 1
(10.0.2.2)
s1 3 3 s2
2 3
1 s3
109
h3
(10.0.3.3)
Copyright © 2018 – P4.org
Source Routing: Packet Format
#define MAX_HOPS 9 • Parse source routes only if etherType is
0x1234
const bit<16> TYPE_IPV4 = 0x800;
const bit<16> TYPE_SRCROUTING = 0x1234;
header srcRoute_t { • The special value bos == 1 indicates the
bit<1> bos; “bottom of stack”
bit<15> port;
}
• Forward packets using source routes,
struct headers { and also decrement IPv4 TTL
ethernet_t ethernet;
srcRoute_t[MAX_HOPS] srcRoutes; • Drop the packet if source routes are not
ipv4_t ipv4; valid
}
2+4=?
h1 s1
(10.0.1.1) 2+4=6
h1 h2
(10.0.1.1) (10.0.2.2)
s1 s2
Virtual IP (10.0.0.1)
shared by h2 and h3
s3
h3
(10.0.3.3)
Copyright © 2018 – P4.org 114
Hashing (V1Model)
enum HashAlgorithm {
csum16,
Computes the hash of data
xor16, (using algo) modulo max
crc32,
crc32_custom, and adds it to base
crc16,
crc16_custom,
random, Uses type variables (like
identity
}
C++ templates / Java
extern void hash<O, T, D, M>( Generics) to allow hashing
out O result,
in HashAlgorithm algo, primitive to be used with
in T base, many different types.
in D data,
in M max);
• Consortium of academic
and industry members
• Membership is free:
contributions are welcome
• Independent, set up as a
California nonprofit