Innovations in Programmability and

Automation
On Catalyst IOS-XE Platforms
Renata Correa
Systems Engineer

June 2023

© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
 Introduction & Overview
IOS XE Device 1 Intro to Programmability
Programmability Innovations 17.x

Programmability
Model Driven Telemetry
TIG_MDT Container Update

New Features & Infrastructure

2 gRPC Tunnel for gNMI
PROTO encoding for gNMI GET/SET
RFC8572 Secure ZTP

Agenda and Topics

YANG Model Innovations
OpenConfig Innovations

Tooling
3 YANG Suite 3.0: SNMP-to-YANG mappings
Terraform for 2023 end-to-end IPsec use case

4 Resources & Closing

© 2019 Cisco and/or its affiliates. All rights reserved. Cisco Confidential

© 2023 Cisco Systems, Inc. All rights reserved.

© 2023 Cisco Systems, Inc. All rights reserved.
 Cisco IOS XE Programmability Innovations
17.1 17.3 17.5 17.7 17.9 17.11
API RBAC with RFC NACM gNMI/YANG SetReplace enhancements for gRPC Tunnel
Zero Touch Provisioning ZTP timing improvements gRPC Dial-Out + mTLS IAC
gNXI CLI simplification NETCONF PKI certificate gNOI reset.proto PROTO encoding for
Guest Shell Python API Conf-D version upgrade
authentication gNMI subscribe
Python 2 to 3 transition gNOI cert.proto Certificate CLI “show run” to YANG XML & AAA Method List for NETCONF/RESTCONF
RFC8572 Secure ZTP
Embedded Event Manager Management API NETCONF performance JSON EVPN YANG Service Module
Guest Shell DNS
Guest Shell HA “folder sync”
RESTCONF Cheat Sheet gNOI cert.proto bootstrapping enhancement for large GET Wireless: YANG Model for CLI RPC, enhancements
ConfD 7.1 stricter XML gNMI wildcard support • YANG for AP & client oper, gRPC Dial Out + mTLS YANG models for
gNMI mixed-schema support Wireless BLE streaming YANG
YANG On-Change models namespace enforcement Mesh RPC Syslog RPC & PTP &
OpenConfig Wireless end-of-support
gRPC Dial-Out with TLS gNOI OS.proto Operating System YANG 1.0 to 1.1 planning
EVPN
API OpenConfig YANG model updates Network
OpenConfig YANG:
NETCONF wait-on-lock Instance & BGP LACP

Cisco IOS XE Programmability Release Timeline

17.2 17.6 17.8 17.10
Guest Shell shared folder NETCONF API from Guest Shell gRPC Dial-Out with Mutual TLS Guest Shell DNS enhancement
ZTP DHCP Client Behavior NETCONF certificate auth CLI Routing: for cloud secret management
simplification • IOS XE Operational IPv6 support for gNMI
gRPC Dial-Out FQDN DNS Consistency: gNMI support YANG 1.1 D-Day
support on Routing platforms OpenConfig YANG: BGP
YANG model for on-change
telemetry
Wireless:
• Telemetry at scale for
periodic & on-change
• Fast Path and Gather Point
architecture for leaf-level
filtering
© 2023 Cisco Systems, Inc. All rights reserved.
 Automate the Device Network Lifecycle
GOAL:
Get device/s into an operational state GOAL:
Apply configuration to the device
Provisioning Automation Tools:
Day 0
PXE Tools:
ZTP
Install YANG Config Data Models
PnP Programmable Interfaces
Python Scripting Python Scripting
Day 1
Day N
Configure
Upgrade & Operate
GOAL:
GOAL: Add dynamic services, optimize
Continuously upgrade network, behavior and trouble shooting
incrementally and safely Day 2
Tools:
Tools: Optimize YANG Operational Data Models
Patching Telemetry
Config / Replace App Hosting

© 2023 Cisco Systems, Inc. All rights reserved.

 IOS XE Programmability & Automation Lifecycle
Pre-boot Execution Network Configuration Protocol
Provisioning Model Driven
Environment (iPXE) (NETCONF), RESTCONF, gNMI
Automation Programmability
Device
Onboarding
RFC8572 Secure Zero Touch YANG “native” Data Models,
Provisioning ZTP OpenConfig,
Day 0
VM Automation YANG Suite, Terraform, Ansible, pyATS
Device tooling
Configuration

Day N Intent-based Day 1

Network Infrastructure

gNOI cert/os/reset proto Device

Optimization
Guest Shell + Python/NETCONF Day 2 TIG_MDT container + guide
Software Image Model Driven
CentOS 8 Python 3 Management Telemetry YANG On-Change support
Device
Application Hosting with Docker Monitoring gRPC Dial-Out + DNS + mTLS

“show run” CLI to XML gNMI/NETCONF Dial-In

© 2023 Cisco Systems, Inc. All rights reserved.
 Programmable Interfaces
CLI
The NETCONF, RETCONF and gNMI are programmatic interfaces that provide additional methods for
SNMP
interfacing with the IOS XE device – Just like the CLI, SNMP, and WebUI is used for configuration changes
and operational metrics so can the programmatic interfaces of NETCONF, RESTCONF and gNMI
WebUI

NETCONF RESTCONF gNMI

gNMI
YANG data models define the data that NETCONF RESTCONF

is available for configuration and YANG

YANGData Models
Data Models
streaming telemetry OpenConfig Cisco Native
OpenConfig Cisco Native
Intent-based
Network Infrastructure
Configuration
Configurationand
and Operation
Operation

Device
DeviceFeatures
Features
SNMP
SNMP
Interface
Interface
BGP
BGP QoS
QoS ACL
ACL …
…

© 2023 Cisco Systems, Inc. All rights reserved.

 Day 0

Day N Intent-based Day 1

Network Infrastructure

IOS XE - YANG Model Coverage on GitHub Day 2

RFC7950 states that “YANG is a data

modeling language used to model
configuration data, state data, Remote
Procedure Calls, and notifications for
network management protocols”

YANG module name.yang Description

Cisco-IOS-XE-native running-config

Cisco-IOS-XE-{feature}-cfg Feature configuration

Cisco-IOS-XE-{feature}-oper Feature operational data

Cisco-IOS-XE-{feature}-rpc Actions

Cisco-evpn-service EVPN service abstraction

OpenConfig-{feature} abstraction for config & oper

https://github.com/YangModels/yang/tree/master/vendor/cisco/xe

© 2023 Cisco Systems, Inc. All rights reserved.

 IOS XE Architecture
Cisco
Cisco IOSIOS XE &17.x
XE 16
IOS Control Plane CAF / IOX
IOS Sub
IOS Sub Docker C8Kv
IOSd Systems
IOS
Systems
sub-systems Docker Guest Shell

Common Infrastructure & HA

Management Interfaces

Module Drivers IOS XE

Kernel
DB
Protected Memory

© 2023 Cisco Systems, Inc. All rights reserved.

 IOS XE Programmability Architecture
IOS XE programmable architecture gNMI GET/SET/Subscribe call flows

gNMI client
WebUI Cisco and Non-Cisco controllers gnmib process confd process

RPC GET JSON <--> XML

SUBSCRIBE SET
Model
CLI/ YANG Mapping
Web Shell NETCONF/RESTCONF Cisco Native pubd process

API
No – standard path
Gather-Point
mapped path JSON <--> XML
Proxy ?
Confd IOS Config DB Yes – Config
JSON <-->
fast path DB
convert data

IOS-XE from internal

structure Shared Memory
DB access
process
CLI Oper + Config
Crimson DBs
WSMA BinOS Apps IOSd (BGP,OSPF, etc) Sync

Config Application Processes

IOSd Oper
(POE, wireless etc)

© 2023 Cisco Systems, Inc. All rights reserved.

 Model Driven Telemetry Interfaces
Dial In: Collector establishes a connection to the device then subscribes to telemetry (pub/sub)

Dial Out: Telemetry is pushed from the device to the collector based off configuration (push)

Publication / Subscription
Di HT DDi ia
al TP gR a l

Di
ov

PC l In
er
GE

al
In Tu In
T

Ou
nn
el

t
NETCONF
NETCONF
RESTCONF
RESTCONF gNMI
gNMI gRPC
gRPC XML, JSON, proto and kvGPB
YANG Data Models encoding
YANG Data Models

Intent-based Open
OpenConfig NativeNative
Cisco
Network Infrastructure
Consistent YANG data models
between interfaces
Configuration
Configurationand
andOperation
Operation

On-change event and time-

Device Features
Device Features
based publication options
SNMP
Interf ace BGP QoS ACL … SNMP
Interface BGP QoS ACL …
© 2023 Cisco Systems, Inc. All rights reserved.
 IOS XE Model Driven Telemetry
CLI
Cisco IOS XE …or with…
YANG

gNMI Dial-In/Dynamic gRPC Dial-Out/Configured

NETCONF Dial-In
Collector/Receiver
Decodes to text

Storage
Time Series Database

Monitoring
and Visualizations
https://hub.docker.com/r/jeremycohoe/tig_mdt https://github.com/jeremycohoe/cisco-ios-xe-mdt
https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/prog/configuration/179/b_179_programmability_cg/m_179_prog_ietf_telemetry.html
© 2023 Cisco Systems, Inc. All rights reserved.
 Updated TIG_MDT container now available! Updated April 2023
Upgrade coming to Telegraf, Influx, and Grafana Model Driven Telemetry
(TIG_MDT) Docker container
Making it easier to consume telemetry in production

Upgraded Telegraf, InlfuxDB, and Grafana tools

Additional dashboards for
Device Health, Wireless Client, Wireless AP, RF etc
Examples for device CLI configuration for telemetry
Details of scale and data storage requirements
docker pull jeremycohoe/tig_mdt
docker run -ti -p 3000:3000 -p 57500:57500 jeremycohoe/tig_mdt

Collector/Receiver
Decodes to text
Storage
Time Series Database
Monitoring
and Visualizations

https://hub.docker.com/r/jeremycohoe/tig_mdt https://github.com/jeremycohoe/cisco-ios-xe-mdt
https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/prog/configuration/179/b_179_programmability_cg/m_179_prog_ietf_telemetry.html
© 2023 Cisco Systems, Inc. All rights reserved.
 Model Driven Telemetry Interface Comparison
NETCONF gRPC (Dial-Out) gNMI

Minimum IOS XE Version 16.6 16.10 Dial-In: 16.12

over gRPC tunnel: 17.11
Network
Recommended Version 17.9 17.9 Dial-In: 17.9 architecture,
over gRPC tunnel 17.11 security posture
Telemetry Direction Dial-In, Dial-Out Dial-In and policy, YANG
IOS XE is server IOS XE is client IOS XE is server data modules,
Dial-Out tools and
gRPC Tunnel language
preferences are
Configuration Dynamic Static Dynamic
some
per session per configuration per session
considerations
Telemetry Collector Client Server Client when leveraging
Encoding XML KV GPB JSON_IETF + PROTO the various MDT
interfaces
Security SSH + PKI mTLS or plain-text mTLS certificates
mTLS cert only or
certificate or password mTLS cert + user/pass authentication

Transport Protocol SSH HTTP2 HTTP2

Data Models YANG YANG YANG
© 2023 Cisco Systems, Inc. All rights reserved.
 Cisco Telemetry Data Broker (Telegraf)
Cisco Telemetry Broker provides many benefits include brokering, filtering, and transforming data. It provides
the ability to replicate telemetry data.

• Cisco Secure Network Analytics (Stealthwatch) UDP Director (UDPD) replicates UDP traffic to
multiple destinations.
• Cisco Telemetry Broker
• Builds upon UDPD
• Optimizes telemetry pipelines for the hybrid cloud
• Simplifies the consumption of telemetry data for customers’ business-critical tools by
brokering hybrid cloud data, filtering unneeded data, and transforming data to a usable
format

https://cs.co/telemetrybroker aka https://www.cisco.com/c/en/us/products/security/telemetry-broker/index.html

https://blogs.cisco.com/security/taking-full-control-of-your-telemetry-with-the-intelligent-telemetry-plane

© 2023 Cisco Systems, Inc. All rights reserved.

 Model Driven Telemetry: usage comparison
60-minute collection sample with 60-second update interval
Interface CPU PCAP file Data byte Data bit Average Average
Impact size/data Rate rate Packet Packet Size
size (MB) Rate (sec) (bytes)
gNMI +3% 23 MB 6 kBps 53 kbps 5 1180
gRPC +3% 69 MB 19 kBps 155 kbps 58 333
NETCONF +2% 83 MB 23 kBps 185 kbps 29 780
RESTCONF +4% 200 MB 35 kBps 281 kbps 37 945 17 xpaths collected at 60 second update interval
/arp-ios-xe-oper:arp-data
/cdp-ios-xe-oper:cdp-neighbor-details
SNMP * +6% 120 / 87 24 kBps 197 kbps 90 273 /environment-ios-xe-oper:environment-sensors
/if:interfaces-state
/interfaces-ios-xe-oper:interfaces/interface
/ios:native
/lldp-ios-xe-oper:lldp-entries
/matm-ios-xe-oper:matm-oper-data
/mdt-oper:mdt-oper-data/mdt-subscriptions
/memory-ios-xe-oper:memory-statistics/memory-statistic
/oc-if:interfaces/interface/state/counters
ch
NETCONF gNMI gRPC /oc-sys:system
/platform-ios-xe-oper:components
/poe-ios-xe-oper:poe-oper-data/poe-switch
/process-cpu-ios-xe-oper:cpu-usage/cpu-utilization
/process-memory-ios-xe-oper:memory-usage-processes

This demonstrates that even when SNMP is only measuring Interfaces the load is still significantly higher
than YANG which is measuring significantly more YANG data
© 2023 Cisco Systems, Inc. All rights reserved.
 Wireless – Programmable support
Platform x feature EWC C9800-CL C9800-L C9800-40/80
ZTP / Guest Shell N/A N/A 17.8 17.3.2a
17.7*
(data port only)

NETCONF 16.12 16.10 16.12 16.10

RESTCONF 16.12 16.11 16.12 16.11
gNMI N/A 17.8 Enabled 17.8

gNOI cert.proto N/A Enabled Enabled Enabled

gNOI factory reset N/A N/A N/A 17.7.1

NETCONF Dial-In MDT 16.12 * Enabled Enabled Enabled
gRPC Dial-Out MDT N/A Enabled Enabled 17.1

gNMI Dial-In MDT N/A Enabled Enabled Enabled

N/A, Not Available Enabled, not TAC or BU Supported since release

supported feature 17.11.1
* NETCONF EWC MDT @ https://www.cisco.com/c/en/us/td/docs/wireless/controller/ewc/16-12/config-guide/ewc_cg_16_12/network_monitoring.html
Confirmed
© 2023 Cisco Systems, Inc.accurate per https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/prog/configuration/1710/b_1710_programmability_cg.html
All rights reserved.
 Routing - Programmable support
Platform x feature ISR 1000 ISR 4000 C8300 CSR 1000v C8KV ASR ASR 1000 C8500 /
C8200 1000 Fixed Modular C8500L
ZTP / Guest Shell 16.9 16.5 17.3 17.2 17.4 16.7 16.8 17.3
NETCONF 16.8 16.3 17.3 16.3 17.4 16.3 16.3 17.3
RESTCONF 16.8 16.6 17.3 16.6 17.4 16.6 16.6 17.3
gNMI 17.8 17.8 17.2 17.2 17.3
gNOI cert.proto

gNOI OS.proto

gNOI reset.proto

NETCONF Dial-In 16.8 16.7 17.3 16.10 17.4 16.7 16.8 17.3
gRPC Dial-Out 16.10 16.10 17.3 16.10 17.4 16.10 16.10 17.3
gNMI Dial-In MDT 17.8 17.8 17.2 17.2 17.3

N/A, Not Available Enabled, not TAC or BU Supported since release

supported feature
© 2023 Cisco Systems, Inc. All rights reserved.
 Switching – Programmable support
Platform x Feature C3650/ 3 C9200CX C9200L C9200 C9300L C9300/ C9500 C9500H C9600
850 9400
ZTP / Guest Shell 16.5 17.8 16.12 16.12 16.6 16.6 16.8 16.12
NETCONF 16.5 17.8 16.9 16.9 16.9 16.6 16.6 16.8 16.11
RESTCONF 16.7 17.8 16.9 16.9 16.9 16.7 16.7 16.8 16.11
gNMI 17.8 16.12 16.12 16.12 16.8 16.8 16.10 16.11
gNOI cert.proto

gNOI OS.proto

gNOI reset.proto

NETCONF Dial-In 16.6 17.8 16.9 16.9 16.9 16.6 16.6 16.8 16.11
gRPC Dial-Out MDT 17.8 16.10 16.10 16.10 16.10 16.10 16.10 16.11
gNMI Dial-In MDT 17.8 16.12 16.12 16.12 16.12 16.12 16.12 16.12

N/A, Not Available Enabled, not TAC or BU Supported since release

supported feature 17.11.1
© 2023 Cisco Systems, Inc. All rights reserved.
 New Programmable Features
and
Infrastructure Updates

© 2023 Cisco Systems, Inc. All rights reserved.

 gRPC Tunnel

aka gNMI tunnel.proto

© 2023 Cisco Systems, Inc. All rights reserved.

 gRPC tunnel
“grpctunnel is an implementation of a TCP-over-gRPC tunnel”
It is very similar to the commonly used “SSH tunnel” concept
https://github.com/openconfig/grpctunnel
Network Device Collector
• The devices makes a secure outbound gnmib
connection to the gRPC tunnel server in order to gRPC core
gNMI Client
gNMI
expose the gNMI API for operational use
• Many devices can connect into a single tunnel Cert Listen
server in order to increase operational efficiency Port
Sockets
• Tunnels can be opened to any number of servers OS
gRPC
as needed and is not limited to a single tunnel Tunnel core
Tunnel Client
Client Ports

Sockets
Tunnel Server

© 2023 Cisco Systems, Inc. All rights reserved.

 17.11
gRPC tunnel

gRPC Dial-Out has seen wide adoption because of the dial-out

architecture. gNMI Subscribe has advantages and now also
HT supports Dial-Out with the “grpc tunnel” proto. The grpctunnel
Di TP Di tooling is an implementation of a TCP-over-gRPC tunnel,
al al

Di
+
gR
In GE In

al
PC
T Tu
written in Go.

Ou
nn
el

t
Customer request: “to create a transparent, bi-directional TCP-
NETCONF RESTCONF gNMI gRPC over-gRPC tunnel supporting gNMI services and other potential
YANG Data Models
protocols such as gNOI in the future”

OpenConfig Cisco Native

gNMI Subscribe is sent within the gNMI tunnel to the device as

Configuration and Operation well as all other operations including GET / SET and gNOI
proto for certificate, reset, and operating system management
Device Features
https://github.com/openconfig/grpctunnel/blob/master/proto/tunnel/tunnel.proto
SNMP
https://github.com/openconfig/grpctunnel
Interface BGP QoS ACL …

© 2023 Cisco Systems, Inc. All rights reserved.

 gRPC Tunnel
Network Device, Cisco IOS XE Tunnel Server Host
GNMIB Network Device, Cisco IOS XE Tunnel Server Host
gNMI Client GNMIB
gRPC core
gNMI gNMI Client
gRPC core
gNMI
Cert Listen
Port Cert Listen
Port Sockets
OS
OS
gRPC
Tunnel Tunnel core
Client
Client Ports

Sockets
Tunnel Server

Without gRPC Tunnel, the gNMI client must connect With gRPC Tunnel the Cisco IOS XE device will
directly into the Cisco IOS XE gNMI listening port. create an outbound connection to the Tunnel
Server Host. From there, the gNMI client can
connect into the Cisco IOS XE device within the
Q: What is the gNMI Client: tunnel.
A: YANG Suite’s gNMI plugin can be used as the gNMI client for
GET/SET/Subscribe – however YANG Suite does not yet support gRPC
Tunnel Server
Q: What is the gNMI Tunnel Server Host software?
A: Currently gNMIc is the recommended tooling for tunnel server testing
and validation
© 2023 Cisco Systems, Inc. All rights reserved.
 gRPC Tunnel quick start - configuration
#1
1. Install gnmic tooling bash -c "$ (curl -sL https://get-
2. Configure gnmic for tunnel server gnmic.openconfig.net)"
3. Enable the gnmic tunnel server #2
4. Enable gNMI API on IOS XE 17.11+ ! tunnel_server_config.yaml
5. Configure and enable gRPC tunnel
insecure: false
#4 conf t skip-verify: true
gnxi log: true
gnxi secure-init username: admin
service internal password: put_yours_here
gnxi secure-allow-self-signed-trustpoint
tunnel-server:
#5 address: ":4000"
gnxi grpctunnel dest ubuntuvm target-wait-time: "10s"
address 10.1.1.3
port 4000 #3
!source-vrf Mgmt-vrf gnmic --config ./tunnel_server_config.yaml
enable --use-tunnel-server subscribe
gnxi grpctunnel target GNMI_GNOI --path system/config/hostname -i 10s
enable --stream-mode sample

Match the port 4000 between the tunnel-server and the grpctunnel config
Update the IP, VRF, and credentials as needed for the environment
Secure gNXI configuration example included with self-signed certificates that are not validated
© 2023 Cisco Systems, Inc. All rights reserved.
 gRPC Tunnel quick start - example
1. Install gnmic as tunnel server
2. Configure gnmic for tunnel service
3. Enable the gnmic server
4. Enable gNMI API #1
5. Configured and enable gRPC tunnel

#2

#4
#3

© 2023 Cisco Systems, Inc. All rights reserved.

#5
 gNMIc as Tunnel Server tooling - example
gnmic \
--config ./tunnel_server_config.yaml \
--use-tunnel-server subscribe \
--path system/config/hostname \
-i 10s \
--stream-mode sample

! tunnel_server_config.yaml

insecure: false
skip-verify: true
log: true
username: admin
password: put_yours_here

tunnel-server:
address: ":4000"
target-wait-time: "10s"

© 2023 Cisco Systems, Inc. All rights reserved.

 gRPC Tunnel quick start - validation
show run | s gnxi
sh gnxi grpc dest

© 2023 Cisco Systems, Inc. All rights reserved.

 PROTO encoding for gNMI
GET/SET

© 2023 Cisco Systems, Inc. All rights reserved.

 17.11
PROTO encoding for gNMI GET/SET
In addition to JSON_IETF encoding, there is also support for PROTO encoding for
gNMI GETs and SETs.
The PROTO encoding mechanism uses the binary encoding format for both path and value to increase the
efficiency of telemetry data transfer. With JSON_IETF the aggregated data is sent to the collector and with
PROTO, there is more granularity in the transmitted data.
Both Telegraf and YANG Suite already support PROTO
The “proto” encoding restriction will be
removed from the guide as it will become
supported in release 17.11

https://github.com/jeremycohoe/cisco-ios-xe-mdt/blob/master/telegraf-gnmi-proto.conf#L5
https://github.com/openconfig/gnmi/blob/master/proto/gnmi/gnmi.proto
https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/prog/configuration/179/b_179_programmability_cg/m_178_prog_gnmi.html
© 2023 Cisco Systems, Inc. All rights reserved.
 PROTO encoding for gNMI GET/SET example

© 2023 Cisco Systems, Inc. All rights reserved.

 PROTO encoding for gNMI GET/SET demo

© 2023 Cisco Systems, Inc. All rights reserved.

 RFC8572 (SZTP)

Secure Zero Touch Provisioning

© 2023 Cisco Systems, Inc. All rights reserved.

 Secure ZTP blog coming soon !

Classic ZTP Overview

IOS XE Device
1. When an IOS XE device (powers on)
boots and no configuration
is present, the device will
issue a DHCP request on the
management port and on
the front panel port.

2. If the DHCP response

contains option 67 then ZTP
is initiated and the device
will retrieve and execute
the python script from
Intent-based
within the Guest Shell Network Infrastructure
Guest Shell
(CentOS 8)
3. Guest Shell is started and Python / YANG / CLI / EEM APIs
networking is automatically
configured IOS XE

ZTP
Complete

https://www.youtube.com/watch?v=EAXnftG6odg
https://blogs.cisco.com/developer/device-provisioning-with-ios-xe-zero-touch-provisioning
https://devnetsandbox.cisco.com/RM/Diagram/Index/f2e2c0ad-844f-4a73-8085-00b5b28347a1?diagramType=Topology
© 2023 Cisco Systems, Inc. All rights reserved.
 Secure ZTP (SZTP) workflow (based on RFC8572)
DHCP Server
ZTP Artifacts
ZTP Server
• Image Download
(Bootstrap Server 1) • CLI Configuration
• ZTP script:
6 Ø Native Python scripts
13
4/v6 143/ Web Server Ø Native bash scripts,
v Restconf Server
CP ion Ø Golang/c++/ pyinstaller
l DH opt (Artifacts) binaries
ti a w
Ini – Ne
e s
g
ssa
Me
1

2 M
ion
Bo ult
eract o ip
de l in t se tst le
AN G mo rv rap
ZTP Y er
IOS-XE s

d
wnl oade
rti fa ct do
A
Reference: https://tools.ietf.org/html/rfc8572
© 2023 Cisco Systems, Inc. All rights reserved. 36
 17.11
RFC8572 Secure ZTP
RFC details: https://www.rfc-editor.org/rfc/rfc8572.html
1. Conveyed Information: used to encode the redirect information and onboarding information (switch config)
2. Ownership Certificate: used by a device to verify the signature over the conveyed information
3. Ownership Voucher: used to verify a device owner as defined by the manufacturer (from the MASA)

Classic Zero Touch Provisioning Secure Zero

Secure ZeroTouch Provisioning
Touch Provisioning RFC8572 (2019)
Device
Turn on
DHCP Server
DHCP Discovery Bootstrapping
(RESTCONF)
Option 143 (136), URL List Server
TLS Handshake; SUDI Client Certificate Validate client
Server Certificate using SUDI

get-bootstrapping-data using YANG-modeled RPC POST Request

Bootstrapping Artifacts:
• Ownership Voucher
• Owner Certificate
• Conveyed Information

Some security requirements for classic ZTP are resolved using Secure ZTP: Validate Server

Update Image Information

• Management system needs to validate the device Ownership Ownership
Configuration Scripts
Conveyed
• Device needs to validate the server Device Trust
Anchor
Voucher Certificate Information NETCONF Configuration
(RFC 8366)
• Device must validate the data is what server sent
Encoded Redirect Traffic
and Onboarding
Information Guestshell

As part of the SZTP RFC, the device supports image upgrade as part of the conveyed information

© 2023 Cisco Systems, Inc. All rights reserved.

 Bootstrapping Server
1. Currently there are no open-source
bootstrapping servers freely available,
but one can easily be written in
Python by following the RFC and API
documentation
Secure Zero Touch Provisioning RFC8572 (2019)
Device
Turn on

2. YANG Suite + Secure ZTP plugin on DHCP Discovery

DHCP Server
Bootstrapping
(RESTCONF)
Option 143 (136), URL List

road map
Server
TLS Handshake; SUDI Client Certificate Validate client
Server Certificate using SUDI

get-bootstrapping-data using YANG-modeled RPC POST Request

3. We will soon be publishing a

Bootstrapping Artifacts:
• Ownership Voucher
• Owner Certificate
• Conveyed Information

bootstrapping server for testing and Validate Server

Update Image Information

validation purposes Device Trust

Anchor
Ownership
Voucher
Ownership
Certificate
Conveyed
Information
Configuration Scripts
NETCONF Configuration
(RFC 8366) Encoded Redirect Traffic
and Onboarding
Information Guestshell

© 2023 Cisco Systems, Inc. All rights reserved.

 Workflows
Secure Zero Touch Provisioning RFC8572 (2019)
Device
Turn on
Single bootstrap DHCP Server
server without DHCP Discovery Bootstrapping
(RESTCONF)
validation. Option 143 (136), URL List Server
TLS Handshake; SUDI Client Certificate Validate client
Redirect to Server Certificate using SUDI
bootstrap server get-bootstrapping-data using YANG-modeled RPC POST Request
with trust anchor
Bootstrapping Artifacts:
validation. • Ownership Voucher
• Owner Certificate
• Conveyed Information
Validate Server

Update Image Information

Configuration Scripts
Ownership Ownership Conveyed
Device Trust Voucher Certificate NETCONF Configuration
Anchor Information
(RFC 8366) Encoded Redirect Traffic
and Onboarding

API call to MASA Information Guestshell

© 2023 Cisco Systems, Inc. All rights reserved.

 MASA and Certificate Signing for OV
(Manufacturer Authorized Signing Authority)

The upstream cloud-based certificate verification service at

https://masa.cisco.com is available for IOS XE Ownership
Voucher (OV) signing workflows

MASA API Docs @ https://masa.cisco.com/docs

Reference XR @ https://xrdocs.io/automation/tutorials/setting-up-crosswork-for-sztp/

© 2023 Cisco Systems, Inc. All rights reserved.

 YANG Model Innovations

© 2023 Cisco Systems, Inc. All rights reserved.

 17.12
YANG Model Innovations
Operational: Multicast Source
Cisco-IOS-XE-msdp-oper
Discovery Protocol
Operational Data
RPC:
Cisco-IOS-XE-logging-ios-actions renamed from exec-ios-actions

Event Notifications:
Cisco-IOS-XE-spanning-tree-events
Cisco-IOS-XE-logging-ios-actions
renamed from
Cisco-IOS-XE-exec-ios-actions-rpc
In order to programmatically generate Syslog notification messages

Cisco-IOS-XE-spanning-tree-events
newly introduced event notifications for STP interface guard events

© 2023 Cisco Systems, Inc. All rights reserved.

 YANG Model Innovations – Product Analytics
https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst9300/software/release/17-11/release_notes/ol-17-11-9300/whats_new_in_cisco_ios_dublin_1710x.html

Config:
Cisco-IOS-XE-cloud-services-cfg

Remote Procedure Calls:

Cisco-IOS-XE-cloud-services-rpc

Feature Models, part of Cisco-IOS-XE-native:

Cisco-IOS-XE-pae

© 2023 Cisco Systems, Inc. All rights reserved.

 YANG Model Innovations - OpenConfig
Cisco IOS XE is committed to support open standards-based
models including OpenConfig by the continually enhancing and
providing new model coverage for campus use cases

OpenConfig provides a consistent and cross vender solution to

configure and manage network devices.

© 2023 Cisco Systems, Inc. All rights reserved.

 Tooling
YANG Suite
Terraform

© 2023 Cisco Systems, Inc. All rights reserved.

 Cisco YANG Suite

• YANG API Testing and Validation Environment

• Construct and test YANG based APIs over NETCONF,

RESTCONF, gRPC and gNMI
Now Generally Available !
• IOS XE / IOS XR / NX OS platforms
developer.cisco.com/yangsuite

YANG Suite TDM . PPT in SalesConnect github.com/CiscoDevNet/yangsuite

https://pubhub.devnetcloud.com/media/yang-suite/docs/YANGSuite%20TDM%20-%20Communities.pdf

© 2023 Cisco Systems, Inc. All rights reserved.

 Dive Deeper into YANG Suite

125-slide TDM in PPT & PDF

Published internally + partners on SalesConnect
Published externally for public as PDF on Github website
https://github.com/CiscoDevNet/yangsuite/

YANG Suite TDM . PPT in SalesConnect

https://pubhub.devnetcloud.com/media/yang-suite/docs/YANGSuite%20TDM%20-%20Communities.pdf

© 2023 Cisco Systems, Inc. All rights reserved. TDM = Technical Decision Maker
 What’s Included
Core plugins
• Initial Release:
• Plugin Manager
• YANG File Manager
• Device Manager
• NETCONF (Python), gRPC Telemetry
• Docker install support with HTTPS
• Second Release:
• RESTCONF
• gNMI
• Python Integrations Additional plugins
• Third Release:
• gRPC Telemetry with TLS Support
• SNMP OID to YANG Xpath Mapping
• Ansible Integrations
• Pip install support
© 2023 Cisco Systems, Inc. All rights reserved.
 SNMP OID to YANG Xpath demo
• This plugin allows mapping of SNMP OID’s to Xpaths
• Engineering resources are currently mapping various feature Xpath’s to OID’s
• This mapping will be included with YANG Suite in future releases

Typical SNMP workflow: YS SNMP Workflow:

1. Find the OID
1. Do a GET for the OID
2. Ex) For CPU utilization
2. Find the YANG model
3. Find the Xpath
4. Do a GET for the Xpath

DevNet Snack Minute Demo:

https://youtu.be/zVsOO9_6rAU?t=
537

© 2023 Cisco Systems, Inc. All rights reserved.

 FY
I
Model Driven Telemetry benefits over SNMP
• devices stream data based on a specified frequency or upon state change
• data is sent as soon as it is available, reducing the need to buffer
• no single large request for all data (unlike SNMP polling)
• data sent incrementally, e.g., only for those data items that have changed
• ability to distribute the telemetry sources (e.g., directly to linecards)
• users issue subscription requests via RPC for data of interest
• data exported in a well-structured, common format, e.g., based on YANG models
• device and collector communicate over a secure, authenticated, reliable channel

© 2023 Cisco Systems, Inc. All rights reserved.

 YANG Suite Resources
Blogs YouTube Videos

https://youtu.be/ https://www.youtube https://www.youtube

smrhjL5Ayz0 .com/watch?v=dTun3 .com/watch?v=3zmN
https://blogs.cisco.com/developer/2 Dfn8b38
3611JA
023yangsuiteupdatesfeatures01

https://blogs.cisco.com/develop
er/363-yangsuite-01

https://www.youtube.co https://www.youtube
m/watch?v=PkbAOzZ1vN https://www.youtube.c
om/watch?v=soyWPr0f .com/watch?v=zVsOO
k 9_6rAU
J0s

https://blogs.cisco.com/developer/l
everageyangsuite01?dtid=osscdc00 Additional Resources
0283
https://blogs.cisco.com/developer/ https://github.com/CiscoDevNet/yangsuite/
yangallthetime01 https://developer.cisco.com/yangsuite/
https://eurl.io/#MaW78CelS YANG Suite General (external)

© 2023 Cisco Systems, Inc. All rights reserved.

 Terraform

© 2023 Cisco Systems, Inc. All rights reserved.

 Terraform is…
Terraform uses the RESTCONF API

NETCONF RESTCONF gNMI

YANG Data Models

Open-source Infrastructure as Code (IaC) Software Tool
OpenConfig Cisco Native
providing a consistent CLI workflow to manage hundreds of
cloud services. Terraform codifies cloud APIs into declarative
configuration files.
Configuration and Operation
• Cloud Native Tooling circa 2014 from HashiCorp
• Agentless, single binary file
• Zero server-side dependencies
Device Features
Resources: SNMP
Ask IOS XE Terraform Provider Webex space: https://eurl.io/#PtsT8eJFl
GitHub Provider Examples: https://github.com/CiscoDevNet/terraform-provider-iosxe/ Interface BGP QoS ACL …
Provider Binary: https://registry.terraform.io/search/providers?namespace=CiscoDevNet
Go Client: https://github.com/CiscoDevNet/iosxe-go-client
Blogs at https://blogs.cisco.com/tag/terraform

Terraform TDM . PPT on SalesConnect

© 2023 Cisco Systems, Inc. All rights reserved.
 Blog and Resources: Terraform
https://github.com/CiscoDevNet/terraform-provider-iosxe/
https://registry.terraform.io/search/providers?namespace=CiscoDevNet

https://salesconnect.cisco.com/#/content-detail/fa072157-
b099-494b-8ec5-2522c6ab2bf6

Intro to IOS XE Terraform Provider Video:

https://www.youtube.com/watch?v=GEY_hyXimbA

Questions? Join the Ask IOS XE

Terraform Provider Webex space:
https://eurl.io/#PtsT8eJFl

Demo Create a Crypto Tunnel Video:

https://www.youtube.com/watch?v=bPS0bhPacDw https://blogs.cisco.com/developer/terraformiosxe01

© 2023 Cisco Systems, Inc. All rights reserved.

 Closing Resources

© 2023 Cisco Systems, Inc. All rights reserved.

 http://cs.co/apiwp

API White Paper

Website: https://www.cisco.com/c/en/us/products/collateral/switches/catalyst-9300-series-switches/nb-06-catalyst-programmability-automation-wp.html
PDF: https://www.cisco.com/c/en/us/products/collateral/switches/catalyst-9300-series-switches/nb-06-catalyst-programmability-automation-wp.pdf
Webinar with live demos & examples: https://www.youtube.com/watch?v=LdcK5PnPu2I
http://cs.co/apiwppdf
© 2023 Cisco Systems, Inc. All rights reserved.
 Cisco IOS XE Programmability – Booksprint Book
http://cs.co/programmabilitybook OR https://www.cisco.com/c/dam/en/us/products/collateral/enterprise-networks/nb-06-ios-xe-prog-ebook-cte-en.pdf

Table of Contents
Authors Telemetry
Acknowledgments Overview
About this Book Operational Data
Introduction Flow Data
Why Programmability Matters Use Cases
Lifecycle of Network Device Operations Subscription Tools
Use Cases Data Collectors
Operational Approaches Python
Next Steps Overview
General Concepts Python WebUI Sandbox
Cisco IOS XE On-Box Python
What is Programmability? Advanced On-Box Python
Application Programming Interfaces (APIs) Common Issues
Programming Languages Guest Shell
Structured Data Introduction
Data Encoding Formats Security
Day 0 Device Onboarding Confuration and Updates
Introduction Resource Allocation
Zero-Touch Provisioning (ZTP) Scenarios Use Cases
Basic ZTP Workow Next Steps
Advanced ZTP Workows Application Hosting
Considerations Introduction
Next Steps Cisco Application-Hosting Framework
YANG Containers and Virtual Machines
Overview Use Case
YANG Concepts Next Steps
YANG Native vs Open Data Models Controllers
YANG Data Model Highlights Introduction
YANG Tools Common Controllers
Network Device APIs Why Use a Controller?
Overview DevOps and NetDevOps
NETCONF Introduction
RESTCONF Continuous Integration and Delivery
Comparison of NETCONF and DevOps Tools
RESTCONF Next Steps
Next Steps Appendices
Additional Resources
Acronyms

© 2023 Cisco Systems, Inc. All rights reserved.

 Programmability Configuration Guide

https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/prog/configuration/1710/b_1710_programmability_cg.html

© 2023 Cisco Systems, Inc. All rights reserved.

 Videos and Tutorials Automation and Learning and Community and
Start Now Sandbox
Code Exchange Certifications Study Groups
Learning Lab

developer.cisco.com

© 2023 Cisco Systems, Inc. All rights reserved.

 Learning Lab and Blogs: Telemetry
https://developer.cisco.com/learning/modules/iosxe_telemetry
https://blogs.cisco.com/developer/model-driven-telemetry-sandbox
https://blogs.cisco.com/developer/getting-started-with-model-driven-telemetry
https://youtu.be/QwwZakkWBng

© 2023 Cisco Systems, Inc. All rights reserved.

 ! new URL!
Programmability & Automation on SalesConnect

https://salesconnect.cisco.com/EnterpriseNetworking/s/programmability-and-automation

© 2023 Cisco Systems, Inc. All rights reserved.

VT