Kubernetes Native Infrastructure and

Operator Framework for 5G Edge Cloud Computing

Hyde Sugiyama, Chief Architect Red Hat K.K.
 OpenShift/K8s on OpenStack NFV session@OSS2018

OpenShift on OpenStack NFV

B2B2X for SoE apps in OpenShift (in
VM) on top of DCN(NFVI)

2
NFV Evolution to Kubernetes

Today Tomorrow Long Term

70％ 30％ 50％ 50％ 30％ 70％
(IT SoE apps） (Stateless & Stateful）

Container VNF
VNF Container VNF VNF Container
CNF VNF
K8s/OCP CNF

Kubevirt
OpenStack OpenStack K8s/OCP4 OpenStack
K8s/OCP4

Bare Metal Bare Metal Bare Metal

Data Lake Data Lake Data Lake

SoR: System of Record OCP4: OpenShift Container Platform 4

SoE: System of Engagement
 Agenda

● Kubernetes on bare metal deployment

● 5GC workloads
● O-RAN alliance
● Edge computing workloads
● Summary

4
 Innovation - 2019

4G
All IP packet
5G
5GC
Carrier Cloud native/Service Based Archtecture
Grade
CP and UPF separation
Linux Network
Function Network slicing
Virtualization UPF offload (FPGA, Edge Switch Fabric)
Distribute vRAN CU-DU split
Compute
Heterogeneous Computing
Node
DPDK(vCPU)
Multiaccess
VNF Edge CNF GPU
FPGA
Computing Autonomous micro edge cloud
Edge AI platform(Intelligent Edge)
Data Lake /Data Hub
5
 OpenShift/k8s on BM
deployment

6
 Performance Sensitive Application Platform

Kubernetes can enhance (with 5G/Edge

adapting from more significant Machine FSI

Learning
open sources) like an OpenShift 4
that becomes the single platform HPC ISVs

to run any application.

Big Data Animation
● Old or new
● Monolithic/Microservice

7
 Possibility for NFV architecture change by adapting
Kubernetes Operator(w/ OpenShift Operator framework)

API management
OSS/BSS NFVO
NFVO/OSS/BSS

=VM= =VM= =VM= =Pod= =Pod= =Pod=

VNF+ VNF+ VNF+ CNF CNF CNF
guest guest guest VNF-M (SMF) (UDM) (...)
OS OS OS
CNF-M CNF-M CNF-M

Cluster Operator OpenShift

= Hypervisor= OpenShift Worker Master
VIM
NFVI (host OS) (host OS)

BM(s) BM(s)

VNF CNF
8
 Kubernetes Operator
● Automate day 2 lifecycle management of
containerized applications in Kubernetes
● Leverage CRDs to deploy Kubernetes native
LIFECYCLE
services that can access Kube API events SDK METERING
MANAGEMENT
● Operator SDK simplifies creation of Operators in
Go (or leverage Helm or Ansible automation)
● Helm Operator allows you to convert Helm Charts
into Operators
○ Deploy Charts without requiring Tiller
○ Leverage Kube RBAC to deploy Charts
○ Automated, over the air updates for Chart
deployed services

Implementation pattern of "SRE (Site Reliability Engineering)"

that incorporates operations by software
https://www.youtube.com/watch?v=LymzLHRbQdk
9
 Full-Stack Automated Operations in OpenShift

APPLICATIONS AND SERVICES

ISV Operators
Custom Operators (built w/Operator SDK)
PLATFORM AND CLUSTER MANAGEMENT
Automated updates for Kubernetes,
monitoring, security, registry and more

LINUX HOST
Over-the-air updates for Red Hat CoreOS

ANY INFRASTRUCTURE

10
 5GC Service Based
Architecture on Kubernetes

11
 5GC system architecture Network
Exposure
Function
Network
Slice
Selection
Function
Access &
Mobility mgmnt
Function
User
Plane
Function
Packet AUthentication
Forwarding Server
Control Function
Protocol Policy
Control
Function
Unified
Data
Management
Session
Data Management
Function
Network

Data Network
12
 KIALI ( GUI for Istio )
5G SBI and Service Mesh

OPENSHIFT SERVICE MESH

● A dedicated network for service to

service communications
● Observability and distributed tracing
● Policy-driven security
● Routing rules & chaos engineering
● Powerful visualization & monitoring
● Will be available via OperatorHub

13
 5GC SBA with Service Mesh Network
KNI(Kubernetes Native Infrastructure) for Edge Exposure
Function
Network
Slice
Selection
Function
Istio Access &
Mobility mgmnt
service Function
mesh Network
Repository
MULTUS CNI setup, Function
Stateful management, AUthentication
5GC CNF LCM, K8s LCM, Server
Function
etc Policy
Control
Function
Unified
Data
Management
Session
User Equipment Management
Function
KNI edge
GPU device plugin, DPDK, FPGA,
Serverless, Data lake, etc
14
 5GC, UPF and Network Slicing (Cloud edge fabric case)

Multi cluster for istio Slice 1 for

service mesh might be B2B2X workloads 1
needed : SMF1 & UPF1
Slice 2 for
B2B2X workloads 2
: SMF2 & UPF2

* https://www.youtube.com/
* * watch?v=1X5U4Jo0Jlw

UE https://www.pilab.jp/ipop20
**
19/exhibition/WhitePaper_i
POP2019.pdf

**

15
 O-RAN alliance &
RAN Cloudification w/
Kubernetes

16
 3GPP &
O-RAN alliance Design Inventory Policy Configuration RAN Intelligent Controller (RIC)non-RT
Orchestration & Automation (e.g. ONAP): MANO, NMS

A1
3GPP (e/g)NB RAN Intelligent Controller (RIC) near-RT
Applications
E
3rd party 2 Radio Connection
Layer Mobility QoS Interference Trained
APP Mgmt Mgmt Mgmt. Mgmt Model

Radio-Network Information Base

3GPP CU E2 :btw RIC near-RT and O-CU/O-DU
Multi-RAT CU-CP CU-UP
RRC E1 SDAP
O-CU Protocol
Stack PDCP-C PDCP-U

Our focus NFVI Platform: Virtualization layer and COTS platform F1

OpenStack,
OpenShift/k8s
O-DU: RLC/MAC/PHY-high
Open Front Haul
3GPP DU
O-RU: PHY-low/RF

17
 O-RAN alliance WG6

Decoupling of software from hardware for all RAN modules in all splits

S/W Flexible O-CU O-DU O-RU All RAN modules

Orch.
Inventory, Support 10,000s
H/W Discovery, H/W abstraction layer
Registration of distributed
Policy,
NFVI stack (containers or VMs), OS, cloud m-plane cloud sites
Blackbox Metrics

BBU Heterogeneous
Common
LCM computing
mechanisms
Multitude of deployment
models: CloudRAN,
Centralized CU/DU CU/DU split
CU-DU split,
Distributed
(C-RAN) CU/DU dRAN on whitebox or DC
(D-RAN)

18
 Candidate WG6 Scenarios B,C

Scenario B: Distributed vO-CU and vO-DU

BBU chassis at cell site
Near-RT vBBU
vO-CU +
To UPF RIC E2 Open fronthaul
vO-DU
1:m, local
O-RU The O-CU/O-DU functionality can be
& MEC NFVI 1:N, remote NFVI Cell site pooled from multiple O-RUs, and meet
Regional cloud Open chassis and blade spec O-DU latency requirements.
K8s or OpenStack Cell site area
Near-RT RIC can serve a very large
number of O-RUs
Scenario C: Centralized vO-CU with distributed vO-DU
O-DU chassis at cell site
Near-RT
To UPF vO-CU F1 vO-DU Open fronthaul
RIC O-RU The O-DU functionality can be pooled
1:N, remote 1:m, local
& MEC NFVI NFVI from multiple O-RUs, and meet O-DU
Cell site
Regional cloud Open chassis and blade spec latency requirements.
K8s or OpenStack Near-RT RIC can serve a very large
Cell site area number of O-RUs, and O-CU is very
centralized.

19
 KNI Edge
Kubernetes
Native
Infrastructure

20
 Akraino Edge Stack project
KNI-Edge Blueprints (in Progress)
Provider Access Edge (PAE) Industrial Edge (IE)
Optimized for real-time and Optimized for small footprint and
networking performance for low-latency for IoT, serverless, and
Containerized vRAN and MEC workloads. machine learning workloads.
MEC Apps VM1 ... VMN ML Apps Srv-less ML Apps IoT Apps
vRAN URLLC
Apps
(RIC) MEC MW KubeVirt Kubeflow Apps Kubeflow EdgeX
kNative
Cluster/ Prometheus Cluster/ Prometheus
Machine OpenShift/OKD Machine OpenShift/OKD + exporters
+ exporters
Operator Operator
Tungsten Ceph CRI-O OVN/OVS Ceph CRI-O
NOS CoreOS and CentOS-rt NOS CoreOS and CentOS-rt
... ...
switch COTS COTS COTS switch COTS COTS COTS
https://wiki.akraino.org/display/AK/Kubernetes-Native+Infrastructure+%28KNI%29+Blueprint+Family
UBI: https://www.redhat.com/en/blog/introducing-red-hat-universal-base-image
21
 IoT Apps - Eclipse IoT project
APPLICATION DEVELOPMENT,
DELIVERY, & INTEGRATION
CONNECTED IoT IoT
“THINGS” EDGE INTEGRATION HUB
App
integration

Management

Telemetry
Telemetry
Edge
analytics
DATA MANAGEMENT & ANALYTICS
Machine Telemetry
Management
learning CLOUDERA’S DISTRIBUTION
INCLUDING HADOOP (CDH)

Machine learning model

22
 ML Apps - OPEN DATA HUB
Collaborate on a Data & AI platform for the Edge Cloud & Core

A collection of open source and

cloud components packaged in a
“machine learning-as-a-service”
platform to solve business
problems.

23
 Collaborate on a Data & AI platform for the Edge Cloud & Core

● AI Library
● Data Science and ETL
Tools
● Streaming and Enriching
Data
● Storing Data
● Managing Data
● Monitoring Infrastructure

24
 Serverless Apps - Knative
• Familiar to Kubernetes users. Native.
• Scale to 0 and autoscale to N based on demand
• Applications and functions. Any container
workload.
• Powerful eventing model with multiple event
sources.
• Operator available via OperatorHub
• Knative v0.6 (v1beta1 APIs)
• No vendor lock in

Learn more
https://www.openshift.com/learn/topics/knative

25
 Red Hat OpenShift Hybrid Serverless

Developer experience Function as a Service Invoker Runtime CLI API

APIs, CLI, service binding Red Hat OpenShift

Knative Events Build Serving
ODO CLI Eclipse Che
Building blocks for serverless
RH MW Services ISV Services
Source-centric and (Operator backed) (Operator backed)
container-based Operator Framework
Automated
Istio
Operations
The leading enterprise
Hybrid Install / Ops Install / Upgrade Network / CNI
Kubernetes platform
Automated Operations Ops & Dev Consoles Security / Auth Storage / CSI

Build an run anywhere (Hybrid

Kubernetes
Cloud)
Red Hat Enterprise Linux or Red Hat CoreOS

26
 Knative Event Sources

Upstream Event Sources:

GitHub
Kubernetes Events
Pub/Sub (AWS SQS, Kafka, NATS, Google PubSub)
Websockets
Expose an ingress

200+ event sources through Camel-K

Camel-K + Knative Demo

https://github.com/knative/eventing-sources

27
 Serverless ：IoT ＆ Sensor

Java Reports

Java Python Java

IoT API
Gateway
Java JavaScript
(NodeJS)
Archiving

EDGE Private Hybrid Public

28
 Summary

29
 Next Generation DX platform
Application portability in any environment and consistency in operation.
Application Portability
Container
* コンテナ
Container
* コンテナ
Container
* Container
Container
* Container
Container
*
Application Application
Application Application
Application Application
Application Application
Application

OS dependencies OS
OS dependencies
dependencies OS
OSdependencies
dependencies OSdependencies
OS dependencies OS
OSdependencies
dependencies

Pod Pod Pod Pod Pod

Easy LCM per microservice, multi-cluster management and federation

Multi Interface, SDN for isolation

RHEL/CoreOS RHEL/CoreOS RHEL/CoreOS RHEL/CoreOS

RHEL/CoreOS
Guest VM Virtual Machine Virtual Machine Virtual Machine

GPU FPGA

LAPTOP VIRTUALIZATION PRIVATE CLOUD BARE METAL PUBLIC CLOUD

Code Ready ** RHV, VMWARE OpenStack NFV Intel, Power, ARM(plan） Azure, AWS, Google, etc.
It is also possible to duplicate and store the same
By using UBI (Universal Base Image),
* developers can concentrate on container container image in geographically distant places
application development at any platform. (utilization of Ceph data lake and QUEY)
** https://github.com/code-ready/crc
30
 Summary

1. Heterogeneous Computing Platform

2. K8s as AI platform at Telco Edge
a. Data Lake/Hub at Telco node
3. Kubernetes Native Infrastructure on Bare metal across Telco
Core and Telco Edge
a. Operator Framework for Site Reliability Engineering and Provider
extension
i. Autonomous micro-cloud at Telco edge
ii. Digital Service Provider driven CNF apps management
b. Service mesh for micro service
c. Serverless at Telco edge node

31
 Thank you!

32