Network Functions Virtualization (NFV)

Testing Best Practices

Contents
1. Scope............................................................................................................................... 3
2. References....................................................................................................................... 3
3. Abbreviations .................................................................................................................. 3
4. Introduction .................................................................................................................... 7
NFV Framework ........................................................................................................... 7
NFV Use Cases ............................................................................................................. 8
5. Assuring compliance to ESTI ISG Standards ...................................................................... 9
6. NFV Test Environment ................................................................................................... 10
7. NFV Test Techniques ..................................................................................................... 11
Static Test.................................................................................................................. 11
Oracle Testing ........................................................................................................... 11
8. Test Case #1: MANO ..................................................................................................... 12
NFV Orchestrator: ..................................................................................................... 12
VNF Manager: ........................................................................................................... 13
Virtualised Infrastructure Manager (VIM): ................................................................. 13
Orchestrator – VNF Manager (Or-Vnfm) .................................................................... 13
Virtualised Infrastructure Manager – VNF Manager (Vi-Vnfm) ................................... 13
Orchestrator – Virtualised Infrastructure Manager (Or-Vi) ......................................... 13
OSS/BSS – NFV MANO (Os-Ma) .................................................................................. 14
VNF – VNF Manager .................................................................................................. 14
9. Test Case #2: Failover Convergence Testing (C.1.2.2) ..................................................... 15
10. Test Case #3: Security .................................................................................................... 16
11. Test Case #4: VNF Migration (C.1.2.3) ............................................................................ 17
12. Test Case #5: Scalability (C.1.2.1) ................................................................................... 18

Page |2
1. Scope
The document provides a list of best practices to be used for ensuring smooth migration of
Network Elements and Services to NFV environment.

It is recognized that certain portion of the best practices and recommendation are not required
in all cases of VNFs migration.

2. References
This document is based on ESTI ISG NFV Standards.

The following referenced documents are served as a baseline for this document:

GS NFV 001 Network Functions Virtualisation (NFV); Use Cases

GS NFV 003 Network Functions Virtualisation (NFV); Terminology for Main Concepts

NFV GS NFV 004 Network Functions Virtualisation (NFV); Virtualisation Requirements

GS NFV-PER 001 Network Functions Virtualisation (NFV); NFV Performance & Portability
Best Practises

3. Abbreviations
API Application Programming Interface
ARP Address Resolution Protocol
AS Application Server
BBU Base Band Unit
BFD Bidirectional Forwarding Detection
BGP Border Gateway Protocol
BIOS Basic Input/Output System
BNG Broadband Network Gateway
BRAS Broadband Remote Access Server
BW Bandwidth
CDN Content Delivery Network
CGNAT Carrier Grade Network Address Translation
CHD Compute Host Descriptor
CIFS Common Internet File System
COTS Commercial Off-The-Shelf
CPE Customer Premises Equipment
CPU Central Processing Unit
C-RAN Cloud-Radio Access Network
CVLAN Customer VLAN
DCB Data Center Bridging

Page |3
DDoS Distributed Denial of Service
DDR2 Double Data Rate type 2
DDR3 Double Data Rate type 3
DHCP Dynamic Host Configuration Protocol
DMA Direct Memory Access
DPI Deep Packet Inspection
DSLAM Digital Subscriber Line Access Multiplexer
DUT Device Under Test
E-CPE Enterprise-Customer Premises Equipment
ERPS Ethernet Ring Protection Switching
FFT Fast Fourier Transform
FIB Forwarding Information Base
FTP File Transfer Protocol
FW Firewall
GB GigaByte
GE Gigabit Ethernet
GGSN Gateway GPRS Support Node
GPRS General Packet Radio Service
GPS Global Positioning System
GRE Generic Routing Encapsulation
GUI Graphical User Interface
GW Gateway
HTML HyperText Markup Language
HTTP HyperText Transfer Protocol
HW Hardware
I/O Input/Output
I-CSCF Interrogating-Call Session Control Function
IMS IP Multimedia Subsystem
IO Input Output
IOMMU Input/Output Memory Management Unit
IOTLB I/O Translation Lookaside Buffer
IP Internet Protocol
IPC Inter-Process Communication
IPoE IP over Ethernet
IPsec IP security
ISIS Intermediate System to Intermediate System
ETSI 9 ETSI GS NFV-PER 001 V1.1.1 (2014-06)
KPI Key Performance Indicator
L4 Layer 4
L7 Layer 7

Page |4
LPM Longest Prefix Match
MAC Media Access Control
MAN Metropolitan Area Network
MANO MANagement and Orchestration
MGCF Media Gateway Controller Function
MME Mobility Management Entity
MMU Memory Management Unit
MOS Mean Opinion Score
MOS-AV MOS-Audio & Video
MPLS Multi-Protocol Label Switching
MSE Mean Square Error
N/A Not Applicable
NAS Network-Attached Storage
NAT Network Address Translation
NF Network Function
NFV Network Functions Virtualisation
NFVI Network Functions Virtualisation Infrastructure
NIC Network Interface Card
NID Network Interface Device
NUMA Non-Uniform Memory Access
OLT Optical Line Terminal
ONT Optical Network Terminal
ONU Optical Network Unit
OS Operating System
OSPF Open Shortest Path First
OVF Open Virtualisation Format
P2P Peer-to-Peer
PCI Peripheral Component Interconnect
PCIe PCI Express
PCIe VF PCIe Virtual Function
PCI-SIG PCI Special Interest Group
PCRF Policy and Charging Rules Function
P-CSCF Proxy-Call Session Control Function
PDN Packet Data Network
PE Provider Edge
P-GW PDN-Gateway
PNF Physical Network Function
PPP Point-to-Point Protocol
PPPoE Point-to-Point Protocol over Ethernet
PSNR Peak Signal-to-Noise Ratio

Page |5
QoE Quality of Experience
QoS Quality of Service
R/W Read/Write
RADIUS Remote Authentication Dial In User Service
RAM Random Access Memory
RAN Radio Access Network
RDMA Remote Direct Memory Access
RGW Residential Gateway
RIB Routing Information Base
RMS Root Mean of Squares
RoCE RDMA over Converged Ethernet
RX Reception
SAN Storage Area Network
S-CSCF Serving-Call Session Control Function
SGSN Serving GPRS Support Node
S-GW Serving-Gateway
SLA Service Level Agreement
SMT Simultaneous Multi-Threading
SR-IOV Single Root I/O Virtualisation
SSIM Structural Similarity
ETSI10 ETSI GS NFV-PER 001 V1.1.1 (2014-06)
STB Set-Top-Box
STP Spanning Tree Protocol
SVLAN Service VLAN
SW Software
TC Test Case
TCP Transmission Control Protocol
TLB Translation Lookaside Buffer
TWAMP Two-Way Active Measurement Protocol
TX Transmission
VF Virtual Function
VIA Virtual Interface Architecture
VIM Virtualised Infrastructure Manager
VLAN Virtual Local Area Network
VM Virtual Machine
VNF Virtualised Network Function
VQM Video Quality Metric
VTA Virtual Test Appliance
EMS Element Management System
KVM Kernel-based Virtual Machine

Page |6
DNS Domain Name System
NTP Network Time Protocol
SSH Secure SHell
NFS Network File System
TA Test Agent
ISV Independent Software Vendor
ABR Available Bit Rate
CBR Constant Bit Rate
VBR Variable Bit Rate
RSS Receive Side Scaling

4. Introduction
Network Functions Virtualization (NFV) is a concept introduced by network operators in 2012
[Network Functions Virtualisation – Introductory White Paper].

Network Functions Virtualization (NFV) is a network architecture concept that proposes using IT
virtualization related technologies to virtualize entire classes of network node functions into
building blocks that may be connected, or chained, together to create communication services.

NFV relies upon, but differs from traditional server virtualization techniques such as those used
in enterprise IT. A virtualized network function, or VNF, may consist of one or more virtual
machines running different software and processes, on top of industry standard high volume
servers, switches and storage, or even cloud computing infrastructure, instead of having custom
hardware appliances for each network function.

The European Telecommunications Standards Institute (ETSI) has formed an Industry

Specification Group on Network Function Virtualization (ISG NFV).

The contributors of the NFV Introductory white paper as well as the ETSI ISG have identified
Testing and QoE monitoring as one of the main use cases and subjects to address when
implementing NFV environment.

NFV Framework
The NFV framework consists of three main components.

1. Virtualized Network Functions (VNF) are software implementations of network

functions that can be deployed on a Network Function Virtualization Infrastructure
(NFVI).
2. NFV Infrastructure (NFVI) is the totality of all hardware and software components
which build up the environment in which VNFs are deployed. The NFV-Infrastructure can

Page |7
 span across several locations. The network providing connectivity between these
locations is regarded to be part of the NFV-Infrastructure.
3. Network Functions Virtualization Management and Orchestration Architectural
Framework (NFV-MANO Architectural Framework) is the collection of all functional
blocks, data repositories used by these functional blocks, and reference points and
interfaces through which these functional blocks exchange information for the purpose
of managing and orchestrating NFVI and VNFs.

The building block for both the NFVI and the NFV-MANO is the NFV platform. In the NFVI role, it
consists of both virtual and physical compute and storage resources, and virtualization software.
In its NFV-MANO role it consists of VNF and NFVI managers and virtualization software
operating on a hardware controller. The NFV platform implements carrier-grade features used
to manage and monitor the platform components, recover from failures and provide effective
security - all required for the public carrier network.

NFV Use Cases

The first standard issued by ESTI identified 9 use cases for NFV in GS NFV 001:

Use Case #1: NFV Infrastructure (NFVI) as a Service

Use Case #2: Virtual Network Functions as a Service (VNFaaS)

Use Case #3: Virtual Network Platform as a Service (VNPaaS)

Use Case #4: VNF Forwarding Graphs

Use Case #5: Virtualisation of Mobile Core Network and IMS

Use Case #6: Virtualisation of Mobile Base Station

Use Case #7: Virtualisation of the Home Environment

Use Case #8: Virtualisation of CDNs (vCDN)

Use Case #9: Fixed Access Network Functions Virtualisation

Each of those use cases requires different level of techniques and has different set of QoS and
QoE KPIs. Nevertheless, from System Test point of view, same workload scenarios may apply for
all use cases

Page |8
5. Assuring compliance to ESTI ISG Standards

Page |9
6. NFV Test Environment
Workload simulation

P a g e | 10
7. NFV Test Techniques
Static Test

Oracle Testing

P a g e | 11
8. Test Case #1: MANO
The first step is to test NFV Management and Orchestration Architectural Framework.

NFV Orchestrator (NFVO)

NS VNF NFV NFV

Catalog Catalog Instance Instance

Or-Vnfm
Or-Vi
Manage
VNF Manager Manag
(VNFM)

Vi-Vnfm

Virtualised Infrastructure Manager

(VIM)

NFV Management and Orchestration Architecture

NFV Orchestrator:
- On-boarding of new Network Service (NS), VNF Forwarding Graphs (VNF-FG) and VNF
Packages
- VNF lifecycle management (including instantiation, scale-out/in, Performance
measurements, event correlation, termination)
- Monitoring and collection of information related to resource usage, including mapping
of usage
- Scheduled request regarding VNF instances
- Global resource management, validation and authorization of NFVI resource requests
- Resources sharing between VNFs
- Policy management
- Constraints management
- SLA parameters
- Network capacity adaptation to load

P a g e | 12
 - Coexistence with legacy network equipment
- Controlling and managing Inventory of versions, releases and patches of all units of
hardware and software
- Maintenance, hardware and software exchange, SW upgrades, Firmware upgrades,
repair
- Manage log of all changes to inventory, unexpected events and maintenance activities

VNF Manager:
- Lifecycle management of VNF instances
- Configuration and event reporting between NFVI and the E/NMS
- Liveness checking of an VNF, e.g. watchdog timer or keepalive
- Failure detection
- Fault remediation of each VNF resiliency category

Virtualised Infrastructure Manager (VIM):

- Controlling and managing Inventory of software (hypervisors), computing, storage and
network resources
- Collection and forwarding of performance measurements and events
- NFV Infrastructure faults collection and remediation
- Management of infrastructure resource and allocation, e.g. increase resource to VMs,
improve energy efficiency and resource reclamation
- Root cause analysis of performance issues from the NFV infrastructure perspective
- Mechanism for time-stamping of hardware (e.g. network interface cards, NICs and NIDs)

Orchestrator – VNF Manager (Or-Vnfm)

- Resource related requests, e.g. authorization, validation, reservation, allocation by VNF
Manager(s)
- Sending configuration information to the VNF Manager, so that the VNF can be
configured appropriately to function within the VNF Forwarding Graph in the NS
- Collecting state information

Virtualised Infrastructure Manager – VNF Manager (Vi-Vnfm)

- Resource allocation requests
- Virtualised hardware resource configuration and state information (e,g. events)
exchange

Orchestrator – Virtualised Infrastructure Manager (Or-Vi)

- Resource reservation and allocation
- Virtualised hardware resource configuration and state information (e,g. events)

P a g e | 13
OSS/BSS – NFV MANO (Os-Ma)
- Requests for network service lifecycle management
- Requests for VNF lifecycle management
- Forwarding of NFV related state information
- Policy management exchanges
- Data analytics exchanges
- Forwarding of NFV related accounting and usage resords
- NFVI capacity and inventory information exchanges

VNF – VNF Manager

- Requests for VNF lifecycle management
- Exchanging configuration information
- Exchanging state information

P a g e | 14
9. Test Case #2: Failover Convergence Testing (C.1.2.2)

P a g e | 15
10. Test Case #3: Security

P a g e | 16
11. Test Case #4: VNF Migration (C.1.2.3)

P a g e | 17
12. Test Case #5: Scalability (C.1.2.1)

P a g e | 18