0% found this document useful (0 votes)

323 views180 pages

CiscoLive! OpenStack Lab LABCLD-2225

CiscoLive! OpenStack Lab

Uploaded by

jakub_zen

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

323 views180 pages

CiscoLive! OpenStack Lab LABCLD-2225

CiscoLive! OpenStack Lab

Uploaded by

jakub_zen

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 180

1. 2. 2015 CiscoLive!

OpenStack Lab LABCLD2225

January 2630,2015 Milan,Italy

Deploying a Private Cloud using OpenStack with Cisco
Plugin
LABCLD2225
Lab Manual

http://openstack.ciscolive.com/printable.php?pod=3 1/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

What is OpenStack?
OpenStack is an opensource cloud operating system designed to control large pools of
compute, storage, and network resources. OpenStack began in 2010 as a joint project of
Rackspace and NASA and is currently managed by the OpenStack Foundation. Hundreds of
companies now participate in the project, developing drivers and components to interact with
their products as well as contributing new features that benefit the entire OpenStack
community. The community collaborates on 6month release cycles. The latest release that
will be used for this lab is named the Juno release.

OpenStack enables application developers to quickly provision the compute, storage, and
network resources they need to develop and deploy their application without the need for
intervention from the IT organization. In many ways, OpenStack was a direct response to the
success of Amazon Web Services for customers who wanted the ability to create a similar
development environment without having to use Amazon's services.

OpenStack has a modular architecture for which each component has specific function. The
following is a list of some of the core OpenStack components:

Component Function

Nova Compute

Cinder Block storage

Neutron Networking

Keystone Identity Services

Horizon Dashboard / Web GUI

Glance Image Service
The focus on this lab will be in familiarizing you with OpenStack with emphasis on the
networking component (Neutron) and Cisco products.

http://openstack.ciscolive.com/printable.php?pod=3 2/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

OpenStack Networking (Neutron)
Pluggable, scalable, APIdriven network and IP management

Today's datacenter networks contain more devices than ever before. Servers, network
equipment, storage systems and security appliances — many of which are further
divided into virtual machines and virtual networks. The number of IP addresses, routing
configurations and security rules can quickly grow into the millions. Traditional network
management techniques fall short of providing a truly scalable, automated approach to
managing these nextgeneration networks. At the same time, users expect more control
and flexibility with quicker provisioning.

OpenStack Networking is a pluggable, scalable and APIdriven system for managing
networks and IP addresses. Like other aspects of the cloud operating system, it can be
used by administrators and users to increase the value of existing datacenter assets.
OpenStack Networking ensures the network will not be the bottleneck or limiting factor
in a cloud deployment and gives users real self service, even over their network
configurations and topologies.

Networking Capabilities
OpenStack provides flexible networking models to suit the needs of different
applications or user groups. Standard models include flat networks or VLANs for
separation of servers and traffic or overlay technologies such as GRE tunnels or
VXLAN encapsulation.
OpenStack Networking manages IP addresses, allowing for dedicated static IPs or
DHCP. Floating IPs provide a layer of abstraction between public IP addresses
and private addresses. A Floating IP is essentially a static NAT translation.
Because the application virtual machines (Instances) have private addresses,
Floating IPs allow traffic to be dynamically rerouted to a different compute
resource. This allows users to redirect traffic during maintenance or in the case of
failure.
Users can create their own networks, control traffic and connect servers and
devices to one or more networks.
The pluggable backend architecture lets users take advantage of commodity
hardware or advanced networking services from supported vendors.
Administrators can take advantage of softwaredefined networking (SDN)
technology like OpenFlow to allow for high levels of multitenancy and massive
scale.
OpenStack Networking has an extension framework allowing additional network

http://openstack.ciscolive.com/printable.php?pod=3 3/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

services, such as intrusion detection systems (IDS), load balancing, firewalls and
virtual private networks (VPN) to be deployed and managed as a service.

OpenStack Compute (Nova)
Provision and manage large networks of virtual machines

The OpenStack cloud operating system enables enterprises and service providers to
offer ondemand computing resources, by provisioning and managing large networks of
virtual machines. Compute resources are accessible via APIs for developers building
cloud applications and via web interfaces for administrators and users. The compute
architecture is designed to scale horizontally on standard hardware, enabling the cloud
economics companies have come to expect.

Flexible Architecture
OpenStack is architected to provide flexibility as you design your cloud, with no
proprietary hardware or software requirements and the ability to integrate with legacy
systems and third party technologies. It is designed to manage and automate pools of
compute resources and can work with widely available virtualization technologies, as
well as bare metal and highperformance computing (HPC) configurations.

Administrators often deploy OpenStack Compute using one of multiple supported
hypervisors in a virtualized environment. KVM and XenServer are the most popular
choices for hypervisor technology and recommended for most use cases. In addition to
different hypervisors, OpenStack supports ARM and alternative hardware architectures.

Popular Use Cases
Service providers offering an IaaS or PaaS platform or delivering SaaS with
OpenStack as the underlying infrastructure for the software
IT departments acting as cloud service providers for business units and project
teams
Processing big data with tools like Hadoop
Scaling compute up and down to meet demand for web resources and
applications
Highperformance computing (HPC) environments processing diverse and

http://openstack.ciscolive.com/printable.php?pod=3 4/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

intensive workloads

Learn more about OpenStack's storage (http://www.openstack.org/software/openstack
storage/) and networking (http://www.openstack.org/software/openstacknetworking/) ,
or take a tour of the dashboard (http://www.openstack.org/software/openstack
dashboard/) .

Features & Benefits
The features below are available in the current release.

Feature Benefit

Manage virtualized Racks of commodity servers as pools of computing
commodity server Improved utilization and automation of resources for
resources greater cost efficiencies
CPU, memory, disk, and
network interfaces

Manage Local Area Programmatically allocate IPs and VLANs
Networks (LAN) (for rapid provisioning of network capabilities and
Flat, Flat DHCP, VLAN security features)
DHCP, IPv6 Flexible networking models
(http://www.openstack.org/software/openstack
networking/) to suit needs of each application and/or
user group

API with rate limiting Designed for automation and security
and authentication (to make it easy for you to manage who has access to
compute resources and prevent users from impacting
each other with excessive API utilization)

Distributed and Massively scalable and highly available system
asynchronous (for increased assurance of system uptime)
architecture

Virtual Machine (VM) Easily store, import, share, and query images
image management (http://www.openstack.org/software/openstackshared
services/)
(to make it easy for you to spin up new standardized
VMs)

Live VM management Increase productivity with lifecycle management
Run, reboot, suspend, (from a single user interface and using the APIs)
resize, terminate
instances

Floating IP addresses Ability to assign (and reassign) IP addresses to VMs

http://openstack.ciscolive.com/printable.php?pod=3 5/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Security Groups Flexibility to assign and control access to VM
instances by creating separation between resource
pools

Role Based Access Ensure security by user, role and project
Control (RBAC)

Projects & Quotas Ability to allocate, track and limit resource utilization

VNC Proxy through Quick and easy CLI administration
web browser

Store and Manage files Automates resource management/provisioning
programmatically via
API

VM Image Caching on Faster provisioning of VMs
compute nodes

Least privileged access Root access separation for management & services
design

Dashboard with fully Learn more about the OpenStack Dashboard
integrated support for (http://www.openstack.org/software/openstack
selfservice dashboard/)
provisioning

OpenStack Storage (Cinder & Swift)
Object and Block storage for use with servers and applications

In addition to traditional enterpriseclass storage technology, many organizations now
have a variety of storage needs with varying performance and price requirements.
OpenStack has support for both Object Storage and Block Storage, with many
deployment options for each depending on the use case.

Object Storage is ideal for cost effective, scaleout storage. It provides a fully
distributed, APIaccessible storage platform that can be integrated directly into

http://openstack.ciscolive.com/printable.php?pod=3 6/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

applications or used for backup, archiving and data retention. Block Storage allows
block devices to be exposed and connected to compute instances for expanded
storage, better performance and integration with enterprise storage platforms, such as
NetApp, Nexenta and SolidFire.

Object Storage Capabilities (Swift)
OpenStack provides redundant, scalable object storage using clusters of
standardized servers capable of storing petabytes of data
Object Storage is not a traditional file system, but rather a distributed storage
system for static data such as virtual machine images, photo storage, email
storage, backups and archives. Having no central "brain" or master point of control
provides greater scalability, redundancy and durability.
Objects and files are written to multiple disk drives spread throughout servers in
the data center, with the OpenStack software responsible for ensuring data
replication and integrity across the cluster.
Storage clusters scale horizontally simply by adding new servers. Should a server
or hard drive fail, OpenStack replicates its content from other active nodes to new
locations in the cluster. Because OpenStack uses software logic to ensure data
replication and distribution across different devices, inexpensive commodity hard
drives and servers can be used in lieu of more expensive equipment.

Block Storage Capabilities (Cinder)
OpenStack provides persistent block level storage devices for use with OpenStack
compute instances.
The block storage system manages the creation, attaching and detaching of the
block devices to servers. Block storage volumes are fully integrated into
OpenStack Compute and the Dashboard allowing for cloud users to manage their
own storage needs.
In addition to using simple Linux server storage, it has unified storage support for
numerous storage platforms including Ceph, NetApp, Nexenta, SolidFire, and
Zadara.
Block storage is appropriate for performance sensitive scenarios such as database
storage, expandable file systems, or providing a server with access to raw block
level storage.
Snapshot management provides powerful functionality for backing up data stored
on block storage volumes. Snapshots can be restored or used to create a new
block storage volume.

Features & Benefits
The features below are available in the current release.

Feature Benefit

No lockin, lower price/GB

http://openstack.ciscolive.com/printable.php?pod=3 7/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Leverages commodity
hardware

HDD/node failure agnostic Self healing
Reliability, data redundancy protecting from
failures

Unlimited storage Huge & flat namespace, highly scalable
read/write access
Ability to serve content directly from storage
system

Multidimensional scalability Backup and archive large amounts of data with
(scale out architecture) linear performance

Scale vertically and horizontally
distributed storage

Account/Container/Object Optimized for scale
structure Scales to multiple petabytes, billions of objects
No nesting, not a traditional file
system

Builtin replication Configurable number of accounts, container and
3x+ data redundancy compared object copies for high availability
to 2x on RAID

Easily add capacity unlike Elastic data scaling with ease
RAID resize

No central database Higher performance, no bottlenecks

RAID not required Handle lots of small, random reads and writes
efficiently

Builtin management utilities Account Management: Create, add, verify,
delete users

Container Management: Upload, download,
verify

Monitoring: Capacity, host, network, log
trawling, cluster health

Drive auditing Detect drive failures preempting data corruption

Expiring objects Users can set an expiration time or a TTL on an
object to control access

http://openstack.ciscolive.com/printable.php?pod=3 8/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Direct object access Enable direct browser access to content, such
as for a control panel

Realtime visibility into client Know what users are requesting
requests

Supports S3 API Utilize tools that were designed for the popular
S3 API

Restrict containers per Limit access to control usage by user
account

Support for NetApp, Nexenta, Unified support for block volumes using a
SolidFire variety of storage systems

Snapshot and backup API for Data protection and recovery for VM data
block volumes

Standalone volume API Separate endpoint and API for integration with
available other compute systems

Integration with Compute Fully integrated to Compute for attaching block
volumes and reporting on usage

OpenStack Shared Services
OpenStack has several shared services that span the three pillars of compute, storage
and networking, making it easier to implement and operate your cloud. These services
— including identity, image management and a web interface — integrate the
OpenStack components with each other as well as external systems to provide a
unified experience for users as they interact with different cloud resources.

Identity Service (Keystone)
OpenStack Identity provides a central directory of users mapped to the OpenStack
services they can access. It acts as a common authentication system across the cloud
operating system and can integrate with existing backend directory services like LDAP.
It supports multiple forms of authentication including standard username and password
credentials, tokenbased systems and AWSstyle logins.

Additionally, the catalog provides a queryable list of all of the services deployed in an
OpenStack cloud in a single registry. Users and thirdparty tools can programmatically
determine which resources they can access.

http://openstack.ciscolive.com/printable.php?pod=3 9/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

As an administrator, OpenStack Identity enables you to:

Configure centralized policies across users and systems
Create users and tenants and define permissions for compute, storage and
networking resources using rolebased access control (RBAC) features
Integrate with an existing directory like LDAP, allowing for a single source of
identity authentication across the enterprise

As a user, OpenStack Identity enables you to:

Get a list of the services that you can access
Make API requests or log into the web dashboard to create resources owned by
your account

Image Service (Glance)
The OpenStack Image Service provides discovery, registration and delivery services for
disk and server images. The ability to copy or snapshot a server image and immediately
store it away is a powerful capability of the OpenStack cloud operating system. Stored
images can be used as a template to get new servers up and running quickly and more
consistently if you are provisioning multiple servers
than installing a server operating
system and individually configuring additional services. It can also be used to store and
catalog an unlimited number of backups.

The Image Service can store disk and server images in a variety of backends, including
OpenStack Object Storage. The Image Service API provides a standard REST interface
for querying information about disk images and lets clients stream the images to new
servers.

Capabilities of the Image Service include:

Administrators can create base templates from which their users can start new
compute instances
Users can choose from available images, or create their own from existing servers
Snapshots can also be stored in the Image Service so that virtual machines can be
backed up quickly

A multiformat image registry, the image service allows uploads of private and public
images in a variety of formats, including:

Raw
Machine (kernel/ramdisk outside of image, a.k.a. AMI)
VHD (HyperV)
VDI (VirtualBox)
qcow2 (Qemu/KVM)
VMDK (VMWare)
OVF (VMWare, others)

Telemetry Service (Ceilometer)

http://openstack.ciscolive.com/printable.php?pod=3 10/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

The OpenStack Telemetry service aggregates usage and performance data across the
services deployed in an OpenStack cloud. This powerful capability provides visibility
and insight into the usage of the cloud across dozens of data points and allows cloud
operators to view metrics globally or by individual deployed resources.

Orchestration Service (Heat)
OpenStack Orchestration is a templatedriven engine that allows application developers
to describe and automate the deployment of infrastructure. The flexible template
language can specify compute, storage and networking configurations as well as
detailed postdeployment activity to automate the full provisioning of infrastructure as
well as services and applications. Through integration with the Telemetry service, the
Orchestration engine can also perform autoscaling of certain infrastructure elements.

Database Service (Trove)
Designed to run entirely on OpenStack, the service has the goal of allowing users to
quickly and easily utilize the features of a relational database without the burden of
handling complex administrative tasks. Cloud users and database administrators can
provision and manage multiple database instances as needed. Initially, the service will
focus on providing resource isolation at high performance while automating complex
administrative tasks including deployment, configuration, patching, backups, restores,
and monitoring.

http://openstack.ciscolive.com/printable.php?pod=3 11/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Configure base Operating System
on Controller
As with any Linux application installation, there are some operating system parameters that
need to be modified to facilitate the operation of your application. Adjustments to the firewall
and NTP are requirements for an operational OpenStack deployment. We have grouped the
requirements ahead of each component installation of OpenStack.

Use of CentOS for this lab
We chose CentOS as the operating system for this
laboratory due to the fact that RedHat Enterprise Linux
(RHEL) is the preferred distribution of many Enterprise
accounts. That being said, OpenStack can be installed
on many different Linux distributions, including
Canonical's Ubuntu. Please refer to our documentation
for instructions on integration with other Cisco plugins at http://www.cisco.com
(http://www.cisco.com).
In this lab, you will have access to two Linux hosts to run OpenStack. The first will serve the
roll of the OpenStack controller which will run most of the OpenStack services, while the
second will be a dedicated Compute Node. In a production environment, you would have
many compute nodes on which to run your virtual machines.

First, perform OS modifications on your Controller Node by connecting to pod3controller in
PuTTY Connection Manager.

Step 1 Configure NTP for proper time
synchronization
In OpenStack it is critical to have proper time synchronization across each of the component
hosts. In CentOS the default time protocol is called Chrony. Chrony is a variation of the
NTPd engine of linux that is designed for disassociation of NTP resources ( for example
laptops that lose access to their network clocks ). In your OpenStack environment you can
run either Chrony or NTPd as needed. For the purpose of this lab we will be using chrony and
it has been preconfigured for you.

For chrony you can use the command chronyc tracking to observe what clock source it's

attached to.

http://openstack.ciscolive.com/printable.php?pod=3 12/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

# chronyc tracking
Reference ID    : 10.0.236.10
Stratum         : 3
Ref time (UTC)  : Tue Jan  6 14:42:10 2015
System time     : 93476.671875000 seconds slow of NTP time
Last offset     : ‐0.000186148 seconds
RMS offset      : 3119.961914062 seconds
Frequency       : 48.261 ppm fast
Residual freq   : ‐0.011 ppm
Skew            : 8.250 ppm
Root delay      : 0.020065 seconds
Root dispersion : 0.023922 seconds
Update interval : 479.2 seconds
Leap status     : Normal

Step 2 Configure HOSTS file for Name Resolution
Setting proper hostname definition plays a key role in the setup of various components of
OpenStack ( in particular RabbitMQ ). There are three things that need to match for
RabbitMQ to properly operate. The first part is that the hostname is correct in the file
/etc/hosts. The second is the environment variable HOSTNAME that is derived in the
system. The third is the hostname definition in /etc/hostname. We have already setup
everything for you via kickstart on the systems you are using such that they have properly
setup the hostname variable.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
cat <<EOF > /etc/hosts
127.0.0.1   localhost localhost.localdomain
localhost4 localhost4.localdomain4
10.0.236.31  pod3‐controller pod3‐
controller.ecatsrtpdmz.cisco.com pod3‐compute1
pod3‐compute1.ecatsrtpdmz.cisco.com
10.0.236.32  pod3‐compute2 pod3‐
compute2.ecatsrtpdmz.cisco.com
EOF

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
cat <<EOF > /etc/hostname
pod3‐controller
EOF

Step 3 Install PLUGIN priorities and EPEL
repository
Install the yumpluginpriorities package to enable assignment of relative priorities within

http://openstack.ciscolive.com/printable.php?pod=3 13/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

repositories and install the epelrelease package to enable the EPEL repository (Extra
Packages for Enterprise Linux).

ALERT!
It is fine if the yum commands return with an ERROR that say's nothing to do.
The servers are built for you from scratch with the yum commands already
applied to save time.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
yum ‐y install yum‐plugin‐priorities

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
yum ‐y install
http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel‐
release‐7‐5.noarch.rpm

Step 4 Add the OpenStack JUNO release
repository
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
yum ‐y install
http://rdo.fedorapeople.org/openstack‐juno/rdo‐
release‐juno.rpm

Step 5 Upgrade system packages
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
yum ‐y upgrade

Step 6 Install OpenStack
On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 14/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
yum ‐y install openstack‐selinux openstack‐utils

http://openstack.ciscolive.com/printable.php?pod=3 15/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Configure base Operating System
on Compute Node
The following steps are similar to those you just performed on the Controller Node. You must
do the same thing on your Compute node. Perform these OS modifications on your Compute
Node by connecting to pod3compute2 in PuTTY Connection Manager.

Step 7 Configure NTP for proper time
synchronization on compute node
In OpenStack it is critical proper time synchronization across each of the component hosts.
In CentOS the default time protocol is called Chrony. Chrony is a variation of the NTPd
engine of linux that is designed for disassociation of NTP resources ( for example laptops
that loose access to their network clocks ). In your OpenStack environment you can run
either Chrony or NTPd as needed. For the purpose of this lab we will be using chrony and it
has been preconfigured for you.

For chrony you can use the command chronyc tracking to observe what clock source it's

attached to.

Step 8 Configure HOSTS file for Name Resolution
Setting proper hostname definition plays a key role in the setup of various components of
OpenStack ( in particular RabbitMQ ). There are three things that need to match for
RabbitMQ to properly operate. The first part is that the hostname is correct in the file
/etc/hosts. The second is the environment variable HOSTNAME that is derived in the
system. The third is the hostname definition in /etc/hostname. We have already setup
everything for you via kickstart on the systems you are using such that they have properly
setup the hostname variable.

On the compute host 10.0.236.32 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 16/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
cat <<EOF > /etc/hostname
pod3‐compute2
EOF

Step 9 Install PLUGIN priorities and EPEL
repository
Install the yumpluginpriorities package to enable assignment of relative priorities within
repositories and install the epelrelease package to enable the EPEL repository (Extra
Packages for Enterprise Linux).

On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
yum ‐y install yum‐plugin‐priorities

Now you must instal the EPEL repository to both hosts.

On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
yum ‐y install
http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel‐
release‐7‐5.noarch.rpm

Step 10 Add the OpenStack JUNO release
repository
On the compute host 10.0.236.32 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 17/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
yum ‐y install
http://rdo.fedorapeople.org/openstack‐juno/rdo‐
release‐juno.rpm

Step 11 Upgrade system packages
On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
yum ‐y upgrade

Step 12 Install OpenStack
On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
yum ‐y install openstack‐selinux openstack‐utils

http://openstack.ciscolive.com/printable.php?pod=3 18/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Database
The database in the OpenStack cluster is utilized as a repository of information for various
components in OpenStack. Different DB engines can be used in OpenStack. We will be
using MariaDB (free version of MySQL) for this lab. The database typically resides on the
Controller Node, so be sure to perform the following actions on your Controller Node.

Step 13 Install MariaDB from repository
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
yum ‐y install mariadb mariadb‐server MySQL‐
python

Step 14 Open Firewall Hole
In a configuration for a single host OpenStack, the firewall in CentOS permits everything on
localhost. Because we will have components scattered across two different hosts for this
lab, we need to open the ports for the various services. MariaDB uses port 3306. To enable
this port, use the command % firewall‐cmd .

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
firewall‐cmd ‐‐add‐port=3306/tcp ‐‐permanent
firewall‐cmd ‐‐reload

Step 15 Configure Network IP Bind
Next, configure MariaDB to accept connections from the network. By default MariaDB only
accepts connections from internal Unix sockets. The following SED command line will insert
the required configuration into the proper location of the my.cnf MariaDB configuration file.
This configuration makes it possible for MariaDB to listen to network connections arriving at
the IP address of the controller 10.0.236.31. It also specifies the default database engine and
UTF types.

[[IDSEQSTEP]]
sed ‐i "4i\bind‐address =
10.0.236.31\nmax_connections = 500\ndefault‐
storage‐engine =
innodb\ninnodb_file_per_table\ncollation‐server
= utf8_general_ci\ninit‐connect = 'SET NAMES
utf8'\ncharacter‐set‐server = utf8\n"
/etc/my.cnf

Once this configuration is completed, you can restart the service to read the new configuration
file. There are two commands. One to enable the service and one to start the service. The

http://openstack.ciscolive.com/printable.php?pod=3 19/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

enable command causes CentOS to activate this service in the system during startup.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
systemctl enable mariadb.service
systemctl stop mariadb.service
systemctl start mariadb.service

To validate that the MariaDB service is properly started you can issue the command %
systemctl status mariadb.service . The output of the command is similar to the following:

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
systemctl status mariadb.service

# systemctl status mariadb.service
mariadb.service ‐ MariaDB database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled)
   Active: active (running) since Wed 2014‐12‐24 13:17:45 EST; 10s ago
  Process: 54009 ExecStartPost=/usr/libexec/mariadb‐wait‐ready $MAINPID (code=exi
ted, status=0/SUCCESS)
  Process: 53980 ExecStartPre=/usr/libexec/mariadb‐prepare‐db‐dir %n (code=exited
, status=0/SUCCESS)
Main PID: 54008 (mysqld_safe)
   CGroup: /system.slice/mariadb.service
           ├─54008 /bin/sh /usr/bin/mysqld_safe ‐‐basedir=/usr
           └─54261 /usr/libexec/mysqld ‐‐basedir=/usr ‐‐datadir=/var/lib/mysql ‐‐
plugin‐dir=/usr/lib64/mysql/plugin ‐‐log‐error=/var/log/mariadb/mariadb.log ‐‐pid
‐file=/var/run/mariad...

Dec 24 13:17:43 localhost.localdomain systemd[1]: Starting MariaDB database serve
r...
Dec 24 13:17:43 localhost.localdomain mysqld_safe[54008]: 141224 13:17:43 mysqld_
safe Logging to '/var/log/mariadb/mariadb.log'.
Dec 24 13:17:43 localhost.localdomain mysqld_safe[54008]: 141224 13:17:43 mysqld_
safe Starting mysqld daemon with databases from /var/lib/mysql
Dec 24 13:17:45 localhost.localdomain systemd[1]: Started MariaDB database server
.

Info
What you are looking for in the status command is the state of the process. There
is a Active line and if the process shows active (running) then the process
has started properly.

Step 16 Configure User Credentials

http://openstack.ciscolive.com/printable.php?pod=3 20/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

With the MariaDB service now running and enabled for startup after reboot, you can configure
the root credentials to access the database. The first step is to configure the root password
credential. By default, the root password is not set so that administrators can access the
database for initial configuration purposes. In some Linux distributions, the installation creates
the default root credentials for you.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
mysql ‐u root

When you issue this command, you are then inside the Maria database cli processor. From
this point you can issue SQL commands against the database store. The first command we
are going to issue is to update the root password. We will be using the password cisco.123
throughout this lab for all access to simplify your experience.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
use mysql;
update user set password=PASSWORD("cisco.123")
where User='root';
flush privileges;

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
use mysql;
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%'
IDENTIFIED BY 'cisco.123' WITH GRANT OPTION;
flush privileges;

The second command we are issuing in MariaDB is to remove to 'blank' users that are tied to
the localhost account. These two blank users need to be removed so that we can access the
system properly from the network. MariaDB has a special command line tool available to
users called mysql_secure_installation that you can use to secure the installation when doing
so in a production network. For this lab we have specific requirements that we are building
directly with these SQL commands.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
USE mysql;
DELETE FROM `user` WHERE User='';
flush privileges;
exit

Once completed you should be able to access to database from the network and also use the
set of credentials to add all the proper databases needed by the various components of

http://openstack.ciscolive.com/printable.php?pod=3 21/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

OpenStack. From this point forward to access the MariaDB using the root account you will
have to specify the password cisco.123.

http://openstack.ciscolive.com/printable.php?pod=3 22/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

RabbitMQ

AMQP (Advanced Message Queue Protocol) is what the various components of OpenStack
like Cinder, Nova or Quantum use to communicate with each other. The AMQP broker sits in
between various OpenStack components and makes communication possible through a
message bus. Components can subscribe to events they are interested in and other
components can publish events to the bug. RabbitMQ takes care of getting the messages to
the interested recipients. The communication between the various components is simplified
via the broker such that little to no knowledge of the separate components is needed,
simplifying development and scalability of the OpenStack platform.

Step 17 Install RabbitMQ
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
yum ‐y install rabbitmq‐server

Once installed, you have to enable the service and start the service. As explained in the
MariaDB section, you enable the service such that it starts automatically during bootup. And
the start parameter starts the service right away.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
systemctl enable rabbitmq‐server.service
systemctl start rabbitmq‐server.service

You can now validate that the RabbitMQ process is properly running on the controller node.

http://openstack.ciscolive.com/printable.php?pod=3 23/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
systemctl status rabbitmq‐server.service

# systemctl status rabbitmq‐server.service
rabbitmq‐server.service ‐ RabbitMQ broker
   Loaded: loaded (/usr/lib/systemd/system/rabbitmq‐server.service; enabled)
   Active: active (running) since Wed 2014‐12‐24 13:33:22 EST; 8s ago
  Process: 54475 ExecStartPost=/usr/lib/rabbitmq/bin/rabbitmqctl wait /var/run/ra
bbitmq/pid (code=exited, status=0/SUCCESS)
  Process: 54443 ExecStartPre=/bin/sh ‐c /usr/lib/rabbitmq/bin/rabbitmqctl status
> /dev/null 2>&1 (code=exited, status=2)
Main PID: 54474 (beam.smp)
   CGroup: /system.slice/rabbitmq‐server.service
           ├─54474 /usr/lib64/erlang/erts‐5.10.4/bin/beam.smp ‐W w ‐K true ‐A30 ‐
P 1048576 ‐‐ ‐root /usr/lib64/erlang ‐progname erl ‐‐ ‐home /var/lib/rabbitmq ‐‐
‐pa /usr/lib/rabbitmq...
           ├─54586 inet_gethost 4
           └─54587 inet_gethost 4

Dec 24 13:33:21 localhost.localdomain rabbitmqctl[54475]: pid is 54474 ...
Dec 24 13:33:21 localhost.localdomain rabbitmq‐server[54474]: RabbitMQ 3.1.5. Cop
yright (C) 2007‐2013 GoPivotal, Inc.
Dec 24 13:33:22 localhost.localdomain rabbitmq‐server[54474]: ##  ##      License
d under the MPL.  See http://www.rabbitmq.com/
Dec 24 13:33:22 localhost.localdomain rabbitmq‐server[54474]: ##  ##
Dec 24 13:33:22 localhost.localdomain rabbitmq‐server[54474]: ##########  Logs: /
var/log/rabbitmq/[email protected]
Dec 24 13:33:22 localhost.localdomain rabbitmq‐server[54474]: ######  ##        /
var/log/rabbitmq/rabbit@localhost‐sasl.log
Dec 24 13:33:22 localhost.localdomain rabbitmq‐server[54474]: ##########
Dec 24 13:33:22 localhost.localdomain rabbitmq‐server[54474]: Starting broker...
completed with 0 plugins.
Dec 24 13:33:22 localhost.localdomain rabbitmqctl[54475]: ...done.
Dec 24 13:33:22 localhost.localdomain systemd[1]: Started RabbitMQ broker.

Step 18 Open Firewall Port
Since all the different nodes in the OpenStack cluster will be using RabbitMQ for inter process
calls, we need to insure that the firewall permits connectivity on the proper ports.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
firewall‐cmd ‐‐add‐port=5672/tcp ‐‐permanent
firewall‐cmd ‐‐reload

More information on ports for RabbitMQ servers RabbitMQ Environment
(https://www.rabbitmq.com/configure.html#defineenvironmentvariables).

http://openstack.ciscolive.com/printable.php?pod=3 24/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Step 19 Change the RabbitMQ password
You have to set the password for the RabbitMQ messaging engine.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
rabbitmqctl change_password guest cisco.123

ALERT!
If you install OpenStack in a production environment, make sure that name
resolution of the RabbitMQ server is configured correctly. RabbitMQ is very
strict with the hostname definitions as it structures the RabbitMQ message
queues based on the hostname. This is one reason we have set up these hosts
to have name resolution from /etc/hosts, DNS and environment variables.

http://openstack.ciscolive.com/printable.php?pod=3 25/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Keystone

Keystone is the identity service of OpenStack and ensures that users making API, CLI, or
GUI calls to the various OpenStack components are authorized to do so. Keystone's
responsibilities include:

Tracking users and their permissions.
Providing a catalog of available services with their API endpoints.

Step 20 Configure Keystone users in database
Keystone will need access to the MariaDB in OpenStack. For this lab, we only have two
nodes and the primary node ( the controller in this case ) has both keystone and MariaDB
running on one. We configure the credentials in the MariaDB such that Keystone will be able
to access MariaDB during it's installation and configures the database.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
mysql ‐‐user=root ‐‐password=cisco.123
CREATE DATABASE keystone;
GRANT ALL PRIVILEGES ON keystone.* TO
'keystone'@'localhost' IDENTIFIED BY
'cisco.123';
GRANT ALL PRIVILEGES ON keystone.* TO
'keystone'@'%' IDENTIFIED BY 'cisco.123';
exit;

Step 21 Install Keystone
As we start configuring OpenStack specific components in this lab, we will be using a
command called % openstack‐config throughout this manual to simplify configuring the
various OpenStack configuration files. OpenStack Config was installed previously. If you
where to install a multinode OpenStack environment, you would have to install the
OpenStack configuration utility via the package openstackutils.

http://openstack.ciscolive.com/printable.php?pod=3 26/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
yum ‐y install openstack‐keystone python‐
keystoneclient

Step 22 Configure Keystone Database Connection
The openstack‐config command requires four parameters:

The configuration file to modify
The section of the document to update
The variable to update
The value of that variable

The advantage of using this method instead of editing the file directly, is that the %
openstack‐config locates the variable and modifies it. If the variable does not already exist,
it will insert it. If you where to accomplish the same task by hand, you would have to first
locate the section and carefully make the change.

To start the Keystone configuration we begin by configuring the connection to the MariaDB
so that keystone can configure the database directly. When a OpenStack process first starts
and is initializing during installation it will directly modify the database to it's requirements.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐set
/etc/keystone/keystone.conf DEFAULT admin_token
cisco.123
openstack‐config ‐‐set
/etc/keystone/keystone.conf database connection
mysql://keystone:[email protected]/keystone
openstack‐config ‐‐set
/etc/keystone/keystone.conf token provider
keystone.token.providers.uuid.Provider
openstack‐config ‐‐set
/etc/keystone/keystone.conf token driver
keystone.token.persistence.backends.sql.Token

Step 23 Configure max Keystone workers to
control mysql connections
This step is mostly for the purpose of this lab to control the amount of connections to the
database from keystone. By default OpenStack bases the amount of workers that are forked
on the amount of CPU resources available on the platform. These servers have 32 CPU
cores and with two worker groups it would total over 64 active connections to MariaDB from
keystone alone. In a production environment you might want to keep the worker values at a
larger amount than what we have setup for this small lab.

On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 27/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
openstack‐config ‐‐set
/etc/keystone/keystone.conf DEFAULT
admin_workers 4
openstack‐config ‐‐set
/etc/keystone/keystone.conf DEFAULT
public_workers 4

Step 24 Configure PKI setup for certificates
To ensure secure connectivity across the OpenStack cluster, keystone needs to prepare the
PKI setup for proper certificate configuration.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
keystone‐manage pki_setup ‐‐keystone‐user
keystone ‐‐keystone‐group keystone
chown ‐R keystone:keystone /var/log/keystone
chown ‐R keystone:keystone /etc/keystone/ssl
chmod ‐R o‐rwx /etc/keystone/ssl
/bin/sh ‐c "keystone‐manage db_sync" keystone

If you wish, you can validate what keystone did to the database. If you goto the MariaDB
database you can see the database that was created and populated by keystone to be ready
for when the service is started.

http://openstack.ciscolive.com/printable.php?pod=3 28/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

# mysql ‐u root ‐p
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 6
Server version: 5.5.40‐MariaDB MariaDB Server

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> use keystone;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with ‐A

MariaDB [keystone]> exit
Bye

Step 25 Open Firewall Holes for Keystone

http://openstack.ciscolive.com/printable.php?pod=3 29/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Keystone is used by various processes across the OpenStack cluster. For this reason we
need to open the firewall port such that different services from different nodes may connect.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
firewall‐cmd ‐‐add‐port=35357/tcp ‐‐permanent
firewall‐cmd ‐‐add‐port=5000/tcp ‐‐permanent
firewall‐cmd ‐‐reload

Step 26 Enable the keystone service
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
systemctl enable openstack‐keystone.service
systemctl start openstack‐keystone.service

You can validate the status of the service also.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
systemctl status openstack‐keystone.service

# systemctl status openstack‐keystone.service
openstack‐keystone.service ‐ OpenStack Identity Service (code‐named Keystone)
   Loaded: loaded (/usr/lib/systemd/system/openstack‐keystone.service; enabled)
   Active: active (running) since Wed 2014‐12‐24 14:36:31 EST; 9s ago
Main PID: 56345 (keystone‐all)
   CGroup: /system.slice/openstack‐keystone.service
           ├─56345 /usr/bin/python /usr/bin/keystone‐all
           ├─56352 /usr/bin/python /usr/bin/keystone‐all
           ├─56353 /usr/bin/python /usr/bin/keystone‐all
           ├─56354 /usr/bin/python /usr/bin/keystone‐all
           └─56355 /usr/bin/python /usr/bin/keystone‐all

Dec 24 14:36:31 localhost.localdomain systemd[1]: Started OpenStack Identity Serv
ice (code‐named Keystone).

Step 27 Create CRON job to purge tokens hourly
By default, the Identity service stores expired tokens in the database indefinitely. The
accumulation of expired tokens considerably increases the database size and might degrade
service performance, particularly in environments with limited resources. For this reason you
will create a cronjob that runs and clears these tokens to keep the database trimmed.

http://openstack.ciscolive.com/printable.php?pod=3 30/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
(crontab ‐l ‐u keystone 2>&1 | grep ‐q
token_flush) || \
echo '@hourly /usr/bin/keystone‐manage
token_flush >/var/log/keystone/keystone‐
tokenflush.log 2>&1' \
>> /var/spool/cron/keystone

You can validate that the crontab was entered into the system via:

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
crontab ‐l ‐u keystone

# crontab ‐l ‐u keystone
@hourly /usr/bin/keystone‐manage token_flush >/var/log/keystone/keystone‐tokenflu
sh.log 2>&1

Step 28 Configure Keystone Admin User and
Tenant
Before configuring Keystone, you must setup some environment variables first. These
environment variables provide a pointer to the password for Keystone and the location that
Keystone is running.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
export OS_SERVICE_TOKEN=cisco.123
export
OS_SERVICE_ENDPOINT=http://10.0.236.31:35357/v2.0

Once completed you can add the Tenant configuration.

On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 31/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
keystone tenant‐create ‐‐name admin ‐‐
description "Admin Tenant"
keystone user‐create   ‐‐name admin ‐‐pass
cisco.123 ‐‐email root@localhost
keystone role‐create   ‐‐name admin
keystone user‐role‐add ‐‐tenant admin ‐‐user
admin ‐‐role admin
keystone role‐create   ‐‐name _member_
keystone user‐role‐add ‐‐tenant admin ‐‐user
admin ‐‐role _member_

When you issue this command the output will look similar to the following.

As you can observe each of these are tables with properties and values. One important value
that we will be utilizing is the id .

Step 29 Configure POD user and Tenant
On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 32/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
keystone tenant‐create ‐‐name TenantPOD3 ‐‐
description "Tenant POD3"
keystone user‐create ‐‐name userpod3 ‐‐pass
cisco.123
keystone user‐role‐add ‐‐tenant TenantPOD3 ‐‐
user userpod3 ‐‐role _member_

The output should be similar to:

Step 30 Configure Service Tenant
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
keystone tenant‐create ‐‐name service ‐‐
description "Service Tenant"
keystone service‐create ‐‐name keystone ‐‐type
identity ‐‐description "OpenStack Identity"

The output of the command should be similar to:

http://openstack.ciscolive.com/printable.php?pod=3 33/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Once this has been created you can use the command # keystone service‐list to get the

ID name that we need to create the endpoint. Luckily we have simplified this for you, but it is
a good idea to see the command.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
keystone service‐list

What you are looking for is that id value hex string. You use that to create the keystone
endpoint. We have provided for you a simple command that uses AWK to capture the output
of that id for the next command that we need for Keystone.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
keystone endpoint‐create \
  ‐‐service‐id $(keystone service‐list | awk '/
identity / {print $2}') \
  ‐‐publicurl http://10.0.236.31:5000/v2.0 \
  ‐‐internalurl http://10.0.236.31:5000/v2.0 \
  ‐‐adminurl http://10.0.236.31:35357/v2.0 \
  ‐‐region regionOne

http://openstack.ciscolive.com/printable.php?pod=3 34/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

And the output for that command should be similar to:

# keystone endpoint‐create \
>   ‐‐service‐id $(keystone service‐list | awk '/ identity / {print $2}') \
>   ‐‐publicurl http://10.0.236.31:5000/v2.0 \
>   ‐‐internalurl http://10.0.236.31:5000/v2.0 \
>   ‐‐adminurl http://10.0.236.31:35357/v2.0 \
>   ‐‐region regionOne
+‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+
|   Property  |              Value               |
+‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+
|   adminurl  |   http://10.0.236.31:35357/v2.0   |
|      id     | 2518b1ff94f24400952dd9047662d67e |
| internalurl |   http://10.0.236.31:5000/v2.0    |
|  publicurl  |   http://10.0.236.31:5000/v2.0    |
|    region   |            regionOne             |
|  service_id | ec4effb75fa0435f96dd871098596d77 |
+‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+

Step 31 Validate Keystone installation
The first step is to remove the environment variables that we had setup previously.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
unset OS_SERVICE_TOKEN OS_SERVICE_ENDPOINT

Then you have to set the values in Keystone.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
keystone ‐‐os‐tenant‐name admin ‐‐os‐username
admin ‐‐os‐password cisco.123    ‐‐os‐auth‐url
http://10.0.236.31:35357/v2.0 token‐get
keystone ‐‐os‐tenant‐name admin ‐‐os‐username
admin ‐‐os‐password cisco.123    ‐‐os‐auth‐url
http://10.0.236.31:35357/v2.0 tenant‐list
keystone ‐‐os‐tenant‐name admin ‐‐os‐username
admin ‐‐os‐password cisco.123    ‐‐os‐auth‐url
http://10.0.236.31:35357/v2.0 user‐list
keystone ‐‐os‐tenant‐name admin ‐‐os‐username
admin ‐‐os‐password cisco.123    ‐‐os‐auth‐url
http://10.0.236.31:35357/v2.0 role‐list

And complete the process for the user that was created

On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 35/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
keystone ‐‐os‐tenant‐name TenantPOD3 ‐‐os‐
username userpod3 ‐‐os‐password cisco.123 ‐‐os‐
auth‐url http://10.0.236.31:35357/v2.0 token‐get
keystone ‐‐os‐tenant‐name TenantPOD3 ‐‐os‐
username userpod3 ‐‐os‐password cisco.123 ‐‐os‐
auth‐url http://10.0.236.31:35357/v2.0 user‐list

This last command will create a ERROR as you are not set as the administrator from the
CLI.

Step 32 Create shell identity files
You will now create two separate files that will be used to change your identity status when
interacting with Keystone.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
cd ~
cat <<EOF > admin.sh
export OS_USERNAME=admin
export OS_TENANT_NAME=admin
export OS_PASSWORD=cisco.123
export OS_AUTH_URL=http://10.0.236.31:5000/v2.0/
export OS_REGION_NAME=regionOne
export PS1='[\u@\h \W(keystone_admin)]# '
EOF
chmod +x admin.sh

You will now create another file that takes the role of the user we have created.

[[IDSEQSTEP]]
cd ~
cat <<EOF > TenantPOD3.sh
export OS_USERNAME=userpod3
export OS_TENANT_NAME=TenantPOD3
export OS_PASSWORD=cisco.123
export OS_AUTH_URL=http://10.0.236.31:5000/v2.0/
export OS_REGION_NAME=regionOne
export PS1='[\u@\h \W(keystone_TenantPOD3)]# '
EOF
chmod +x TenantPOD3.sh

Step 33 Verify identity shell
You can validate that each shell is working properly by executing the shell command via the
#source command.

On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 36/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
cd ~
source admin.sh

The output should be minimal, yet the environment variables should be set.

[root@localhost ~(keystone_admin)]# env
OS_REGION_NAME=regionOne
OS_PASSWORD=cisco.123
PS1=[\u@\h \W(keystone_admin)]#
SELINUX_LEVEL_REQUESTED=
OS_AUTH_URL=http://10.0.236.31:5000/v2.0/
OS_USERNAME=admin
OS_TENANT_NAME=admin

You can assume the identity of the POD Tenant you are using with the same command.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
cd ~
source TenantPOD3.sh

http://openstack.ciscolive.com/printable.php?pod=3 37/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Glance

Glance provides an image store for users of OpenStack to create instances. Once glance is
configured, the administrator of the system can upload various disk images that have been
prebuilt with an OS such as Ubuntu, CentOS or Windows. The images may contain a plain
OS image or may contain preinstalled applications. The primary consumer of Glance
services is the Nova service that manages the compute resources in OpenStack.

Step 34 Open Firewall Hole
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
firewall‐cmd ‐‐add‐port=9292/tcp ‐‐permanent
firewall‐cmd ‐‐add‐port=9191/tcp ‐‐permanent
firewall‐cmd ‐‐add‐port=873/tcp ‐‐permanent
firewall‐cmd ‐‐add‐port=3260/tcp ‐‐permanent
firewall‐cmd ‐‐reload

Step 35 Configure Glance database and access to
database
Just as you completed previously, you have to provide Glance with the privileges to access
the database to configure it's tables.

On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 38/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
mysql ‐‐user=root ‐‐password=cisco.123
CREATE DATABASE glance;
GRANT ALL PRIVILEGES ON glance.* TO
'glance'@'localhost' IDENTIFIED BY 'cisco.123';
grant all privileges on glance.* to 'glance'@'%'
identified by 'cisco.123';
exit

Step 36 Add glance users to keystone
Once the database access credentials is completed, you will add the glance user to
keystone. For every OpenStack service we have to add, we add the service user into
Keystone. Previously you had created a script to assume the identity of the administrator.
You have to execute this command before you create the glance users such that it has
proper permissions to add the users.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
cd ~
source admin.sh

Now that you have assumed the identity of the admin user, you can issue the following
commands to create the user and the service for Glance in Keystone.

On the controller host 10.0.236.31 issue the commands:

[[IDSEQSTEP]]
keystone user‐create    ‐‐name glance ‐‐pass
cisco.123
keystone user‐role‐add  ‐‐user glance ‐‐tenant
service ‐‐role admin
keystone service‐create ‐‐name glance ‐‐type
image ‐‐description "OpenStack Image Service"
keystone endpoint‐create \
  ‐‐service‐id $(keystone service‐list | awk '/
image / {print $2}') \
  ‐‐publicurl http://10.0.236.31:9292 \
  ‐‐internalurl http://10.0.236.31:9292 \
  ‐‐adminurl http://10.0.236.31:9292 \
  ‐‐region regionOne

Step 37 Install Glance
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
yum ‐y install openstack‐glance python‐
glanceclient

http://openstack.ciscolive.com/printable.php?pod=3 39/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Step 38 Configure Glance API
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/glance/glance‐
api.conf database connection
mysql://glance:[email protected]/glance
openstack‐config ‐‐set /etc/glance/glance‐
api.conf DEFAULT workers 4
openstack‐config ‐‐set /etc/glance/glance‐
api.conf keystone_authtoken auth_uri
http://10.0.236.31:5000/v2.0
openstack‐config ‐‐set /etc/glance/glance‐
api.conf keystone_authtoken identity_uri
http://10.0.236.31:35357
openstack‐config ‐‐set /etc/glance/glance‐
api.conf keystone_authtoken admin_tenant_name
service
openstack‐config ‐‐set /etc/glance/glance‐
api.conf keystone_authtoken admin_user glance
openstack‐config ‐‐set /etc/glance/glance‐
api.conf keystone_authtoken admin_password
cisco.123
openstack‐config ‐‐set /etc/glance/glance‐
api.conf paste_deploy flavor keystone
openstack‐config ‐‐set /etc/glance/glance‐
api.conf glance_store default_store file
openstack‐config ‐‐set /etc/glance/glance‐
api.conf glance_store filesystem_store_datadir
/var/lib/glance/images/

Step 39 Configure Glance Registry
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/glance/glance‐
registry.conf database connection
mysql://glance:[email protected]/glance
openstack‐config ‐‐set /etc/glance/glance‐
registry.conf DEFAULT workers 4
openstack‐config ‐‐set /etc/glance/glance‐
registry.conf keystone_authtoken auth_uri
http://10.0.236.31:5000/v2.0
openstack‐config ‐‐set /etc/glance/glance‐
registry.conf keystone_authtoken identity_uri
http://10.0.236.31:35357
openstack‐config ‐‐set /etc/glance/glance‐
registry.conf keystone_authtoken
admin_tenant_name service
openstack‐config ‐‐set /etc/glance/glance‐
registry.conf keystone_authtoken admin_user
glance
openstack‐config ‐‐set /etc/glance/glance‐
registry.conf keystone_authtoken
admin_password cisco.123
http://openstack.ciscolive.com/printable.php?pod=3 40/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

admin_password cisco.123
openstack‐config ‐‐set /etc/glance/glance‐
registry.conf paste_deploy flavor keystone

Step 40 Synchronize Glance database
The following command will connect glance to the MariaDB on the controller host and insert
the proper table SQL structure that glance needs.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
su ‐s /bin/sh ‐c "glance‐manage db_sync" glance

Step 41 Enable Glance service and startup
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
systemctl enable openstack‐glance‐api.service
openstack‐glance‐registry.service
systemctl start openstack‐glance‐api.service
openstack‐glance‐registry.service

You can validate that Glance was correctly installed and configured.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
systemctl status systemctl status openstack‐
glance‐api.service

# systemctl status openstack‐glance‐api.service
openstack‐glance‐api.service ‐ OpenStack Image Service (code‐named Glance) API se
rver
   Loaded: loaded (/usr/lib/systemd/system/openstack‐glance‐api.service; enabled)
   Active: active (running) since Fri 2014‐12‐26 21:49:50 EST; 11s ago
Main PID: 33550 (glance‐api)
   CGroup: /system.slice/openstack‐glance‐api.service
           ├─33550 /usr/bin/python /usr/bin/glance‐api
           ├─33566 /usr/bin/python /usr/bin/glance‐api
           └─33567 /usr/bin/python /usr/bin/glance‐api

Dec 26 21:49:50 localhost.localdomain systemd[1]: Started OpenStack Image Service
(code‐named Glance) API server.

On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 41/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
systemctl status openstack‐glance‐
registry.service

# systemctl status openstack‐glance‐registry.service
openstack‐glance‐registry.service ‐ OpenStack Image Service (code‐named Glance) R
egistry server
   Loaded: loaded (/usr/lib/systemd/system/openstack‐glance‐registry.service; ena
bled)
   Active: active (running) since Fri 2014‐12‐26 21:49:50 EST; 20s ago
Main PID: 33551 (glance‐registry)
   CGroup: /system.slice/openstack‐glance‐registry.service
           ├─33551 /usr/bin/python /usr/bin/glance‐registry
           ├─33564 /usr/bin/python /usr/bin/glance‐registry
           └─33565 /usr/bin/python /usr/bin/glance‐registry

Dec 26 21:49:49 localhost.localdomain systemd[1]: Starting OpenStack Image Servic
e (code‐named Glance) Registry server...
Dec 26 21:49:50 localhost.localdomain systemd[1]: Started OpenStack Image Service
(code‐named Glance) Registry server.
Hint: Some lines were ellipsized, use ‐l to show in full.

Step 42 Download ISO image for Glance Image
Repository
Since Glance is operational, we will download a small linux image called Cirros.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
mkdir /tmp/images
cd /tmp/images
wget http://cdn.download.cirros‐
cloud.net/0.3.3/cirros‐0.3.3‐x86_64‐disk.img

Step 43 Add ISO image to Glance Image
Repository
Using glance, we will take the Cirros image and add it to the glance repository. Once glance
has the image in the repository we can create instances in OpenStack of the Cirros image. If
you had this in a production environment, you could download various different operating
system flavors and insert them into glance for use by the Tenants of OpenStack.

On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 42/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
glance image‐create ‐‐name "cirros‐0.3.3‐x86_64"
‐‐file cirros‐0.3.3‐x86_64‐disk.img ‐‐disk‐
format qcow2 ‐‐container‐format bare ‐‐is‐public
True ‐‐progress
cd
rm ‐rf /tmp/images

Step 44 List image repository in glance
The command glance image‐list will provide you the list of all images that are available
for users in this OpenStack system.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
glance image‐list

Now that the image has been added to glance, you can look at the list of images in glance
with the command glance image‐list

http://openstack.ciscolive.com/printable.php?pod=3 43/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Configure Nova on Controller Host

As you read earlier, Nova is the compute provisioning service of OpenStack. It manages
scheduling and placement of instances in the OpenStack system.

For this lab you will be configuring two separate nodes as compute. The controller is
performing a dual role as both a controller and a compute node while the second compute
node that is only a compute node. Note that the Nova service itself is not a hypervisor. Nova
provides an API abstract the hypervisor from the applications and then leverages underlying
hypervisors like KVM.

Step 45 Open Firewall Holes
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
firewall‐cmd ‐‐add‐port=8774/tcp ‐‐permanent
firewall‐cmd ‐‐add‐port=8773/tcp ‐‐permanent
firewall‐cmd ‐‐add‐port=8775/tcp ‐‐permanent
firewall‐cmd ‐‐add‐port=6080/tcp ‐‐permanent
firewall‐cmd ‐‐add‐port=6081/tcp ‐‐permanent
firewall‐cmd ‐‐add‐port=6082/tcp ‐‐permanent
firewall‐cmd ‐‐add‐port=5900‐5999/tcp ‐‐
permanent
firewall‐cmd ‐‐reload

Step 46 Configure Nova database and access to
database

http://openstack.ciscolive.com/printable.php?pod=3 44/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
mysql ‐‐user=root ‐‐password=cisco.123
CREATE DATABASE nova;
GRANT ALL PRIVILEGES ON nova.* TO
'nova'@'localhost' IDENTIFIED BY 'cisco.123';
GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'%'
IDENTIFIED BY 'cisco.123';
exit

Step 47 Add Nova users to keystone
Once the database is ready, you can add the Nova users to keystone.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
cd ~
source admin.sh

Now that you have assumed the identity of the admin user, you can issue the following
commands to create the user and the service for Nova in Keystone.

On the controller host 10.0.236.31 issue the commands:

[[IDSEQSTEP]]
keystone user‐create    ‐‐name nova ‐‐pass
cisco.123
keystone user‐role‐add  ‐‐user nova ‐‐tenant
service ‐‐role admin
keystone service‐create ‐‐name nova ‐‐type
compute ‐‐description "OpenStack Compute"
keystone endpoint‐create \
  ‐‐service‐id $(keystone service‐list | awk '/
compute / {print $2}') \
  ‐‐publicurl http://10.0.236.31:8774/v2/%\
(tenant_id\)s \
  ‐‐internalurl http://10.0.236.31:8774/v2/%\
(tenant_id\)s \
  ‐‐adminurl http://10.0.236.31:8774/v2/%\
(tenant_id\)s \
  ‐‐region regionOne

Step 48 Install Nova
On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 45/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
yum ‐y install openstack‐nova‐api openstack‐
nova‐cert openstack‐nova‐conductor \
openstack‐nova‐console openstack‐nova‐
novncproxy openstack‐nova‐scheduler \
python‐novaclient openstack‐nova‐compute
sysfsutils

Step 49 Configure Nova INI file
On the controller host 10.0.236.31 issue the command:
The first command is to establish the SQL connection base parameters. This provides Nova
with the credentials needed to reach the database and it's default database (nova).

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/nova/nova.conf
database connection
mysql://nova:[email protected]/nova

Limit the workers for Nova in this system. In production environments with servers that have
large amount of CPU cores, you might want to reduce the worker size. The reason is that for
each CPU core, OpenStack instantiates a thread that connects to MySQL. In large
datacenter environments the amount of workers keeping established sessions against the
database can cause scaling issues. For this lab we are limiting them to 4 as the server has
many CPU cores.

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT ec2_workers 4
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT osapi_compute_workers 4
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT metadata_workers 4
openstack‐config ‐‐set /etc/nova/nova.conf
conductor workers 4

You will now configure the keystone authentication and RabbitMQ backend.

http://openstack.ciscolive.com/printable.php?pod=3 46/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT rpc_backend rabbit
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT rabbit_host 10.0.236.31
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT rabbit_password cisco.123
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT auth_strategy keystone
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT my_ip 10.0.236.31
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT network_api_class
nova.network.neutronv2.api.API
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT security_group_api neutron
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT linuxnet_interface_driver
nova.network.linux_net.LinuxOVSInterfaceDriver
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT firewall_driver
nova.virt.firewall.NoopFirewallDriver
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT verbose True
openstack‐config ‐‐set /etc/nova/nova.conf
keystone_authtoken auth_uri
http://10.0.236.31:5000/v2.0
openstack‐config ‐‐set /etc/nova/nova.conf
keystone_authtoken identity_uri
http://10.0.236.31:35357
openstack‐config ‐‐set /etc/nova/nova.conf
keystone_authtoken admin_tenant_name service
openstack‐config ‐‐set /etc/nova/nova.conf
keystone_authtoken admin_user nova
openstack‐config ‐‐set /etc/nova/nova.conf
keystone_authtoken admin_password cisco.123

Define the VNC parameters to reach the instances created by Nova.

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT vnc_enabled True
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT vnc_keymap en‐us
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT vncserver_listen 0.0.0.0
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT vncserver_proxyclient_address
10.0.236.31
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT novncproxy_base_url
http://10.0.236.31:6080/vnc_auto.html

Define the glance host so that Nova can reach ISO images for instance creation.

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/nova/nova.conf
glance host 10.0.236.31

http://openstack.ciscolive.com/printable.php?pod=3 47/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Specify the virtualization driver (hypervisor) to use for creating instances.

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/nova/nova.conf
libvirt virt_type kvm

Configure neutron access and metadata. This makes it possible for Nova to learn how to
configure the network components of instances that it creates.

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/nova/nova.conf
neutron url http://10.0.236.31:9696
openstack‐config ‐‐set /etc/nova/nova.conf
neutron auth_strategy keystone
openstack‐config ‐‐set /etc/nova/nova.conf
neutron admin_auth_url
http://10.0.236.31:35357/v2.0
openstack‐config ‐‐set /etc/nova/nova.conf
neutron admin_tenant_name service
openstack‐config ‐‐set /etc/nova/nova.conf
neutron admin_username neutron
openstack‐config ‐‐set /etc/nova/nova.conf
neutron admin_password cisco.123
openstack‐config ‐‐set /etc/nova/nova.conf
neutron service_metadata_proxy True
openstack‐config ‐‐set /etc/nova/nova.conf
neutron metadata_proxy_shared_secret cisco.123

Step 50 Initialize the Database for Nova
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
su ‐s /bin/sh ‐c "nova‐manage db sync" nova

Step 51 Configure system to start Nova
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
systemctl enable openstack‐nova‐api.service
openstack‐nova‐cert.service \
  openstack‐nova‐consoleauth.service openstack‐
nova‐scheduler.service \
  openstack‐nova‐conductor.service openstack‐
nova‐novncproxy.service
systemctl start openstack‐nova‐api.service
openstack‐nova‐cert.service \
  openstack‐nova‐consoleauth.service openstack‐
nova‐scheduler.service \
  openstack‐nova‐conductor.service openstack‐
nova‐novncproxy.service

http://openstack.ciscolive.com/printable.php?pod=3 48/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Step 52 Validate Nova system status
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
systemctl status openstack‐nova‐api.service
openstack‐nova‐cert.service \
openstack‐nova‐consoleauth.service openstack‐
nova‐scheduler.service \
openstack‐nova‐conductor.service openstack‐
nova‐novncproxy.service

# systemctl status openstack‐nova‐api.service openstack‐nova‐cert.service \
>   openstack‐nova‐consoleauth.service openstack‐nova‐scheduler.service \
>   openstack‐nova‐conductor.service openstack‐nova‐novncproxy.service
openstack‐nova‐api.service ‐ OpenStack Nova API Server
   Loaded: loaded (/usr/lib/systemd/system/openstack‐nova‐api.service; enabled)
   Active: active (running) since Thu 2015‐01‐15 11:31:34 EST; 23h ago
Main PID: 13448 (nova‐api)
   CGroup: /system.slice/openstack‐nova‐api.service
           ‚î‚îÄ13448 /usr/bin/python /usr/bin/nova‐api
           ‚î‚îÄ13455 /usr/bin/python /usr/bin/nova‐api
           ‚î‚îÄ13456 /usr/bin/python /usr/bin/nova‐api
           ‚î‚îÄ13457 /usr/bin/python /usr/bin/nova‐api
           ‚î‚îÄ13458 /usr/bin/python /usr/bin/nova‐api
           ‚î‚îÄ13459 /usr/bin/python /usr/bin/nova‐api
           ‚î‚îÄ13460 /usr/bin/python /usr/bin/nova‐api
           ‚î‚îÄ13461 /usr/bin/python /usr/bin/nova‐api
           ‚î‚îÄ13462 /usr/bin/python /usr/bin/nova‐api
           ‚î‚îÄ13469 /usr/bin/python /usr/bin/nova‐api
           ‚î‚îÄ13470 /usr/bin/python /usr/bin/nova‐api
           ‚î‚îÄ13471 /usr/bin/python /usr/bin/nova‐api
           ‚îî‚îÄ13472 /usr/bin/python /usr/bin/nova‐api

Jan 15 11:31:33 pod4‐controller systemd[1]: Starting OpenStack Nova API Server...
Jan 15 11:31:34 pod4‐controller sudo[13463]: nova : TTY=unknown ; PWD=/ ; USER=ro
ot ; COMMAND=/bin/nova‐rootwrap /etc/nova/rootwrap.conf iptables‐save ‐c
Jan 15 11:31:34 pod4‐controller sudo[13466]: nova : TTY=unknown ; PWD=/ ; USER=ro
ot ; COMMAND=/bin/nova‐rootwrap /etc/nova/rootwrap.conf iptables‐restore ‐c
Jan 15 11:31:34 pod4‐controller systemd[1]: Started OpenStack Nova API Server.

openstack‐nova‐cert.service ‐ OpenStack Nova Cert Server
   Loaded: loaded (/usr/lib/systemd/system/openstack‐nova‐cert.service; enabled)
   Active: active (running) since Thu 2015‐01‐15 11:29:46 EST; 23h ago
Main PID: 12629 (nova‐cert)
   CGroup: /system.slice/openstack‐nova‐cert.service
           ‚îî‚îÄ12629 /usr/bin/python /usr/bin/nova‐cert

Jan 15 11:29:45 pod4‐controller systemd[1]: Starting OpenStack Nova Cert Server..
.
Jan 15 11:29:46 pod4‐controller systemd[1]: Started OpenStack Nova Cert Server.

openstack‐nova‐consoleauth.service ‐ OpenStack Nova VNC console auth Server
Loaded: loaded (/usr/lib/systemd/system/openstack‐nova‐consoleauth.service; en
abled)
Active: active (running) since Thu 2015‐01‐15 11:29:46 EST; 23h ago
Main PID: 12630 (nova‐consoleaut)

http://openstack.ciscolive.com/printable.php?pod=3 49/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

CGroup: /system.slice/openstack‐nova‐consoleauth.service
‚îî‚îÄ12630 /usr/bin/python /usr/bin/nova‐consoleauth

Jan 15 11:29:45 pod4‐controller systemd[1]: Starting OpenStack Nova VNC console a
uth Server...
Jan 15 11:29:46 pod4‐controller systemd[1]: Started OpenStack Nova VNC console au
th Server.

openstack‐nova‐scheduler.service ‐ OpenStack Nova Scheduler Server
   Loaded: loaded (/usr/lib/systemd/system/openstack‐nova‐scheduler.service; enab
led)
   Active: active (running) since Thu 2015‐01‐15 11:31:18 EST; 23h ago
Main PID: 13338 (nova‐scheduler)
   CGroup: /system.slice/openstack‐nova‐scheduler.service
           ‚îî‚îÄ13338 /usr/bin/python /usr/bin/nova‐scheduler

Jan 15 11:31:18 pod4‐controller systemd[1]: Started OpenStack Nova Scheduler Serv
er.

openstack‐nova‐conductor.service ‐ OpenStack Nova Conductor Server
   Loaded: loaded (/usr/lib/systemd/system/openstack‐nova‐conductor.service; enab
led)
   Active: active (running) since Thu 2015‐01‐15 11:31:18 EST; 23h ago
Main PID: 13336 (nova‐conductor)
   CGroup: /system.slice/openstack‐nova‐conductor.service
           ‚î‚îÄ13336 /usr/bin/python /usr/bin/nova‐conductor
           ‚î‚îÄ13359 /usr/bin/python /usr/bin/nova‐conductor
           ‚î‚îÄ13360 /usr/bin/python /usr/bin/nova‐conductor
           ‚î‚îÄ13361 /usr/bin/python /usr/bin/nova‐conductor
           ‚îî‚îÄ13362 /usr/bin/python /usr/bin/nova‐conductor

[CUT]

openstack‐nova‐novncproxy.service ‐ OpenStack Nova NoVNC Proxy Server
   Loaded: loaded (/usr/lib/systemd/system/openstack‐nova‐novncproxy.service; ena
bled)
   Active: active (running) since Thu 2015‐01‐15 11:29:45 EST; 23h ago
Main PID: 12633 (nova‐novncproxy)
   CGroup: /system.slice/openstack‐nova‐novncproxy.service
           ‚îî‚îÄ12633 /usr/bin/python /usr/bin/nova‐novncproxy ‐‐web /usr/share/
novnc/

[CUT]
Hint: Some lines were ellipsized, use ‐l to show in full.

Step 53 Determine system hardware CPU type
For virtualization we have to determine the hardware type of the CPU. If you are running
OpenStack on a virtual machine, it might not be possible to run KVM in itself as it requires
hardware acceleration features from the CPU. You can determine the hardware type by
issuing the command.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
egrep ‐c '(vmx|svm)' /proc/cpuinfo

http://openstack.ciscolive.com/printable.php?pod=3 50/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

# egrep ‐c '(vmx|svm)' /proc/cpuinfo
2
#

Based on the result from the command you can derive the following from the table:

Result Required Hypervisor

0 Qemu

>1 KVM
You can now then validate that the KVM module is loaded.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
lsmod | grep kvm

# lsmod | grep kvm
kvm_intel 138567 0
kvm 441119 1 kvm_intel

Step 54 Configure system to start virtualization
engine
Now that you have validated the virtualization requirements, you can start that functionality. By
default KVM is the default hypervisor used in OpenStack.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
systemctl enable libvirtd.service openstack‐
nova‐compute.service
systemctl start libvirtd.service
systemctl start openstack‐nova‐compute.service

Step 55 Validate Nova instance
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
nova service‐list

http://openstack.ciscolive.com/printable.php?pod=3 51/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

# nova service‐list
+‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐
‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+
| Id | Binary           | Host            | Zone     | Status  | State | Updated_
at                 | Disabled Reason |
+‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐
‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+
| 1  | nova‐cert        | pod3‐controller | internal | enabled | up    | 2015‐01‐
11T02:48:12.000000 | ‐               |
| 2  | nova‐consoleauth | pod3‐controller | internal | enabled | up    | 2015‐01‐
11T02:48:13.000000 | ‐               |
| 3  | nova‐conductor   | pod3‐controller | internal | enabled | up    | 2015‐01‐
11T02:48:08.000000 | ‐               |
| 4  | nova‐scheduler   | pod3‐controller | internal | enabled | up    | 2015‐01‐
11T02:48:08.000000 | ‐               |
| 5  | nova‐compute     | pod3‐controller | nova     | enabled | up    | 2015‐01‐
11T02:48:08.000000 | ‐               |
+‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐
‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+

Nova should at this point also see the image resources provided by glance. The command
nova image‐list should present you with the same information we got from glance. Yet this
is Nova interacting with glance retrieving that information.

[[IDSEQSTEP]]
nova image‐list

http://openstack.ciscolive.com/printable.php?pod=3 52/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Configure Nova on Compute Host

Next you must configure the second host in the cluster as a compute host. The controller
host contains the first compute host. In a large scale OpenStack environment that wouldn't
be the case. Separate hosts would be required for scalability. The first compute host contains
the control processes for the compute host. Each additional compute host has a smaller
configuration footprint than the controller.

NOTE: This commands for this section should be entered on the second host the compute
node, not the controller node. Ensure you connect to the correct PuTTY session before
entering these commands.

Step 56 Open Firewall Holes
As a first step in the compute host you need to open the firewall ports for Nova.

On the compute host 10.0.236.32 issue the command:

Step 57 Install Nova on compute node
On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
yum ‐y install openstack‐nova‐compute sysfsutils

Step 58 Configure Nova on compute
On the compute host 10.0.236.32 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 53/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT rpc_backend rabbit
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT rabbit_host 10.0.236.31
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT rabbit_password cisco.123
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT auth_strategy keystone
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT my_ip 10.0.236.32
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT vnc_enabled True
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT vncserver_listen 0.0.0.0
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT vncserver_proxyclient_address
10.0.236.32
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT novncproxy_base_url
http://10.0.236.31:6080/vnc_auto.html
openstack‐config ‐‐set /etc/nova/nova.conf
keystone_authtoken auth_uri
http://10.0.236.31:5000/v2.0
openstack‐config ‐‐set /etc/nova/nova.conf
keystone_authtoken identity_uri
http://10.0.236.31:35357
openstack‐config ‐‐set /etc/nova/nova.conf
keystone_authtoken admin_tenant_name service
openstack‐config ‐‐set /etc/nova/nova.conf
keystone_authtoken admin_user nova
openstack‐config ‐‐set /etc/nova/nova.conf
keystone_authtoken admin_password cisco.123
openstack‐config ‐‐set /etc/nova/nova.conf
glance host 10.0.236.31
openstack‐config ‐‐set /etc/nova/nova.conf
libvirt virt_type kvm

Step 59 Determine system hardware CPU type
For virtualization we have to determine the hardware type of the CPU. If you are running
OpenStack on a virtual machine, it might not be possible to run KVM in itself as it requires
hardware acceleration features from the CPU. You can determine the hardware type by
issuing the command.

On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
egrep ‐c '(vmx|svm)' /proc/cpuinfo

# egrep ‐c '(vmx|svm)' /proc/cpuinfo
2
#

Based on the result from the command you can derive the following from the table:

http://openstack.ciscolive.com/printable.php?pod=3 54/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Result Required Hypervisor

0 Qemu

>1 KVM
You can now then validate that the KVM module is loaded.

On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
lsmod | grep kvm

# lsmod | grep kvm
kvm_intel 138567 0
kvm 441119 1 kvm_intel

Step 60 Configure system to start virtualization
engine
Now that you have validated the virtualization requirements, you can start that functionality. By
default KVM is the default hypervisor used in OpenStack.

On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
systemctl enable libvirtd.service openstack‐
nova‐compute.service
systemctl start libvirtd.service
systemctl start openstack‐nova‐compute.service

Step 61 Validate Nova system status
Run the following commands to check the status of the virtualization driver. Don't worry about
the errors that show up in red. This is because we have not loaded drivers for Linux Containers
as we are not using them in this lab.

On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
systemctl status libvirtd.service
systemctl status openstack‐nova‐compute.service

http://openstack.ciscolive.com/printable.php?pod=3 55/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

# systemctl status libvirtd.service
libvirtd.service ‐ Virtualization daemon
   Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled)
   Active: active (running) since Thu 2015‐01‐15 10:30:32 EST; 24h ago
Main PID: 3879 (libvirtd)
   CGroup: /system.slice/libvirtd.service
           ‚îî‚îÄ3879 /usr/sbin/libvirtd

Jan 15 10:30:32 pod4‐compute2.ecatsrtpdmz.cisco.com libvirtd[3879]: libvirt versi
on: 1.1.1, package: 29.el7_0.4 (CentOS Build...org)
Jan 15 10:30:32 pod4‐compute2.ecatsrtpdmz.cisco.com libvirtd[3879]: Module /usr/l
ib64/libvirt/connection‐driver/libvirt_drive...ible
Jan 15 10:30:32 pod4‐compute2.ecatsrtpdmz.cisco.com systemd[1]: Started Virtualiz
ation daemon.
Jan 15 11:30:04 pod4‐compute2 systemd[1]: Started Virtualization daemon.
Jan 15 11:35:28 pod4‐compute2 libvirtd[3879]: End of file while reading data: Inp
ut/output error
Jan 16 10:58:45 pod4‐compute2 systemd[1]: Started Virtualization daemon.
Hint: Some lines were ellipsized, use ‐l to show in full.
[root@pod4‐compute2 ~]# systemctl status openstack‐nova‐compute.service
openstack‐nova‐compute.service ‐ OpenStack Nova Compute Server
   Loaded: loaded (/usr/lib/systemd/system/openstack‐nova‐compute.service; enable
d)
   Active: active (running) since Thu 2015‐01‐15 11:35:29 EST; 23h ago
Main PID: 17151 (nova‐compute)
   CGroup: /system.slice/openstack‐nova‐compute.service
           ‚îî‚îÄ17151 /usr/bin/python /usr/bin/nova‐compute

Jan 16 06:02:47 pod4‐compute2 sudo[15029]: nova : TTY=unknown ; PWD=/ ; USER=root
; COMMAND=/bin/nova‐rootwrap /etc/nova/ro...221da6
Jan 16 06:42:48 pod4‐compute2 sudo[19882]: nova : TTY=unknown ; PWD=/ ; USER=root
; COMMAND=/bin/nova‐rootwrap /etc/nova/ro...221da6
Jan 16 07:23:59 pod4‐compute2 sudo[24839]: nova : TTY=unknown ; PWD=/ ; USER=root
; COMMAND=/bin/nova‐rootwrap /etc/nova/ro...221da6
Jan 16 08:04:14 pod4‐compute2 sudo[29699]: nova : TTY=unknown ; PWD=/ ; USER=root
; COMMAND=/bin/nova‐rootwrap /etc/nova/ro...221da6
Jan 16 08:46:07 pod4‐compute2 sudo[2315]: nova : TTY=unknown ; PWD=/ ; USER=root
; COMMAND=/bin/nova‐rootwrap /etc/nova/roo...221da6
Jan 16 09:26:30 pod4‐compute2 sudo[7266]: nova : TTY=unknown ; PWD=/ ; USER=root
; COMMAND=/bin/nova‐rootwrap /etc/nova/roo...221da6
Jan 16 10:06:41 pod4‐compute2 sudo[12099]: nova : TTY=unknown ; PWD=/ ; USER=root
; COMMAND=/bin/nova‐rootwrap /etc/nova/ro...221da6
Jan 16 10:47:57 pod4‐compute2 sudo[17156]: nova : TTY=unknown ; PWD=/ ; USER=root
; COMMAND=/bin/nova‐rootwrap /etc/nova/ro...221da6
Jan 16 10:58:45 pod4‐compute2 systemd[1]: Started OpenStack Nova Compute Server.
Jan 16 10:59:26 pod4‐compute2 systemd[1]: Started OpenStack Nova Compute Server.
Hint: Some lines were ellipsized, use ‐l to show in full.

Step 62 Create shell identity files
You will now create to separate files that will be used to change your identity status when
interacting with Keystone.

On the compute host 10.0.236.32 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 56/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

You will now create another file that takes the role of the user we have created.

Step 63 Assume admin identity for CLI
Once the database is ready, you can add the Nova users to keystone.

On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
cd ~
source admin.sh

Step 64 Validate Nova instance
In the output of the following command, verify that the dedicated compute node (pod3
compute2) is online and enabled.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
nova service‐list

http://openstack.ciscolive.com/printable.php?pod=3 57/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

# nova service‐list
+‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐
‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+
| Id | Binary           | Host            | Zone     | Status  | State | Updated_
at                 | Disabled Reason |
+‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐
‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+
| 1  | nova‐cert        | pod3‐controller | internal | enabled | up    | 2015‐01‐
11T02:50:13.000000 | ‐               |
| 2  | nova‐consoleauth | pod3‐controller | internal | enabled | up    | 2015‐01‐
11T02:50:13.000000 | ‐               |
| 3  | nova‐conductor   | pod3‐controller | internal | enabled | up    | 2015‐01‐
11T02:50:08.000000 | ‐               |
| 4  | nova‐scheduler   | pod3‐controller | internal | enabled | up    | 2015‐01‐
11T02:50:10.000000 | ‐               |
| 5  | nova‐compute     | pod3‐controller | nova     | enabled | up    | 2015‐01‐
11T02:50:08.000000 | ‐               |
| 6  | nova‐compute     | pod3‐compute2   | nova     | enabled | up    | 2015‐01‐
11T02:50:12.000000 | ‐               |
+‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐
‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+

http://openstack.ciscolive.com/printable.php?pod=3 58/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Configure Neutron Controller Host

OpenStack Networking (neutron) manages all networking functions for the Virtual Networking
Infrastructure (VNI) and the access layer aspects of the Physical Networking Infrastructure
(PNI) in your OpenStack environment. OpenStack Networking enables tenants to create
advanced virtual network topologies including services such as firewalls, load balancers, and
virtual private networks (VPNs).

Neutron provides the networks, subnets, and routers object abstractions. Each abstraction
has functionality that mimics its physical counterpart: networks contain subnets, and routers
route traffic between different subnet and networks.

Each router has one gateway that connects to a network, and many interfaces connected to
subnets. Subnets can access machines on other subnets connected to the same router.

Any given Neutron set up has at least one external network. Unlike the other networks, the
external network is not merely a virtually defined network. Instead, it represents a view into a
slice of the physical, external network accessible outside the OpenStack installation. IP
addresses on the external network are accessible by anybody physically on the outside
network. Because the external network merely represents a view into the outside network,
DHCP is disabled on this network.

In addition to external networks, any Neutron set up has one or more internal networks called
Tenant networks. These softwaredefined networks connect directly to the VMs. Only the
VMs on a given internal network, or those on subnets connected through interfaces to a
similar router, can access VMs connected to that network directly.

For the outside network to access VMs, and vice versa, routers between the networks are
needed. Each router has one gateway that is connected to a network and many interfaces
that are connected to subnets. Like a physical router, subnets can access machines on other
subnets that are connected to the same router, and machines can access the outside
network through the gateway for the router.

Additionally, you can allocate IP addresses on external networks to ports on the internal
network. Whenever a virtual device (a virtual NIC on an instance) is connected to a subnet,
that connection is called a port. You can associate external network IP addresses with ports

http://openstack.ciscolive.com/printable.php?pod=3 59/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

to VMs. This way, entities on the outside network can access VMs. This functionality is
called a Floating IP in OpenStack and basically provides a static NAT translation between an
external network and a port on a Tenant network.

Networking also supports security groups. Security groups enable administrators to define
basic firewall rules in groups. A VM can belong to one or more security groups, and Neutron
applies the rules in those security groups to block or unblock ports, port ranges, or traffic
types for that VM.

Each plugin that Neutron uses has its own concepts. While not vital to operating the VNI
and OpenStack environment, understanding these concepts can help you set up Neutron. All
Neutron installations use a core plugin and a security group plugin (or just the NoOp
security group plugin). Additionally, FirewallasaService (FWaaS) and LoadBalancerasa
Service (LBaaS) plugins are available.

NOTE: Remember to switch back to your Controller node in PuTTY for this section.

Step 65 Open Firewall Holes
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
firewall‐cmd ‐‐add‐port=9696/tcp ‐‐permanent
firewall‐cmd ‐‐reload

Step 66 Configure Neutron database and access
to database
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
mysql ‐‐user=root ‐‐password=cisco.123
CREATE DATABASE neutron;
GRANT ALL PRIVILEGES ON neutron.* TO
'neutron'@'localhost' IDENTIFIED BY 'cisco.123';
GRANT ALL PRIVILEGES ON neutron.* TO
'neutron'@'%' IDENTIFIED BY 'cisco.123';
exit

Step 67 Add Neutron users to keystone
Once the database is ready, you can add the Neutron users to keystone.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
cd ~
source admin.sh

http://openstack.ciscolive.com/printable.php?pod=3 60/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Now that you have assumed the identity of the admin user, you can issue the following
commands to create the user and the service for Neutron in Keystone.

On the controller host 10.0.236.31 issue the commands:

[[IDSEQSTEP]]
keystone user‐create    ‐‐name neutron ‐‐pass
cisco.123
keystone user‐role‐add  ‐‐user neutron ‐‐tenant
service ‐‐role admin
keystone service‐create ‐‐name neutron ‐‐type
network ‐‐description "OpenStack Networking"
keystone endpoint‐create \
  ‐‐service‐id $(keystone service‐list | awk '/
network / {print $2}') \
  ‐‐publicurl http://10.0.236.31:9696 \
  ‐‐adminurl http://10.0.236.31:9696 \
  ‐‐internalurl http://10.0.236.31:9696 \
  ‐‐region regionOne

Step 68 Install Neutron
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
yum ‐y install openstack‐neutron openstack‐
neutron‐ml2 python‐neutronclient openstack‐
neutron‐openvswitch which

Step 69 Configure Neutron INI
On the controller host 10.0.236.31 issue the command:
Configure the database connection and limit Neutron workers to 4.

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/neutron/neutron.conf
database connection
mysql://neutron:[email protected]/neutron
openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT osapi_compute_workers 4
openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT workers 4
openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT ec2_workers 4
openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT metadata_workers 4

Configure the connection to the RabbitMQ server.

http://openstack.ciscolive.com/printable.php?pod=3 61/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT rpc_backend rabbit
openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT rabbit_host 10.0.236.31
openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT rabbit_password cisco.123

Configure the keystone server auth URI parameters.

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT auth_strategy keystone
openstack‐config ‐‐set /etc/neutron/neutron.conf
keystone_authtoken auth_uri
http://10.0.236.31:5000/v2.0
openstack‐config ‐‐set /etc/neutron/neutron.conf
keystone_authtoken identity_uri
http://10.0.236.31:35357
openstack‐config ‐‐set /etc/neutron/neutron.conf
keystone_authtoken admin_tenant_name service
openstack‐config ‐‐set /etc/neutron/neutron.conf
keystone_authtoken admin_user neutron
openstack‐config ‐‐set /etc/neutron/neutron.conf
keystone_authtoken admin_password cisco.123

Configure Neutron to Nova parameters.

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT notify_nova_on_port_status_changes True
openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT notify_nova_on_port_data_changes True
openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT nova_url http://10.0.236.31:8774/v2
openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT nova_admin_auth_url
http://10.0.236.31:35357/v2.0
openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT nova_region_name regionOne
openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT nova_admin_username nova
openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT nova_admin_tenant_id $(keystone tenant‐
get service | awk '/ id / {print $4}')
openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT nova_admin_password cisco.123

Define the ML2 Plugin.

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT core_plugin ml2
openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT service_plugins router
openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT allow_overlapping_ips True

http://openstack.ciscolive.com/printable.php?pod=3 62/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

The Modular Layer 2 plugin allows OpenStack Networking to simultaneously utilize the
variety of layer 2 networking technologies found in complex realworld data centers. It
currently includes drivers for the local, flat, VLAN, GRE and VXLAN network types and
works with the existing Open vSwitch, Linux Bridge , and HyperV L2 agents. The ML2 plugin
can be extended through mechanism drivers, allowing multiple mechanisms to be used
simultaneously.

Step 70 Configure ML2 Plugin
On the controller host 10.0.236.31 issue the command:

openstack‐config ‐‐set /etc/neutron/plugins/ml2/ml2_conf.ini
securitygroup enable_security_group True
openstack‐config ‐‐set /etc/neutron/plugins/ml2/ml2_conf.ini
securitygroup enable_ipset True
openstack‐config ‐‐set /etc/neutron/plugins/ml2/ml2_conf.ini
securitygroup firewall_driver
neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver

Step 71 Configure Nova Neutron integration
On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 63/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT network_api_class
nova.network.neutronv2.api.API
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT security_group_api neutron
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT linuxnet_interface_driver
nova.network.linux_net.LinuxOVSInterfaceDriver
openstack‐config ‐‐set /etc/nova/nova.conf
DEFAULT firewall_driver
nova.virt.firewall.NoopFirewallDriver

openstack‐config ‐‐set /etc/nova/nova.conf
neutron url http://10.0.236.31:9696
openstack‐config ‐‐set /etc/nova/nova.conf
neutron auth_strategy keystone
openstack‐config ‐‐set /etc/nova/nova.conf
neutron admin_auth_url
http://10.0.236.31:35357/v2.0
openstack‐config ‐‐set /etc/nova/nova.conf
neutron admin_tenant_name service
openstack‐config ‐‐set /etc/nova/nova.conf
neutron admin_username neutron
openstack‐config ‐‐set /etc/nova/nova.conf
neutron admin_password cisco.123

Step 72 Link ML2 plugin to neutron
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
ln ‐s /etc/neutron/plugins/ml2/ml2_conf.ini
/etc/neutron/plugin.ini

Step 73 Database build for Neutron
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
su ‐s /bin/sh ‐c "neutron‐db‐manage ‐‐config‐
file /etc/neutron/neutron.conf ‐‐config‐file
/etc/neutron/plugins/ml2/ml2_conf.ini upgrade
juno" neutron

Step 74 Restart Nova to reread configuration file
On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 64/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
systemctl restart openstack‐nova‐api.service
openstack‐nova‐scheduler.service openstack‐nova‐
conductor.service

Step 75 Enable and start Neutron service
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
systemctl enable neutron‐server.service
systemctl start neutron‐server.service

Step 76 Validate Neutron service status
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
systemctl status neutron‐server.service

# systemctl status neutron‐server.service
neutron‐server.service ‐ OpenStack Neutron Server
   Loaded: loaded (/usr/lib/systemd/system/neutron‐server.service; enabled)
   Active: active (running) since Fri 2015‐01‐09 13:21:57 EST; 17s ago
Main PID: 23998 (neutron‐server)
   CGroup: /system.slice/neutron‐server.service
           ‚îî‚îÄ23998 /usr/bin/python /usr/bin/neutron‐server ‐‐config‐file /usr
/share/neutron/neutron‐dist.conf ‐‐config‐file /etc/...

Jan 09 13:21:57 pod4‐controller systemd[1]: Started OpenStack Neutron Server.

Step 77 Configure IP forwarding on linux system
control
Ip Forwarding allows the network packets on one network interface (eth0) to be forwarded to
another network interface (eth1). This will allow the Linux computer to connect (“ethernet
bridge”) or route network traffic.

Filter selects the default RPF filtering setting for IPV4 networking with three possible options.

0 = No source validation
1 = Strict mode as defined in RFC3704 Strict Reverse Path Each incoming packet is
tested against the FIB and if the interface is not the best reverse path the packet check
will fail. By default failed packets are discarded.
2 = Loose mode as defined in RFC3704 Loose Reverse Path Each incoming packet's
source address is also tested against the FIB and if the source address is not reachable

http://openstack.ciscolive.com/printable.php?pod=3 65/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

via any interface the packet check will fail.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
cat <<EOF >> /etc/sysctl.conf
net.ipv4.ip_forward=1
net.ipv4.conf.all.rp_filter=0
net.ipv4.conf.default.rp_filter=0
EOF

Validate changes to sysctl file.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
sysctl ‐p

Understanding OpenStack Networking
There are four distinct type of virtual
networking devices: TAP devices, veth pairs,
Linux bridges, and Open vSwitch bridges. For
an Ethernet frame to travel from eth0 of a
virtual machine to the physical network, it
must pass through nine virtual devices inside
of the host: TAP vnet0, Linux bridge qbrNNN,
veth pair (qvbNNN, qvoNNN), Open vSwitch
bridge brint, veth pair (intbrex, phybreth1),
and, finally, the physical network interface
card enp8s0.

A TAP device, such as vnet0 is how
hypervisors such as KVM implement a virtual
network interface card (typically called a VIF or vNIC). An Ethernet frame sent to a TAP
device is received by the guest operating system.

A veth pair is a pair of directly connected virtual network interfaces. An Ethernet frame sent
to one end of a veth pair is received by the other end of a veth pair. Networking uses veth
pairs as virtual patch cables to make connections between virtual bridges.

A Linux bridge behaves like a simple MAC learning switch: you can connect multiple
(physical or virtual) network interfaces devices to a Linux bridge. The Linux bridge uses a
MAC caching table to record which interface on the bridge is used to communicate with a
host on the link. For any Ethernet frames that come in from one interface attached to the
bridge, the host MAC address and port on which the frame was received is recorded in the
MAC caching table for a limited time. When the bridge needs to forward a frame, it will check
to see if the frame's destination MAC address is recorded in the table. If so, the Linux bridge
forwards the frame through only that port. If not, the frame is flooded to all network ports in

http://openstack.ciscolive.com/printable.php?pod=3 66/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

the bridge, with the exception of the port where the frame was received. The sole purpose of
the Linux bridge here is to user the Linux iptables functionality to enforce OpenStack security
groups. Note that this Linux bridge is actually managed by the Nova service, not by Neutron.

The Open vSwitch plugin is one of the most popular core plugins. Open vSwitch
configurations consist of bridges and ports. Ports represent connections to other things, such
as physical interfaces and patch cables. Packets from any given port on a bridge are shared
with all other ports on that bridge. Bridges can be connected through Open vSwitch virtual
patch cables or through Linux virtual Ethernet cables (veth). Additionally, bridges appear as
network interfaces to Linux, so you can assign IP addresses to them.

In Neutron, the integration bridge, called brint, connects to the virtual machines (Instances)
and associated services via the Linux Bridges that are used for security group functionality.
The external bridge, called brex, connects to the external network when using flat or VLAN
networking. If an overlay such as GRE networking is defined for Tenant networks, a brtun
bridge is used instead. Finally, the VLAN configuration of the Open vSwitch plugin uses
bridges associated with each physical network.

In addition to defining bridges, Open vSwitch uses OpenFlow, which enables you to define
networking flow rules. Certain configurations use these rules to transfer packets between
VLANs.

Neutron uses a Linux feature called network namespaces that enable Linux to group adapters
into unique namespaces that are not visible to other namespaces (the equivalent of a VRF in
the routing world). This allows the same network node to manage multiple Neutron routers
with each namespace representing a virtual router function.

With Open vSwitch, you can use two different technologies to create the virtual networks:
GRE or VLANs.

Generic Routing Encapsulation (GRE) is an overlay tunneling technology. It encapsulates IP
packets inside another IP packet to create entirely new packets with different routing
information. When the new packet reaches its destination, it is unwrapped, and the
underlying packet is routed. To use GRE with Open vSwitch, Neutron creates GRE tunnels.
These tunnels are ports on a bridge and enable bridges on different systems to act as though
they were one bridge. This allows for the extension of a layer 2 broadcast domain (subnet)
across various nodes and thereby allows the compute and network nodes to act as one for
the purposes of routing.

Virtual LANs (VLANs), on the other hand, use a special modification to the Ethernet header.
They add a 4byte VLAN tag that ranges from 1 to 4094 (the 0 tag is special, and the 4095
tag, made of all ones, is equivalent to an untagged packet). Special NICs, switches, and
routers know how to interpret the VLAN tags, as does Open vSwitch. Packets tagged for one
VLAN are only shared with other devices configured to be on that VLAN, even through all
devices are on the same physical network.

The most common security group driver used with Open vSwitch is the Hybrid
IPTables/Open vSwitch plugin. It uses a combination for IPTables and OpenFlow rules. Use
the IPTables tool to create firewalls and set up NATs on Linux. This tool uses a complex rule
system and chains of rules to accommodate the complex rules required by Neutron security
groups.

Step 78 Install Open vSwitch
On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 67/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
yum install openstack‐neutron openstack‐neutron‐
ml2 openstack‐neutron‐openvswitch

Step 79 Configure ML2 for Open vSwitch
In this step you will create a mapped relationship between the Open vSwitch bridges and the
Physical Networks names that you will then use in the Neutron API or OpenStack GUI. This
name is an arbitrary name representing a physical network connection (via an Open vSwitch
bridge). This is the list of mappings you will configure:

Open vSwitch Physical Network
Bridge Value

brex pub

brnexus nexus

braci aci
This relationship allows you to have separate physical networks abstracted to a name that
Tenants can use to chose the appropriate network without needing to know about the
underlying infrastructure.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf.ini ovs
bridge_mappings pub:br‐ex,nexus:br‐nexus,aci:br‐
aci

The relationship will map as follows. While
you haven't yet been to the OpenStack
dashboard, this is a quick peek to understand
the relationships.

Step 80 Configure VLAN Maps to each TAG
After you have defined the relationships of the Physical Networks to each Open vSwitch, you
will have to also do that map of the VLANS that you will be using.

http://openstack.ciscolive.com/printable.php?pod=3 68/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/neutron/plugins/ml2/ml2_conf.ini
ml2_type_vlan network_vlan_ranges
pub:300:305,nexus:310:315,nexus:330:335,nexus:370:375,aci:390:395

Physical Network VLAN Range

pub 300 <> 305

nexus 310 <> 315
330 <> 335
370 <> 375

aci 390 <> 395

Step 81 Configure L3 Agent for Neutron
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/neutron/l3_agent.ini
DEFAULT interface_driver
neutron.agent.linux.interface.OVSInterfaceDriver
openstack‐config ‐‐set /etc/neutron/l3_agent.ini
DEFAULT use_namespaces True
openstack‐config ‐‐set /etc/neutron/l3_agent.ini
DEFAULT external_network_bridge

Step 82 Configure DHCP Agent for Neutron
On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 69/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
openstack‐config ‐‐set
/etc/neutron/dhcp_agent.ini  DEFAULT
interface_driver
neutron.agent.linux.interface.OVSInterfaceDriver
openstack‐config ‐‐set
/etc/neutron/dhcp_agent.ini  DEFAULT dhcp_driver
neutron.agent.linux.dhcp.Dnsmasq
openstack‐config ‐‐set
/etc/neutron/dhcp_agent.ini  DEFAULT
use_namespaces True

Step 83 Configure metadata for Neutron
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐set
/etc/neutron/metadata_agent.ini DEFAULT auth_url
http://10.0.236.31:5000/v2.0
openstack‐config ‐‐set
/etc/neutron/metadata_agent.ini DEFAULT
auth_region regionOne
openstack‐config ‐‐set
/etc/neutron/metadata_agent.ini DEFAULT
admin_tenant_name service
openstack‐config ‐‐set
/etc/neutron/metadata_agent.ini DEFAULT
admin_user neutron
openstack‐config ‐‐set
/etc/neutron/metadata_agent.ini DEFAULT
admin_password cisco.123

openstack‐config ‐‐set
/etc/neutron/metadata_agent.ini DEFAULT
nova_metadata_ip 10.0.236.31

openstack‐config ‐‐set
/etc/neutron/metadata_agent.ini DEFAULT
metadata_proxy_shared_secret cisco.123

openstack‐config ‐‐set /etc/nova/nova.conf
neutron service_metadata_proxy True
openstack‐config ‐‐set /etc/nova/nova.conf
neutron metadata_proxy_shared_secret cisco.123

Step 84 Restart Nova
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
systemctl restart openstack‐nova‐api.service

http://openstack.ciscolive.com/printable.php?pod=3 70/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Step 85 Enable and activate Open vSwitch
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
systemctl enable openvswitch.service
systemctl start openvswitch.service

Step 86 Configure Open vSwitch bridges
An Open vSwitch bridge behaves like a
virtual switch: network interface devices
connect to Open vSwitch bridge's ports, and
the ports can be configured much like a
physical switch's ports, including VLAN
configurations.

The brint Open vSwitch bridge is the
integration bridge: all guests running on the
compute host connect to this bridge.
Networking implements isolation across
these guests by configuring the brint ports.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
cat <<EOF > /etc/sysconfig/network‐
scripts/ifcfg‐br‐int
DEVICE=br‐int
ONBOOT=yes
DEVICETYPE=ovs
TYPE=OVSBridge
BOOTPROTO=static
EOF

[[IDSEQSTEP]]
cat <<EOF > /etc/sysconfig/network‐
scripts/ifcfg‐br‐ex
DEVICE=br‐ex
ONBOOT=yes
DEVICETYPE=ovs
TYPE=OVSBridge
BOOTPROTO=static
EOF

http://openstack.ciscolive.com/printable.php?pod=3 71/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
cat <<EOF > /etc/sysconfig/network‐
scripts/ifcfg‐br‐nexus
DEVICE=br‐nexus
ONBOOT=yes
DEVICETYPE=ovs
TYPE=OVSBridge
BOOTPROTO=static
EOF

[[IDSEQSTEP]]
cat <<EOF > /etc/sysconfig/network‐
scripts/ifcfg‐br‐aci
DEVICE=br‐aci
ONBOOT=yes
DEVICETYPE=ovs
TYPE=OVSBridge
BOOTPROTO=static
EOF

Step 87 Configure Open vSwitch physical NIC
ports
These last 3 ports are to configure the actual
physical interfaces of the server that is
running this OpenStack instance.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
cat <<EOF > /etc/sysconfig/network‐
scripts/ifcfg‐enp8s0
DEVICE="enp8s0"
ONBOOT=yes
DEVICETYPE=ovs
TYPE=OVSPort
OVS_BRIDGE=br‐ex
EOF

http://openstack.ciscolive.com/printable.php?pod=3 72/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
cat <<EOF > /etc/sysconfig/network‐
scripts/ifcfg‐enp10s0
DEVICE="enp10s0"
ONBOOT=yes
DEVICETYPE=ovs
TYPE=OVSPort
OVS_BRIDGE=br‐nexus
EOF

[[IDSEQSTEP]]
cat <<EOF > /etc/sysconfig/network‐
scripts/ifcfg‐enp11s0
DEVICE="enp11s0"
ONBOOT=yes
DEVICETYPE=ovs
TYPE=OVSPort
OVS_BRIDGE=br‐aci
EOF

Step 88 Restart Operating System Network
Service
You have to restart the network stack on this server to reread the configuration and add all
the interfaces.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
systemctl restart network.service

You can validate that the Open vSwitch controller has all the bridge defined. The command
ovs‐vsctl show shows you the status of all configured Open vSwitch bridges and their ports.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
ovs‐vsctl show

http://openstack.ciscolive.com/printable.php?pod=3 73/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

# ovs‐vsctl show
bf4bccee‐b0ba‐41e7‐b53a‐bb108b78ca70
    Bridge br‐int
        Port br‐int
            Interface br‐int
                type: internal
    Bridge "br‐nexus"
        Port "br‐nexus"
            Interface "br‐nexus"
                type: internal
        Port "enp10s0"
            Interface "enp10s0"
    Bridge br‐aci
        Port br‐aci
            Interface br‐aci
                type: internal
        Port "enp11s0"
            Interface "enp11s0"
    Bridge br‐ex
        Port "enp8s0"
            Interface "enp8s0"
        Port br‐ex
            Interface br‐ex
                type: internal
    ovs_version: "2.1.3"

Step 89 Restart Neutron

ALERT!
Due to a packaging bug, the Open vSwitch agent initialization script explicitly
looks for the Open vSwitch plugin configuration file rather than a symbolic link
/etc/neutron/plugin.ini pointing to the ML2 plugin configuration file. Run the
following commands to resolve this issue:

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
cp /usr/lib/systemd/system/neutron‐openvswitch‐
agent.service /usr/lib/systemd/system/neutron‐openvswitch‐
agent.service.orig
sed ‐i
's,plugins/openvswitch/ovs_neutron_plugin.ini,plugin.ini,g'
/usr/lib/systemd/system/neutron‐openvswitch‐agent.service

On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 74/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
systemctl enable neutron‐openvswitch‐
agent.service neutron‐l3‐agent.service neutron‐
dhcp‐agent.service neutron‐metadata‐
agent.service neutron‐ovs‐cleanup.service
systemctl start neutron‐openvswitch‐
agent.service neutron‐l3‐agent.service neutron‐
dhcp‐agent.service neutron‐metadata‐
agent.service

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
systemctl restart neutron‐server.service

You can look at systemctl to validate that neutron server is operational.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
systemctl status neutron‐server.service

# systemctl status neutron‐server.service
neutron‐server.service ‐ OpenStack Neutron Server
   Loaded: loaded (/usr/lib/systemd/system/neutron‐server.service; enabled)
   Active: active (running) since Sun 2015‐01‐04 23:56:24 EST; 9s ago
Main PID: 46682 (neutron‐server)
   CGroup: /system.slice/neutron‐server.service
           └─46682 /usr/bin/python /usr/bin/neutron‐server ‐‐config‐file /usr/sha
re/neutron/neutron‐dist.conf ‐‐...

Jan 04 23:56:23 controller systemd[1]: Starting OpenStack Neutron Server...
Jan 04 23:56:24 controller systemd[1]: Started OpenStack Neutron Server.

And finally if you look at the same command issued earlier to see the bridge definitions
active, you will get more information as the paths should now be built between the different
bridges.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
ovs‐vsctl show

http://openstack.ciscolive.com/printable.php?pod=3 75/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

# ovs‐vsctl show
bf4bccee‐b0ba‐41e7‐b53a‐bb108b78ca70
    Bridge br‐int
        fail_mode: secure
        Port "int‐br‐n7k"
            Interface "int‐br‐n7k"
                type: patch
                options: {peer="phy‐br‐nexus"}
        Port br‐int
            Interface br‐int
                type: internal
        Port int‐br‐ex
            Interface int‐br‐ex
                type: patch
                options: {peer=phy‐br‐ex}
        Port int‐br‐aci
            Interface int‐br‐aci
                type: patch
                options: {peer=phy‐br‐aci}
    Bridge "br‐nexus"
        Port "phy‐br‐nexus"
            Interface "phy‐br‐nexus"
                type: patch
                options: {peer="int‐br‐nexus"}
        Port "br‐n7k"
            Interface "br‐nexus"
                type: internal
        Port "enp10s0"
            Interface "enp10s0"
    Bridge br‐aci
        Port br‐aci
            Interface br‐aci
                type: internal
        Port "enp11s0"
            Interface "enp11s0"
        Port phy‐br‐aci
            Interface phy‐br‐aci
                type: patch
                options: {peer=int‐br‐aci}
    Bridge br‐ex
        Port phy‐br‐ex
            Interface phy‐br‐ex
                type: patch
                options: {peer=int‐br‐ex}
        Port "enp8s0"
            Interface "enp8s0"
        Port br‐ex
            Interface br‐ex
                type: internal
    ovs_version: "2.1.3"

When you observe this command you will see that each bridge now has a port definition that
ties it to specific interfaces and other bridges.

Step 90 Validate Neutron operation
[[IDSEQSTEP]]
neutron agent‐list

http://openstack.ciscolive.com/printable.php?pod=3 76/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Step 91 Enable Cisco Nexus Plugin
On the controller host 10.0.236.31 issue the command:

Install git and enable Python library for NETCONF clients

[[IDSEQSTEP]]
yum install ‐y git
git clone
https://github.com/CiscoSystems/ncclient.git
cd ncclient/
python ./setup.py install

Step 92 Enable Cisco Nexus Plugin
On the controller host 10.0.236.31 issue the command:

The following section adds 'cisco_nexus' plugin to the ml2 mechanism driver, configured the
vlan prefix to be use and the provider network that the plugin will manage.

[[IDSEQSTEP]]
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf.ini ml2
mechanism_drivers openvswitch,cisco_nexus
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_cisco vlan_name_prefix POD3‐neutron‐
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_cisco managed_physical_network nexus
openstack‐config ‐‐set
/etc/neutron/dhcp_agent.ini DEFAULT

http://openstack.ciscolive.com/printable.php?pod=3 77/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225
/etc/neutron/dhcp_agent.ini DEFAULT
enable_isolated_metadata True

Step 93 Add Cisco Nexus 3000 Switch and
associate it with Compute1 (Controller)
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_mech_cisco_nexus:n3k‐os‐pod3 username admin
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_mech_cisco_nexus:n3k‐os‐pod3 password
cisco.123
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_mech_cisco_nexus:n3k‐os‐pod3 ssh_port 22
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_mech_cisco_nexus:n3k‐os‐pod3 pod3‐controller
1/1

Step 94 Add Cisco Nexus 7000 Switch and
associate it with Compute2
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_mech_cisco_nexus:n7k‐os‐pod3 username admin
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_mech_cisco_nexus:n7k‐os‐pod3 password
cisco.123
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_mech_cisco_nexus:n7k‐os‐pod3 ssh_port 22
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_mech_cisco_nexus:n7k‐os‐pod3 pod3‐compute2
3/5

Step 95 Add Cisco ML2 ConfigFile To Neutron
Server
On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 78/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
sed ‐i 's/plugin.ini/plugin.ini ‐‐config‐file
\/etc\/neutron\/plugins\/ml2\/ml2_conf_cisco.ini/'
/usr/lib/systemd/system/neutron‐server.service
systemctl daemon‐reload

Step 96 Add RSA key fingerprint of Nexus3k to
neutron.
In this step we need to get the RSA key for the Nexus3000 stored under the neutron user
account. This

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
sudo ‐u neutron mkdir /var/lib/neutron/.ssh
sudo ‐u neutron ssh‐keyscan 10.0.226.103 >
/var/lib/neutron/.ssh/known_hosts
sudo ‐u neutron ssh‐keyscan 10.0.226.119 >>
/var/lib/neutron/.ssh/known_hosts
sudo ‐u neutron ssh‐keyscan n3k‐os‐pod3 >>
/var/lib/neutron/.ssh/known_hosts
sudo ‐u neutron ssh‐keyscan n7k‐os‐pod3 >>
/var/lib/neutron/.ssh/known_hosts

Step 97 SSH into Nexus 3000 for configuration
Configuration
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
sudo ‐u neutron sshpass ‐p cisco.123 ssh
admin@n3k‐os‐pod3

Step 98 Nexus 3000 Interface Configuration
On the n3kospod3 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 79/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
config t

feature hsrp
feature interface‐vlan
feature ospf
!
interface Ethernet1/1
  switchport
  description UCS
  switchport mode trunk
  switchport trunk allowed vlan none
!
interface Vlan330
  no shutdown
  ip address 10.3.30.3/24
  hsrp version 2
  hsrp 330
    ip 10.3.30.1
!
interface Vlan370
  no shutdown
  ip address 10.3.70.3/24
  hsrp version 2
  hsrp 370
    ip 10.3.70.1
!
interface ethernet 1/48
  no switchport
  no shutdown
  ip address 10.3.90.1/24
!
ip route 10.3.91.0/24 10.3.90.2
ip route 10.3.92.0/24 10.3.90.2
end
copy running‐config startup‐config
exit

Step 99 Nexus 7000 Interface Configuration
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
sudo ‐u neutron sshpass ‐p cisco.123 ssh
admin@n7k‐os‐pod3

Step 100 Nexus 7000 Configuration
On the n7kospod3 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 80/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
config t

feature hsrp
feature interface‐vlan
feature ospf
!
interface Ethernet3/5
  switchport
  description UCS
  switchport mode trunk
  switchport trunk allowed vlan none
!
interface Vlan330
  no shutdown
  ip address 10.3.30.7/24
  hsrp version 2
  hsrp 330
    ip 10.3.30.1
!
interface Vlan370
  no shutdown
  ip address 10.3.70.7/24
  hsrp version 2
  hsrp 370
    ip 10.3.70.1
!
end
copy running‐config startup‐config
exit

Step 101 Restart Neutron Server
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐service restart neutron

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐service status neutron

[root@pod12‐controller ~(keystone_admin)]# openstack‐service status neutron
neutron‐dhcp‐agent (pid 29951) is active
neutron‐l3‐agent (pid 29953) is active
neutron‐metadata‐agent (pid 29959) is active
neutron‐openvswitch‐agent (pid 29983) is active
neutron‐server (pid 29949) is active

http://openstack.ciscolive.com/printable.php?pod=3 81/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Configure Neutron Compute Host

You have to configure the compute host network component.

NOTE: This commands for this section should be entered on the second host the compute
node, not the controller node. Ensure you connect to the correct PuTTY session before
entering these commands.

Step 102 Open Firewall Holes
On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
firewall‐cmd ‐‐add‐port=9696/tcp ‐‐permanent
firewall‐cmd ‐‐reload

Step 103 Configure IP forwarding on linux system
control
On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
cat <<EOF >> /etc/sysctl.conf
net.ipv4.ip_forward=1
net.ipv4.conf.all.rp_filter=0
net.ipv4.conf.default.rp_filter=0
EOF

Validate changes to sysctl file.

On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
sysctl ‐p

Step 104 Install Neutron on Compute host

http://openstack.ciscolive.com/printable.php?pod=3 82/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
yum ‐y install openstack‐neutron openstack‐
neutron‐ml2 python‐neutronclient openstack‐
neutron‐openvswitch which

Step 105 Configure Neutron for compute node
On the compute host 10.0.236.32 issue the command:

openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT auth_strategy keystone

openstack‐config ‐‐set /etc/neutron/neutron.conf
keystone_authtoken auth_uri
http://10.0.236.31:5000/v2.0
openstack‐config ‐‐set /etc/neutron/neutron.conf
keystone_authtoken identity_uri
http://10.0.236.31:35357
openstack‐config ‐‐set /etc/neutron/neutron.conf
keystone_authtoken admin_tenant_name service
openstack‐config ‐‐set /etc/neutron/neutron.conf
keystone_authtoken admin_user neutron
openstack‐config ‐‐set /etc/neutron/neutron.conf
keystone_authtoken admin_password cisco.123

openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT core_plugin ml2
openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT service_plugins router
openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT allow_overlapping_ips True

Step 106 Configure Neutron ML2 for compute
node
On the compute host 10.0.236.32 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 83/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/neutron/plugins/ml2/ml2_conf.ini ml2
type_drivers local,flat,vlan,gre,vxlan
openstack‐config ‐‐set /etc/neutron/plugins/ml2/ml2_conf.ini ml2
tenant_network_types vlan
openstack‐config ‐‐set /etc/neutron/plugins/ml2/ml2_conf.ini ml2
mechanism_drivers openvswitch
openstack‐config ‐‐set /etc/neutron/plugins/ml2/ml2_conf.ini
securitygroup enable_security_group True
openstack‐config ‐‐set /etc/neutron/plugins/ml2/ml2_conf.ini
securitygroup enable_ipset True
openstack‐config ‐‐set /etc/neutron/plugins/ml2/ml2_conf.ini
securitygroup firewall_driver
neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver

Step 107 Configure ML2 and L3 Agent for OVS
On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf.ini ovs
bridge_mappings nexus:br‐nexus,aci:br‐aci
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf.ini ml2_type_vlan
network_vlan_ranges
nexus:310:315,nexus:330:335,nexus:370:375,aci:390:395

Step 108 Configure Neutron NOVA components
On the compute host 10.0.236.32 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 84/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Step 109 Link ML2 to Neutron
On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
ln ‐s /etc/neutron/plugins/ml2/ml2_conf.ini
/etc/neutron/plugin.ini

Step 110 Copy OpenVSwitch service configuration
files
On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
cp /usr/lib/systemd/system/neutron‐openvswitch‐
agent.service \
/usr/lib/systemd/system/neutron‐openvswitch‐
agent.service.orig
sed ‐i
's,plugins/openvswitch/ovs_neutron_plugin.ini,plugin.ini,g'
\
/usr/lib/systemd/system/neutron‐openvswitch‐agent.service

Step 111 Enable OpenVSwitch service and start
http://openstack.ciscolive.com/printable.php?pod=3 85/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
systemctl enable openvswitch.service
systemctl start openvswitch.service

Step 112 Configure OpenVSwitch bridges
On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
cat <<EOF > /etc/sysconfig/network‐
scripts/ifcfg‐br‐int
DEVICE=br‐int
ONBOOT=yes
DEVICETYPE=ovs
TYPE=OVSBridge
BOOTPROTO=static
EOF

[[IDSEQSTEP]]
cat <<EOF > /etc/sysconfig/network‐
scripts/ifcfg‐br‐nexus
DEVICE=br‐nexus
ONBOOT=yes
DEVICETYPE=ovs
TYPE=OVSBridge
BOOTPROTO=static
EOF

[[IDSEQSTEP]]
cat <<EOF > /etc/sysconfig/network‐
scripts/ifcfg‐br‐aci
DEVICE=br‐aci
ONBOOT=yes
DEVICETYPE=ovs
TYPE=OVSBridge
BOOTPROTO=static
EOF

Step 113 Configure OpenVSwitch physical NIC
ports
On the compute host 10.0.236.32 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 86/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
cat <<EOF > /etc/sysconfig/network‐
scripts/ifcfg‐enp10s0
DEVICE="enp10s0"
ONBOOT=yes
DEVICETYPE=ovs
TYPE=OVSPort
OVS_BRIDGE=br‐nexus
EOF

[[IDSEQSTEP]]
cat <<EOF > /etc/sysconfig/network‐
scripts/ifcfg‐enp11s0
DEVICE="enp11s0"
ONBOOT=yes
DEVICETYPE=ovs
TYPE=OVSPort
OVS_BRIDGE=br‐aci
EOF

Step 114 Restart network services
On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
systemctl restart network.service

Validate that network service is operational.

On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
systemctl status network.service

http://openstack.ciscolive.com/printable.php?pod=3 87/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

# systemctl status network.service
network.service ‐ LSB: Bring up/down networking
Loaded: loaded (/etc/rc.d/init.d/network)
Active: active (exited) since Fri 2015‐01‐09 13:32:28 EST; 1min 18s ago

Jan 09 13:32:25 pod4‐compute2 network[21023]: Bringing up interface enp6s0:  Conn
ection successfully activated (D‐Bus activ...ion/6)
Jan 09 13:32:25 pod4‐compute2 network[21023]: [  OK  ]
Jan 09 13:32:26 pod4‐compute2 ovs‐vsctl[21876]: ovs|00001|vsctl|INFO|Called as ov
s‐vsctl ‐t 10 ‐‐ ‐‐may‐exist add‐port br‐t...enp7s0
Jan 09 13:32:26 pod4‐compute2 network[21023]: Bringing up interface enp7s0:  [  O
K  ]
Jan 09 13:32:27 pod4‐compute2 ovs‐vsctl[22012]: ovs|00001|vsctl|INFO|Called as ov
s‐vsctl ‐t 10 ‐‐ ‐‐may‐exist add‐port br‐n3k enp9s0
Jan 09 13:32:27 pod4‐compute2 network[21023]: Bringing up interface enp9s0:  [  O
K  ]
Jan 09 13:32:27 pod4‐compute2 ovs‐vsctl[22147]: ovs|00001|vsctl|INFO|Called as ov
s‐vsctl ‐t 10 ‐‐ ‐‐may‐exist add‐port br‐n...np10s0
Jan 09 13:32:28 pod4‐compute2 network[21023]: Bringing up interface enp10s0:  [
OK  ]
Jan 09 13:32:28 pod4‐compute2 ovs‐vsctl[22290]: ovs|00001|vsctl|INFO|Called as ov
s‐vsctl ‐t 10 ‐‐ ‐‐may‐exist add‐port br‐a...np11s0
Jan 09 13:32:28 pod4‐compute2 network[21023]: Bringing up interface enp11s0:  [
OK  ]
Jan 09 13:32:28 pod4‐compute2 systemd[1]: Started LSB: Bring up/down networking.
Hint: Some lines were ellipsized, use ‐l to show in full.

Step 115 Restart NOVA Compute services
On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
systemctl restart openstack‐nova‐compute.service

Validate that NOVA service is operational.

On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
systemctl status openstack‐nova‐compute.service

http://openstack.ciscolive.com/printable.php?pod=3 88/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

# systemctl status openstack‐nova‐compute.service
openstack‐nova‐compute.service ‐ OpenStack Nova Compute Server
   Loaded: loaded (/usr/lib/systemd/system/openstack‐nova‐compute.service; enable
d)
   Active: active (running) since Fri 2015‐01‐09 13:36:00 EST; 7s ago
Main PID: 22916 (nova‐compute)
   CGroup: /system.slice/openstack‐nova‐compute.service
           ‚îî‚îÄ22916 /usr/bin/python /usr/bin/nova‐compute

Jan 09 13:36:00 pod4‐compute2 systemd[1]: Started OpenStack Nova Compute Server.

Step 116 Enable and start OpenVSwitch on
compute node
On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
systemctl enable neutron‐openvswitch‐
agent.service
systemctl start neutron‐openvswitch‐
agent.service

Validate that OpenVSwitch service is operational.

On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
systemctl status neutron‐openvswitch‐
agent.service

http://openstack.ciscolive.com/printable.php?pod=3 89/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

# systemctl status neutron‐openvswitch‐agent.service
neutron‐openvswitch‐agent.service ‐ OpenStack Neutron Open vSwitch Agent
   Loaded: loaded (/usr/lib/systemd/system/neutron‐openvswitch‐agent.service; ena
bled)
   Active: active (running) since Fri 2015‐01‐09 13:32:36 EST; 4min 21s ago
Main PID: 22381 (neutron‐openvsw)
   CGroup: /system.slice/neutron‐openvswitch‐agent.service
           ‚î‚îÄ22381 /usr/bin/python /usr/bin/neutron‐openvswitch‐agent ‐‐config
‐file /usr/share/neutron/neutron‐dist.conf ‐‐config...
           ‚î‚îÄ22558 sudo neutron‐rootwrap /etc/neutron/rootwrap.conf ovsdb‐clie
nt monitor Interface name,ofport ‐‐format=json
           ‚î‚îÄ22560 /usr/bin/python /usr/bin/neutron‐rootwrap /etc/neutron/root
wrap.conf ovsdb‐client monitor Interface name,ofpor...
           ‚îî‚îÄ22563 /bin/ovsdb‐client monitor Interface name,ofport ‐‐format=j
son

Jan 09 13:36:39 pod4‐compute2 sudo[23013]: neutron : TTY=unknown ; PWD=/ ; USER=r
oot ; COMMAND=/bin/neutron‐rootwrap /etc/n...ble=23
Jan 09 13:36:41 pod4‐compute2 sudo[23016]: neutron : TTY=unknown ; PWD=/ ; USER=r
oot ; COMMAND=/bin/neutron‐rootwrap /etc/n...ble=23
Jan 09 13:36:43 pod4‐compute2 sudo[23019]: neutron : TTY=unknown ; PWD=/ ; USER=r
oot ; COMMAND=/bin/neutron‐rootwrap /etc/n...ble=23
Jan 09 13:36:45 pod4‐compute2 sudo[23022]: neutron : TTY=unknown ; PWD=/ ; USER=r
oot ; COMMAND=/bin/neutron‐rootwrap /etc/n...ble=23
Jan 09 13:36:47 pod4‐compute2 sudo[23025]: neutron : TTY=unknown ; PWD=/ ; USER=r
oot ; COMMAND=/bin/neutron‐rootwrap /etc/n...ble=23
Jan 09 13:36:49 pod4‐compute2 sudo[23028]: neutron : TTY=unknown ; PWD=/ ; USER=r
oot ; COMMAND=/bin/neutron‐rootwrap /etc/n...ble=23
Jan 09 13:36:51 pod4‐compute2 sudo[23031]: neutron : TTY=unknown ; PWD=/ ; USER=r
oot ; COMMAND=/bin/neutron‐rootwrap /etc/n...ble=23
Jan 09 13:36:53 pod4‐compute2 sudo[23034]: neutron : TTY=unknown ; PWD=/ ; USER=r
oot ; COMMAND=/bin/neutron‐rootwrap /etc/n...ble=23
Jan 09 13:36:55 pod4‐compute2 sudo[23037]: neutron : TTY=unknown ; PWD=/ ; USER=r
oot ; COMMAND=/bin/neutron‐rootwrap /etc/n...ble=23
Jan 09 13:36:57 pod4‐compute2 sudo[23040]: neutron : TTY=unknown ; PWD=/ ; USER=r
oot ; COMMAND=/bin/neutron‐rootwrap /etc/n...ble=23
Hint: Some lines were ellipsized, use ‐l to show in full.

http://openstack.ciscolive.com/printable.php?pod=3 90/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Cinder

Step 117 Open Firewall Holes
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
firewall‐cmd ‐‐add‐port=8776/tcp ‐‐permanent
firewall‐cmd ‐‐reload

Step 118 Configure Cinder database and access
to database
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
mysql ‐‐user=root ‐‐password=cisco.123
CREATE DATABASE cinder;
GRANT ALL PRIVILEGES ON cinder.* TO
'cinder'@'localhost' IDENTIFIED BY 'cisco.123';
GRANT ALL PRIVILEGES ON cinder.* TO 'cinder'@'%'
IDENTIFIED BY 'cisco.123';
exit

Step 119 Add Cinder users to keystone
Once the database is ready, you can add the Cinder users to keystone.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
cd ~
source admin.sh

Now that you have assumed the identity of the admin user, you can issue the following
commands to create the user and the service for Cinder in Keystone.

http://openstack.ciscolive.com/printable.php?pod=3 91/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

On the controller host 10.0.236.31 issue the commands:

[[IDSEQSTEP]]
keystone user‐create ‐‐name cinder ‐‐pass
cisco.123
keystone user‐role‐add ‐‐user cinder ‐‐tenant
service ‐‐role admin

Step 120 Add Cinder services to Keystone
On the controller host 10.0.236.31 issue the commands:

[[IDSEQSTEP]]
keystone service‐create ‐‐name cinder ‐‐type
volume ‐‐description "OpenStack Block Storage"
keystone service‐create ‐‐name cinderv2 ‐‐type
volumev2 ‐‐description "OpenStack Block Storage"
keystone endpoint‐create \
  ‐‐service‐id $(keystone service‐list | awk '/
volume / {print $2}') \
  ‐‐publicurl http://10.0.236.31:8776/v1/%\
(tenant_id\)s \
  ‐‐internalurl http://10.0.236.31:8776/v1/%\
(tenant_id\)s \
  ‐‐adminurl http://10.0.236.31:8776/v1/%\
(tenant_id\)s \
  ‐‐region regionOne
keystone endpoint‐create \
  ‐‐service‐id $(keystone service‐list | awk '/
volumev2 / {print $2}') \
  ‐‐publicurl http://10.0.236.31:8776/v2/%\
(tenant_id\)s \
  ‐‐internalurl http://10.0.236.31:8776/v2/%\
(tenant_id\)s \
  ‐‐adminurl http://10.0.236.31:8776/v2/%\
(tenant_id\)s \
  ‐‐region regionOne

Step 121 Install OpenStack Cinder
On the controller host 10.0.236.31 issue the commands:

[[IDSEQSTEP]]
yum ‐y install openstack‐cinder python‐
cinderclient python‐oslo‐db

Step 122 Configure Cinder
On the controller host 10.0.236.31 issue the commands:

http://openstack.ciscolive.com/printable.php?pod=3 92/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/cinder/cinder.conf
database connection
mysql://cinder:[email protected]/cinder
openstack‐config ‐‐set /etc/cinder/cinder.conf
DEFAULT osapi_volume_workers 4
openstack‐config ‐‐set /etc/cinder/cinder.conf
DEFAULT rpc_backend rabbit
openstack‐config ‐‐set /etc/cinder/cinder.conf
DEFAULT rabbit_host 10.0.236.31
openstack‐config ‐‐set /etc/cinder/cinder.conf
DEFAULT rabbit_password cisco.123
openstack‐config ‐‐set /etc/cinder/cinder.conf
DEFAULT auth_strategy keystone
openstack‐config ‐‐set /etc/cinder/cinder.conf
keystone_authtoken auth_uri
http://10.0.236.31:5000/v2.0
openstack‐config ‐‐set /etc/cinder/cinder.conf
keystone_authtoken identity_uri
http://10.0.236.31:35357
openstack‐config ‐‐set /etc/cinder/cinder.conf
keystone_authtoken admin_tenant_name service
openstack‐config ‐‐set /etc/cinder/cinder.conf
keystone_authtoken admin_user cinder
openstack‐config ‐‐set /etc/cinder/cinder.conf
keystone_authtoken admin_password cisco.123
openstack‐config ‐‐set /etc/cinder/cinder.conf
DEFAULT my_ip 10.0.236.31
openstack‐config ‐‐set /etc/cinder/cinder.conf
DEFAULT glance_host 10.0.236.31
openstack‐config ‐‐set /etc/cinder/cinder.conf
DEFAULT iscsi_helper lioadm

Step 123 Synchronize Cinder database schema
On the controller host 10.0.236.31 issue the commands:

[[IDSEQSTEP]]
su ‐s /bin/sh ‐c "cinder‐manage db sync" cinder

Step 124 Enable and start Cinder service
On the controller host 10.0.236.31 issue the commands:

[[IDSEQSTEP]]
systemctl enable openstack‐cinder‐api.service
openstack‐cinder‐scheduler.service
systemctl start openstack‐cinder‐api.service
openstack‐cinder‐scheduler.service

Step 125 Install targetcli

http://openstack.ciscolive.com/printable.php?pod=3 93/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

targetcli is the general management platform for the LinuxIO Target. It comprises a shell
that uses the LIO library through a welldefined API. LinuxIO Target is based on a SCSI
engine that implements the semantics of a SCSI target as described in the SCSI
Architecture Model (SAM), and supports its comprehensive SPC3/SPC4 feature set in a
fabricagnostic way. Native support in OpenStack (setup, code), starting with the Grizzly
release, makes LIO also an attractive storage option for cloud deployments.

On the controller host 10.0.236.31 issue the commands:

[[IDSEQSTEP]]
yum ‐y install targetcli

Step 126 Enable and start Cinder volume service
On the controller host 10.0.236.31 issue the commands:

[[IDSEQSTEP]]
systemctl enable openstack‐cinder‐volume.service
target.service
systemctl start openstack‐cinder‐volume.service
target.service

Validate that OpenVSwitch service is operational.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
systemctl status openstack‐cinder‐volume.service

http://openstack.ciscolive.com/printable.php?pod=3 94/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

# systemctl status openstack‐cinder‐volume.service target.service
openstack‐cinder‐volume.service ‐ OpenStack Cinder Volume Server
   Loaded: loaded (/usr/lib/systemd/system/openstack‐cinder‐volume.service; enabl
ed)
   Active: active (running) since Fri 2015‐01‐09 13:39:15 EST; 1min 43s ago
Main PID: 28775 (cinder‐volume)
   CGroup: /system.slice/openstack‐cinder‐volume.service
           ‚î‚îÄ28775 /usr/bin/python2 /usr/bin/cinder‐volume ‐‐config‐file /usr/
share/cinder/cinder‐dist.conf ‐‐config‐file /etc/ci...
           ‚îî‚îÄ28797 /usr/bin/python2 /usr/bin/cinder‐volume ‐‐config‐file /usr
/share/cinder/cinder‐dist.conf ‐‐config‐file /etc/ci...

Jan 09 13:39:16 pod4‐controller cinder‐volume[28775]: 2015‐01‐09 13:39:16.457 287
97 TRACE cinder.volume.manager Command: sud...lumes
Jan 09 13:39:16 pod4‐controller cinder‐volume[28775]: 2015‐01‐09 13:39:16.457 287
97 TRACE cinder.volume.manager Exit code: 5
Jan 09 13:39:16 pod4‐controller cinder‐volume[28775]: 2015‐01‐09 13:39:16.457 287
97 TRACE cinder.volume.manager Stdout: u''
Jan 09 13:39:16 pod4‐controller cinder‐volume[28775]: 2015‐01‐09 13:39:16.457 287
97 TRACE cinder.volume.manager Stderr: u' ...es\n'
Jan 09 13:39:16 pod4‐controller cinder‐volume[28775]: 2015‐01‐09 13:39:16.457 287
97 TRACE cinder.volume.manager
Jan 09 13:39:16 pod4‐controller cinder‐volume[28775]: 2015‐01‐09 13:39:16.748 287
97 INFO oslo.messaging._drivers.impl_rabbit...:5672
Jan 09 13:39:16 pod4‐controller cinder‐volume[28775]: 2015‐01‐09 13:39:16.764 287
97 INFO oslo.messaging._drivers.impl_rabbit...:5672
Jan 09 13:39:16 pod4‐controller cinder‐volume[28775]: /usr/lib/python2.7/site‐pac
kages/amqp/channel.py:616: VDeprecationWarn...moved
Jan 09 13:39:16 pod4‐controller cinder‐volume[28775]: from py‐amqp v1.5.0.
Jan 09 13:39:16 pod4‐controller cinder‐volume[28775]: warn(VDeprecationWarning(EX
CHANGE_AUTODELETE_DEPRECATED))

target.service ‐ Restore LIO kernel target configuration
   Loaded: loaded (/usr/lib/systemd/system/target.service; enabled)
   Active: active (exited) since Fri 2015‐01‐09 13:39:16 EST; 1min 43s ago
  Process: 28776 ExecStart=/usr/bin/targetctl restore (code=exited, status=0/SUCC
ESS)
Main PID: 28776 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/target.service

Jan 09 13:39:15 pod4‐controller systemd[1]: Starting Restore LIO kernel target co
nfiguration...
Jan 09 13:39:16 pod4‐controller target[28776]: No saved config file at /etc/targe
t/saveconfig.json, ok, exiting
Jan 09 13:39:16 pod4‐controller systemd[1]: Started Restore LIO kernel target con
figuration.
Hint: Some lines were ellipsized, use ‐l to show in full.

http://openstack.ciscolive.com/printable.php?pod=3 95/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Horizon

Step 127 Open Firewall Holes
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
firewall‐cmd ‐‐add‐port=80/tcp ‐‐permanent
firewall‐cmd ‐‐add‐port=8080/tcp ‐‐permanent
firewall‐cmd ‐‐add‐port=443/tcp ‐‐permanent
firewall‐cmd ‐‐reload

Step 128 Install Horizon
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
yum ‐y install openstack‐dashboard httpd
mod_wsgi memcached python‐memcached
yum ‐y install python‐pbr

Step 129 Configure Horizon local settings
The following commands will make changes to the file /etc/openstackdashboard/local
settings. They are designed specifically around line numbers and will only work with a un
modified file.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
sed ‐i 's/OPENSTACK_HOST =
"127.0.0.1"/OPENSTACK_HOST = "10.0.236.31"/g'
/etc/openstack‐dashboard/local_settings

What this command did was go into the file, find the specific line that is configured as
OPENSTACK_HOST = "127.0.0.1" and swap it to OPENSTACK_HOST = "10.0.236.31".

The same command will be used to modify the ALLOWED_HOSTS definition.

http://openstack.ciscolive.com/printable.php?pod=3 96/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
sed ‐i "s/ALLOWED_HOSTS = \
['horizon.example.com',
'localhost'\]/ALLOWED_HOSTS = '*'/g"
/etc/openstack‐dashboard/local_settings

The last configuration change for the Horizon dashboard settings is to uncomment a section
and recomment another.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
sed ‐i '104,109 s/#//g' /etc/openstack‐
dashboard/local_settings
sed ‐i '104,109 s/^\s//g' /etc/openstack‐
dashboard/local_settings
sed ‐i '111,115 s/^/#/g' /etc/openstack‐
dashboard/local_settings

Step 130 Set HTTP
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
setsebool ‐P httpd_can_network_connect on
chown ‐R apache:apache /usr/share/openstack‐
dashboard/static

Step 131 Enable and start HTTP service
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
systemctl enable httpd.service memcached.service
systemctl start httpd.service memcached.service

Step 132 Validate HTTP service is enabled
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
systemctl status httpd.service

http://openstack.ciscolive.com/printable.php?pod=3 97/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

# systemctl status httpd.service
httpd.service ‐ The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled)
   Active: active (running) since Mon 2015‐01‐05 04:29:45 EST; 14s ago
Main PID: 83322 (httpd)
   Status: "Total requests: 0; Current requests/sec: 0; Current traffic:   0 B/se
c"
   CGroup: /system.slice/httpd.service
           ├─83322 /usr/sbin/httpd ‐DFOREGROUND
           ├─83329 /usr/sbin/httpd ‐DFOREGROUND
           ├─83330 /usr/sbin/httpd ‐DFOREGROUND
           ├─83331 /usr/sbin/httpd ‐DFOREGROUND
           ├─83332 /usr/sbin/httpd ‐DFOREGROUND
           ├─83333 /usr/sbin/httpd ‐DFOREGROUND
           └─83334 /usr/sbin/httpd ‐DFOREGROUND

Step 133 Validate Horizon is operational
From your desktop you should be able to point your browser to the controller host with the url:

http://10.0.236.31/dashboard (http://10.0.236.31/dashboard)

This should open a page that looks like the following.

You can login into the interface with the credentials

Username: admin

http://openstack.ciscolive.com/printable.php?pod=3 98/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Password: cisco.123

http://openstack.ciscolive.com/printable.php?pod=3 99/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

OpenStack Host Aggregates
Step 134 Login into dashboard with admin
account
Under the administrator account you will goto

On the horizon web interface on 10.0.236.31:

After you login into to the Horizon dashboard with the admin account, you should see a
screen similar to the following:

The first step that you will perform is to set up host aggregates so that you can specify which
host you want to place your virtual machine instances. By default, OpenStack has a series of
algorithms to select the most appropriate host to place created instances. It does an analysis
of CPU, memory and other factors. For the purpose of this lab, we want to force the virtual
machine instances created by you to reside in specific hosts of the topology so you can test
the network connectivity between the hosts.

Step 135 Go to Host Aggregates

http://openstack.ciscolive.com/printable.php?pod=3 100/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

To access the Host Aggregates configuration section, select Host Aggregates on the left
menu, then click the +Create Host Aggregate button on the upper right corner.

On the horizon web interface on 10.0.236.31:

Step 136 Create compute1 Aggregate
On the Create Host Aggregate configuration page, enter the name compute1 for both Name
and Availability Zone as shown below. This will match the OpenStack naming convention for
our hosts.

On the horizon web interface on 10.0.236.31:

Now you have to select the host that will be part of this aggregate. In this case we want to
include the controller host into this aggregate.

http://openstack.ciscolive.com/printable.php?pod=3 101/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Once created, the aggregate should look similar to the following. You have created the first
aggregate. You will repeat the process to create another aggregate but for the second host in
your cluster.

Step 137 Create compute2 Aggregate
On the horizon web interface on 10.0.236.31:

Once the popup is up enter the following information:

http://openstack.ciscolive.com/printable.php?pod=3 102/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

You should now have two separate aggregates called compute1 and compute2 matching the
appropriate host of this OpenStack cluster.

http://openstack.ciscolive.com/printable.php?pod=3 103/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

OpenStack base network
Step 138 Log in to dashboard with admin account
If you are not already logged into the OpenStack dashboard, connect to
http://10.0.236.31/dashboard (http://10.0.236.31/dashboard) and log in with the following
credentials:

On the horizon web interface on 10.0.236.31:

Step 139 Create Tenant Public Network
Locate the Networks tab under Admin > System and click on +Create Network

On the horizon web interface on 10.0.236.31:

Step 140 Tenant Public Network Parameters
Fill out the form as follows:

On the horizon web interface on 10.0.236.31:

http://openstack.ciscolive.com/printable.php?pod=3 104/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

When finished you should see a screen similar to the following. Now you need to click on the
Network Name to add the subnet for this network.

Click on the Network Name, and the following screen will appear. On this screen click on the
+Create Subnet

Step 141 Create Subnet for Public Network
In this step you will be creating the subnet for the public admin network. When creating a
subnet it involves two parts.

http://openstack.ciscolive.com/printable.php?pod=3 105/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

ALERT!
Make sure to disable DHCP for this subnet.

Step 142 Create Tenant Private Network
Click on the OpenStack Horizon Dashboard Networks Tab to return to the Network List. It
should look similar to this:

http://openstack.ciscolive.com/printable.php?pod=3 106/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Click the +Create Network button again.

On the horizon web interface on 10.0.236.31:

Step 143 Signout and Signin as Tenant User
Logout from the admin account as the next steps will be conducted as you assume the role
of the Tenant of this OpenStack network.

Step 144 Login to dashboard with poduser
account
On the horizon web interface on 10.0.236.31:

Step 145 Add subnet to Tenant VLAN
http://openstack.ciscolive.com/printable.php?pod=3 107/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

The administrator created the two networks. You are going to use the Tenant account to
create the subnet under the tenant network Vlan310. You will notice that you can't make
changes to the network Vlan300. The reason for this is that that network belongs to the
admin side while the network Vlan310 belongs to the Tenant Project.

Now that you have assumed the entity of the Tenant User, you will configure the subnet for
the Tenant VLAN. Under the Network>Networks tab in Horizon Dashboard, you should be
able to add a Subnet under actions for Vlan310.

On the horizon web interface on 10.0.236.31:

Step 146 Create Subnet for Tenant Private
Network
Once you click add Subnets a popup will appear as the following. Fill in with the following
values.

For this network we will be creating a DHCP server on the subnet such that the instances
that we create will receive IP addresses. Enable DHCP and enter the scope in the range that
list listed as 10.3.10.101,10.3.10.199.

http://openstack.ciscolive.com/printable.php?pod=3 108/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Once completed we can proceed to create the router for the Tenant Network

Step 147 Configure Tenant Router
To configure the Router, please proceed to the following screen and click on the
+Create Router .

When the popup appears please enter the name TenantRouterPOD3 as the router name.

1 Router Name:
TenantRouterPOD3

Once created you will select the gateway for this router. Click on the Set Gateway as you

would see below.

http://openstack.ciscolive.com/printable.php?pod=3 109/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

The following screen should appear. Select Vlan300 for the External Network then click Set
Gateway

1 External Network:
Vlan300

Now you need to configure the port of the router that is attached to the internal private
network. For this you need to click on the router network name.

And a screen similar to the following will be presented. Click on the + Add interface button.

The following screen will be presented. Select the option that says Vlan310:10.3.10.0/24 and
click the + Add interface button.

1 Subnet:
Vlan310:10.3.10.0/24

http://openstack.ciscolive.com/printable.php?pod=3 110/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

With that you have concluded creating the network for this Tenant. Now you will create two
separate instances that will be placed in the private Tenant network and will be capable of
reaching the external network.

Step 148 Create Instance on compute1
You will be creating two separate instances and each instance will be created in a separate
compute node of your cluster. You will start with creating the instance on compute1. First
make sure you are logged into the Horizon Dashboard as userpod3 and go to the Instances
tab as shown below. The Instances tab is under the Compute section on the left of the
screen.

Once there click on the  Launch Instance button. The following screen will popup and

you will need to fill out the form as shown below making sure not to click the submit button
until after you have selected the network adapter:

1 Availability Zone: compute1

2 Instance Name: POD3VM1

3 Instance Boot Source: Boot
from image

4 Image Name: cirros0.X.X
x86_64 (XX.X MB)

http://openstack.ciscolive.com/printable.php?pod=3 111/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

When you click on the network tab the following will be presented.

1 Click on the + button for the
option presented as Vlan310. This
selects the VLAN as the NIC
interface for this instance.

2 Click on the Launch button

At this point OpenStack will start the process of creating your instance attached to the
network Vlan310.

Step 149 Create Instance on compute2
You will now create the second instance (VM) on compute2.

Click on the  Launch Instance button. The following screen will popup and you will need

to fill out as follows making sure not to click the submit button until after you have selected
the network adapter:

1 Availability Zone: compute2

2 Instance Name: POD3VM2

3 Instance Boot Source: Boot
from image

4 Image Name: cirros0.X.X
x86_64 (XX.X MB)

When you click on the network tab the following will be presented.

http://openstack.ciscolive.com/printable.php?pod=3 112/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

1 Click on the + button for the
option presented as Vlan310. This
selects the VLAN as the NIC
interface for this instance.

2 Click on the Launch button

At this point you should have two instances running in OpenStack! Congrats! After a few
seconds, you should see the following:

Your network should now look something like this:

ALERT!
This might be a good time to take a small break! The system is creating the

http://openstack.ciscolive.com/printable.php?pod=3 113/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

instances and takes a couple of minutes until the instances are ready for use.

Step 150 Connect to instance consoles and
validate network
The best way to connect to the consoles of the instances is via the Network Topology TAB
in the Horizon Dashboard. Head over to your Tenant Network tab and select Network
Topology.

Once there you can see the topology you have created up to this point. If you hover over one
of the computer icons you will see the following:

From there you can now click on Open Console to open the console of the specific instance.
A popup window will appear similar to the following.

Log into the two separate instances. And from them issue the command ip a . This

command will list the interface eth0 ip address.

http://openstack.ciscolive.com/printable.php?pod=3 114/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

# ip a
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0:  mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:25:b5:bc:00:14 brd ff:ff:ff:ff:ff:ff
    inet 10.XX.XX.XX/24 brd 10.0.236.255 scope global eth0
    inet6 fe80::225:b5ff:febc:14/64 scope link
       valid_lft forever preferred_lft forever

You should be able to ping from instance to instance and also the default gateways.

On one of the instances issue the command:

ALERT!
You will not be able to copy into this VNC window.

ping 10.3.10.1

# ping 10.3.10.1
PING 10.3.10.1 (10.3.10.1) 56(84) bytes of data.
64 bytes from 10.3.10.1: icmp_seq=1 ttl=255 time=0.290 ms
64 bytes from 10.3.10.1: icmp_seq=2 ttl=255 time=0.207 ms
64 bytes from 10.3.10.1: icmp_seq=3 ttl=255 time=0.257 ms
64 bytes from 10.3.10.1: icmp_seq=4 ttl=255 time=0.207 ms

You should also be capable of pinging the northbound ASR1K that is depicted in the topology
diagram above.

On one of the instances issue the command:

ping 10.3.0.1

# ping 10.3.0.1
PING 10.3.0.1 (10.3.0.1) 56(84) bytes of data.
64 bytes from 10.3.0.1: icmp_seq=1 ttl=255 time=0.290 ms
64 bytes from 10.3.0.1: icmp_seq=2 ttl=255 time=0.207 ms
64 bytes from 10.3.0.1: icmp_seq=3 ttl=255 time=0.257 ms
64 bytes from 10.3.0.1: icmp_seq=4 ttl=255 time=0.207 ms

http://openstack.ciscolive.com/printable.php?pod=3 115/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Create CSR1K Flavor in
OpenStack
This procedure explains the process to create the CSR1K flavor. In OpenStack you create
flavors to define what resources an instance will consume. For example, the instance image,
CPU requirements and memory requirements. When you complete this procedure, you will
provide your Tenant with the ability to create a instance based on the Cisco CSR (Cloud
Services Router) in their networks.

Step 151 Log in to the OpenStack dashboard with
admin account
If you are not already logged into the OpenStack dashboard, connect to
http://10.0.236.31/dashboard (http://10.0.236.31/dashboard) and log in with the following
credentials:

On the horizon web interface on 10.0.236.31:

Step 152 Create Flavor
Go to Admin > Flavors and click on the +Create Flavor button as shown below.

You will need to fill out the following form with this information:

1 Name: CSR1K
2VCPU

http://openstack.ciscolive.com/printable.php?pod=3 116/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

2 ID: auto

3 VPCU: 2

4 RAM: 4096

5 Root Disk: 0

6 Ephermeral Disk: 0

7 Swap Disk: 0

Step 153 Create Image
Next, navigate to Admin > System > Images and click on the +Create Image button as
shown below:

Fill out the Create An Image form with the following information:

1 Name: CSR1KV3.13.1S

2 Image Source: image location

3 Image Location:
http://10.0.226.7/tftpboot/iso/csr1000v
universalk9.03.13.01.S.1543.S1
ext.qcow2

4 Format: QCOW2 QEMU Emulator

5 Public: (CHECKED)

ALERT!

http://openstack.ciscolive.com/printable.php?pod=3 117/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

This process will take a few minutes to complete as the image needs to be
downloaded from our internal server into your OpenStack topology.

Once completed, you should now have the ability to create CSR1KV instances. You will be
doing this as part of your Nexus integration later in this lab.

http://openstack.ciscolive.com/printable.php?pod=3 118/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

OpenStack Nexus3k/7k network
configuration
Step 154 Login into dashboard with admin
account
If you are not already logged into the OpenStack dashboard, connect to
http://10.0.236.31/dashboard (http://10.0.236.31/dashboard) and log in with the following
credentials:

On the horizon web interface on 10.0.236.31:

After you login into to the Horizon dashboard with the admin account, you should see a
screen similar to the following:

Step 155 Create Nexus Public Network
Navigate to Admin > System > Networks and click on +Create Network

http://openstack.ciscolive.com/printable.php?pod=3 119/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

On the horizon web interface on 10.0.236.31:

Step 156 Nexus Public Network Parameters
Enter the information as shown below to create the network:

On the horizon web interface on 10.0.236.31:

When finished you should see a screen similar to the following. Now you need to click on the
Network Name to add the subnet for this network.

Click on the Network Name, and the following screen will appear.

On this screen click on the +Create Subnet

Step 157 Create Subnet for Public Network

http://openstack.ciscolive.com/printable.php?pod=3 120/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

In this step you will be creating the subnet for the public admin network. When creating a
subnet it involves two parts.

ALERT!
Make sure to disable DHCP for this subnet.

Step 158 Create Network for Nexus Private
Network
Navigate to Admin > System > Networks to return to the network list and click on the
+Create Network button.

http://openstack.ciscolive.com/printable.php?pod=3 121/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Enter the information as follows:

When finished you should see a screen similar to the following. Now you need to click on the
Network Name to add the subnet for this network.

You should be back to the Network pane. On this screen click on the +Create Subnet

You will now create the private network for the Nexus component.

http://openstack.ciscolive.com/printable.php?pod=3 122/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Step 159 Signout and Signin as Tenant User
Log out from the admin account as the next steps will be conducted as you assume the role
of a Tenant of the OpenStack network.

Step 160 Login to dashboard with poduser
account
On the horizon web interface on 10.0.236.31:

http://openstack.ciscolive.com/printable.php?pod=3 123/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Step 161 Create a CSR1Kv instance
After you log in to the Horizon dashboard with the tenant account, navigate to Compute >
Instances on the Horizon Dashboard. Click on the Launch Instance button.

In the Instance Create form, enter the following information:

1 Availability Zone: compute1
2 Instance Name: CSR1KvR1

ALERT!
This is case sensitive.
Type exactly or next steps
will fail. Copy provided to
help.

CSR1KvR1

3 Flavor: CSR1K2VCPU
4 Instance Boot Source: Boot
from image
5 Image Name: CSR1KV
3.13.1S
6 Click on Networking tab

Once you click on the Networking tab you should see a screen similar to the following:

1 Select Vlan330

http://openstack.ciscolive.com/printable.php?pod=3 124/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

2 Click Launch

In the background, the Nexus plugin has done some work on the Nexus switches to allow
the required VLANs to reach the hosts. You can take a look at this now via the CLI as
follows.

Step 162 Access Nexus 3k to validate
interface/VLAN creation.
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
sshpass ‐p cisco.123 ssh ‐o
StrictHostKeyChecking=no admin@n3k‐os‐pod3

You should be able to see the creation of both the VLAN and the interface.

http://openstack.ciscolive.com/printable.php?pod=3 125/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

N3K‐OS‐POD4# show vlan

VLAN Name                             Status    Ports
‐‐‐‐ ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐ ‐‐‐‐‐‐‐‐‐ ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐
1    default                          active    Eth1/2, Eth1/3, Eth1/4, Eth1/5
                                                Eth1/6, Eth1/7, Eth1/8, Eth1/9
                                                Eth1/10, Eth1/11, Eth1/12
                                                Eth1/13, Eth1/14, Eth1/15
                                                Eth1/16, Eth1/17, Eth1/18
                                                Eth1/19, Eth1/20, Eth1/21
                                                Eth1/22, Eth1/23, Eth1/24
                                                Eth1/25, Eth1/26, Eth1/27
                                                Eth1/28, Eth1/29, Eth1/30
                                                Eth1/31, Eth1/32, Eth1/33
                                                Eth1/34, Eth1/35, Eth1/36
                                                Eth1/37, Eth1/38, Eth1/39
                                                Eth1/40, Eth1/41, Eth1/42
                                                Eth1/43, Eth1/44, Eth1/45
                                                Eth1/46, Eth1/47, Eth1/48
                                                Eth1/49/1, Eth1/49/2, Eth1/49/3
                                                Eth1/49/4, Eth1/50/1, Eth1/50/2
                                                Eth1/50/3, Eth1/50/4, Eth1/51/1
                                                Eth1/51/2, Eth1/51/3, Eth1/51/4
                                                Eth1/52/1, Eth1/52/2, Eth1/52/3
                                                Eth1/52/4
310  POD3‐neutron‐310                 active    Eth1/1
330  POD3‐neutron‐330                 active    Eth1/1

VLAN Type  Vlan‐mode
‐‐‐‐ ‐‐‐‐‐ ‐‐‐‐‐‐‐‐‐‐
1    enet  CE
310  enet  CE
330  enet  CE
Primary  Secondary  Type             Ports
‐‐‐‐‐‐‐  ‐‐‐‐‐‐‐‐‐  ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐  ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐

N3K‐OS‐POD3# show ip int brief
IP Interface Status for VRF "default"(1)
Interface            IP Address      Interface Status
Vlan330              10.3.30.3       protocol‐up/link‐up/admin‐up
Vlan370              10.3.70.3       protocol‐down/link‐down/admin‐up

Step 163 Attach port to CSR1Kv
If you are still logged in to the Nexus switch, exit the SSH session to return to the terminal
prompt on your controller node. Next, you must assume the identity of the Tenant to issue
these commands.

On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 126/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
cd ~
. TenantPOD3.sh

You can list all the instances that are active in this OpenStack network with the command:

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
nova list

That should provide output similar to the following:

You can see the instance for the CSR1Kv. From here we can derive the command to create
the port in OpenStack.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
nova interface‐attach ‐‐port‐id $(neutron port‐
create ‐‐fixed‐ip ip_address=10.3.31.1 Vlan331 |
awk '/ id / {print $4}') CSR1Kv‐R1

You should be able to now look at the CSR1Kv from the CLI and see both connected ports to
OpenStack.

On the controller host 10.0.236.31 issue the command:

nova show CSR1Kv‐R1

http://openstack.ciscolive.com/printable.php?pod=3 127/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

# nova show CSR1Kv‐R1
+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐
‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+
| Property                             | Value
                 |
+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐
‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+
| OS‐DCF:diskConfig                    | AUTO
                 |
| OS‐EXT‐AZ:availability_zone          | compute1
                 |
| OS‐EXT‐STS:power_state               | 1
                 |
| OS‐EXT‐STS:task_state                | ‐
                 |
| OS‐EXT‐STS:vm_state                  | active
                 |
| OS‐SRV‐USG:launched_at               | 2015‐01‐15T19:31:41.000000
                 |
| OS‐SRV‐USG:terminated_at             | ‐
                 |
| Vlan330 network                      | 10.3.30.2
                 |
| Vlan331 network                      | 10.3.31.1
                 |
| accessIPv4                           |
                 |
| accessIPv6                           |
                 |
| config_drive                         |
                 |
| created                              | 2015‐01‐15T19:31:28Z
                 |
| flavor                               | CSR1K‐2VCPU (ef11df23‐a196‐4786‐a841‐5b9
b42105ebe)       |
| hostId                               | 4b8af0b563c2933d9991ffe261f6b594b7c45307
c07c430eed4eb04d |
| id                                   | 8b2d7d9f‐692e‐4c70‐b2b9‐0717bffa987a
                 |
| image                                | CSR1KV‐3.13.1S (ac958f45‐85e0‐4984‐864b‐
f7970a303210)    |
| key_name                             | ‐
                 |
| metadata                             | {}
                 |
| name                                 | CSR1Kv‐R1
                 |
| os‐extended‐volumes:volumes_attached | []
                 |
| progress                             | 0
                 |
| security_groups                      | default
                 |
| status                               | ACTIVE
                 |
| tenant_id                            | f3b50c1031b64ab38488d8aa5b3497fc
                 |
| updated                              | 2015‐01‐15T19:31:41Z
                 |
| user_id                              | bcdb00a1ace34bf9adc567d0c6dca133
                 |

http://openstack.ciscolive.com/printable.php?pod=3 128/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐
‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+

Step 164 Validate CSR instance and network.
Navigate to Network > Network Topology as the Tenant user. You should see a diagram
like this:

Step 165 Configure IOS interfaces for CSR1Kv
Now that both interfaces of the CSR1Kv are attached to the proper networks, you can
configure the CSR directly via CLI. To do this you will need to connect to the console of the
CSR1Kv directly. The best way to attach to the console of the CSR is via the Network >
Network Topology OpenStack Dashboard tab. Once there, hover over the CSR1Kv
instance and click on the Open Console link.

This will open a screen similar to the following:

Because of the nature of the VNC console you will not be able to COPY/PASTE into it. You
will now configure the CSR router functionality via the CLI.

On the CSR1Kv instance issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 129/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

enable
show ip interface brief

# show ip interface brief
Interface                       IP‐Address       OK? Method Status
  Protocol
GigabitEthernet1                unassigned       YES unset  administratively down
  down
GigabitEthernet2                unassigned       YES unset  administratively down
  down

Your next step is to configure the IP Addresses of both interfaces. The first interface we will
configure will be for GigabitEthernet2 that is attached to Vlan331.

On the CSR1Kv instance issue the command:

[[IDSEQSTEP]]
config t
int GigabitEthernet2
ip address dhcp
no shutdown
end

The interface of the router should become operational and you should see a message that it
has acquired a IP address. If you issue the command show ip interface brief it will look
similar to:

# show ip interface brief
Interface                       IP‐Address       OK? Method Status
  Protocol
GigabitEthernet1                unassigned       YES unset  administratively down
  down
GigabitEthernet2                10.3.31.1        YES DHCP   up
  up

For the interface that is attached to Vlan330, you can locate the IP Address that was
assigned to it via the CLI command nova show CSR1Kv‐R1 .

On the controller host 10.0.236.31 issue the command:

nova show CSR1Kv‐R1

# nova show CSR1Kv‐R1
+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐

http://openstack.ciscolive.com/printable.php?pod=3 130/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+
| Property                             | Value
                 |
+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐
‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+
| OS‐DCF:diskConfig                    | AUTO
                 |
| OS‐EXT‐AZ:availability_zone          | compute1
                 |
| OS‐EXT‐STS:power_state               | 1
                 |
| OS‐EXT‐STS:task_state                | ‐
                 |
| OS‐EXT‐STS:vm_state                  | active
                 |
| OS‐SRV‐USG:launched_at               | 2015‐01‐15T19:31:41.000000
                 |
| OS‐SRV‐USG:terminated_at             | ‐
                 |
| Vlan430 network                      | 10.3.30.2
                 |
| Vlan431 network                      | 10.3.31.1
                 |
| accessIPv4                           |
                 |
| accessIPv6                           |
                 |
| config_drive                         |
                 |
| created                              | 2015‐01‐15T19:31:28Z
                 |
| flavor                               | CSR1K‐2VCPU (ef11df23‐a196‐4786‐a841‐5b9
b42105ebe)       |
| hostId                               | 4b8af0b563c2933d9991ffe261f6b594b7c45307
c07c430eed4eb04d |
| id                                   | 8b2d7d9f‐692e‐4c70‐b2b9‐0717bffa987a
                 |
| image                                | CSR1KV‐3.13.1S (ac958f45‐85e0‐4984‐864b‐
f7970a303210)    |
| key_name                             | ‐
                 |
| metadata                             | {}
                 |
| name                                 | CSR1Kv‐R1
                 |
| os‐extended‐volumes:volumes_attached | []
                 |
| progress                             | 0
                 |
| security_groups                      | default
                 |
| status                               | ACTIVE
                 |
| tenant_id                            | f3b50c1031b64ab38488d8aa5b3497fc
                 |
| updated                              | 2015‐01‐15T19:31:41Z
                 |
| user_id                              | bcdb00a1ace34bf9adc567d0c6dca133
                 |
+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐
‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+

http://openstack.ciscolive.com/printable.php?pod=3 131/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Validate from the output of the command that Vlan330 has an assigned IP address of
10.3.30.2. If it doesn't, use the value that is specified by NOVA in the previous command.

On the CSR1Kv instance issue the command:

[[IDSEQSTEP]]
config t
int GigabitEthernet1
ip address 10.3.30.2 255.255.255.0
no shutdown
end

With this both interfaces should be operational. You can then ping the northbound SVI
interface on the Nexus3k.

On the CSR1Kv instance issue the command:

ping 10.3.30.1

# ping 10.3.30.1
Type escape sequence to abort.
Sending 5, 100‐byte ICMP Echos to 10.3.30.1, timeout is 2 seconds:
.!!!!
Success rate is 100 percent (5/5), round‐trip min/avg/max = 1/1/1 ms

Step 166 Configure OSPF on CSR1Kv
Your first task is to configure OSPF process on the CSR1Kv

On the CSR1Kv instance issue the command:

[[IDSEQSTEP]]
conf t
router ospf 1
router‐id 10.3.30.2
network 10.3.30.0 255.255.255.0 area 0
network 10.3.31.0 255.255.255.0 area 0
exit
interface gigabitethernet1
ip ospf network broadcast
exit
interface gigabitethernet2
ip ospf network broadcast

Step 167 Configure OSPF on Nexus3k
You will need to connect to the Nexus3k.

http://openstack.ciscolive.com/printable.php?pod=3 132/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
sshpass ‐p cisco.123 ssh ‐o
StrictHostKeyChecking=no admin@n3k‐os‐pod3

Once in the Nexus3k, you will have to configure OSPF.

On the nexus3k issue the command:

[[IDSEQSTEP]]
conf t
router ospf 1
router‐id 10.3.30.1
network 10.3.30.0 255.255.255.0 area 0
interface Vlan330
ip ospf network broadcast
end
copy running‐config startup‐config

With this configuration the Nexus3k is configured for OSPF. You should be able to already
establish a neighbor relationship with the CSR1Kv.

On the nexus3k issue the command:

show ip ospf neighbors

N3K‐OS‐PODPOD3# show ip ospf neighbors

OSPF Process ID 1 VRF default
Total number of neighbors: 1
Neighbor ID Pri State Up Time Address Interface
10.3.30.2 1 FULL/DR 00:02:21 10.3.30.2 Vlan330

You can also validate that the Nexus3k has learned the routes after the CSR1Kv

show ip route

http://openstack.ciscolive.com/printable.php?pod=3 133/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

N3K‐OS‐POD3# show ip route
IP Route Table for VRF "default"
'*' denotes best ucast next‐hop
'**' denotes best mcast next‐hop
'[x/y]' denotes [preference/metric]
'%' in via output denotes VRF

[CUT]
10.3.31.0/24, ubest/mbest: 1/0
*via 10.3.30.2, Vlan330, [110/41], 00:04:20, ospf‐1, intra

Step 168 Create Instance on compute1
You will be creating two separate instances and each instance will be created in a separate
compute node of your cluster. You will start with creating the instance on compute1. First
make sure you are logged into the Horizon Dashboard as userpod3 and goto the following
tab.

Once there click on the  Launch Instance button. The following screen will popup and

you will need to fill out as follows making sure not to click the submit button until after you
have selected the network adapter:

1 Availability Zone: compute1

2 Instance Name: POD3VM3

3 Instance Boot Source: Boot
from image

4 Image Name: cirros0.X.X
x86_64 (XX.X MB)

When you click on the network tab the following will be presented.

1 Click on the + button for the
option presented as Vlan331. This

http://openstack.ciscolive.com/printable.php?pod=3 134/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

selects the VLAN as the NIC
interface for this instance.

2 Click on the Launch button

At this point OpenStack will start the process of creating your instance attached to the
network Vlan331. Please click on Launch Instance

Step 169 Create second Instance on compute2
You will now create the second instance (VM) on compute2.

Click on the  Launch Instance button. The following screen will popup and you will need

to fill out as follows making sure not to click the submit button until after you have selected
the network adapter:

1 Availability Zone: compute2

2 Instance Name: POD3VM4

3 Instance Boot Source: Boot
from image

4 Image Name: cirros0.X.X
x86_64 (XX.X MB)

When you click on the network tab the following will be presented.

1 Click on the + button for the
option presented as Vlan331. This

http://openstack.ciscolive.com/printable.php?pod=3 135/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

selects the VLAN as the NIC
interface for this instance.

2 Click on the Launch button

At this point you should have two instances running in OpenStack! Congrats!. The following
screen you should be seeing at this point is as follows.

Your network should now look something like this:

ALERT!
This might be a good time to take a small break! The system is creating the
instances and takes a couple of minutes until the instances are ready for use.

http://openstack.ciscolive.com/printable.php?pod=3 136/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Step 170 Connect to instance consoles and
validate network
The best way to connect to the consoles of the instances is via the Network Topology tab in
the Horizon Dashboard. Head over to your Tenant Network tab and an option is Network
Topology. Select that option.

Once there you can see the topology you have created up to this point. If you hover over that
computer icon you can then see the following:

From there you can now click on Open Console to open the console of the specific instance.
A popup window will appear similar to the following.

Log into the two separate instances. And from them issue the command ip a . This

command will list the interface eth0 ip address.

http://openstack.ciscolive.com/printable.php?pod=3 137/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

You should be able to ping from instance to instance and also the default gateways.

On one of the instances issue the command:

ALERT!
You will not be able to copy into this VNC window.

ping 10.3.31.1

# ping 10.3.31.1
PING 10.3.31.1 (10.3.10.1) 56(84) bytes of data.
64 bytes from 10.3.31.1: icmp_seq=1 ttl=255 time=0.290 ms
64 bytes from 10.3.31.1: icmp_seq=2 ttl=255 time=0.207 ms
64 bytes from 10.3.31.1: icmp_seq=3 ttl=255 time=0.257 ms
64 bytes from 10.3.31.1: icmp_seq=4 ttl=255 time=0.207 ms

You should also be capable of pinging the northbound ASR1K that is depicted in the topology
diagram above.

On one of the instances issue the command:

ping 10.3.30.1

# ping 10.3.30.1
PING 10.3.30.1 (10.3.30.1) 56(84) bytes of data.
64 bytes from 10.3.30.1: icmp_seq=1 ttl=255 time=0.290 ms
64 bytes from 10.3.30.1: icmp_seq=2 ttl=255 time=0.207 ms
64 bytes from 10.3.30.1: icmp_seq=3 ttl=255 time=0.257 ms
64 bytes from 10.3.30.1: icmp_seq=4 ttl=255 time=0.207 ms

http://openstack.ciscolive.com/printable.php?pod=3 138/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Congrats! You
Finished!
If you wish to continue the lab with the Cisco Nexus ACI integration, on the next step a
special script will be used to remove the Tenant instances and network definitions. This will
not remove the underlying Neutron configuration. You will be replacing some values for ACI
in that step. If you just wish to continue playing with the lab without ACI stop here.

http://openstack.ciscolive.com/printable.php?pod=3 139/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Wipe Tenant OpenStack
configuration
This script is available on the internet and available via GitHub. It goes through an
OpenStack Tenant and removes all the configuration. The purpose of this script is to remove
all resources related to the Tenant before it get's cleared. For this lab we plan to run the
command with the options to keep the Tenant but remove all it's underlying resources so that
you can proceed to the ACI integration with a clear OpenStack setup.

Step 171 Login into dashboard with admin
account
Under the administrator account you will goto

On the horizon web interface on 10.0.236.31:

Step 172 Run dry run of purge all elements under
TenantPOD3
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
cd ~
. admin.sh
ospurge ‐‐dry‐run ‐‐cleanup‐project TenantPOD3

http://openstack.ciscolive.com/printable.php?pod=3 140/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

# ospurge ‐‐dry‐run ‐‐cleanup‐project TenantPOD3

* Resources type: CinderSnapshots

* Resources type: CinderBackups

* Resources type: NovaServers
server POD3VM4 (id 7e3198a3‐438e‐46f4‐adf0‐7da270f2b299)
server POD3VM3 (id 1de71e78‐fdda‐4024‐a5d4‐240d303dbfd5)
server CSR1Kv‐R1 (id 2aecc3de‐799f‐464d‐963b‐19d62d4b1d1a)
server POD3VM2 (id 4af9c718‐6e6d‐42c9‐a3b7‐41fb4a6e3f37)
server POD3VM1 (id 18b87292‐0cd0‐4c27‐95aa‐187a1d41b9a4)

* Resources type: NeutronFloatingIps

* Resources type: NeutronInterfaces
interface (id 31745c03‐391d‐4feb‐9631‐6c9e3c15c22f)

* Resources type: NeutronRouters
router TenantRouterPOD4 (id 35f2e546‐df5e‐4d39‐a0d2‐ce852b540e9f)

* Resources type: NeutronPorts
port  (id 2ca7a0b6‐c1c8‐4334‐ba54‐d64e129765f1)
port  (id 5f11d032‐d011‐4776‐b6d6‐16507cc1407a)
port  (id 5f3693bd‐2640‐4930‐b40b‐c3c8486aaf86)
port  (id 746e2593‐d469‐490a‐b631‐ca7d9e2cc13c)
port  (id a8df498d‐8c8a‐4959‐8eec‐0837f5793ca5)
port  (id dd12b4d4‐2ce4‐4e89‐8b0e‐e1befc315678)

* Resources type: NeutronNetworks
network Vlan431 (id 610813a8‐bed7‐4249‐85b0‐0dff0eaf57a4)
network Vlan430 (id 698416c8‐d2d7‐4765‐a823‐3a0b2b6d0c1b)
network Vlan410 (id 875e731d‐3c08‐4928‐aa9f‐2d2ed3c66a40)

* Resources type: NeutronSecgroups

* Resources type: GlanceImages

* Resources type: CinderVolumes

Step 173 Purge all elements under TenantPOD3

ALERT!
This is the last change to STOP. Once you enter the following command if you
wish to spend more time with just the Nexus3k integration you will have to redo
all the GUI based steps.

On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 141/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
cd ~
. admin.sh
ospurge ‐‐verbose ‐‐dont‐delete‐project ‐‐
cleanup‐project TenantPOD3

Step 174 Login into dashboard with tenant
account
Under the tenant account you will goto

On the horizon web interface on 10.0.236.31:

You should be able to browse through the Tenant and see that everything has been removed.

http://openstack.ciscolive.com/printable.php?pod=3 142/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

ACI Integration
Step 175 Configure a service for the Cisco APIC
host agent
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐del
/etc/neutron/plugins/ml2/ml2_conf.ini ml2
mechanism_drivers
openstack‐config ‐‐del
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_cisco vlan_name_prefix
openstack‐config ‐‐del
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_cisco managed_physical_network
openstack‐config ‐‐del
/etc/neutron/dhcp_agent.ini
enable_isolated_metadata True
openstack‐config ‐‐del
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_mech_cisco_nexus:n3k‐os‐pod3 username
openstack‐config ‐‐del
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_mech_cisco_nexus:n3k‐os‐pod3 password
openstack‐config ‐‐del
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_mech_cisco_nexus:n3k‐os‐pod3 ssh_port
openstack‐config ‐‐del
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_mech_cisco_nexus:n3k‐os‐pod3 pod3‐controller
openstack‐config ‐‐del
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_mech_cisco_nexus:n7k‐os‐pod3 username
openstack‐config ‐‐del
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_mech_cisco_nexus:n7k‐os‐pod3 password
openstack‐config ‐‐del
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_mech_cisco_nexus:n7k‐os‐pod3 ssh_port
openstack‐config ‐‐del
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_mech_cisco_nexus:n7k‐os‐pod3 pod3‐compute2

Step 176 Create a service file for systemd Cisco
APIC host agent
On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 143/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
cat <<EOF > /etc/systemd/system/multi‐
user.target.wants/neutron‐cisco‐apic‐host‐
agent.service
[Unit]
Description=OpenStack APIC Host Agent
After=syslog.target network.target

[Service]
Type=simple
User=neutron
ExecStart=/usr/bin/neutron‐cisco‐apic‐host‐agent
‐‐config‐file=/etc/neutron/neutron.conf ‐‐
config‐
file=/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
‐‐log‐file=/var/log/neutron/cisco‐apic‐host‐
agent.log
PrivateTmp=false
KillMode=process

[Install]
Wanted=multi‐user.target
EOF

Step 177 Create a service file for systemd Cisco
APIC service agent
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
cat <<EOF > /etc/systemd/system/multi‐
user.target.wants/neutron‐cisco‐apic‐service‐
agent.service
[Unit]
Description=OpenStack APIC Service Agent
After=syslog.target network.target

[Service]
Type=simple
User=neutron
ExecStart=/usr/bin/neutron‐cisco‐apic‐service‐
agent ‐‐config‐file=/etc/neutron/neutron.conf ‐‐
config‐
file=/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
‐‐log‐file=/var/log/neutron/cisco‐apic‐service‐
agent.log
PrivateTmp=false
KillMode=process

[Install]
WantedBy=multi‐user.target
EOF

Step 178 Create a service file for systemd for
neutron service
http://openstack.ciscolive.com/printable.php?pod=3 144/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
cat <<EOF > /etc/systemd/system/multi‐
user.target.wants/neutron‐server.service
[Service]
Type=notify
User=neutron
ExecStart=/usr/bin/neutron‐server ‐‐config‐file
/usr/share/neutron/neutron‐dist.conf ‐‐config‐
file /etc/neutron/neutron.conf ‐‐config‐file
/etc/neutron/plugin.ini ‐‐log‐file
/var/log/neutron/server.log ‐‐config‐file
/etc/neutron/plugins/ml2/ml2_conf.ini ‐‐config‐
file /etc/neutron/plugins/ml2/ml2_conf_cisco.ini
PrivateTmp=true
NotifyAccess=all
KillMode=process

[Install]
WantedBy=multi‐user.target
EOF

Step 179 Create a service file for systemd for
OpenVSwitch agent
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
cat <<EOF > /etc/systemd/system/multi‐
user.target.wants/neutron‐openvswitch‐
agent.service
[Unit]
Description=OpenStack Neutron Open vSwitch Agent
After=syslog.target network.target

[Service]
Type=simple
User=neutron
ExecStart=/usr/bin/neutron‐openvswitch‐agent ‐‐
config‐file /usr/share/neutron/neutron‐dist.conf
‐‐config‐file /etc/neutron/neutron.conf ‐‐
config‐file
/etc/neutron/plugins/ml2/ml2_conf.ini ‐‐log‐file
/var/log/neutron/openvswitch‐agent.log
PrivateTmp=true
KillMode=process

[Install]
WantedBy=multi‐user.target
EOF

Step 180 Configure Cisco ML2 INI for ACI
integration
http://openstack.ciscolive.com/printable.php?pod=3 145/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Configure the systemID that will be used to prefix all configuration towards the ACI fabric.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
DEFAULT apic_system_id POD3

Now you have to define the connectivity parameters for OpenStack to connect to the APIC
Controller.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_cisco_apic apic_hosts
10.0.226.59:80,10.0.226.60:80,10.0.226.61:80
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_cisco_apic apic_username admin
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_cisco_apic apic_password cisco.123
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_cisco_apic apic_name_mapping use_name
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_cisco_apic apic_use_ssl False

Define the use of SUDO for neutron ML2 Cisco APIC driver.

Step 181 Configure Cisco ML2 INI Attached
devices
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
ml2_cisco_apic root_helper 'sudo'

The following command will add to the SUDOERS file the ability for neutron to be a sudoer.

On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 146/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
sed ‐i 's/commands\.\.\./comands\.\.\. \n##
Neutron sudoers as needed for APIC following
installation instructions\nneutron ALL=(ALL)
NOPASSWD: ALL/g' /etc/sudoers

Finally define the APIC network definitions for this OpenStack POD towards APIC.

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
apic_external_network:POD3‐ext switch 201
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
apic_external_network:POD3‐ext port 1/19
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
apic_external_network:POD3‐ext cidr_exposed
10.3.90.2/24
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
apic_external_network:POD3‐ext gateway_ip
10.3.90.1
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf_cisco.ini
apic_switch:201 pod3‐controller,pod3‐compute2
1/5

Step 182 Configure Neutron keystone auth
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/neutron/neutron.conf
keystone_authtoken auth_protocol http
openstack‐config ‐‐set /etc/neutron/neutron.conf
keystone_authtoken auth_host 10.0.236.31
openstack‐config ‐‐set /etc/neutron/neutron.conf
keystone_authtoken auth_port 35357

Step 183 Change service plugin
On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 147/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT service_plugins
neutron.services.l3_router.l3_apic.ApicL3ServicePlugin
openstack‐config ‐‐set /etc/neutron/neutron.conf
DEFAULT core_plugin ml2

Step 184 Configure the ML2 Security Groups
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/neutron/plugins/ml2/ml2_conf.ini
securitygroup enable_security_group True
openstack‐config ‐‐set /etc/neutron/plugins/ml2/ml2_conf.ini
securitygroup enable_ipset True
openstack‐config ‐‐set /etc/neutron/plugins/ml2/ml2_conf.ini
securitygroup firewall_driver
neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver

Step 185 Configure the ML2 agent
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf.ini agent
polling_interval 2
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf.ini agent
l2_population False
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf.ini agent
arp_responder False

Step 186 Configure the ML2 driver type and
network type
On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 148/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Step 187 Configure the Cisco APIC Mechanism
driver
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf.ini ml2
mechanism_drivers openvswitch,cisco_apic

Step 188 Configure Cisco ML2 OVS Vlan Ranges
aci 390 <> 395

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf.ini
ml2_type_vlan network_vlan_ranges aci:390:395

Step 189 Configure Cisco ML2 OVS Bridge
Mapping
The relationship will map as follows. While you haven't yet been to the OpenStack
dashboard, this is a quick peek to understand the relationships.

http://openstack.ciscolive.com/printable.php?pod=3 149/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf.ini ovs
bridge_mappings aci:br‐aci
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf.ini ovs
enable_tunneling False
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf.ini ovs
integration_bridge br‐int

Step 190 Restart Neutron Services
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
systemctl daemon‐reload
systemctl restart neutron‐server.service
systemctl status neutron‐server.service

http://openstack.ciscolive.com/printable.php?pod=3 150/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

ACI Integration
Step 191 Create a service file for systemd Cisco
APIC host agent
On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
cat <<EOF > /etc/systemd/system/multi‐
user.target.wants/neutron‐cisco‐apic‐host‐
agent.service
[Unit]
Description=OpenStack APIC Host Agent
After=syslog.target network.target

[Install]
Wanted=multi‐user.target
EOF

Step 192 Configure Cisco ML2 OVS Vlan Ranges
On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐set
/etc/neutron/plugins/ml2/ml2_conf.ini
ml2_type_vlan network_vlan_ranges aci:390:395

Step 193 Configure Cisco ML2 OVS Bridge
Mapping
On the compute host 10.0.236.32 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 151/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Step 194 Enable and start OpenVSwitch on
compute node
On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
systemctl enable neutron‐openvswitch‐
agent.service
systemctl start neutron‐openvswitch‐
agent.service

Validate that OpenVSwitch service is operational.

On the compute host 10.0.236.32 issue the command:

[[IDSEQSTEP]]
systemctl status neutron‐openvswitch‐
agent.service

http://openstack.ciscolive.com/printable.php?pod=3 152/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

http://openstack.ciscolive.com/printable.php?pod=3 153/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

OpenStack base network
Step 195 Login into dashboard with admin
account
Under the administrator account you will goto

On the horizon web interface on 10.0.236.31:

After you login into to the Horizon dashboard with the admin account, you should see a
screen similar to the following:

Step 196 Create ACI Network POD3ext
Locate the network tab under admin and click on +Create Network

On the horizon web interface on 10.0.236.31:

http://openstack.ciscolive.com/printable.php?pod=3 154/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

When finished you should see a screen similar to the following. Now you need to click on the
Network Name to add the subnet for this network.

Click on the Network Name, and the following screen will appear.

On this screen click on the +Create Subnet

In this step you will be creating the subnet for the public admin network. When creating a
subnet it involves two parts.

http://openstack.ciscolive.com/printable.php?pod=3 155/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

ALERT!
Make sure to disable DHCP for this subnet.

Step 197 Create ACI Network Vlan391
Locate the network tab under admin and click on +Create Network

On the horizon web interface on 10.0.236.31:

http://openstack.ciscolive.com/printable.php?pod=3 156/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

When finished you should see a screen similar to the following. Now you need to click on the
Network Name to add the subnet for this network.

Click on the Network Name, and the following screen will appear.

On this screen click on the +Create Subnet

Step 198 Create Subnet for ACI Network Vlan391
In this step you will be creating the subnet for the public admin network. When creating a
subnet it involves two parts.

http://openstack.ciscolive.com/printable.php?pod=3 157/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Step 199 Create Network for ACI network Vlan392
Click on the Network Tab to return to the network list and click on +Create Network
button.

http://openstack.ciscolive.com/printable.php?pod=3 158/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

When finished you should see a screen similar to the following. Now you need to click on the
Network Name to add the subnet for this network.

You should be back to the Network pane. On this screen click on the +Create Subnet

You will now create the private network for the Nexus component.

http://openstack.ciscolive.com/printable.php?pod=3 159/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Step 200 Login into ACI
Open a new TAB in your browser and point your browser to:

http://10.0.226.59 (http://10.0.226.59)

On the ACI web interface on 10.0.226.59:

Once you have logged into the ACI admin, click on the Tenants TAB at the top of the page.

Once in the Tenants TAB you need to click on All tenants.

http://openstack.ciscolive.com/printable.php?pod=3 160/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

You will be presented with a list of Tenants that are part of the system. Depending on where
other members of the lab are in completion, depends on the list of Tenants. You will need to
locate and click on the tenant name _POD3_TenantPOD3 . The following screen will be
presented.

Here you can click on the Networking PLUS sign on the left of the screen to expand
Networking.

In the following screen you can observe that OpenStack has created for this Tenant the two
Bridge Domains in ACI for both networks Vlan391 and Vlan392.

If you expand on the Application Profile, you will see that OpenStack has also created two
separate End Point Groups (EPGs) correlated to each of the networks that you built.

The OpenStack ACI plugin will always create a Bridge Domain and an End Point Group for
every Network that is built in OpenStack. At this point any instance that is built in these
networks would exist inside these EPGs in ACI.

http://openstack.ciscolive.com/printable.php?pod=3 161/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Tenant is probably the most important construct of the ACI Fabric. A Tenant can mean
different things to different organizations. For example for a service provider, a tenant might
mean different customers. For a single company a Tenant might mean different parts of the
organization ( engineering, marketing ). For a group of developers a Tenant might mean single
developer.

A Tenant is a logical construct that isolates a series of networks from other tenants in the
fabric. Each tenant is an overlay in the network. It let's you organize the network in such a
way that each Tenant perceives itself as part of a unique network. Each Tenant is
compromised of different elements ( or objects ) that are part of the Managed Information
Tree.

End Point Group
An EPG is a managed object, a named logical entity that contains a collection of end
points. End points are devices connected to the network directly or indirectly.
Application Network Profiles
Application network profiles provide a convenient way to model application
requirements. The application profile contains as many (or as few) EPGs as necessary
that are logically related to providing the capabilities of an application.
Contracts
Contracts are the rules that specify what and how communication between EPGs takes
place. If there is no contract, interEPG communication is disabled by default.
Subject
Contracts contain one or more subjects. Subjects specify what information can be
communicated and how; for example, HTTPS messages (the what), the ports allowed,
and the direction (the how).
Filters
Filters are TCP/IP header fields such as L3 protocol type, L4 ports, etc. Subjects
determine if filters are uni or bidirectional. A unidirectional filter is used in one

http://openstack.ciscolive.com/printable.php?pod=3 162/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

direction. Unidirectional filters define in or out communications but not the same for
both. Bidirectional filters are the same for both directions; they define both in and out.
According to its related contract, an EPG provider dictates the protocols and ports in
both the in and out directions. According to its related contract, a consumer EPG
specifies the protocols and ports for use in the inbound direction (from the provider).
Contracts contain all of the filters (and their directions) that will be applied between
EPGs that produce and consume the contract.
Contexts
A context defines a layer 3 address domain. All of the endpoints within the layer 3
domain must have unique IP addresses because it is possible to directly forward
packets between these devices should the policy allow it. It is equivalent to a VRF in
the networking world. A tenant may contain multiple contexts.
Bridge Domain
While a context defines a unique IP address space, that address space can consist of
multiple subnets. Those subnets are defined in one or more bridge domains that
reference the corresponding context. Each bridge domain must be linked to a context
and have at least one subnet.

The next step will be to create a Router in OpenStack. While traditionally a router would
create a entity that does routing inbetween segments, for ACI the Router is going to
establish a contract between the two EPGs that have been created.

Step 201 Signout and Signin as Tenant User
Logout from the admin account as the next steps will be conducted as you assume the role
of the Tenant of this OpenStack network.

Step 202 Login to dashboard with poduser
account
On the horizon web interface on 10.0.236.31:

Step 203 Configure Tenant Router
To configure the Router, please proceed to the following screen and click on the
+Create Router .

http://openstack.ciscolive.com/printable.php?pod=3 163/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

When the popup appears please enter the name ACIRouterPOD3 as the router name.

1 Router Name:
ACIRouterPOD3

Once created you will select the gateway for this router. Click on the Set Gateway as you

would see below.

The following screen should have popped up to you.

1 External Network:
POD3ext

This will setup the external connectivity for the ACI network. Click on the Router Name
ACIRouterPOD3.

http://openstack.ciscolive.com/printable.php?pod=3 164/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Click on the + Add interface button. The following screen will be presented. Select the

option that says Vlan391:10.3.91.0/24 and click the + Add interface button.

1 Subnet:
Vlan391:10.3.91.0/24

You will have to repeat these steps to create another interface on this router for the second
network.

Click on the + Add interface button. The following screen will be presented. Select the

option that says Vlan392:10.3.92.0/24 and click the + Add interface button.

1 Subnet:
Vlan392:10.3.92.0/24

Now that you have completed this step, you will go back to ACI to see that OpenStack has
now created a contract between the two End Point Groups (EPGs).

Step 204 Login into ACI
http://openstack.ciscolive.com/printable.php?pod=3 165/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Open the previous TAB in your browser to reach the ACI interface at:

http://10.0.226.59 (http://10.0.226.59)

On the ACI web interface on 10.0.226.59:

In ACI you have to look under the Application Profile. Expand the POD3_app that will list you
the EPGs that we had seen before. Now though, the EPGs are connected to each other via a
Contract.

The following image provides a visual representation of the completed Application Profile that
OpenStack built in the fabric.

You will now return to OpenStack and you will create some instances in OpenStack that will
utilize the ACI fabric.

http://openstack.ciscolive.com/printable.php?pod=3 166/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Step 205 Login to dashboard with poduser
account
On the horizon web interface on 10.0.236.31:

Step 206 Create Instance on compute1
You will be creating two separate instances and each instance will be created in a separate
compute node of your cluster. You will start with creating the instance on compute1. First
make sure you are logged into the Horizon Dashboard as userpod3 and goto the following
TAB.

Once there click on the  Launch Instance button. The following screen will popup and

you will need to fill out as follows making sure not to click the submit button until after you
have selected the network adapter:

1 Availability Zone: compute1

2 Instance Name: POD3ACI
VM1

3 Instance Boot Source: Boot
from image

4 Image Name: cirros0.X.X
x86_64 (XX.X MB)

http://openstack.ciscolive.com/printable.php?pod=3 167/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

When you click on the network tab the following will be presented.

1 Click on the + button for the
option presented as Vlan391. This
selects the VLAN as the NIC
interface for this instance.

2 Click on the Launch
button

At this point OpenStack will start the process of creating your instance attached to the
network Vlan391. Please click on Launch Instance

Step 207 Create second Instance on compute2
You will now create the second instance (VM) on compute2.

Click on the  Launch Instance button. The following screen will popup and you will need

to fill out as follows making sure not to click the submit button until after you have selected
the network adapter:

1 Availability Zone: compute2

2 Instance Name: POD3ACI
VM2

3 Instance Boot Source: Boot
from image

4 Image Name: cirros0.X.X
x86_64 (XX.X MB)

When you click on the network tab the following will be presented.

http://openstack.ciscolive.com/printable.php?pod=3 168/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

1 Click on the + button for the
option presented as Vlan392. This
selects the VLAN as the NIC
interface for this instance.

2 Click on the Launch button

At this point you should have two instances running in OpenStack! Congrats!. The following
screen you should be seeing at this point is as follows.

Your network should now look like this:

ALERT!
This might be a good time to take a small break! The system is creating the
instances and takes a couple of minutes until the instances are ready for use.

http://openstack.ciscolive.com/printable.php?pod=3 169/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Step 208 Connect to instance consoles and
validate network
The best way to connect to the consoles of the instances is via the Network Topology TAB
in the Horizon Dashboard. Head over to your Tenant Network tab and an option is Network
Topology. Select that option.

Once there you can see the topology you have created up to this point. If you hover over that
computer icon you can then see the following:

From there you can now click on Open Console to open the console of the specific instance.
A popup window will appear similar to the following.

Log into the two separate instances. And from them issue the command ip a . This

command will list the interface eth0 ip address.

http://openstack.ciscolive.com/printable.php?pod=3 170/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

You should be able to ping from instance to instance and also the default gateways.

On one of the instances issue the command:

ALERT!
You will not be able to copy into this VNC window.

ping 10.3.91.1

# ping 10.3.91.1
PING 10.3.91.1 (10.3.10.1) 56(84) bytes of data.
64 bytes from 10.3.91.1: icmp_seq=1 ttl=255 time=0.290 ms
64 bytes from 10.3.91.1: icmp_seq=2 ttl=255 time=0.207 ms
64 bytes from 10.3.91.1: icmp_seq=3 ttl=255 time=0.257 ms
64 bytes from 10.3.91.1: icmp_seq=4 ttl=255 time=0.207 ms

You should also be capable of pinging the northbound ASR1K.

On one of the instances issue the command:

ping 10.3.92.1

# ping 10.3.92.1
PING 10.3.92.1 (10.3.92.1) 56(84) bytes of data.
64 bytes from 10.3.92.1: icmp_seq=1 ttl=255 time=0.290 ms
64 bytes from 10.3.92.1: icmp_seq=2 ttl=255 time=0.207 ms
64 bytes from 10.3.92.1: icmp_seq=3 ttl=255 time=0.257 ms
64 bytes from 10.3.92.1: icmp_seq=4 ttl=255 time=0.207 ms

You should also be capable of pinging the northbound external network.

http://openstack.ciscolive.com/printable.php?pod=3 171/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

On one of the instances issue the command:

ping 10.3.90.1

# ping 10.3.90.1
PING 10.3.90.1 (10.3.90.1) 56(84) bytes of data.
64 bytes from 10.3.90.1: icmp_seq=1 ttl=255 time=0.290 ms
64 bytes from 10.3.90.1: icmp_seq=2 ttl=255 time=0.207 ms
64 bytes from 10.3.90.1: icmp_seq=3 ttl=255 time=0.257 ms
64 bytes from 10.3.90.1: icmp_seq=4 ttl=255 time=0.207 ms

Congrats you Finished!

http://openstack.ciscolive.com/printable.php?pod=3 172/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Configure Heat Controller Host
NOTE: Remember to switch back to your Controller node in PuTTY for this section.

Step 209 Open Firewall Holes
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
firewall‐cmd ‐‐add‐port=8000/tcp ‐‐permanent
firewall‐cmd ‐‐add‐port=8004/tcp ‐‐permanent
firewall‐cmd ‐‐reload

Step 210 Configure Heat database and access to
database
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
mysql ‐‐user=root ‐‐password=cisco.123
CREATE DATABASE heat;
GRANT ALL PRIVILEGES ON heat.* TO
'heat'@'localhost' IDENTIFIED BY 'cisco.123';
GRANT ALL PRIVILEGES ON heat.* TO 'heat'@'%'
IDENTIFIED BY 'cisco.123';
exit

Step 211 Add Heat users to keystone
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
cd ~
source admin.sh

On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 173/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
keystone user‐create ‐‐name heat ‐‐pass
cisco.123

keystone user‐role‐add ‐‐user heat ‐‐tenant
service ‐‐role admin
keystone role‐create ‐‐name heat_stack_owner
keystone user‐role‐add ‐‐user demo ‐‐tenant demo
‐‐role heat_stack_owner
keystone role‐create ‐‐name heat_stack_user
keystone service‐create ‐‐name heat ‐‐type
orchestration ‐‐description "Orchestration"
keystone service‐create ‐‐name heat‐cfn ‐‐type
cloudformation ‐‐description "Orchestration"
keystone endpoint‐create \
  ‐‐service‐id $(keystone service‐list | awk '/
orchestration / {print $2}') \
  ‐‐publicurl http://10.0.236.31:8004/v1/%\
(tenant_id\)s \
  ‐‐internalurl http://10.0.236.31:8004/v1/%\
(tenant_id\)s \
  ‐‐adminurl http://10.0.236.31:8004/v1/%\
(tenant_id\)s \
  ‐‐region regionOne
keystone endpoint‐create \
  ‐‐service‐id $(keystone service‐list | awk '/
cloudformation / {print $2}') \
  ‐‐publicurl http://10.0.236.31:8000/v1 \
  ‐‐internalurl http://10.0.236.31:8000/v1 \
  ‐‐adminurl http://10.0.236.31:8000/v1 \
  ‐‐region regionOne

Step 212 Install Heat
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
yum ‐y install openstack‐heat‐api openstack‐
heat‐api‐cfn openstack‐heat‐engine python‐
heatclient

Step 213 Heat INI File
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/heat/heat.conf
database connection
mysql://heat:[email protected]/heat

On the controller host 10.0.236.31 issue the command:

http://openstack.ciscolive.com/printable.php?pod=3 174/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/heat/heat.conf
DEFAULT rpc_backend rabbit
openstack‐config ‐‐set /etc/heat/heat.conf
DEFAULT rabbit_host 10.0.236.31
openstack‐config ‐‐set /etc/heat/heat.conf
DEFAULT rabbit_password cisco.123
openstack‐config ‐‐set /etc/heat/heat.conf
DEFAULT heat_metadata_server_url
http://10.0.236.31:8000
openstack‐config ‐‐set /etc/heat/heat.conf
DEFAULT heat_waitcondition_server_url
http://10.0.236.31:8000/v1/waitcondition

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
openstack‐config ‐‐set /etc/heat/heat.conf
keystone_authtoken auth_uri
http://10.0.236.31:5000/v2.0
openstack‐config ‐‐set /etc/heat/heat.conf
keystone_authtoken identity_uri
http://10.0.236.31:35357
openstack‐config ‐‐set /etc/heat/heat.conf
keystone_authtoken admin_tenant_name service
openstack‐config ‐‐set /etc/heat/heat.conf
keystone_authtoken admin_user heat
openstack‐config ‐‐set /etc/heat/heat.conf
keystone_authtoken admin_password cisco.123

openstack‐config ‐‐set /etc/heat/heat.conf
ec2authtoken auth_uri
http://10.0.236.31:5000/v2.0

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
su ‐s /bin/sh ‐c "heat‐manage db_sync" heat

On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
systemctl enable openstack‐heat‐api.service
openstack‐heat‐api‐cfn.service openstack‐heat‐
engine.service
systemctl start openstack‐heat‐api.service
openstack‐heat‐api‐cfn.service openstack‐heat‐
engine.service
systemctl status openstack‐heat‐api.service
openstack‐heat‐api‐cfn.service openstack‐heat‐
engine.service

http://openstack.ciscolive.com/printable.php?pod=3 175/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Step 214 Add Heat Template
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
cd ~
. TenantPOD15.sh

test‐stack.yml << EOF >
description: Test Template

parameters:
  ImageID:
    type: string
    description: Image use to boot a server
  NetID:
    type: string
    description: Network ID for the server

resources:
  server1:
    type: OS::Nova::Server
    properties:
      name: "Test server"
      image: { get_param: ImageID }
      flavor: "m1.tiny"
      networks:
      ‐ network: { get_param: NetID }

outputs:
  server1_private_ip:
    description: IP address of the server in the
private network
    value: { get_attr: [ server1, first_address
] }
EOF

Step 215 Create Heat Stack
On the controller host 10.0.236.31 issue the command:

[[IDSEQSTEP]]
NET_ID=$(nova net‐list | awk '/ demo‐net / {
print $2 }')
heat stack‐create ‐f test‐stack.yml ‐P
"ImageID=cirros‐0.3.3‐x86_64;NetID=$NET_ID"
testStack
heat stack‐list

http://openstack.ciscolive.com/printable.php?pod=3 176/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

http://openstack.ciscolive.com/printable.php?pod=3 177/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Troubleshooting Commands
Locating only changed variables in OpenStack
configuration files
One valuable command line grep to identify only the configuration lines in a OpenStack INI
file that are not set ( meaning they don't start with a # or are a blank line ).

grep ‐v "^#" <filename.conf> | grep ‐v "^\s*$"

# grep ‐v "^#" /etc/glance/glance‐api.conf | grep ‐v "^\s*$"
[DEFAULT]
[database]
connection = mysql://glance:cisco.123@controller/glance
[keystone_authtoken]
auth_uri = http://controller:5000/v2.0
identity_uri = http://controller:35357
admin_tenant_name = service
admin_user = glance
admin_password = cisco.123
[paste_deploy]
flavor = keystone
[store_type_location_strategy]
[profiler]
[task]
[glance_store]
default_store = file
filesystem_store_datadir = /var/lib/glance/images/

Looking at log files
All the logs that we are interested on are on /var/log. They are orginized in directories by
component. For example for neutron server the corresponding log file is
'/var/log/neutorn/server.log'.

When troubleshooting, we need to look at differet logs file. It is helpfull to look at them at the
same time. Here is a way to do this:

Controller logs:
tail ‐f
/var/log/{ceilometer,cinder,glance,keystone,mysql,neutron,nova,openvswitch,rabbitmq}/*.log
/var/log/messages

Compute logs:
tail ‐f /var/log/{ceilometer,neutron,nova,openvswitch}/*.log /var/log/messages

http://openstack.ciscolive.com/printable.php?pod=3 178/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Reference Material
Important Web References
Synopsis URL

Detailed document on the https://openstack.redhat.com/Networking_in_too_much_detail
networking components of (https://openstack.redhat.com/Networking_in_too_much_detail)
OpenStack from the
RedHat team

NOVA Scheduler options http://docs.openstack.org/juno/config
and configuration reference/content/section_computescheduler.html
(http://docs.openstack.org/juno/config
reference/content/section_computescheduler.html)

RabbitMQ and Nova http://docs.openstack.org/developer/nova/devref/rpc.html
(http://docs.openstack.org/developer/nova/devref/rpc.html)

Openstack TCP Ports
Default ports that OpenStack components use

OpenStack service Default ports Port type

Block Storage ( cinder ) 8776 publicurl and

adminurl

Compute ( nova ) endpoints 8774 publicurl and

adminurl

Compute API ( nova‐api ) 8773, 8775

Compute ports for access to virtual machine 59005999
consoles

Compute VNC proxy for browsers ( 6080
openstack‐nova‐novncproxy )

Compute VNC proxy for traditional VNC 6081
clients ( openstack‐nova‐xvpvncproxy )

Proxy port for HTML5 console used by 6082
Compute service

Identity service ( keystone ) administrative 35357 adminurl

endpoint

Identity service public endpoint 5000 publicurl

http://openstack.ciscolive.com/printable.php?pod=3 179/180
1. 2. 2015 CiscoLive! OpenStack Lab LABCLD2225

Image Service ( glance ) API 9292 publicurl and

adminurl

Image Service registry 9191

Networking ( neutron ) 9696 publicurl and

adminurl

Object Storage ( swift ) 6000, 6001, 6002

Orchestration ( heat ) endpoint 8004 publicurl and

adminurl

Orchestration AWS CloudFormation 8000
compatible API ( openstack‐heat‐api‐cfn )

Orchestration AWS CloudWatchcompatible 8003
API ( openstack‐heat‐api‐cloudwatch )

Telemetry ( ceilometer ) 8777 publicurl and

adminurl

Default ports that secondary services related to OpenStack components use

Service Default port Used by

HTTP 80 OpenStack dashboard ( Horizon ) when it is

not configured to use secure access.

HTTP alternate 8080 OpenStack Object Storage ( swift ) service.

HTTPS 443 Any OpenStack service that is enabled for

SSL, especially secureaccess dashboard.

rsync 873 OpenStack Object Storage. Required.

iSCSI target 3260 OpenStack Block Storage. Required.

MySQL database 3306 Most OpenStack components.

service

Message Broker 5672 OpenStack Block Storage, Networking,

(AMQP traffic) Orchestration, and Compute.

http://openstack.ciscolive.com/printable.php?pod=3 180/180

Red Hat Openstack Platform-17.1-Installing and Managing Red Hat Openstack Platform With Director-En-Us222
No ratings yet
Red Hat Openstack Platform-17.1-Installing and Managing Red Hat Openstack Platform With Director-En-Us222
192 pages
Cloud Train DevOps Instructor Led Training Curriculum 2023
No ratings yet
Cloud Train DevOps Instructor Led Training Curriculum 2023
25 pages
Openstack Laboratory Guide v5.0.1 Pike Release
100% (2)
Openstack Laboratory Guide v5.0.1 Pike Release
204 pages
Openstack Technical 101
100% (2)
Openstack Technical 101
31 pages
OpenStack Installation Checklist 1.1
No ratings yet
OpenStack Installation Checklist 1.1
85 pages
OVN Kubernetes Presentation OCTO October6th2020
No ratings yet
OVN Kubernetes Presentation OCTO October6th2020
43 pages
Red Hat Enterprise Linux OpenStack Platform-5-PDF-Installation and Configuration Guide-Red Hat Enterprise Linux OpenStack Platform-5-Installation and Configuration
No ratings yet
Red Hat Enterprise Linux OpenStack Platform-5-PDF-Installation and Configuration Guide-Red Hat Enterprise Linux OpenStack Platform-5-Installation and Configuration
232 pages
Open Stack Deployment Design Guide
No ratings yet
Open Stack Deployment Design Guide
100 pages
Red Hat OpenStack Platform 13 - Technical Update - 301 - TDM
No ratings yet
Red Hat OpenStack Platform 13 - Technical Update - 301 - TDM
142 pages
OpenStack Fundamentals Training Part 2 - Compute Presentation
No ratings yet
OpenStack Fundamentals Training Part 2 - Compute Presentation
29 pages
Build Public/private Cloud Using Openstack: by Framgia Server Lab
No ratings yet
Build Public/private Cloud Using Openstack: by Framgia Server Lab
32 pages
What Is Cloud Computing
No ratings yet
What Is Cloud Computing
21 pages
InstallGuide OpenStack
100% (1)
InstallGuide OpenStack
129 pages
How To Configure or Build Ceph Storage Cluster in Openstack
No ratings yet
How To Configure or Build Ceph Storage Cluster in Openstack
30 pages
Openstack Deployment Ops Guide PDF
No ratings yet
Openstack Deployment Ops Guide PDF
197 pages
Openstack Super Bootcamp
100% (2)
Openstack Super Bootcamp
96 pages
OpenStack NVM 2021
No ratings yet
OpenStack NVM 2021
120 pages
OpenStack 101
No ratings yet
OpenStack 101
36 pages
Openstack Super Bootcamp PDF
No ratings yet
Openstack Super Bootcamp PDF
96 pages
Red Hat Openstack Platform 13: Transitioning To Containerized Services
No ratings yet
Red Hat Openstack Platform 13: Transitioning To Containerized Services
31 pages
Building High Availability in OpenStack Sergii Golovatiuk Mirantis
No ratings yet
Building High Availability in OpenStack Sergii Golovatiuk Mirantis
23 pages
Red Hat OpenShift With NetApp
No ratings yet
Red Hat OpenShift With NetApp
110 pages
Openstack 888
No ratings yet
Openstack 888
68 pages
5000 Programming Books.
57% (7)
5000 Programming Books.
160 pages
Ceph at CSC
No ratings yet
Ceph at CSC
27 pages
CL210 Rhosp16
No ratings yet
CL210 Rhosp16
6 pages
Openstack Redhat
No ratings yet
Openstack Redhat
24 pages
OpenStack Lab
No ratings yet
OpenStack Lab
7 pages
OpenStack Commissioning - MSS Cloud CFP5
No ratings yet
OpenStack Commissioning - MSS Cloud CFP5
142 pages
Dell Emc Ready Architecture Guide Red Hat v16 1
No ratings yet
Dell Emc Ready Architecture Guide Red Hat v16 1
77 pages
LFS252 Openstack OCA
No ratings yet
LFS252 Openstack OCA
119 pages
OpenStack Virtual Machine Image Guide
No ratings yet
OpenStack Virtual Machine Image Guide
64 pages
Fuel For OpenStack 3.1 UserGuide
100% (1)
Fuel For OpenStack 3.1 UserGuide
126 pages
Classroom Requirements: Red Hat Training & Certification
100% (1)
Classroom Requirements: Red Hat Training & Certification
11 pages
Red Hat OpenStack - CL210 Training
No ratings yet
Red Hat OpenStack - CL210 Training
10 pages
Openstack Horizon
No ratings yet
Openstack Horizon
89 pages
5 210 249 137MB
100% (1)
5 210 249 137MB
249 pages
Openstack HP
No ratings yet
Openstack HP
15 pages
Kolla Ansible
No ratings yet
Kolla Ansible
210 pages
Zimbra Collaboration System Administration - Jan2014
No ratings yet
Zimbra Collaboration System Administration - Jan2014
262 pages
Red Hat OpenStack Administration Certification Training
No ratings yet
Red Hat OpenStack Administration Certification Training
10 pages
Red Hat OpenStack Platform-9-Director Installation and Usage-En-US
No ratings yet
Red Hat OpenStack Platform-9-Director Installation and Usage-En-US
190 pages
OCP 4 Install Workshop-2
100% (2)
OCP 4 Install Workshop-2
97 pages
Red Hat Ceph Storage 5 Installation Guide en Us
No ratings yet
Red Hat Ceph Storage 5 Installation Guide en Us
111 pages
SaltStack Administration I Summary
No ratings yet
SaltStack Administration I Summary
5 pages
TechGuide OpenShift Deployment Dell RedHat OpenStack v5
No ratings yet
TechGuide OpenShift Deployment Dell RedHat OpenStack v5
42 pages
Red Hat Openstack Platform-16.1-Networking Guide-En-Us
No ratings yet
Red Hat Openstack Platform-16.1-Networking Guide-En-Us
208 pages
OpenStack Bootcamp OS100
No ratings yet
OpenStack Bootcamp OS100
2 pages
Software-Defined Networking (SDN) Deployment Guide
No ratings yet
Software-Defined Networking (SDN) Deployment Guide
38 pages
Learning OpenStack Networking (Neutron) - Second Edition - Sample Chapter
No ratings yet
Learning OpenStack Networking (Neutron) - Second Edition - Sample Chapter
28 pages
Oca - Openstack PDF
No ratings yet
Oca - Openstack PDF
5 pages
Troubleshooting Open Stack
No ratings yet
Troubleshooting Open Stack
20 pages
Mastering Openstack: Controller Nodes
No ratings yet
Mastering Openstack: Controller Nodes
21 pages
TXLF2011 OpenStack
No ratings yet
TXLF2011 OpenStack
83 pages
Install Your Own OpenStack Cloud Essex Edition
No ratings yet
Install Your Own OpenStack Cloud Essex Edition
49 pages
Huawei Distributed Cloud Data Center Technical White Paper
No ratings yet
Huawei Distributed Cloud Data Center Technical White Paper
51 pages
TR Openshift Learning Path Infographic f23466 en
No ratings yet
TR Openshift Learning Path Infographic f23466 en
1 page
Red Hat Ceph Storage Hardware Selection Guide
No ratings yet
Red Hat Ceph Storage Hardware Selection Guide
23 pages
Unix / Linux FAQ: with Tips to Face Interviews
From Everand
Unix / Linux FAQ: with Tips to Face Interviews
Prof. N.B. Venkateswarlu
No ratings yet
RHEL 9.3 - Configuring A Redhat High Availability Cluster On Redhat Openstack Platform
No ratings yet
RHEL 9.3 - Configuring A Redhat High Availability Cluster On Redhat Openstack Platform
25 pages
Openstack
No ratings yet
Openstack
19 pages
1 Openstack Neutron Distributed Virtual Router
No ratings yet
1 Openstack Neutron Distributed Virtual Router
11 pages
Exam JN0-211: IT Certification Guaranteed, The Easy Way!
No ratings yet
Exam JN0-211: IT Certification Guaranteed, The Easy Way!
24 pages
TripleO OpenStack
No ratings yet
TripleO OpenStack
36 pages
Configuring Cisco Nexus 9000 Switches in ACI Mode: Duration: 5 Days Course Code: DCAC9K Version: 2.0
No ratings yet
Configuring Cisco Nexus 9000 Switches in ACI Mode: Duration: 5 Days Course Code: DCAC9K Version: 2.0
2 pages
Prashanth Dollu: Page 1 of 4
No ratings yet
Prashanth Dollu: Page 1 of 4
4 pages
Junos OS: Network Address Translation Feature Guide For Security Devices
No ratings yet
Junos OS: Network Address Translation Feature Guide For Security Devices
388 pages
Implementing NetScaler VPX™ - Second Edition
From Everand
Implementing NetScaler VPX™ - Second Edition
Sandbu Marius
No ratings yet
OpenShift Container Platform-4.17-Authentication and authorization-en-US
No ratings yet
OpenShift Container Platform-4.17-Authentication and authorization-en-US
224 pages
Emerging Cloud Software Environments
No ratings yet
Emerging Cloud Software Environments
19 pages
Murex Services Factsheet
100% (2)
Murex Services Factsheet
16 pages
Learning SaltStack - Second Edition
From Everand
Learning SaltStack - Second Edition
Colton Myers
No ratings yet
Red Hat 3scale API Management 2.5
No ratings yet
Red Hat 3scale API Management 2.5
38 pages
Arista Cavium 25G
No ratings yet
Arista Cavium 25G
8 pages
vWAF Deployment Guide
No ratings yet
vWAF Deployment Guide
90 pages
Comparing Openstack
No ratings yet
Comparing Openstack
13 pages
Cloud Computing Lab Experiments
No ratings yet
Cloud Computing Lab Experiments
2 pages
Flowmon Solution: Supported Technologies in A Nutshell
No ratings yet
Flowmon Solution: Supported Technologies in A Nutshell
3 pages
Ccie Dump PDF
No ratings yet
Ccie Dump PDF
3 pages
Cloud Computing Practical
No ratings yet
Cloud Computing Practical
32 pages
HP - Actualtests.HP2Z31.vmay19.by .Lagi .90q
No ratings yet
HP - Actualtests.HP2Z31.vmay19.by .Lagi .90q
84 pages
Virtualized Research Environments On Bwforcluster Nemo: Zki Arbeitskreis Supercomputing, 17.03.2017, Duisburg
No ratings yet
Virtualized Research Environments On Bwforcluster Nemo: Zki Arbeitskreis Supercomputing, 17.03.2017, Duisburg
18 pages
Veritas Netbackup™ For Openstack Administrator'S Guide: Unix, Windows, and Linux With Parallel Stream Framework
No ratings yet
Veritas Netbackup™ For Openstack Administrator'S Guide: Unix, Windows, and Linux With Parallel Stream Framework
44 pages
An Introduction To Openstack Networking
No ratings yet
An Introduction To Openstack Networking
21 pages
Suse-3125
No ratings yet
Suse-3125
1 page
JN0-214 Juniper Exam Practice Questions
No ratings yet
JN0-214 Juniper Exam Practice Questions
10 pages
Cloud Computing Notes
No ratings yet
Cloud Computing Notes
14 pages
Sicredi Case Study - 986069
No ratings yet
Sicredi Case Study - 986069
4 pages
CBIS Resources Not Working Properly / Dentry Cache (CBIS Release 19/19A)
No ratings yet
CBIS Resources Not Working Properly / Dentry Cache (CBIS Release 19/19A)
6 pages
Resume PDF
No ratings yet
Resume PDF
3 pages
VMware Horizon View Essentials
From Everand
VMware Horizon View Essentials
Peter von Oven
No ratings yet
Rachid Chaouch CV
No ratings yet
Rachid Chaouch CV
1 page
Infrastructure As A Code
No ratings yet
Infrastructure As A Code
28 pages
Kubernetes A Complete Guide
From Everand
Kubernetes A Complete Guide
Gerardus Blokdyk
No ratings yet

CiscoLive! OpenStack Lab LABCLD-2225

Uploaded by

CiscoLive! OpenStack Lab LABCLD-2225

Uploaded by

1. 2. 2015 CiscoLive!

For chrony you can use the command chronyc tracking to observe what clock source it's

For chrony you can use the command chronyc tracking to observe what clock source it's

Once this has been created you can use the command # keystone service‐list to get the

Click the +Create Network button again.

Once created you will select the gateway for this router. Click on the Set Gateway as you

And a screen similar to the following will be presented. Click on the + Add interface button.

Once there click on the  Launch Instance button. The following screen will popup and

2 Click on the Launch button

Click on the  Launch Instance button. The following screen will popup and you will need

2 Click on the Launch button

Log into the two separate instances. And from them issue the command ip a . This

Once there click on the  Launch Instance button. The following screen will popup and

2 Click on the Launch button

Click on the  Launch Instance button. The following screen will popup and you will need

2 Click on the Launch button

Log into the two separate instances. And from them issue the command ip a . This

Once created you will select the gateway for this router. Click on the Set Gateway as you

Click on the + Add interface button. The following screen will be presented. Select the

option that says Vlan391:10.3.91.0/24 and click the + Add interface button.

Click on the + Add interface button. The following screen will be presented. Select the

option that says Vlan392:10.3.92.0/24 and click the + Add interface button.

Once there click on the  Launch Instance button. The following screen will popup and

Click on the  Launch Instance button. The following screen will popup and you will need

2 Click on the Launch button

Log into the two separate instances. And from them issue the command ip a . This

OpenStack service Default ports Port type

Block Storage ( cinder ) 8776 publicurl and

Compute ( nova ) endpoints 8774 publicurl and

Compute API ( nova‐api ) 8773, 8775

Identity service ( keystone ) administrative 35357 adminurl

Identity service public endpoint 5000 publicurl

Image Service ( glance ) API 9292 publicurl and

Networking ( neutron ) 9696 publicurl and

Object Storage ( swift ) 6000, 6001, 6002

Orchestration ( heat ) endpoint 8004 publicurl and

Telemetry ( ceilometer ) 8777 publicurl and

Service Default port Used by

HTTP 80 OpenStack dashboard ( Horizon ) when it is

HTTP alternate 8080 OpenStack Object Storage ( swift ) service.

HTTPS 443 Any OpenStack service that is enabled for

rsync 873 OpenStack Object Storage. Required.

iSCSI target 3260 OpenStack Block Storage. Required.

MySQL database 3306 Most OpenStack components.

Message Broker 5672 OpenStack Block Storage, Networking,

You might also like