Study and Development of an OpenStack solution

A Degree Thesis
Submitted to the Faculty of the
Escola Tècnica d'Enginyeria de Telecomunicació de
Barcelona
Universitat Politècnica de Catalunya
by
Maria Jorba Brosa

In partial fulfilment
of the requirements for the degree in Telematics
ENGINEERING

Advisor: Jose Luis Muñoz Tapias

Barcelona, June 2014

Abstract
This thesis explores the possibilities that OpenStack Cloud Computing software offers.
OpenStack consists of several projects that provide resources to create an Infrastructure
Service cloud.
All the services and their architecture have been documented, giving the knowledge of
their use in the OpenStack environment.
During the project development an OpenStack deployment with its basic service to
provide an stable infrastructure. This basic infrastructure provides an scalable system
which is able to control every resource of the environment.
As a result of the thesis, the software deployment infrastructure has been documented
and an access management system based on policies has been configured.

1
Resum
La tesi explora les possibilitats que ofereix el software de Cloud Computing anomenat
OpenStack. OpenStack consisteix de diversos projectes que proveeixen recursos per
crear una infrastructura de service cloud.
Tots els serveis i la seva infrastructura ha estat documentada, donant la visió del seu ús
en l'ambient del OpenStack.
Durant el desenvolupament del projecte, s'ha desplegat el software OpenStack amb els
seus serveis bàsics per tal d'oferir una infrastructura estable. Aquesta infrastructura
ofereix un sistema escalable, el que permet controlar tots els recursos de l'ambient.
Com a resultat de la tesi, el desplegament de la infrastructura ha estat documentat i s'ha
configurat un sistema de control d'accessos basat en polítiques.

2
Resumen
La tesis explora las posibilidades que ofrece el software de Cloud Computing llamado
OpenStack. Openstack consiste en varios proyectos que proveen recursos para poder
crear una infraestructura de service cloud.
Todos los servicios y su infraestructura ha estado documentada, dando la visión sobre su
uso en el ambiente de OpenStack.
Durante el desarrollo del proyecto, se ha desplegado el software OpenStack con sus
servicios básicos para poder ofrecer una infraestructura estable. Esta infraestructura
ofrece un sistema escalable, lo que permite controlar todos los recursos del ambiente.
Como resultado de la tesis, el despliegue de la infraestructura ha estado documentada y
se ha configura un sistema de control de accesos basado en políticas.

3
Acknowledgements
I want to thank my thesis director, Jose Luis Muñoz, for introducing me to the cloud
computing technologies and guiding me through the process of this thesis.
I also want to thank my fellows who helped me through this time and my family who
supported all this time.

4
 Revision history and approval record

Revision Date Purpose

0 13/10/2015 Document creation

1 14/10/2015 Document revision 1

2 15/10/2015 Document revision 2

DOCUMENT DISTRIBUTION LIST

Name e-mail

Maria Jorba Brosa [email protected]

Jose Luis Muñoz Tapia [email protected]

Written by: Reviewed and approved by:

Date 13/10/2015 Date 15/10/2015

Name Maria Jorba Brosa Name Jose Luis Muñoz Tapias

Position Project Author Position Project Supervisor

5
Table of contents
Abstract ............................................................................................................................ 1
Resum .............................................................................................................................. 2
Resumen .......................................................................................................................... 3
Acknowledgements .......................................................................................................... 4
Revision history and approval record ................................................................................ 5
Table of contents .............................................................................................................. 6
1. Introduction................................................................................................................ 7
1.1. Objectives .......................................................................................................... 7
1.2. Requirements and specifications ........................................................................ 7
1.3. Resources .......................................................................................................... 8
1.4. Work plan ........................................................................................................... 8
1.5. Project deviations ............................................................................................. 11
2. State of the art of the technology used or applied in this thesis:............................... 12
2.1. OpenStack ....................................................................................................... 12
2.2. Basic services .................................................................................................. 12
2.3. Policy based authorization ................................................................................ 12
3. Methodology / project development: ........................................................................ 13
4. Results .................................................................................................................... 14
5. Budget ..................................................................................................................... 15
6. Conclusions and future development: ...................................................................... 16
6.1. Conclusions ...................................................................................................... 16
6.2. Future developments ........................................................................................ 16
Bibliography: ................................................................................................................... 17
Annexes ......................................................................................................................... 18

6
1. Introduction
The goal of this document is to provide an executive summary of the project. The
documentation of the technologies used during the project and the performed
configurations are explained in detail at the annex of this memory.

1.1. Objectives
The purpose of this project is to study and deploy an OpenStack environment in order to
explore the capabilities that provide an infrastructure service cloud. Each OpenStack
service uses different technologies, such as QEMU/KVM hypervisors for the compute
service or OpenvSwitch for implementing the networking.
During the course of the project, there has been a focus on the Identity service which
provides the authentication to the environment and to services but it also has been
studied and used the policy based authorisation system of each service.
Every service provided by OpenStack and the documentation of the deployment and
authentication/authorisation configuration has been detailed in the annex.
The following features have been deployed configured in this project:
 Deployment and configuration of Identity service (Keystone), Compute service
(Nova) and Network service (Neutron)
 Configuration of the Identity service profiles
 Policy based access authorisation for Identity and Network services.

1.2. Requirements and specifications

The requirements of the project are based on conceptual and technical requirements.
The conceptual requirements are those needed to understand how the OpenStack
system works and the technical are those needed to install and configure them.
Technical specifications:
 Identity access management.
 Use of OpenStack configuration files.
 Use of OpenvSwitch.
 Use of QEMU/KVM virtualization.
 Use of the LaTeX editor to document.

Conceptual specifications:
 Bridging.
 AMQP protocol.
 Linux kernel-based OS knowledge.
 OpenStack services.

7
 1.3. Resources

The OpenStack software and all its services, used through the entire project, was
developed by Openstack and was open sourced under and apache license.
The basic configuration implemented is based on the suggested deployment proposed by
the OpenStack organization, which can be found in its web page referenced in [15].

1.4. Work plan

Work Packages:

Project: Study and development of an OpenStack solution WP ref: 1

Major constituent: Research of OpenStack services Sheet n of m

Short description: Planned start date: 09-03-2015

In order to understand the whole concept of cloud computing and Planned end date: 31-03-2015
the OpenStack cloud computing software and its services.
Start event:
End event:

Internal task T1: Investigate about cloud computing Deliverables: Dates:

Internal task T2: Research of OpenStack services

Project: Study and development of an OpenStack solution WP ref: 2

Major constituent: Research of OpenStack deployment Sheet n of m

Short description: Planned start date: 31-03-2015

Research of how OpenStack services can be installed in an Planned end date: 27-04-2015
enviroment and the basic needed configuration.
Start event:
End event:

Internal task T1: Installation and configuration research of Deliverables: Dates:

Identity service.
Internal task T2: Installation and configuration research of
Compute service
Internal task T3: Installation and configuration research of
Network service

8
Project: Study and development of an OpenStack solution WP ref: 3

Major constituent: Keystone deployment Sheet n of m

Short description: Planned start date: 28-04-2015

Deployment of the Identity service (Keystone) Planned end date: 30-05-2015

Start event:
End event:

Deliverables: Dates:

Project: Study and development of an OpenStack solution WP ref: 4

Major constituent: Nova deployment Sheet n of m

Short description: Planned start date: 01-06-2015

Deployment of the Compute service (Nova) Planned end date: 30-06-2015

Start event:
End event:

Deliverables: Dates:

Project: Study and development of an OpenStack solution WP ref: 5

Major constituent: Neutron deployment Sheet n of m

Short description: Planned start date: 01-07-2015

Deployment of the Network service (Neutron) Planned end date: 14-08-2015

Start event:
End event:

Deliverables: Dates:

9
Project: Study and development of an OpenStack solution WP ref: 6

Major constituent: Documentation Sheet n of m

Short description: Planned start date: 15-08-2015

Documentation of the deployment and the issues that arised Planned end date: 25-09-2015
during the process.
Start event:
End event:

Deliverables: Dates:

Project: Study and development of an OpenStack solution WP ref: 7

Major constituent: Access management configuration Sheet n of m

Short description: Planned start date: 26-09-2015

Configure the users for controlling the access to the system and Planned end date: 09-10-2015
configure the policy based authorization of Neutron and
Keystone service. Start event:
End event:

Deliverables: Dates:

Milestones
WP# Task# Short title Milestone / deliverable Date (week)
3 - Keystone configuration 10
4 - Nova configuration 14
5 - Neutron configuration 20
7 - Access management 28
configuration

10
Time plan

1.5. Project deviations

At the beginning of the project the access management wasn't an objective of the project
and there was the objective of deploying OpenStack using different methods but during
the process these options were discarded.

11
2. State of the art of the technology used or applied in this
thesis:

2.1. OpenStack
OpenStack is an open source cloud operating system that is comprised of several open
source sub-projects or services which control large pools of compute, storage, and
networking resources throughout a datacenter, all managed through a dashboard that
gives administrators control while empowering their users to provision resources through
a web interface. These services are provided in order to create an Infrastructure Service
cloud.
All OpenStack services expose a RESTful API for communication among them and use
HTTP protocol for each data exchange. In addition, by using this type of API, scalability
and fault-tolerance is provided to the system.
However, the real communication is provided by workers, which are separate processes
dedicated for certain atoms execution, possibly running on other machines, connected via
AMQP and using a message bus which is RabbitMQ by default.

2.2. Basic services

An OpenStack environment can consist of many services, however, its basic services are
the Identity service (Keystone), the Compute service (Nova), the Network service
(Neutron) and the Image service (Glance).
 Keystone: Keystone service provides a common authentication and authorisation
store for the other OpenStack services.
 Nova: Nova is responsible for hosting and managing cloud computing systems,
which is the major part of an Infrastructure-as-a-Service (IaaS) system.
 Neutron: Neutron provides the networking service between interface devices
managed by other Openstack services.
 Glance: Glance provides a service where users can upload and discover data
assets that are meant to be used with other services. This currently includes
images and metadata definitions.

2.3. Policy based authorization

Each OpenStack service has its own role-based access policies. They determine which
user can access which objects in which way, and are defined in the service's policy.json
file stored in /etc/name_of_the_service/.
Whenever an API call to an OpenStack service is made, the service's policy engine uses
the appropriate policy definitions to determine if the call can be accepted. Any changes to
policy.json are effective immediately, which allows new policies to be implemented while
the service is running.

12
3. Methodology / project development:
One of the most main objectives of the project has been the deployment and
configuration of OpenStack. To do so, it has been used the following method:
1. Download the module packages
2. Configure the service
3. Verify the proper service operation
4. Document the found issues

This process has been followed when deploying each service. Every step made in any of
the previous steps has also been documented in detail, as it can be seen in the annex.

The other main objective of the project, has been to configure an access management
system using the Identity service. To do so, it has been followed this method:
1. Create users
2. Create user's tenants
3. Create roles
4. Attach users to a tenant and role
5. Configure the policy file using the created roles
6. Verify the proper behaviour
7. If the verification is not successful, return to step 5 to reconfigure the policy file.

13
4. Results
As a result of the project, OpenStack capabilities have been analysed and it has been
proved that it has a great potential for creating an Infrastructure service cloud. At the end
of the project it has been installed the Identity, Compute, Network and Image service
providing a basic virtual environment.
The results achieved can be summarized in the following lines:

 It has been possible to create instances (Chapter 11 of the annex) and connect to
them from the web browser, which provides the capacity of managing instances
remotely.
 It has been configured the desired network infrastructure, which connects all the
instances of a single tenant (Chapter 8 and chapter 10 of the annex). The network
infrastructure can be modified at any moment if needed, which provides a big
capacity of networking management.
 In terms of security, it can be concluded that the default configuration doesn't
supply the access security that a production environment should have do to the
fact that there is just one user with administration privileges.
 It has been configured an access system which divides the power of the
administration used in several users which can only administrate a single service.
This type of configuration provides security to the system and makes the
operations traceable (Chapter 12 of the annex).

14
5. Budget
OpenStack used is an open-source software and licensed under the Apache license so it
hadn't had any cost to the project.

The platform used to develop the environment, meaning the Operative System, is the
Linuxmint distribution which is a GNU/Linux distribution so it doesn't had any cost.

Finally, the estimated hours dedicated to the project are 450h. The cost of an engineer
without experience is 9.5€/h. This represents a final cost of 4275€ for the project.

15
6. Conclusions and future development:

6.1. Conclusions
The OpenStack system has proved that it is able to provide an Infrastructure service
cloud with great scalability and manageability. It is able to provide all the services needed
to create any type of production environment to several clients.
However, for the moment OpenStack may not provide all the same usability as it could
provide a configuration created directly with the same tools. This is caused by the fact
that the drivers used to implement technologies over OpenStack don’t offer the same
capabilities.

6.2. Future developments

As a future development for the OpenStack environment created in this thesis, could be
the deployment of the rest of the available services. However, to do so, it would be
needed a bigger infrastructure with bigger computational resources.
Another approach to a future development is the generation of an orchestration system
powered by the Orchestration service (Heat) of OpenStack. This orchestration system
could be used to deploy other OpenStack environments or any other type.

16
Bibliography:
[1] Amqp and rabbitmq. http://ilearnstack.com/2013/04/16/ introduction-to-amqp-
messaging-with-rabbitmq/.
[2] Amqp protocol. https://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol.
[3] Ceilometer service documentation. http://docs.openstack.org/developer/ceilometer/
architecture.html.
[4] Designate service documentation. http://docs.openstack.org/developer/designate/
architecture.html.
[5] Glance service documentation. html. http://docs.openstack.org/developer/glance/
architecture.
[6] Hadoop documentation. https://en.wikipedia.org/wiki/Apache\_Hadoop.
[7] Heat service documentation. http://docs.openstack.org/developer/heat/architecture.
html#heat-services.
[8] Horizon service documentation.http://docs.openstack.org/developer/horizon/intro.html.
[9] Ironic service documentation. http://docs.openstack.org/developer/ironic/deploy/user-
guide.html.
[10] Keystone service documentation. http://es.slideshare.net/openstackindia/ openstack-
keystone-identity-service?related=1.
[11] Magnum service documentation. http://docs.openstack.org/developer/magnum/.
[12] Manila service documentation. http://docs.openstack.org/developer/manila/adminref/
intro.html.
[13] Neutron service documentation. http://docs.openstack.org/admin-guide-cloud/
networking_arch.html.
[14] Nova service documentation. ttp://docs.openstack.org/developer/nova/project_scope.
html.
[15] Openstack basic architecture. http://docs.openstack.org/juno/install-guide/install/
apt/content/ch_basic_environment.html.
[16] Openstack messaging. http://ilearnstack.com/2013/04/24/ messaging-in-openstack-
using-rabbitmq/.
[17] Policy documentation. http://docs.openstack.org/kilo/config-reference/content/ policy-
json-file.html.

17
Annexes

18
 Maria Jorba Brosa

UPC Telematics Department

Study and Development of an OpenStack solution

Jose Luis Muñoz Tapias

Contents

1 Cloud Computing 9
1.1 Cloud Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2 OpenStack technologies 11
2.1 Haddop clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.1.2 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.1.3 File systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.1.4 JobTracker and TaskTracker: the MapReduce engine . . . . . . . . . . . . . . . . . . . . . . 13

3 OpenStack 15
3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.1.2 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.1.3 AMQP protocol and RabbitMQ implementation . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.2 OpenStack messaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.2.1 RabbitMQ message broker node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.2.2 OpenStack RPC modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.3 Keystone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.3.1 Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.3.2 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.4 Nova . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.4.1 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.4.2 Nova resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.4.3 Filter scheduler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.4.4 Conductor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.4.5 Virtual Machines states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.5 Neutron . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.5.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
 3.5.2 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.5.3 Network modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.5.4 Network namespaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.5.5 External and tenant networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.6 Glance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.6.1 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.6.2 Disk and container formats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.7 Swift . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.7.1 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.8 Cinder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.8.1 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.9 Heat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.10 Trove . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.11 Ironic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.11.1 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.11.2 Key technologies for Ironic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.12 Sahara . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.12.1 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.12.2 Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.13 Zaqar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.13.1 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.14 Designate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.14.1 Designate API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.14.2 Designate Central . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.14.3 Designate MiniDNS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.14.4 Designate Pool Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.14.5 Designate Sink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.14.6 DNS Backend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.14.7 Database/Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.15 Manila . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.16 Magnum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.17 Horizon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.18 Ceilometer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.18.1 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.18.2 Data gathering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.18.3 Data processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.18.4 Data storage - Collector service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
 3.18.5 Data access - API service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.18.6 Data evaluation - Alarming service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.19 Policiy files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.19.1 Policy syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.19.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

4 Project deployment pre-configuration 53

4.1 Infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.1.1 Virtual Machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.1.2 Virtual Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.2 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.2.1 Controller node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.2.2 Compute node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.2.3 Network node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.3 Networking configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.3.1 Controller node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.3.2 Network node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.3.3 Compute node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.3.4 Name resolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
4.3.5 Connectivity verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
4.4 Network Time Protocol (NTP) configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.4.1 NTP service verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
4.5 OpenStack repository . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.6 SQL Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.7 Messaging server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4.7.1 Messaging server configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

5 Identity service: Keystone 61

5.1 Identity database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
5.2 Service installation and configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
5.2.1 Required modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
5.2.2 Service configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
5.3 Tenants, Users and Roles creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
5.3.1 Environment variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
5.3.2 Administrative profile creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
5.3.3 UPC profile creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
5.3.4 Service tenant creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
5.4 Service entity and API Endpoint creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
 5.4.1 Service entity’s catalog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
5.4.2 API Endpoint’s catalog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
5.5 Verify operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
5.5.1 Admin user verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
5.5.2 Demo user verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
5.6 Client environment scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
5.6.1 admin-openrc.sh file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
5.6.2 demo-openrc.sh file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

6 Image service: Glance 69

6.1 Image database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
6.2 Image service configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
6.2.1 Credentials creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
6.2.2 Image service entity and API Endpoint creation . . . . . . . . . . . . . . . . . . . . . . . . . 70
6.3 Service installation and configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
6.3.1 Required modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
6.3.2 Service configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
6.4 Verify operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

7 Compute service: Nova 73

7.1 Compute database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
7.2 Nova service configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
7.2.1 Credentials creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
7.2.2 Nova service entity and API Endpoint creation . . . . . . . . . . . . . . . . . . . . . . . . . 74
7.3 Service installation and configuration for Controller node . . . . . . . . . . . . . . . . . . . . . . . . 74
7.3.1 Required modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
7.3.2 Service configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
7.4 Service installation and configuration for Compute node . . . . . . . . . . . . . . . . . . . . . . . . 76
7.4.1 Required modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
7.4.2 Service configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
7.5 Verify operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

8 Networking component: Neutron 79

8.1 Neutron database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
8.2 Neutron service configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
8.2.1 Credentials creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
8.2.2 Neutron service entity and API Endpoint creation . . . . . . . . . . . . . . . . . . . . . . . . 80
8.3 Service installation and configuration for Controller node . . . . . . . . . . . . . . . . . . . . . . . . 80
 8.3.1 Required modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
8.3.2 Service configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
8.3.3 Networking Compute configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
8.3.4 Finish the configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
8.3.5 Verify operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
8.4 Service installation and configuration for Network node . . . . . . . . . . . . . . . . . . . . . . . . . 83
8.4.1 Prerequisites configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
8.4.2 Required modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
8.4.3 Service configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
8.4.4 Metadata agent Controller configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
8.4.5 Open vSwitch service configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
8.4.6 Finish the configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
8.4.7 Verify operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
8.5 Service installation and configuration for Compute node . . . . . . . . . . . . . . . . . . . . . . . . 88
8.5.1 Prerequisites configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
8.5.2 Required modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
8.5.3 Service configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
8.5.4 Finish the configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
8.5.5 Verify operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

9 Dashboard: Horizon 93
9.1 Dashboard Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
9.1.1 Required modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
9.1.2 Service configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
9.1.3 Finish the configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
9.1.4 Verify operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

10 Network configuration 95
10.1 Network configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
10.2 External network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
10.3 Tenant network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
10.4 Tenant router . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
10.5 Final virtual network infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

11 Instance creation 99
11.1 Instance launching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
11.1.1 Previous requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
11.1.2 Instance creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
12 Policy configuration 103
12.1 Identity configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
12.1.1 Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
12.1.2 Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
12.2 Policy configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

13 Troubleshooting 105
13.1 Keystone troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
13.2 Nova troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
13.3 Neutron troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
13.3.1 Rabbit module error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
13.3.2 Controller node unreachable error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
13.4 Instance creation troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
13.4.1 Low RAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
13.4.2 RPC version not supported . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

8
Chapter 1

Cloud Computing

1.1 Cloud Computing

The technology named as cloud computing is a model for enabling obiquitous, convenient, on-demand network ac-
cessto a shared poolof configurable computing resources that can be rapidly provisioned and released with minimal
management effort or service provider interaction. This resources can be of any kind such as servers, node, services,
storage or applications.
This tipe of cloud is composed by five essential characteristics [23]:

• On-demand Self-service: A consumer can uniterally provision computing capabilities automatically without
requiring human interaction with each service provider.

• Broad network access: Capabilities are available over the network and accessed through standard mechanisms
that promoteuse by heterogeneous client platforms.

• Resource pooling: The provider’s computing resources are pooled to serve multiple consumers as a multi-
tenant model, each of them with different physical and virtual resources dynamically assigned and reassigned
according to the consumer demand. This characteristic implies that the consumer generally has no controll over
the exact location of the provided resources but may be able to specify location at a high level of abstraction
such as country, state or datacenter.

• Rapid elacticity: Capabilities can be elastically provisioned and released, in some cases automatically, to scale
rapidly outward and inward commensurate with demand. To the consumer the capabilities available for provi-
sioning often appear to be unlimited and can be at any appropiated in any quantity at any time.

• Measured Service: Cloud systems automatically control and optimize resource use by leveraging a metering
capability at some level of abstraction appropiate to the type of service.

The cloud computing can also be provided to the consumer using three service models:

• Software as a Service (SaaS): This model provides to the consumer the capability to use the provider’s appli-
cations running on a cloud infrastructure. The applications are accessible from various client devices through a
client interface. The consumer doesn’t manage or control the underlying cloud infrastructure, with the possible
exception of limited user-specific application configuration settings.

• Platform as a Service (PaaS): This model provides to the consumer the capability to deploy onto the cloud infras-
tructure consumer-related or acquired applications created using programming languages, libraries, services and

9
 tools supported by the provider. The consumer doesn’t manage or control the underlying cloud infrastructure
but it has control over the deployed applications and possibly configuration settings for the application-hosting
environment.
• Infrastructure as a Service (IaaS): This model provides to the consumer the capability to provision processing,
storage, networks and other fundamental computing resources where the consumer is able to deploy and run
arbitrary software, which can include operating systems and applications. The consumer doesn’t manage or
control the underlying cloud infrastructure but has control over operating systems, storage and deployed appli-
cations and possibly limited control of select networking components.

Apart from the service models, the cloud computing can also be deployed following several models:

• Private cloud: The cloud infrastructure is provisioned for exclusive use bya single organization comprising mul-
tiple consumers. It may be owned, managed and operated by the organization, a third party, or some combination
of them, and it may exist on or off premises.
• Community cloud: The cloud infrastructure is provisioned for exclusive use by a specific community of con-
sumers from organizations that have shared concerns. It may be owned, managed and operated by one or more
of the organizations in the organizations in the community, a third party, or some combination of them, and it
may exist on or off premises.

• Public cloud: The cloud infrastructure is provisioned for open use by the general public. It may be owned,
managed and operated by a bussiness, academic or goverment organization or some combination of them. It
exists on the premises of the cloud provider.
• Hybrid cloud: The cloud infrastructure is a composition of two or more distinct cloud infrastructure that remain
unique entities, but are bound together by standardized or propietary technology that enables data and application
portability.

10
Chapter 2

OpenStack technologies

2.1 Haddop clusters

2.1.1 Overview

Apache Hadoop [6]is an open-source software framework written in Java for distributed storage and distributed pro-
cessing of very large data sets on computer clusters built from commodity hardware. All the modules in Hadoop are
designed with a fundamental assumption that hardware failures are commonplace and thus should be automatically
handled in software by the framework.
The core of Apache Hadoop consists of a storage part (Hadoop Distributed File System (HDFS)) and a processing part
(MapReduce). Hadoop splits files into large blocks and distributes them amongs the nodes in the cluster. To process
the data, Hadoop MapReduce transfers packaged code for nodes to process in parallel, based on the data each node
needs to process. This approach takes advantage of data locality (nodes manipulating the data that they have on hand)
to allow the data to be processed faster and more efficiently than it would be in a more conventional supercomputer
architecture that relies on a parallel file system where computation and data are connected via high-speed networking.
The base Apache Hadoop framework is composed of the following modules:

• Hadoop Common: Contains libraries and utilities needed by other Hadoop modules.

• Hadoop Distributed File System (HDFS): A distributed file-system that stores data on commodity machines,
providing very high aggregate bandwidth across the cluster.

• Hadoop YARN: A resource-management platform responsible for managing computing resources in clusters
and using them for scheduling of users’ applications.

• Hadoop MapReduce: A programming model for large scale data processing.

2.1.2 Architecture

Hadoop consists of the Hadoop Common package, which provides filesystem and OS level abstractions, a MapReduce
engine and the Hadoop Distributed File System (HDFS) as it can be seen in figure 2.1.
For effective scheduling of work, every Hadoop-compatible file system should provide location awareness: the name
of the rack where a worker node is. Hadoop applications can use this information to run work on the node where the
data is reducing backbone traffic. HDFS uses this method when replicating data to try to keep different copies of the

11
 Figure 2.1: Hadoop architecture

data on different racks. The goal is to reduce the impact of a rack power outage or switch failure, so that even if these
events occur, the data may still be readable.
A Hadoop cluster includes a single master and multiple worker nodes. The master node consists of a JobTracker,
TaskTracker, NameNode, and DataNode. A slave or worker node acts as both a DataNode and TaskTracker, though it
is possible to have data-only worker nodes and compute-only worker nodes.
Hadoop requires Java Runtime Environment (JRE) 1.6 or higher. The standard startup and shutdown scripts require
that Secure Shell (ssh) be set up between nodes in the cluster.
In largest clusters, the HDFS is managed through a dedicated NameNode server to host the file system index, and a
secondary NameNode that can generate snapshots of the namenode’s memory structures, thus preventing file-system
corruption and reducing loss of data. Similarly, a standalone JobTracker server can manage job scheduling. In clus-
ters where the Hadoop MapReduce engine is deployed against an alternate file system, the NameNode, secondary
NameNode, and DataNode architecture of HDFS are replaced by the file-system-specific equivalents.

2.1.3 File systems

The Hadoop distributed file system (HDFS) is a distributed, scalable, and portable file-system written in Java for
the Hadoop framework. A Hadoop cluster has nominally a single namenode plus a cluster of datanodes, although
redundancy options are available for the namenode due to its criticality. Each datanode serves up blocks of data over
the network using a block protocol specific to HDFS. The file system uses TCP/IP sockets for communication and
clients use remote procedure call (RPC) to communicate between each other.
HDFS stores large files across multiple machines which achieves reliability by replicating the data across multiple

12
hosts. With the default replication value, which is 3, data is stored on three nodes: two on the same rack, and one on a
different rack. Data nodes can talk to each other to rebalance data, to move copies around, and to keep the replication
of data high.
The HDFS file system includes a so-called secondary namenode, a misleading name that some might incorrectly in-
terpret as a backup namenode for when the primary namenode goes offline. In fact, the secondary namenode regularly
connects with the primary namenode and builds snapshots of the primary namenode’s directory information, which
the system then saves to local or remote directories. These checkpointed images can be used to restart a failed primary
namenode without having to replay the entire journal of file-system actions, then to edit the log to create an up-to-date
directory structure.

2.1.4 JobTracker and TaskTracker: the MapReduce engine

Above the file systems comes the MapReduce Engine, which consists of one JobTracker, to which client applications
submit MapReduce jobs. The JobTracker pushes work out to available TaskTracker nodes in the cluster, striving to
keep the work as close to the data as possible. With a rack-aware file system, the JobTracker knows which node
contains the data, and which other machines are nearby. If the work cannot be hosted on the actual node where the
data resides, priority is given to nodes in the same rack. This reduces network traffic on the main backbone network. If
a TaskTracker fails or times out, that part of the job is rescheduled. The TaskTracker on each node spawns a separate
Java Virtual Machine process to prevent the TaskTracker itself from failing if the running job crashes its JVM. A
heartbeat is sent from the TaskTracker to the JobTracker every few minutes to check its status. The Job Tracker and
TaskTracker status and information is exposed by Jetty and can be viewed from a web browser.
The known limitations of this approach are:

• The allocation of work to TaskTrackers is very simple. Every TaskTracker has a number of available slots (such
as "4 slots"). Every active map or reduce task takes up one slot. The Job Tracker allocates work to the tracker
nearest to the data with an available slot. There is no consideration of the current system load of the allocated
machine, and hence its actual availability.
• If one TaskTracker is very slow, it can delay the entire MapReduce job—especially towards the end of a job,
where everything can end up waiting for the slowest task. With speculative execution enabled, however, a single
task can be executed on multiple slave nodes.

Scheduling

By default Hadoop uses FIFO scheduling, and optionally 5 scheduling priorities to schedule jobs from a work queue.
Fair scheduler
The goal of the fair scheduler is to provide fast response times for small jobs and QoS for production jobs. The fair
scheduler has three basic concepts.

1. Jobs are grouped into pools.

2. Each pool is assigned a guaranteed minimum share.
3. Excess capacity is split between jobs.

By default, jobs that are uncategorized go into a default pool. Pools have to specify the minimum number of map slots,
reduce slots, and a limit on the number of running jobs.
Capacity scheduler
The capacity scheduler supports several features that are similar to the fair scheduler.

13
1. Queues are allocated a fraction of the total resource capacity.
2. Free resources are allocated to queues beyond their total capacity.
3. Within a queue a job with a high level of priority has access to the queue’s resources.

14
Chapter 3

OpenStack

3.1 Overview

3.1.1 Introduction
OpenStack is an open source cloud operating system that is comprised of several open source sub-projects or services
which control large pools of compute, storage, and networking resources throughout a datacenter, all managed through
a dashboard that gives administrators control while empowering their users to provision resources through a web
interface. These services are provided in order to create an Infrastructure Service cloud.
All OpenStack services expose a RESTful API for communication among them and use HTTP protocol for each data
exchange. In addition, by using this type of API, scalability and fault-tolerance is provided to the system.
However, the real communication is provided by workers, which are separate processes dedicated for certain atoms
execution, possibly running on other machines, connected via AMQP and using a message bus which is RabbitMQ by
default.
Regarding the service data storage, each OpenStack service has its own SQL based database for storing state informa-
tion, but in order to provide resilience and throughput it is possible to implement a replicated multi-master databases.
In order to provide all the resources needed to create an Infrastructure service cloud, there are available the following
services in OpenStack system:

• Keystone: Identity service

• Nova: Compute service
• Neutron: Networking service
• Glance: Image service
• Swift: Object Storage service
• Cinder: Block Storage service
• Heat: Orchestration service
• Trove: Database service
• Ironic: Bare Metal service
• Sahara: Data processing service

15
 • Zaqar: Message service

• Barbican: Key management service

• Designate: DNS service

• Manila: Shared Filesystems service

• Magnum: Containers service

• Murano: Application catalog

• Congress: Governance service

• Mistral: Workflow service

• MagnetoDB: Key-value store as a service

• Horizon: Dashboard

• Ceilometer: Telemetry service

This list present all the available services but to provide a basic production enviroment Keystone, Nova, Neutron,
Heat, Glance, Cinder, Horizon and Ceilometer services are the ones needed. The figure 3.1 provides the conceptual
architecture of a typical OpenStack environment.

3.1.2 Architecture

OpenStack system follows a simple process when a user is trying to access to a service:

1. The user who is trying to perform an action to a service send its credentials to Keystone

2. Keystone sends to the user a temporary token and a list of tenants if the previous credentials authentication
successes

3. The user sends the credentials to Keystone together with the desired tenant

4. If the authentication of the credentials and the desired tenant is correct, Keystone sends a tenant token and a list
of available service for this tenant.

5. The user determines the correct endpoint depending on the action that is desired to be performed and makes the
request to the endpoint along with the tenant token acquired for authentication.

6. To avoid connections from non authorized entities, the service request Keystone whether the received token is
correct and is allowed to use the service.

7. If keystone authenticates the token and the right to use the service, this checks to its own policy if the user is
allowed to make the request.

8. If the policy validation is correct the request will be executed.

In the figure 3.2 it can bee seen the described process which will be followed each time a user wants to make a request
granting a high level of security [15].

16
 Figure 3.1: OpenStack services

3.1.3 AMQP protocol and RabbitMQ implementation

AMQP protocol

The Advanced Message Queuing Protocol (AMQP) [2] is an open standard for passing business messages between
applications or organizations. It connects systems, feeds business processes with the information they need and reliably
transmits onward the instructions that achieve their goals.
AMQP is a binary, application layer protocol, designed to efficiently support a wide variety of messaging applica-
tions and communication patterns. It provides flow controlled,message-oriented communication with the following
message-delivery guarantees:

• at-most-once: Each message is delivered once or never.

• at-least-once: Each message is certain to be delivered, but may do so multiple times.
• exactly-once: The message will always certainly arrive and do so only once.

The authentication and encryption is based on SASL and TLS and it assumes an underlying reliable transport layer
protocol such as Transmission Control Protocol (TCP).

17
 Figure 3.2: OpenStack architecture

The AMQP specification is defined in several layers:

• Type system: It is used to define a message format allowing standard and extended meta-data to be expressed
and understood by processing entities. It is also used to define the communication primitives through which
messages are exchanged between such entities.

• Link protocol: The basic unit of data in AMQP is a frame. There are nine AMQP frame bodies defined that are
used to initiate, control and tear down the transfer of messages between two peers:

– open: It is used to initiate connections, which are multiplexed sessions.

– begin: It is used to initiate sessions, which are multiple links grouped together.
– attach: It is sent to initiate a new unidirectional link.
– transfer: It is used to sent a message.
– flow: It is used to manage the flow control scheme.
– disposition: It is used for state changes of a transfer.
– detach: It is sent to tear down a link.
– end: It is used to terminate sessions.
– close: t is used to terminate connections.

• Message format: The message format is defined by two blocks. The first one is the bare message which is the
part of the message that is created by the sending application and it is immutable as the message is transferred
between one or more processes. The second one is the header which is a standard set of delivery related
annotations that can be requested or indicated for a message and includes time to live, durability, priority.

18
 • Message capabilities: The AMQP assumes that the peer nodes have some common behaviours:

– Some standard outcomes for transfers.

– A mechanism for indicating or requesting one of the two basic distribution patterns, competing and non-
competing consumers, through the distribution modes.
– The ability to create nodes on-demand.
– The ability to refine the set of message of interest to a receiver through filters.

RabbitMQ message broker

The default implementation for AMQP protocol in OpenStack system is the RabbitMQ broker message queue[1].
RabbitMQ is a messaging broker that gives to applications a common platform to send and receive messages, and
to messages a safe place to live until received. Applications can connect to each other, as components of a larger
application, or to user devices and data and the messaging is asynchronous, decoupling applications by separating
sending and receiving data.
In the OpenStack system, RabbitMQ platform uses the protocol AMQP to perform the messaging and has the following
features that increases the messaging capabilities:

• Reliability: RabbitMQ offers a variety of features to let you trade off performance with reliability, including
persistence, delivery acknowledgements, publisher confirms, and high availability.

• Flexible Routing: RabbitMQ features several built-in exchange types for typical routing logic.

• Clustering: Several RabbitMQ servers on a local network can be clustered together, forming a single logical
broker.

• Highly Available Queues: Queues can be mirrored across several machines in a cluster, ensuring that even in
the event of hardware failure the messages are safe.

• Many Clients: There are RabbitMQ clients for almost any programming language.

• Tracing: RabbitMQ offers tracing support to find out what’s going on.

• Plugin System: RabbitMQ ships with a variety of plugins extending it in different ways.

RabbitMQ message broker deals with publishers and consumers and has to ensure that the messages from a publisher
go to the right consumers. In order to do that, the broker uses two key components: exchanges and queues. The figure
3.3 shows how they connect a publisher to a consumer.

Figure 3.3: RabbitMQ architecture

19
A queue is the name for a mailbox of messages an it lives inside RabbitMQ. Although messages flow through message
broker and applications, they can be stored only inside a queue which is an infinite buffer so it is not bound by any
limits and it can store as many messages as needed. Many publishers can send messages that go to one queue, many
consumers can try to receive data from one queue.
The core idea in the messaging model in RabbitMQ is that the producer never sends any messages directly to a queue.
Actually, quite often the publisher does not even know if a message will be delivered to any queue at all.
Instead, the publisher can only send messages to an exchange. Exchanges then distribute message copies to queues
which are binded to them. The rules for determining what the exchange will do with the message are defined by the
exchange type:

• Direct Exchange: Direct exchange, showed in the figure 3.4, delivers messages to queues based on the message
routing key. A direct exchange is ideal for the unicast routing of messages.

Figure 3.4: Direct exchange

• Fanout Exchange: A fanout exchange, showed in the figure 3.5, routes messages to all of the queues that are
bound to it and the routing key is ignored. If N queues are bound to a fanout exchange, when a new message is
published to that exchange a copy of the message is delivered to all N queues. Fanout exchanges are ideal for
the broadcast routing of messages.

Figure 3.5: Fanout exchange

• Topic Exchange: Topic exchanges, showed in figure 3.6, route messages to one or many queues based on
matching between a message routing key and the pattern that was used to bind a queue to an exchange. The
topic exchange type is often used to implement various publish/subscribe pattern variations. Topic exchanges
are commonly used for the multicast routing of messages.
• Headers Exchange: A headers exchange is designed to for routing on multiple attributes that are more easily
expressed as message headers than a routing key. Headers exchanges ignore the routing key attribute. Instead,
the attributes used for routing are taken from the headers attribute. A message is considered matching if the
value of the header equals the value specified upon binding.

20
 Figure 3.6: Topic exchange

3.2 OpenStack messaging

AMQP is the messaging technology chosen by OpenStack [16]system whose services communicates internally via
AMQP and through each other using REST-call.
The AMQP broker,RabbitMQ , sits between any two internal Openstack services and allows them to communicate to
one another using Remote Procedure Calls (RPC) and using direct, fanout, and topic-based message broker exchanges.

3.2.1 RabbitMQ message broker node

Among OpenStack components some are invokers, which send messages in the queuing system via two operations
(rpc.call and rpc.cast) and some other are workers, which receive messages from the queuing system and reply accord-
ingly to rpc.call operations.
The following are the elements of the RabbitMQ message broker node :

• Topic Publisher: Deals with an rpc.call or an rpc.cast operation and used to push a message to the queuing
system. Every publisher connects always to the same topic-based exchange whose life-cycle is limited to the
message delivery.

• Direct Consumer: Deals with only rpc.call operation used to receive a response message from the queuing
system. Every consumer connects to a unique direct-based exchange via a unique exclusive queue whose life-
cycle is limited to the message delivery.

• Topic Consumer: It is activated when a Worker is instantiated and exists throughout its life-cycle,this is used
to receive messages from the queue and it invokes the appropriate action as defined by the Worker role. A Topic
Consumer connects to the same topic-based exchange either via a shared queue or via a unique exclusive queue.
Every Worker has two topic consumers, one that is addressed only during rpc.cast operations and the other that
is addressed only during rpc.call operations.

• Direct Publisher: It comes to life only during rpc.call operations and it is instantiated to return the message
required by the request/response operation. The object connects to a direct-based exchange whose identity is
dictated by the incoming message.

3.2.2 OpenStack RPC modes

RPC Openstack Messaging uses two modes:

• rpc.cast: The messaging don’t wait for result.

21
 • rpc.call: The messaging wait for result when there is something to return.

Some OpenStack services such as Nova and Cinder provides an adapter class which take cares of marshaling and
unmarshaling of messages into function calls. Each service components create two queues at the initialization time,
one which accepts messages with routing keys ‘NODE-TYPE.NODE-ID’ and another, which accepts messages with
routing keys as generic ‘NODE-TYPE’.

RPC calls

The rpc.call operation, showed in figure 3.7, performs the following fases to be completed:

1. a Topic Publisher is instantiated to send the message request to the queuing system. Immediately before the
publishing operation, a Direct Consumer is instantiated to wait for the response message.
2. Once the message is dispatched by the exchange, it is fetched by the Topic Consumer dictated by the routing
key (such as ‘topic.host’) and passed to the Worker in charge of the task.
3. Once the task is completed, a Direct Publisher is allocated to send the response message to the queuing system.
4. Once the message is dispatched by the exchange, it is fetched by the Direct Consumer dictated by the routing
key (such as ‘msg_id’) and passed to the Invoker.

Figure 3.7: RPC call

RPC cast

The rpc.cast operation, showed in 3.8, performs the following fases to be completed:

1. A Topic Publisher is instantiated to send the message request to the queuing system.
2. Once the message is dispatched by the exchange, it is fetched by the Topic Consumer dictated by the routing
key (such as ‘topic’) and passed to the Worker in charge of the task.

The publisher sends the message to a topic exchange and a consumer retrieves the message from the queue. No
response is expected as it is a cast and not a call.

22
 Figure 3.8: RPC cast

3.3 Keystone
Keystone service[10] provides a common authentication and authorization store for OpenStack services. Keystone is
responsible for users, their roles, and to which project(tenants) they belong to. Moreover it provides a catalogue of all
other OpenStack services which verify on Keystone a user’s request.
Essentially, Keystone has as primary functions to control the authentication and authorization of a user:

• User management: Keystone keep track of users and of what they are allowed to do. This task is made by
checking and administrating the existing associating among users, roles and tenant.

• Service catalog: Keystone provides a catalog of available services and where their API endpoints are located.

3.3.1 Components
The components that Keystone needs to perform its purpose are the following:

• User: Users are digital representations of a person, system, or service that uses OpenStack cloud services.
Keystone ensures that incoming requests are coming from a valid login user assigned to a particular tenant with
specific role that can be assigned resource-access tokens.

• Tenant: A tenant is a group used to isolate resources and/or users. Groups can be mapped to customers, projects
or organizations.

• Role: A role includes a set of assigned user rights and privileges for performing a specific set of operations. A
user token issued by Keystone includes a list of that user’s roles. Services then determine how to interpret those
roles by their internal policies stored in each policy.json file showed in 3.19 section.

• Credentials: Credentials are data known only by a specific user who proves his or her identity.

• Token: A token is an arbitrary bit of text used to access resources. Each token has a scope describing accessible
resources which may be revoked at any time and is valid for a finite duration.

• Endpoint: An endpoint is a network-accessible address, usually described by URL, from which services are
accessed.

23
3.3.2 Architecture

Keystone’s architecture is organized in a group of internal services exposed on one or many endpoints which can be
seen in figure 3.9:

• Identity: The identity service provides auth credential validation and data about Users, Tenants and Roles as
well as any associated metadata.

• Token: The token service validates and manages Tokens used for authenticating requests once a user/tenant’s
credentials have already verified.

• Catalog: The Catalog service provides an endpoint registry used for endpoint discovery.

• Policy: The policy service provides a rule-based authorization engine.

Each of these services can used several backends options to allow Keystone to fit a variety of enviroment and needs.
The available options are the following:

• KVS Backend: A simple backend interface meant to be further backended on anything that can support primary
key lookups.

• SQL Backend: A SQL backend to store data permanently.

• PAM Backend: Extra simple backend that uses the current system’s PAM service to authenticate, providing a
one-to-one relationship between users and tenants.

• LDAP Backend: The LDAP backend to store users ans tenants in separate subtrees.

• Templated Backend: A simple Template used to configure Keystone

KEYSTONE API OpenStack

services

Identity Token Catalog Policy

database database database database

Figure 3.9: Keystone architecture

24
3.4 Nova
Nova [14]is responsible for hosting and managing cloud computing systems, which is the major part of an Infrastructure-
as-a-Service (IaaS) system and most complicated and distributed service of OpenStack. Nova interacts with Keystone
for authentication and Glance for disk and server images whose access is limited by tenants and users.
Its main modules are implemented in Python and configure a large number of processes which cooperate to turn API
requests into running virtual machines.

3.4.1 Architecture
Nova is built on a shared-nothing, messaging-based architecture. All of the major nova components can be run on
multiple servers which means that most component to component communication must go via message queue. In
order to avoid blocking each component while waiting for a response, we use deferred objects, with a callback that
gets triggered when a response is received.
Nova uses a sql-based central database that is shared by all components in the system whose data fits into a sql database
quite well. For small deployments this seems like an optimal solution but for larger ones, and especially if security is
a concern, it will be used multiple data stores with some kind of aggregation system.
Nova architecture, showed in figure 3.10, is defined by several areas:

• API: This area is responsible for receiving HTTP requests, converting commands and communicating with
other components via the oslo.messaging queue or HTTP. This area has the following components:

– nova-api service: Accepts and responds to end user compute API calls.
– nova-api-metadata service: Accepts metadata requests from instances.

• Compute: The compute area is responsible for managing communication with hypervisor and virtual machines.
It has the following components:

– nova-compute service: It’s a worker daemon that creates and terminates virtual machine instances through
available hypervisor APIs, such as XenAPI for XenServer/XCP, libvirt for KVM or QEMU or VMwareAPI
for VMware.
– nova-scheduler service: It takes a virtual machine instance request from the queue and determines on
which compute server host it runs.
– nova-conductor module: It mediates interactions between the nova-compute service and the database and
it eliminates direct accesses to the cloud database made by the nova-compute service.

• Networking: This area is responsible for managing ip forwarding, bridges, and vlans and it has the following
components:

– nova-network worker daemon: It accepts networking tasks from the queue and manipulates the network.
Performs tasks such as setting up bridging interfaces or changing IPtables rules.

• Console: This area allows end users to access their virtual instance’s console through a proxy. Its components
are:

– nova-consoleauth daemon: It authorizes tokens for users that console proxies provide.
– nova-novncproxy daemon: It provides a proxy for accessing running instances through a VNC connection.
– nova-cert daemon: It provides x509 certificates.

• Image: This area manages the interaction with Glance for image use. The components for this interaction are:

25
 – nova-objectstore daemon: It is an S3 interface for registering images with the OpenStack Image Service.
– euca2ools client: It is a set of command-line interpreter commands for managing cloud resources.
• Database: This are is responsible for storing most build-time and run-time states for a cloud infrastructure such
as available instance types, instances in use, available networks and projects.

KEYSTONE

NOVA API

NOVA
CONSOLEAUTH
NOVA
DATABASE MESSAGE
QUEUE NOVA
NOVNC PROXY

NOVA
CONDUCTOR NOVA GLANCE
SCHEDULER

NOVA NEUTRON
COMPUTE
CINDER

HYPERVISOR

Figure 3.10: Nova architecture

3.4.2 Nova resources

Nova supports multiple different implementations for compute via plugins or drivers:

• Virtual servers: In most cases, Nova only to provide access to virtual servers from a single hypervisor, however,
it is possible to have a Nova deployment include multiple different types of hypervisors.
• Containers: Nova allows to use containers based on LXC or Docker in a similar way to how would use it on
demand virtual machines.
• Bare Metal servers:Nova allows to use physical machines in a similar way to on demand virtual machines.

3.4.3 Filter scheduler

The Filter Scheduler supports filtering and weighting to make informed decisions on where a new instance should be
created. This Scheduler supports only working with Compute Nodes.

Filtering

During its work Filter Scheduler firstly makes dictionary of unfiltered hosts, then filters them using filter properties and
finally chooses hosts for the requested number of instances (each time it chooses the most weighed host and appends
it to the list of selected hosts).
If it turns up, that it can’t find candidates for the next instance, it means that there are no more appropriate hosts where
the instance could be scheduled. This process is showed in figure 3.11.

26
 Figure 3.11: Filtering

Weighting

Filter Scheduler uses the so called weights during its work. A weighter is a way to select the best suitable host from a
group of valid hosts by giving weights to all the hosts in the list. In order to prioritize one weigher against another, all
the weighters have to define a multiplier that will be applied before computing the weight for a node.
Filter Scheduler finds local list of acceptable hosts by repeated filtering and weighting. Each time it chooses a host, it
virtually consumes resources on it, so subsequent selections can adjust accordingly. In the end Filter Scheduler sorts
selected hosts by their weight and provisions instances on them. The figure 3.12 shows the weighting process.

3.4.4 Conductor
The conductor service serves as a centralized place to manage the execution of workflows which involve the scheduler.
Rebuild, resize/migrate, and building an instance are managed here. This provides a better separation of responsibil-
ities between what compute nodes should handle and what the scheduler should handle, and to clean up the path of
execution.
The answer to this was to change to process to be the following:

1. API receives request to build an instance.

2. API sends an RPC cast to the conductor to build an instance.

3. Conductor sends an RPC call to the scheduler to pick a compute and waits for the response. If there is a scheduler
fail it stops the build at the conductor.

4. Conductor sends an RPC cast to the compute to build the instance.

• If the build succeeds it stops here.

• If the build fails then compute sends an RPC cast to conductor to build an instance. This is the same RPC
message that was sent by the API.

27
 Figure 3.12: Weighting

This process means the scheduler only deals with scheduling, the compute only deals with building an instance, and
the conductor manages the workflow.

3.4.5 Virtual Machines states

An instance can be in several states that show its actual status. The figure 3.13 show the required virtual machine states
and the possible transition to other ones.

Figure 3.13: VM states

28
3.5 Neutron
Neutron [13]provides “networking as a service” between interface devices managed by other Openstack services (e.g.,
nova). The core Neutron API includes support for Layer 2 networking and IP address management(IPAM), as well as
an extension for a Layer 3 router construct that enables routing between Layer 2 networks and gateways to external
networks. Neutron includes a list of plugins that enable interoperability with various commercial and open source
network technologies, including routers, switches, virtual switches and software-defined networking (SDN)controllers.

3.5.1 Overview

Neutron manages all networking facets for the Virtual Networking Infrastructure (VNI) and the access layer aspects
of the Physical Networking Infrastructure (PNI) in an OpenStack environment. Neutron service enables tenants to
create advanced virtual network topologies including services such as firewalls, load balancers, and virtual private
networks (VPNs).In addition, it provides the networks, subnets, and routers object abstractions. Each abstraction has
functionality that mimics its physical counterpart.
Each router has one gateway that connects to a network, and many interfaces connected to subnets. Subnets can access
machines on other subnets connected to the same router.
Any given Networking set up has at least one external network. Unlike the other networks, the external network is not
merely a virtually defined network. Instead, it represents a view into a slice of the physical, external network accessible
outside the OpenStack environment. IP addresses on the external network are accessible by anybody physically on the
outside network. Because the external network merely represents a view into the outside network, DHCP is disabled
on this network.
In addition to external networks, any Networking set up has one or more internal networks. These software-defined
networks connect directly to the VMs. Only the VMs on any given internal network, or those on subnets connected
through interfaces to a similar router, can access VMs connected to that network directly.
For the outside network to access VMs, and vice versa, routers between the networks are needed. Each router has
one gateway that is connected to a network and many interfaces that are connected to subnets. Like a physical router,
subnets can access machines on other subnets that are connected to the same router, and machines can access the
outside network through the gateway for the router.
Additionally, you can allocate IP addresses on external networks to ports on the internal network. Whenever something
is connected to a subnet, that connection is called a port.You can associate external network IP addresses with ports to
VMs. This way, entities on the outside network can access VMs.
Networking also supports security groups. Security groups enable administrators to define firewall rules in groups. A
VM can belong to one or more security groups, and Networking applies the rules in those security groups to block or
unblock ports, port ranges, or traffic types for that VM.

3.5.2 Architecture

Neutron is a standalone component in the OpenStack modular architecture. It’s positioned alongside OpenStack
services such as Nova, Glance or Keystone. Like those services, the Neutron deployment often involves deploying
several services to a variety of hosts.
Neutron server uses the neutron-server daemon to expose the Networking API and enable administration of the con-
figured Networking plug-in which requires access to a database for persistent storage. Neutron is configured by the
following components:

• neutron-server: Accepts and routes API requests to the appropriate Neutron plug-in for action.

29
 • Neutron plug-ins and agents: Plugs and unplugs ports, creates networks or subnets, and provides IP address-
ing. These plug-ins and agents differ depending on the vendor and technologies used in the particular cloud.
Neutron ships with plug-ins and agents for Cisco virtual and physical switches, NEC OpenFlow products, Open
vSwitch, Linux bridging, Ryu Network Operating System, and the VMware NSX product. The common agents
for a typical environment are L3, DHCP, and a plug-in agent.

• Messaging queue: Route information between the neutron-server and various agents, as well as a database to
store networking state for particular plug-ins.

The figure 3.14 shows neutron components in its architecture diagram:

Management network

NEUTRON
PLUGIN AGENT
NEUTRON NEUTRON
NEUTRON PLUGIN AGENT SERVER
L3 AGENT

NEUTRON
DHCP AGENT

Neutron node Compute node Controller node

Internal network

External network API network

Figure 3.14: Neutron architecture

3.5.3 Network modes

Neutron service offers the following network modes:

• Flat mode: Flat mode is the simplest networking mode. Each instance receives a fixed IP from the pool and all
instances are attached to the same bridge by default although it has to be configured manually. The networking
configuration is injected into the instance before it is booted and there is no floating IP feature in this mode.

• Flat DHCP mode: This is similar to the flat mode as all instances are attached to the same bridge. In this
mode Nova does a bit more configuration because it will attempt to bridge into an Ethernet device. It will also
run dnsmasq as a dhcpserver listening on this bridge. Instances receive their fixed IPs by doing a dhcpdiscover.
Moreover, floating IP feature is provided.

30
 • VLAN network mode: It is the default mode for Nova. It provides a private network segment for each project’s
instances that can be accessed via a dedicated VPN connection from the Internet. In this mode, each project gets
its own VLAN, Linux networking bridge, and subnet. The subnets are specified by the network administrator,
and are assigned dynamically to a project when required. A DHCP Server is started for each VLAN to pass out
IP addresses to VM instances from the subnet assigned to the project. All instances belonging to one project are
bridged into the same VLAN for that project.

3.5.4 Network namespaces

A namespace is a way of scoping a particular set of identifiers. Using a namespace, you can use the same identifier
multiple times in different namespaces. You can also restrict an identifier set visible to particular processes.
In a network namespace, the scoped ‘identifiers’ are network devices; so a given network device, such as eth0, exists in
a particular namespace. Linux starts up with a default network namespace,but it is also possible to create further non-
default namespaces, and create new devices in those namespaces, or to move an existing device from one namespace
to another.
Each network namespace also has its own routing table, and in fact this is the main reason for namespaces to exist.
A routing table is keyed by destination IP address, so network namespaces are what you need if you want the same
destination IP address to mean different things at different times - which is something that OpenStack Networking
requires for its feature of providing overlapping IP addresses in different virtual networks.
Each network namespace also has its own set of iptables. So, you can apply different security to flows with the same
IP addressing in different namespaces, as well as different routing.

3.5.5 External and tenant networks

To provide connectivity to instances, there is the need of a virtual network infrastructure which is based on an external
network and one or more internal networks.

Internal networks

The tenant network provides internal network access for instances. The architecture isolates this type of network from
other tenants. This means that there can be many internal networks for different tenants which won’t be connected
between them.
Users and tenants that own an internal network have full control over the network topology which are able to create
virtual routers between networks including external networks in order to provide Internet access. Apart from creating
virtual routers, the network’s owners can manage subnets, DHCP and DNS services, L3 routing and firewall services.

External networks

The external network provides Internet access for the instances. By default, this network only allows Internet access
from instances using Network Address Translation (NAT) and enabled to individual instances using a floating IP
address and suitable security group rules.

Network topology

To implement the network topology, Neutron stores the logical representation of networks in its database and plugins
such as Open vSwitch translates these logical representations. Agents on each host or hypervisor makes the necessary
modifications to the instances in order to implement the networking.

31
To do so, the plugins use the message queue for communication and scale out but it also can be used an specific
SDN/Network controller.
The figure 3.15 shows the typical network topology in an OpenStack environment:

Figure 3.15: Neutron network

As said before, plugins such as Open vSwitch, the default one, provide networking access to instances and the tenant
networks are implemented by flows received from the Neutron server. PLugins use virtual switches/bridges to form
the networks on each host. In a typical OpenStack enviroment there are the following bridges:

• br-int (Integration bridge): All local ports interconnect here, e.g. vNICs or Neutron services (DHCP,L3...).
• br-tun (Tunnel Bridge): If tunnels are enabled, this bridge is used to link other vSwitches together for tenant
traffic
• br-ex (External Bridge): Used to route external traffic to/from tenant networks and requires a physical ethernet
adapter.

The figure 3.16 shows this bridge architecture.

3.6 Glance
Glance [5]provides a service where users can upload and discover data assets that are meant to be used with other
services. This currently includes images and metadata definitions. Glance image services include discovering, regis-
tering, and retrieving virtual machine images. It has a RESTful API that allows querying of VM image metadata as
well as retrieval of the actual image. VM images made available through Glance can be stored in a variety of locations
from simple filesystems to object-storage systems like the OpenStack Swift project.

3.6.1 Architecture
OpenStack Glance has a client-server architecture, showed in figure 3.17, and provides a user REST API through
which requests to the server are performed. The internal server operations are managed by a Glance Domain Controller
divided into layers and each layer implements its own task. All the files operations are performed using glance_store
library which is responsible for interaction with external storage back ends or local filesystem, and provides a uniform
interface to access. Glance uses an sql-based central database that is shared with all the components in the system.
Glance architecture consists of several components:

32
 Figure 3.16: Neutron bridge architecture

• REST API: It exposes Glance functionalities via REST.

• Database Abstraction Layer (DAL): It is an application programming interface which unifies the communi-
cation between Glance and databases.

• Glance Domain Controller: It is a middleware that implements the main Glance functionalities: authorization,
notifications, policies, database connections.

• Glance Store: It organizes interactions between Glance and various data stores.

• Registry Layer: It is a layer that organizes secure communication between the domain and the DAL by using a
separate service.

3.6.2 Disk and container formats

When adding an image to Glance it is needed to specify what the virtual machine image’s disk format and container
format are.

Disk formats

The disk format of a virtual machine image is the format of the underlying disk image. There are the following
available disk formats:

• raw: An unstructured disk image format.

• vhd: VHD disk format, a common disk format used by virtual machine monitors from VMWare, Xen, Microsoft,
VirtualBox, and others.

• vmdk: A common disk format supported by many common virtual machine monitors.

• vdi: A disk format supported by VirtualBox virtual machine monitor and the QEMU emulator.

33
 GLANCE
CLIENT GLANCE API DATABASE

REGISTRY
GLANCE LAYER
KEYSTONE DOMAIN
CONTROLLER
DATABASE
ABSTRACTION
LAYER
GLANCE STORE

Figure 3.17: Glance architecture

• iso: An archive format for the data contents of an optical disc (e.g. CDROM).

• qcow2: A disk format supported by the QEMU emulator that can expand dynamically and supports Copy on
Write.

• aki: This indicates what is stored in Glance is an Amazon kernel image.

• ari: This indicates what is stored in Glance is an Amazon ramdisk image.

• ami: This indicates what is stored in Glance is an Amazon machine image.

Container formats

The container format refers to whether the virtual machine image is in a file format that also contains metadata about
the actual virtual machine. The available formats are the following:

• bare: There is no container or metadata envelope for the image.

• ovf: OVF container format.

• aki: This indicates what is stored in Glance is an Amazon kernel image.

• ari: This indicates what is stored in Glance is an Amazon ramdisk image.

• ami: This indicates what is stored in Glance is an Amazon machine image.

• ova: This indicates what is stored in Glance is an OVA tar archive file.

34
3.7 Swift
OpenStack Object Storage [20]is a highly available, distributed, eventually consistent object store. It is possible to
create, modify, and get objects and metadata by using the Object Storage API, which is implemented as a set of
Representational State Transfer (REST) web services.
The Object Storage system organizes data in a hierarchy:

• Account: Represents the top-level of the hierarchy. Each account owns all resources in it. The account, which
is synonymous with a tenant, defines namespace for containers. It is possible to have the same name for a
container if they are in two different accounts.

• Container: There can be any number of containers for each account. Each container defines a namespace for
objects which can have the same if they are in different containers.In addition to containing objects, it can also
be used the container to control access to objects by using an access control list (ACL).

• Object: Stores data content, such as documents, images, and so on. It can also be stored custom metadata with
an object.

3.7.1 Architecture

OpenStack Swift has a client-server architecture, showed in figure 3.18, and provides a user REST API through which
requests to the server are performed. Swift proxy receives and manages the API requests and communicates with the
Storage service node to perform the action within the proper Account/Container/Object. The components that form
these architecture are described below.

CLIENT

KEYSTONE Account
SWIFT-ACCOUNT database

Container
SWIFT-CONTAINER database
SWIFT PROXY
Object
SWIFT-OBJECT database

Figure 3.18: Swift architecture

Proxy Server

The Proxy Server is the central component which ties together the rest of the Swift architecture. For each request,
it will look up the location of the account, container, or object in the ring and route the request accordingly. When
objects are streamed to or from an object server, they are streamed directly through the proxy server to or from the
user, the proxy server does not spool them. The Proxy server can handle a large number of failures as if an action is
unavailable for a certain server, it will ask the ring for a handoff server and route there instead.

35
The Ring

A ring represents a mapping between the names of entities stored on disk and their physical location. There are separate
rings for accounts, containers, and one object ring per storage policy. When other components need to perform any
operation on an object, container, or account, they need to interact with the appropriate ring to determine its location
in the cluster.
The Ring maintains this mapping using zones, devices, partitions, and replicas. Each partition in the ring is replicated,
by default, 3 times across the cluster, and the locations for a partition are stored in the mapping maintained by the ring.
The ring is also responsible for determining which devices are used for handoff in failure scenarios.
Data is evenly distributed across the capacity available in the cluster as described by the devices weight. Weights can
be used to balance the distribution of partitions on drives across the cluster. This can be useful when different sized
drives are used in a cluster. Device weights can also be used when adding or removing capacity or failure domains to
control how many partitions are reassigned during a rebalance to be moved as soon as replication bandwidth allows.
When partitions need to be moved around (for example if a device is added to the cluster), the ring ensures that a
minimum number of partitions are moved at a time, and only one replica of a partition is moved at a time.

Storage Policies

Storage Policies provide a way for object storage providers to differentiate service levels, features and behaviors of
a Swift deployment. Each Storage Policy configured in Swift is exposed to the client via an abstract name. Each
device in the system is assigned to one or more Storage Policies. This is accomplished through the use of multiple
object rings, where each Storage Policy has an independent object ring, which may include a subset of hardware
implementing a particular differentiation.
This mapping is then exposed on a per-container basis, where each container can be assigned a specific storage policy
when it is created, which remains in effect for the lifetime of the container. Applications require minimal awareness
of storage policies to use them, once a container has been created with a specific policy all objects stored in it will be
done so in accordance with that policy.

Replication

Replication is designed to keep the system in a consistent state in the face of temporary error conditions like network
outages or drive failures.
The replication processes compare local data with each remote copy to ensure they all contain the latest version. Object
replication uses a hash list to quickly compare subsections of each partition, and container and account replication use
a combination of hashes and shared high water marks.
The replicator also ensures that data is removed from the system. When an item is deleted, a tombstone is set as the
latest version of the item. The replicator will see the tombstone and ensure that the item is removed from the entire
system.

Object Server

The Object Server is a very simple storage server that can store, retrieve and delete objects stored on local devices.
Objects are stored as binary files on the filesystem with metadata stored in the file’s extended attributes (xattrs). This
requires that the underlying filesystem choice for object servers support xattrs on files.
Each object is stored using a path derived from the object name’s hash and the operation’s timestamp. Last write
always wins, and ensures that the latest object version will be served. A deletion is also treated as a version of the

36
file. This ensures that deleted files are replicated correctly and older versions don’t magically reappear due to failure
scenarios.

Container Server

The Container Server’s primary job is to handle listings of objects. It doesn’t know where those objects are, just what
objects are in a specific container. The listings are stored as sqlite database files, and replicated across the cluster
similar to how objects are. Statistics are also tracked which include the total number of objects, and total storage usage
for that container.

Account Server

The Account Server is very similar to the Container Server, excepting that it is responsible for listings of containers
rather than objects.

3.8 Cinder
OpenStack Block Storage provides the software to create and centrally manage a service that provisions storage in
the form of block devices known as Cinder volumes. Under the most common scenario, the Cinder volumes provide
persistent storage to instances that are managed by Nova. Cinder can also be used independently of other OpenStack
services.
Cinder permits organizations that deploy it to make available a catalog of block-based storage devices with differing
characteristics. For example, one potential storage volume type might be a high-performance option for database
storage. Cinder also features basic storage capabilities such as snapshot management and volume clones, which are
often enhanced through vendor-specific drivers.
The physical storage media, whether disks or solid-state drives, can be located within or directly attached to the Cinder
server nodes, or they can be located in external storage systems from third-party vendors. Third-party storage vendors
use Cinder’s plug-in architecture to do the necessary integration work. The back-end path from the external storage
systems to the compute nodes where the hypervisors are located can be iSCSI, Network File System (NFS) or Fibre
Channel.

3.8.1 Architecture
Cinder architecture, showed in figure 3.19, is organized in a union several components which use the message queue
to exchange data among them:

• Cinder API: It exposes Cinder functionalities via REST which enables the manipulation of volumes and snap-
shots.
• Cinder-volume: This module is responsible for reading an writing the Cinder database to manage the volume
state. It also responsible for interacting with cinder-scheduler module and the backend driver used for storing.
• Cinder-scheduler: This module is responsible for selecting the best storing block when a volume is created.
• Cinder database: The database sis responsible for storing the volumes states.

The backend used for the volume storage can be changed in order to use a third-party storage option such as CEPH,
Red Hat, Storage (GlusterFS), IBM XIV, HP Leftland or 3Par. When a third-party volume storage option is used the
cinder-volume module will use the ISCI protocol to communicate with the indicated API driver.

37
 CINDER API

CINDER-SCHEDULER
Cinder DB

CINDER-VOLUME

MESSAGE
QUEUE

Figure 3.19: Cinder architecture

3.9 Heat
Heat [7]provides an AWS CloudFormation implementation for OpenStack that orchestrates an AWS CloudFormation
template describing a cloud application by executing appropriate OpenStack API calls to generate running cloud
applications.
The software integrates other core components of OpenStack into a one-file template system. The templates allow
creation of most OpenStack resource types (such as instances, floating IPs, volumes, security groups and users), as
well as some more advanced functionality such as instance high availability, instance autoscaling, and nested stacks.
By providing very tight integration with other OpenStack core projects, all OpenStack core projects could receive a
larger user base.
Heat consists of the following components:

• heat: The heat tool is a CLI which communicates with the heat-api to execute AWS CloudFormation APIs.

• heat-api: The heat-api component provides an OpenStack-native REST API that processes API requests by
sending them to the heat-engine over RPC.

• heat-api-cfn: The heat-api-cfn component provides an AWS Query API that is compatible with AWS Cloud-
Formation and processes API requests by sending them to the heat-engine over RPC.

• heat-engine: The heat-engine’s main responsibility is to orchestrate the launching of templates and provide
events back to the API consumer.

3.10 Trove
Trove[21] is Database as a Service for OpenStack. It’s designed to run entirely on OpenStack with the goal of allow-
ing users to quickly and easily utilize the features of a relational database without the burden of handling complex
administrative tasks. Cloud users and database administrators can provision and manage multiple database instances
as needed.

38
Initially, the service will focus on providing resource isolation at high performance while automating complex admin-
istrative tasks including deployment, configuration, patching, backups, restores, and monitoring.
Its architecture, shown in figure 3.20, consists of the following components:

• Trove-api: Trove-api service provides a RESTful API that supports JSON and XML to provision and manage
Trove instances.
• Trove-taskmanager: Trove-taskmanager service does the heavy lifting as far as provisioning instances, man-
aging the lifecycle of instances, and performing operations on the Database instance.
• Trove-guestagent: Guestagent is a service that runs within the guest instance, responsible for managing and
performing operations on the Database itself. The Guestagent listens for RPC messages through the message
bus and performs the requested operation.
• Trove-conductor: Conductor is a service that runs on the host, responsible for receiving messages from guest
instances to update information on the host.

CLIENT API TASKMANAGER

GUESTAGENT Database engine

DATABASE
VM

Figure 3.20: Trove architecture

3.11 Ironic
Ironic[9] is an OpenStack project which provisions physical hardware as opposed to virtual machines. Ironic provides
several reference drivers which leverage common technologies to cover a wide range of hardware. Ironic’s pluggable
driver architecture also allows vendor-specific drivers to be added for improved performance or functionality not
provided by reference drivers. Ironic is thought as an hypervisor API gluing together multiple drivers, each of which
implement some portion of that functionality with respect to physical hardware which makes physical servers as easy
to provision as virtual machines in cloud.
The reasons for bare metal provisioning are the possibility of high-performance of computing clusters and the possi-
bility to make computing tasks onto devices which can’t be virtualized. In terms of performance, the database hosting
is improved when hosting it in a physical device. However, not every reason is focused on the device’s performance
but Ironic also provides the possibility to have a tenant which is fully dedicated for hardware.

3.11.1 Architecture
The Ironic architecture, showed in figure 3.21, is composed of the following components:

• API service: An API to interact with the managed bare metal servers.

39
 • Conductor: This service does all the operations received through the API. The communication among them is
via RPC.

• Drivers: The drivers are essential for supporting heterogeneous hardware.

• Database: The database stores information about the resources. Among other things, it includes the state of the
conductors, nodes and drivers.

IRONIC

NOVA NOVA IRONIC

API API

NOVA
SCHEDULER IRONIC
CONDUCTOR DRIVERS
NEUTRON

GLANCE

IRONIC
OTHER DATABASE
OPENSTACK
SERVICES

Figure 3.21: Ironic architecture

When a user requests to boot an instance is passed to the Nova Compute service via Nova API and Nova Scheduler.
The Compute service hands over this request to the Ironic service, where the request passes from the Ironic API, to
the Conductor, to a Driver to successfully provision a physical server for the user.
Just as the Nova Compute service talks to various OpenStack services to provision a virtual machine instance, the
Ironic service talks to the same OpenStack services for image, network and other resource needs to provision a bare
metal instance.

3.11.2 Key technologies for Ironic

In order to provide the Ironic service properly, the following technologies are essential.

PXE

Preboot Execution Environment (PXE) is part of the Wired for Management (WfM) specification developed by Intel
and Microsoft. The PXE enables system’s BIOS and network interface card (NIC) to bootstrap a computer from the
network in place of a disk. Bootstrapping is the process by which a system loads the OS into local memory so that

40
it can be executed by the processor. This capability of allowing a system to boot over a network simplifies server
deployment and server management for administrators.

DHCP

Dynamic Host Configuration Protocol (DHCP) is a standardized networking protocol used on Internet Protocol (IP)
networks for dynamically distributing network configuration parameters, such as IP addresses for interfaces and ser-
vices. Using PXE, the BIOS uses DHCP to obtain an IP address for the network interface and to locate the server that
stores the network bootstrap program (NBP).

NBP

Network Bootstrap Program (NBP) is equivalent to GRUB (GRand Unified Bootloader) or LILO (LInux LOader)
which are drivers that are traditionally used in local booting. Like the boot program in a hard drive environment, the
NBP is responsible for loading the OS kernel into memory so that the OS can be bootstrapped over a network.

TFTP

Trivial File Transfer Protocol (TFTP) is a simple file transfer protocol that is generally used for automated transfer of
configuration or boot files between machines in a local environment. In a PXE environment, TFTP is used to download
NBP over the network using information from the DHCP server.

IPMI

Intelligent Platform Management Interface (IPMI) is a standardized computer system interface used by system admin-
istrators for out-of-band management of computer systems and monitoring of their operation. It is a method to manage
systems that may be unresponsive or powered off by using only a network connection to the hardware rather than to
an operating system.

3.12 Sahara
Sahara[19] aims to provide users with a simple means to provision Hadoop, Spark, and Storm clusters by specifying
several parameters such as the version, cluster topology, hardware node details and more. After a user fills in all the
parameters, Sahara deploys the cluster in a few minutes. Also Sahara provides means to scale an already provisioned
cluster by adding or removing worker nodes on demand. Sahara provides the following key features to the OpenStack
environment:

• It is designed as an OpenStack component, so it provides scalability and compatibility with other OpenStack
services.

• It can be managed through a REST API with a user interface(UI) available as part of OpenStack Dashboard.

• It supports a variety of data processing frameworks:

– Multiple Hadoop vendor distributions.

– Apache Spark and Storm.
– Pluggable system of Hadoop installation engines.

41
 – Integration with vendor specific management tools, such as Apache Ambari and Cloudera Management
Console.

• There are predefined configuration templates with the ability to modify parameters easily.

3.12.1 Architecture

Sahara service interacts with several OpenStack services when it is operating. The figure 3.22 shows its architecture
and the relationships between the other services. Specifically these the following:

• Horizon: It provides a GUI with ability to use all of Sahara’s features.

• Keystone: It authenticates users and provides security tokens that are used to work with OpenStack, limiting a
user’s abilities in Sahara to their OpenStack privileges.

• Nova: It is used to provision VMs for data processing clusters.

• Heat: It is used to provision and orchestrate the deployment of data processing clusters.

• Glance: It stores VM images, each image containing an operating system and a pre-installed data processing
distribution or framework.

• Swift: It can be used as storage for job binaries and data that will be processed or created by framework jobs.

• Cinder: It can be used to provision block storage for VM instances.

• Neutron: It provides networking services to data processing clusters.

• Ceilometer: It is used to collect measures of cluster usage for metering and monitoring purposes.

• Manila: It can be used for storage of framework job binaries and data that will be processed or created by jobs.

Components

The Sahara architecture consists of several components:

• Auth component: It is responsible for client authentication and authorization and communicates with the Key-
stone.

• Data Access Layer(DAL): It persists internal models in DB.

• Provisioning Engine: This component is responsible for communication with Nova, Heat, Cinder and Glance.

• Vendor Plugins: These are pluggable mechanisms responsible for configuring and launching data processing
frameworks on provisioned VMs.

• Elastic Data Processing (EDP): It is responsible for scheduling and managing data processing jobs on clusters
provisioned by Sahara.

• API: It exposes Sahara functionality via REST HTTP interface.

• Sahara pages: The GUI for the Sahara is located in the OpenStack Dashboard.

42
 Figure 3.22: Sahara architecture

3.12.2 Workflow
Sahara provides two use cases in its workflow. These are the cluster provisioning and analytics as a service.

Cluster provisioning

For fast cluster provisioning a generic workflow will be as following:

1. A Hadoop version is selected.

2. A base image with or without pre-installed data processing framework is selected.
3. Cluster configuration, including cluster size, topology, and framework parameters are defined.
4. The cluster is provisioned: Sahara will provision VMs, install and configure the data processing framework.
5. Operations on the cluster (add or remove nodes) are performed.
6. The cluster is terminated when it is no longer needed.

Analytics as a service

1. One of the predefined data processing framework versions is selected.

2. A job is chosen.
3. The limit for the cluster size is set.
4. The job is executed.
5. The results of computations are received.

43
3.13 Zaqar
Zaqar[22] is a multi-tenant cloud messaging and notification service for web and mobile developers.
The service features a REST API, which developers can use to send messages between various components of their
SaaS and mobile applications, by using a variety of communication patterns. Underlying this API is an efficient
messaging engine designed with scalability and security in mind.
Other OpenStack components can integrate with Zaqar to surface events to end users and to communicate with guest
agents that run in the “over-cloud” layer.

3.13.1 Architecture

Zaqar is composed of two layers, the transport drivers layer and the storage drivers layer:

• Transport layer: The transport drivers are responsible for interacting with Zaqar clients. Every query made by
clients is processed by the transport layer, which is in charge of passing this information to the backend and then
returning the response in a format understandable by the client.

• Storage layer: The storage drivers are responsible for interacting with the storage backends and, that way, store
or retrieve the data coming from the transport layer.

In order to keep these layers decoupled, it has been established that checks should be performed in the appropriate
layer. In other words, transport drivers must guarantee that the incoming data is well-formed and storage drivers must
enforce their data model stays consistent.

3.14 Designate
Designate[4] provides DNSaaS services for OpenStack. The figure 3.23 shows the logical architecture for this service
and of which components it is composed.

POOL BACKEND
MANAGER

CUSTOMER
FACING
API CENTRAL DB
SERVER

SINK
MINI DNS

Figure 3.23: Designate architecture

44
3.14.1 Designate API

Designate-api provides the standard OpenStack style REST API service, accepting HTTP requests, validating authen-
tication tokens with Keystone and passing them to the Designate Central service over AMQP. Multiple versions of the
API can be hosted, as well as API extensions, allowing for pluggable extensions to the core API.

3.14.2 Designate Central

Designate-central is the service that handles RPC requests via the MQ, it coordinates the persistent storage of data and
applies business logic to data from the API.

3.14.3 Designate MiniDNS

Designate-mdns is the service that sends DNS NOTIFY and answers zone transfer requests. This allows Designate to
integrate with any DNS server that supports these very standard methods of communicating.

3.14.4 Designate Pool Manager

Designate-pool-manager is a service that manages the states of the DNS servers Designate manages. The Pool Manager
is configured to know which DNS servers Designate manages, and their type of backend. It can also divide those
servers into ‘Pools’ so that zones within Designate can be split across different sets of backend servers. The Pool
Manager is also responsible for making sure that backend DNS servers are in sync with the Designate database.

3.14.5 Designate Sink

Designate-sink is an optional service which listens for event notifications, handlers are available for Nova and Neutron.
Notification events can then be used to trigger record creation and deletion.

3.14.6 DNS Backend

Backends are drivers for a particular DNS server.

3.14.7 Database/Storage

Storage drivers are drivers for a particular SQL/NoSQL server. Designate needs a SQLAlchemy-supported storage
engine for the persistent storage of data.

3.15 Manila
Manila[12] is the File Share service project for OpenStack. To administer the OpenStack File Share service. When
configuring the File Share service, it is required to declare at least one back end. Manila can be configured to run in
a single-node configuration or across multiple nodes. Manila can be configured to provision shares from one or more
back ends. The OpenStack File Share service allows offering file-share services to users of an OpenStack installation.

45
3.16 Magnum
Magnum[11] is an OpenStack project which offers container orchestration engines for deploying and managing con-
tainers as first class resources in OpenStack.
The Magnum architecture has two binaries work together to compose the magnum system. The first binary is the
magnum-api REST server. The REST server may run as one process or multiple processes. When a REST request is
sent to the client API, the request is sent via AMQP to the magnum-conductor process. The REST server is horizontally
scalable.
The magnum-conductor process runs on a controller machine and connects to a Kubernetes or Docker REST API
endpoint. The Kubernetes and Docker REST API endpoints are managed by the bay object.
When service or pod objects are created, Kubernetes may be directly contacted via the Kubernetes REST API. When
container objects are acted upon, the Docker REST API may be directly contacted.
There are several different types of objects in the magnum system:

• Bay: A collection of node objects where work is scheduled.

• BayModel: An object stores template information about the bay which is used to create new bays consistent.

• Node: A baremetal or virtual machine where work is executed.

• Pod: A collection of containers running on one physical or virtual machine.

• Service: An abstraction which defines a logical set of pods and a policy by which to access them.

• ReplicationController: An abstraction for managing a group of pods to ensure a specified number of resources
are running.

• Container: A Docker container.

3.17 Horizon
Horizon[8] ships with three central dashboards, a “User Dashboard”, a “System Dashboard”, and a “Settings” dash-
board. Between these three they cover the core OpenStack applications and deliver on Core Support.
The Horizon application also ships with a set of API abstractions for the core OpenStack projects in order to provide
a consistent, stable set of reusable methods for developers. Using these abstractions, developers working on Horizon
don’t need to be intimately familiar with the APIs of each OpenStack project.

3.18 Ceilometer

3.18.1 Architecture

Each of Ceilometer’s services[3] are designed to scale horizontally. Additional workers and nodes can be added
depending on the expected load. Ceilometer offers five core services, the data agents designed to work independently
from collection and alarming, but also designed to work together as a complete solution as it can be seen in figure
3.24:

• polling agent: It is the daemon designed to poll OpenStack services and build Meters.

46
 • notification agent: It is the daemon designed to listen to notifications on message queue, convert them to Events
and Samples, and apply pipeline actions.

• collector: It is the daemon designed to gather and record event and metering data created by notification and
polling agents.

• api: It is the service to query and view data recorded by collector service.

• alarming: It is the daemons to evaluate and notify based on defined alarming rules.

Figure 3.24: Ceilometer architecture

3.18.2 Data gathering

Data collection

Ceilometer uses two methods to collect data due to not all services send events on the message bus and it will be
needed to instrument other tools which may not use the same bus as OpenStack has defined:

• Bus listener agent: It takes events generated on the notification bus and transforms them into Ceilometer samples.
This is the preferred method of data collection. If you are working on some OpenStack related project and are
using the Oslo library, you are kindly invited to come and talk to one of the project members to learn how you
could quickly add instrumentation for your project.

• Polling agents: It is the less preferred method, will poll some API or other tool to collect information at a regular
interval. Where the option exists to gather the same data by consuming notifications, then the polling approach
is less preferred due to the load it can impose on the API services.

47
The first method is supported by the ceilometer-notification agent, which monitors the message queues for notifica-
tions. Polling agents can be configured either to poll local hypervisor or remote APIs.
The figure 3.25 shows the two described methods.

Figure 3.25: Ceilometer data collection

Notification agents

The heart of the system is the notification daemon (agent-notification) which monitors the message bus for data being
provided by other OpenStack components such as Nova, Glance, Cinder, Neutron, Swift, Keystone, and Heat, as well
as Ceilometer internal communication.
The notification daemon loads one or more listener plugins, which can listen to any topics. The listeners grab messages
off the defined topics and redistributes them to the appropriate plugins to be processed into Events and Samples.
Sample-oriented plugins provide a method to list the event types they’re interested in and a callback for processing
messages accordingly. The registered name of the callback is used to enable or disable it using the pipeline of the
notification daemon. The incoming messages are filtered based on their event type value before being passed to the
callback so the plugin only receives events it has expressed an interest in seeing.
The described process is shown in figure 3.26

Figure 3.26: Ceilometer notification

48
Data polling

Polling for compute resources is handled by a polling agent running on the compute node, often referred to as the
compute-agent. Polling via service APIs for non-compute resources is handled by an agent running on a cloud con-
troller node, often referred to the central-agent. A single agent can fulfill both roles in an all-in-one deployment.
Conversely, multiple instances of an agent may be deployed, in which case the workload is shared. The polling agent
daemon is configured to run one or more pollster plugins.
The agents periodically ask each pollster for instances of Sample objects. The frequency of polling is controlled via
the pipeline configuration. The agent framework then passes the samples to the pipeline for processing.
The figure 3.27 shows the polling method.

Figure 3.27: Ceilometer polling

3.18.3 Data processing

Pipeline manager

Ceilometer offers the ability to take data gathered by the agents, manipulate it, and publish it in various combinations
via multiple pipelines. This functionality is handled by the notification agents.

Data transformation

The data gathered from the polling and notifications agents contains a wealth of data and if combined with historical
or temporal context, can be used to derive even more data. Ceilometer offers various transformers which can be used
to manipulate data in the pipeline.

Data publishing

Processed data can be published using 3 different transports:

• notifier: a notification based publisher which pushes samples to a message queue which can be consumed by
the collector or an external system.

49
 • udp: publishes samples using UDP packets.

• kafka: publishes data to a Kafka message queue to be consumed by any system that supports Kafka.

3.18.4 Data storage - Collector service

The collector daemon gathers the processed event and metering data captured by the notification and polling agents.
It validates the incoming data and then writes the messages to a declared target: database, file, or http.

3.18.5 Data access - API service

If the collected data from polling and notification agents are stored in supported databases, it is possible that the schema
of these databases may evolve over time. For accessing the collected data it is better to use the available Ceilometer
API rather than accessing to it by the underlying database directly.

3.18.6 Data evaluation - Alarming service

The alarming component of Ceilometer, first delivered in the Havana version, allows you to set alarms based on
threshold evaluation for a collection of samples. An alarm can be set on a single meter, or on a combination. It can
only be set alarms on meters for the tenant’s owned components.
The are two forms of action:

• HTTP callback: you provide a URL to be called whenever the alarm has been set off. The payload of the request
contains all the details of why the alarm was triggered.

• log: mostly useful for debugging, stores alarms in a log file.

3.19 Policiy files

As it has been described in section 3.1.1, each OpenStack service, Identity, Compute, Networking and so on has
its own role-based access policies[17]. They determine which user can access which objects in which way, and are
defined in the service’s policy.json file.
Whenever an API call to an OpenStack service is made, the service’s policy engine uses the appropriate policy defini-
tions to determine if the call can be accepted. Any changes to policy.json are effective immediately, which allows new
policies to be implemented while the service is running.
A policy.json file is a text file in JSON (Javascript Object Notation) format. Each policy is defined by a one-line
statement in the form "<target>" : "<rule>". The policy target, also named "action", represents an API call like "start
an instance" or "attach a volume".
Action names are usually qualified, for example OpenStack Compute features API calls to list instances, volumes
and networks. In policy.json file stored in /etc/nova/policy.json, these APIs are represented by compute:get_all, vol-
ume:get_all and network:get_all, respectively.
The policy rule determines under which circumstances the API call is permitted. Usually this involves the user who
makes the call and often the object on which the API call operates. A typical rule checks if the API user is the object’s
owner but it can be edited for checking other aspects of the requesting user.

50
3.19.1 Policy syntax
A policy.json file consists of policies and aliases of the form target:rule or alias:definition, separated by commas and
enclosed in curly braces:
"alias 1" : "definition 1", "alias 2" : "definition 2", ... "target 1" : "rule 1", "target 2" : "rule 2", ....
Targets are APIs and are written "service:API" or simply "API". For example, "compute:create" or "add_image" and
the rules determine whether the API call is allowed. This rules can be:

• always true. The action is always permitted. This can be written as "" (empty string), [], or "@".
• always false. The action is never permitted. Written as "!".
• a special check. These can be:
– <role>:<role name>, a test whether the API credentials contain this role.
– <rule>:<rule name>, the definition of an alias.
– http:<target URL>, which delegates the check to a remote server. The API is authorized when the server
returns True.
• a comparison of two values. These values are compared using the syntax "value1 : value2". The possible values
for comparing are:
– constants: Strings, numbers, true, false.
– API attributes. They can be project_id, user_id or domain_id.
– target object attributes.
– the flag is_admin
• boolean expressions based on simpler rules

Target object attributes are fields from the object description in the database. For example in the case of the "com-
pute:start" API, the object is the instance to be started. The policy for starting instances could use the %(project_id)s
attribute, that is the project that owns the instance. The trailing s indicates this is a string.
The attribute is_admin indicates that administrative privileges are granted via the admin token mechanism (the –os-
token option of the keystone command). The admin token allows initialisation of the identity database before the
admin role exists.
The alias construct exists for convenience. An alias is short name for a complex or hard to understand rule. It is
defined in the same way as a policy:
alias name : alias definition
Once an alias is defined, use the rule keyword to use it in a policy rule.

3.19.2 Examples
A simple rule might be the following:
"compute:get_all" : ""
The target is "compute:get_all", the "list all instances" API of the Compute service. The rule is an empty string
meaning "always". This policy allows anybody to list instances.
Permission to use an API can also be declined:
"compute:shelve": "!"

51
The exclamation mark stands for "never" or "nobody", which effectively disables the Compute API "shelve an in-
stance".
Many APIs can only be called by admin users. This can be expressed by the rule "role:admin". The following policy
ensures that only administrators can create new users in the Identity database:
"identity:create_user" : "role:admin"
An API can be limited to any role, for example, an action cabe allowed to be used by any user except the user belonging
to an specific role:
"stacks:create": "not role:heat_stack_user"
This rule makes use of the boolean operator not. More complex rules can be built using operators and, or and paren-
theses.
The aliases help to assing a rule or a combination of rules to several action by only using the name of that aliase.
"deny_stack_user": "not role:heat_stack_user"
The policy engine understands that "deny_stack_user" is not an API and consequently interprets it as an alias. The use
of that alise can be written the following way:
"stacks:create": "rule:deny_stack_user"
As it has been explained in section 3.19.1, rules can compare API attributes to object attributes:
"compute:start" : "user_id:%(user_id)s"
This rule states that only the owner of an instance can start it up. The user_id string before the colon is an API attribute,
namely the user ID of the API user. It is compared with the user ID of the object. More precisely, it is compared with
the user_id field of that object in the database. If the two values are equal, permission is granted.
An admin user always has permission to call APIs. THe following example shows how it is set:

"admin_required": "role:admin or is_admin:1",

"owner" : "user_id:%(user_id)s",
"admin_or_owner": "rule:admi_required or rule:owner",
"identity:change_password": "rule:admin_or_owner"

• The first line defines an alias for "user is an admin user". The is_admin flag is only used when setting up the
Identity service for the first time. It indicates that the user has admin privileges granted by the service token.
• The second line creates an alias for "user owns the object" by comparing the API’s user ID with the object’s
user ID.
• The third line defines a third alias admin_or_owner, combining the two first aliases with the Boolean operator
or.
• The forth line sets up the policy that a password can only be modified by its owner or an admin user.

The rule can be more complex by using a combination of several boolean operators and parentheses:
"identity:ec2_delete_credential":"rule:admin_required or
(rule:owner and user_id:%(target.credential.user_id)s)"
In this case,boolean operators and parentheses combine three simpler rules. admin_required and owner are the same
aliases as in the previous example. user_id:%(target.credential.user_id)s compares the API user with the user ID of
the credential object associated with the target.

52
Chapter 4

Project deployment pre-configuration

In this part, it is displayed the needed configuration of the system in order to deploy OpenStack properly.

4.1 Infrastructure
The base infrastructure used to deploy the software is a basic laptop. Its principal features are:

• RAM: 4GB

• Number of CPU’s: 8

• Processor: Intel core i7

However, OpenStack needs a minimum of three nodes to work as desired. This nodes are virtual machines that will
run over the physical host. The virtual infrastructure needed is the following:

• Virtual machines: Each node needed in the deployment will be a virtual machine. In this deployment, it has
been used three virtual machines, one for each node named controller, network and compute.

• Virtual networks: This networks will be used to connect the virtual machines. Two networks have been defined
in this deployment, one for management purpose and the other for internal information exchange between nodes.

4.1.1 Virtual Machines

All virtual machines run over KVM hypervisor and have installed a plain Ubuntu 14.04 Thrusty Tair as base image.
The technical features for each instance are the following:

• Number of CPU’s: 2

• RAM: 1GB

• Storage space: 8GB

The number of interfaces in each node will vary depending on the node.

53
4.1.2 Virtual Networks
Basically, these virtual network are virtual bridges that connect the host with the nodes and between them.
As it has been referenced in 4.1 section, there will be two virtual networks defined:

• Management network: This network will have the 192.168.122.0/24 range. It will be used for the management
exchange between the nodes. This management exchange consists of all the data which will be sent from the
controller node to the compute and network nodes in order to execute functionalities.
• Internal network: This network will have the 192.168.123.0/24 range.This network will be an isolated virtual
network, so there won’t be any connection with an external host. The purpose of this network is to provide the
exchange of the created instances data, emulating a private network for them.

4.2 Architecture
In this deployment the architecture of the system, which can be seen in figure 4.1 will be based on three nodes that
will include all the modules needed for every functionality desired. These nodes are the base of any system that works
with OpenStack due to the fact that they gather the main services.
As it has been said before, these nodes will be named Controller, Compute and Network. The following picture shows
the architecture:

4.2.1 Controller node

The controller node runs the Identity service, Image service, management portions of Compute and Networking service
and the Networking plug-in which are the main services for an OpenStack system.
It also contains an SQL database, a message queue and Network Time Protocol (NTP) in order to sinchronize all the
nodes.

4.2.2 Compute node

The compute node runs the hypervisor portion of the Compute service that operates virtual machines or instances. It
is possible to use different hypervisors, but KVM is the default option. This node also runs the Networking plug-in
and an agent that connect tenant networks to instances and provide firewalling.
Due to the fact that Compute node needs lots of resources, it is possible to have more than one compute node in order
to satisfy all the tenant requirements.

4.2.3 Network node

The network node runs the Networking plug-in and several agents that provide tenant networks and provide switch-
ing, NAT, DHCP and routing services. This node also handles the external connectivity for tenant virtual machines
instances.

4.3 Networking configuration

Before installing the modules, the networking configuration has to be set up in each node as it is showed in figure 4.2.
This part is necessary in order to ensure the connectivity between the nodes and to the internet.

54
 Controller node Compute node Network node

KEYSTONE
NOVA
NOVA
NEUTRON
NEUTRON
NEUTRON

GLANCE

Figure 4.1: Node-service architecture

4.3.1 Controller node

The Controller node will only have one interface that will be configured in the /etc/network/interfaces file:

• eth0: the IP address is set to 192.168.122.11/24 and the default gateway to 192.168.122.1. This interface is
connected to the management network that will exchange all the data with the Compute and the Network nodes.

After configuring the networking, the hostname have to be set to controller by editing the /etc/hostname file by setting
controller as the node name.

4.3.2 Network node

The Network node will have three interfaces that will be configured in the /etc/network/interfaces file:

• eth0: the IP address is set to 192.168.122.21/24 and the default gateway to 192.168.122.1. This interface is
connected to the management network that will exchange all the data with the Controller and the management
data of the Compute node.
• eth1: the IP address is set to 192.168.123.21/24 and the default gateway to 192.168.123.1. This interface is
connected to the internal network and only exchanges data with the Compute node.
• eth2: It set with an special configuration without an IP assigned to it. It is configured as the external network
that enables connectivity to the internet.

After configuring the networking, the hostname have to be set to network by editing the /etc/hostname file by setting
network as the node name.

4.3.3 Compute node

The Compute node will have two interfaces that will be configured in the /etc/network/interfaces file:

• eth0: the IP address is set to 192.168.122.31/24 and the default gateway to 192.168.122.1. his interface is
connected to the management network that will exchange all the data with the Controller and the management
data of the Network node.

55
 • eth1: the IP address is set to 192.168.123.31/24 and the default gateway to 192.168.123.1.This interface is
connected to the internal network and only exchanges data with the Compute node.

After configuring the networking, the hostname have to be set to compute by editing the /etc/hostname file by setting
compute as the node name.

192.168.122.0/24

Controller node Compute node Network node

eth0
eth0 192.168.122.21
eth0 192.168.122.31 eth1
192.168.122.11 192.168.123.21
eth1
192.168.123.31
eth2

192.168.123.0/24

Internet

Figure 4.2: Network architecture

4.3.4 Name resolution

After configuring the networking and setting the hostnames, it also has to be edited the /etc/hosts file of each host to
configure the name resolution of the other ones by adding the following lines:

• 127.0.0.1 localhost

• 192.168.122.11 controller

• 192.168.122.21 network

• 192.168.122.31 compute

4.3.5 Connectivity verification

To verify that the network configuration is working properly, the connection is tried with the ping command from
every node to the others. The results of the verification are shown in the figures 4.3, 4.4 and 4.5:
As it can be seen all the nodes are reachable from every node and the external connection of the network node is
working.
At this point each node can exchange data with the others without any trouble.

56
 Figure 4.3: Ping from Controller node

Figure 4.4: Ping from Compute node

Figure 4.5: Ping from Network node

4.4 Network Time Protocol (NTP) configuration

In order to synchronize all the nodes NTP has to be installed in every node. Network and Compute nodes will
reference to Controller node and this will reference to a more accurate server. This configuration provides a higher
synchronization between Controller, Compute and Network.
The first node to configure will be the Controller as it has to be the reference to the other ones. To install NTP service

57
the following command has to be executed:
# apt-get install ntp
To configure the service the /etc/ntp.conf file has to be edited in order to specify the server that will be the reference
for Controller node. In Controller node’s the NTP server that has been used are the following:

• 0.ubuntu.pool.ntp.org

• 1.ubuntu.pool.ntp.org

• 2.ubuntu.pool.ntp.org

• 3.ubuntu.pool.ntp.org

This servers are from the ntp pool project which offer a reliable NTP service and are used in the main Linux based
distributions.
Before starting the Compute and Network node’s configuration, the Controller’s NTP service has to be restarted with
the # service ntp restart.
The NTP service configuration for Compute and Network nodes are the same, as both of them are referencing to
Controller node. In this case the NTP service is installed using the previous command (# apt-get install ntp).
The reference server configured in the NTP configuration file will be the Controller node and will be set with the
following line in both nodes:
server Controller iburst
As in Controller node, the services have to be restarted before verifying that the configuration is working properly.

4.4.1 NTP service verification

In first place, the Controller has to be associated to a minimum of one NTP server. To check it, we use the following
commands:
# ntpq -c peers # ntpq -c assoc
In figure 4.6 we see the servers from which Controller is actually polling information. In this case, Controller is
available to have information from all the configured servers but the symbol indicates that it is having information
from the server dnscache-madrid. That is also shown in figure 4.7, in which we see that the server has the condition of
system peer.

Figure 4.6: Controller peer command

To check the NTP service on the two other nodes, it is only necessary to check that the nodes are associated with the
Controller node with the following command:
# ntpq -c assoc
As it can be seen in figures 4.8 and 4.9, the symbol that appears in the output indicates that the nodes are associated
with Controller node.

58
 Figure 4.7: Controller assoc command

Figure 4.8: Computer assoc command

Figure 4.9: Network assoc command

4.5 OpenStack repository

Distributions release OpenStack packages as part of the distribution or using other methods because of differing
release schedules. This differences between releases can impact on the performance of the environment. In this case
the release installed has been the Juno release using the following actions:
# apt-get install ubuntu-cloud-keyring
echo "deb http://ubuntu-cloud.archive.canonical.com/ubuntu" "trusty-updates/juno
main" > /etc/apt/sources.list.d/cloudarchive-juno.list
After installing the Juno repository it has been needed to update the system’s packages to implement the new one:
# apt-get update && apt-get dist-upgrade
This action has to be performed in the three node in order to install the same release in each of them.

4.6 SQL Database

Most OpenStack services use an SQL database to store information. The database only runs on the Controller node as
it manages all the services. In this case the database installed is MariaDB 5.5.41.
To install it, the following command was used:
# apt-get mariadb-server python-mysqldb
As the other nodes have to access to the database through the management network there is the need to set a bind
address in the sql service configuration file in order to enable it. The edited file is /etc/mysql/my.conf and to enable
the bind address it has been added the following line:
bind-address = 192.168.122.11
Besides this key, it is also needed to set other keys to set options automatically when a OpenStack service make a

59
change in the database information:

• default-storage-engine=innodb: Sets the innodb as the storage engine. This storage engine is transactional,
which ensures that an operation on the database can be rolled back if an error occurs during the process.
• innodb_file_per_table: It will be created one file for each table, which give a better capacity of management.

• collation-server=utf8_general_ci: Sets the general utf8 rules for comparing characters.

• init-connect=’SET NAMES utf8’: Indicates that each time a client connects to the server it will use the utf8
character set,
• character-set-server=utf8: Sets the utf8 encoding character system.

These options ensure a proper behaviour of the database. In order to make these changes operative the server has to
be restarted using the # service mysql restart command.

4.7 Messaging server

OpenStack system runs several services/applications which share information. Therefore, there is a need to connect
these applications and to coordinate and manage the operations among them. To do so, it is installed the RabbitMQ
Message Broker service software.

4.7.1 Messaging server configuration

To configure RabbitMQ messaging server, the following packets have been installed:
# apt-get install rabbitmq-server
It is created a default account that uses g̈uestäs username and password. This account will be used in OpenStack
services, but the password has to be changed in order to secure the account with the following command:
# rabbitmqctl change-password guest RABBITMQKEY
Finally, to enable the changes and finalize the system’s configuration the RabbitMQ message broker service has to be
restarted with the # service rabbitmq-server restart command.

60
Chapter 5

Identity service: Keystone

5.1 Identity database

As every service in OpenStack, to store and retrieve its information there’s the need of creating a database.
To create the Identity service database, it is needed to be connected to the database server running on Controller node.
The connection is made as a root user with the following command:
mysql -u root -p
The database’s name will be keystone as it is the name of the service and all the privileges will be granted for the
access to the database from the localhost or any external host. The password to access to the keystone’s database will
be ’KEYSTONEKEY’. The following commands are the necessary to be executed in the database server in order to
grant the features described before:
CREATE DATABASE keystone;
GRANT ALL PRIVILEGES ON keystone.* TO ’keystone@’localhost’ IDENTIFIED BY ’KEYSTONEKEY’;
GRANT ALL PRIVILEGES ON keystone.* TO ’keystone@’%’ IDENTIFIED BY ’KEYSTONEKEY’;

5.2 Service installation and configuration

5.2.1 Required modules

To run Keystone service the following modules have to be installed:

• keystone: This module contains the daemons which control keystone service.
• python-keystoneclient: This module contains the client library for the OpenStack Keystone API.

To install them the following command has to be executed:

# apt-get install keystone python-keystoneclient

5.2.2 Service configuration

The file that configures the behaviour of Keystone is the keystone.conf which is stored in /etc/keystone/keystone.conf.
This file is distributed in several sections [nameofthesection] in order to sort the configuration keys.

61
In the DEFAULT section there have been defined the following keys:

• admin_token=KEYSTONEKEY: This configures the value of the initial administration token that is used
during the initial configuration of the system.
• verbose=True: This key enables the verbose logging which gives more information about a possible error.

In the database section there are defined the keys regarding the connection between the Identity service and the Key-
stone database. The keys defined to configure the database connection are the following:

• connection=mysql://keystone:KEYSTONEKEY@controller/keystone: This keys configures the access to

the database created previously in which will be stored the service’s information.

In the token section there are described the token’s configuration options. The keys defined in this section are the
following:

• provider=keystone.token.providers.uuid.Provider: This keys controls the token construction, validation, and

revocation operations.
• driver=keystone.token.backends.sql.Token: This keys configures the token persistence backend driver.

In the revoke section there is defined the SQL token revocation driver:

• driver=keystone.contrib.revoke.backends.sql.Revoke: This key configures an implementation of the backend

for persisting revocation events.

To finalize the configuration, the database configured in the configuration file has been populated using the following
command:
su -s /bin/sh -c k̈eystone-manage db_synck̈eystone
Once the database is populated the service has to be restarted in order to enable all the configured settings.
In addition, the file SQLite file stored in /var/lib/keystone/keystone.db is removed as it is being used the database
server.

5.3 Tenants, Users and Roles creation

It is able to create several environments for different purposes. In this case it has been created an administrative
environment for administration operations and a demo environment for typical operations.

5.3.1 Environment variables

To create tenants, users and roles for a particular environment it is needed to use the temporary token enabled in
Keystone configuration file and configure the location of the identity service.
The administration token and the endpoint of the Identity service will be configured by exporting them as environment
variables with the following commands:

• export OS_SERVICE_TOKEN=KEYSTONEKEY
• export OS_SERVICE_ENDPOINT=http://controller:35357/v2.0

62
5.3.2 Administrative profile creation

Tenant creation

The following command is executed to create the administration tenant:

keystone tenant-create --name admin --description "Admin Tenant"
The figure 5.1 shows the output of the created tenant:

Figure 5.1: admin tenant

User creation

The following command is executed to create the administration user:

keystone user-create --name admin --pass ADMIN_PASS --email [email protected]
The figure 5.2 shows the output of the created demo:

Figure 5.2: admin user

Role creation

The following command is executed to create the administration role:

keystone role-create --name admin
The figure 5.3 shows the output of the created role:
Once the role is created, the admin user has to be added to the administration role to be its member:
keystone user-role-add --user admin --tenant admin --role admin

63
 Figure 5.3: admin role

5.3.3 UPC profile creation

Tenant creation

The following command is executed to create the upc tenant:

keystone tenant-create --name upc --description "UPC Tenant"
The figure 5.4 shows the output of the created tenant:

Figure 5.4: upc tenant

User creation

The following command is executed to create the demo user:

keystone user-create --name upcuser --pass UPCUSER --email [email protected]
The figure 5.5 shows the output of the created demo:

Figure 5.5: upc user

As the user upcuser wouldn’t have the permission to execute any administration commands, it is not added to any role.

64
5.3.4 Service tenant creation

Each OpenStack service needs a tenant, user and role to interact with the other services. Therefore, it is required
to create one or more users with the admin role under the service tenant that has been created with the following
command:
keystone tenant-create --name service --description "Service Tenant"

Figure 5.6: service tenant

5.4 Service entity and API Endpoint creation

5.4.1 Service entity’s catalog

The Identity service manages a catalog of services running in an individual OpenStack environment. This catalog is
used by all the services in the environment in order to locate the other services and exchange data among them.
Every time a service is installed in the OpenStack environment, the service entity has to be created in the catalog with
the following command:
keystone service-create --name keystone --type identity --description
"OpenStack Identity"

Figure 5.7: keystone service

5.4.2 API Endpoint’s catalog

The Identity service also manages a catalog of API endpoints associated with the services running in an OpenStack
environment. This catalog is used by the services to determine how to communicate with other services.
OpenStack provides three API endpoint variation for each service which are admin, internal and public. In this
deployment all the variations use the management network, but it is possible to place the variations on separated
networks for security reasons.
The following command have to be executed to create the Identity service endpoint:

65
keystone endpoint-create --service-id $(keystone service-list | awk ’/
identity / print $2 --publicurl http://controller:5000/v2.0 --internalurl
http://controller:5000/v2.0 --adminurl http://controller:35357/v2.0 --region
regionOne

5.5 Verify operation

To execute keystone commands simulating the profiles created previously and verify the proper operation of Keystone
module, the environment variables exported in the section 5.3.1 have to be disabled using the following command:
$ unset OS_SERVICE_TOKEN OS_SERVICE_ENDPOINT

5.5.1 Admin user verification

As it has been configured in the policy.json file, the admin user can execute any command. Therefore, the admin user
should be able to list all the created tenants to verify the proper behaviour:
$ keystone --os-tenant-name admin --os-username admin --os-password
’ADMIN_PASS’ --os-auth-url http://controller:35357/v2.0 tenant-list
As the environment variables have been disabled, it is needed to include the tenant, username, password and endpoint
in the keystone command in order to identify who is requesting the information.
The figure 5.8 shows that the admin profile is able to list all the existing tenants:

Figure 5.8: admin verification

5.5.2 Demo user verification

In the case of the demo environment, the demo user can’t execute admin-only commands. Therefore, when as the demo
user tries to list all the created users with the following command, the Identity service returns a message indicating
that the user isn’t authorized to execute the command:
$ keystone --os-tenant-name demo --os-username demo --os-password ’DEMO_PASS’
--os-auth-url http://controller:35357/v2.0 tenant-list
The figure 5.9 shows the output message:

Figure 5.9: upc verification

66
During the verification of these operations, there was an error that prevented the proper execution of any command.
The resolution of this issue is explained in section 13.1.

5.6 Client environment scripts

To increase the efficiency of clients operations and to set the required environment variables for the OpenStack
command-lines it is needed to create an environment file. This files have the name PROJECT-openrc.sh.
This type of files have to mandatorily contain the following environment variables:

• OS_TENANT_NAME
• OS_USERNAME
• OS_PASSWORD
• OS_AUTH_URL

As in this deployment there are two clients (admin and demo) there is the need to create one file for each client.

5.6.1 admin-openrc.sh file

export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=ADMIN_PASS
export OS_AUTH_URL=http://controller:35357/v2.0

5.6.2 demo-openrc.sh file

export OS_TENANT_NAME=demo
export OS_USERNAME=demo
export OS_PASSWORD=DEMO_PASS
export OS_AUTH_URL=http://controller:5000/v2.0

67
68
Chapter 6

Image service: Glance

6.1 Image database

For the Image service, there is the need of a database to store disk or server images that will be used when OpenStack
creates an instance.
The steps to create the Image database are very similar as the ones used when creating the Identity database once
connected to the database server in the controller node as root user.
The database’s name will be glance and all the privileges will be granted to access to this database from the localhost
or any other host. The password to access to the keystone’s database will be ’GLANCEKEY’. The commands used
were the following:
CREATE DATABASE glance;
GRANT ALL PRIVILEGES ON keystone.* TO ’glance@’localhost’ IDENTIFIED BY ’GLANCEKEY’;
GRANT ALL PRIVILEGES ON keystone.* TO ’glance@’%’ IDENTIFIED BY ’GLANCEKEY’;

6.2 Image service configuration

6.2.1 Credentials creation

Every OpenStack service needs its own admin role credentials under the service tenant to interact with the other
services. To create the Image service credentials we need to source the admin-openrc file created during Keystone
configuration to gain access to admin-only command.
The following command is executed to create the Image service user:
$ keystone user-create --name glance --password GLANCE_PASS
The figure 6.1 show the output of the command:
At this point glance user isn’t added to the admin role nor to the service tenant in order to be able to communicate with
other services. It is needed to add this user to the service tenant and the admin role to be fully configured:
$ keystone user-role-add --user glance --tenant service --role admin

69
 Figure 6.1: glance user

6.2.2 Image service entity and API Endpoint creation

As it was done with the Identity service previously, the Image service entity has to be created to be located by the other
services running in the OpenStack environment. It has to be the same way with its API endpoint in order to enable the
running services to know how to communicate with it.
The commands used are the following:
$ keystone service-create --name glance --type image --description “OpenStack Image
service”
$keystone keystone endpoint-create --service-id $(keystone service-list |
awk ’/ image / print $2’) --publicurl http://controller:9292 --internalurl
http://controller:9292 --adminurl http://controller:9292 --region regionOne

6.3 Service installation and configuration

6.3.1 Required modules

To run Glance service the following modules have to be installed in Controller node:

• glance: This module contains the daemons which control glance service.

• python-glanceclient: This module contains the client library for the OpenStack Glance API.

To download and install these modules the next command has to be executed:
# apt-get install glance python-glanceclient

6.3.2 Service configuration

The behaviour of Glance is configured by two file named glance-api.conf and glance-registry.conf stored in /etc/glance/
directory. As all the OpenStack configuration files, these are also distributed in several sections [nameofthesection]
in order to sort the configuration options.
The first configured file is glance-api.conf which is responsible for accepting Image API calls for image discovery,
retrieval, and storage.
In the DEFAULT section there have been defined these options:

• verbose=True: It enables the verbose logging which gives more information about a possible error.

70
 • default_store=file: It is the default scheme to use to store image data.
• bind_host=0.0.0.0: It sets controller node as the API server.
• bind_port=9292: It binds the port 9292 to the API server.
• log_file=/var/log/glance/api.log: It sets the api.log file as the logging registry.
• notification_driver=noop: It disables the the options to set notifications as it is an option of another OpenStack
service.

In the database section there have been defined these options:

• connection=mysql://glance:GLANCEKEY@controller/glance: This option sets the database access to the

Image service database created previously.

In the keystone_authtoken section there have been defined these options:

• auth_uri=http://controller:5000/v2.0: It sets the complete public Identity API endpoint.

• identity_uri=http://controller:35357: It sets the complete admin Identity API endpoint.
• admin_tenant_name=service: It sets service as the Glance admin tenant.
• admin_user=glance: It sets glance as the Glance admin user.
• admin_password=GLANCE_PASS: It sets the Glance user’s password in order to authenticate to Keystone.

In the glance_store section there have been defined these options:

• filesystem_store_file=/var/lib/glance/images/: This directory is used by Filesystem backend store to write

image data. In the default_store option, it is set the filesystem backend store to be a plain file.

The second configured file is glance-registry.conf which is responsible for storing, processing, and retrieving metadata
about images where metadata can be any information about the images. It has similar options as glance-api.conf file
because both services has to behaviour similarly.
In the DEFAULT section there have been defined these options:

• verbose=True: It enables the verbose logging which gives more information about a possible error.
• bind_host=0.0.0.0: It sets controller node as the API server.
• bind_port=9191: It binds the port 9191 to the API server.
• notification_driver=noop: It disables the the options to set notifications as it is an option of another OpenStack
service.

In the database section there have been defined these options:

• connection=mysql://glance:GLANCEKEY@controller/glance: This option sets the database access to the

Image service database created previously.

In the keystone_authtoken section there have been defined these options:

71
 • auth_uri=http://controller:5000/v2.0: It sets the complete public Identity API endpoint.
• identity_uri=http://controller:35357: It sets the complete admin Identity API endpoint.
• admin_tenant_name=service: It sets service as the Glance admin tenant.
• admin_user=glance: It sets glance as the Glance admin user.

• admin_password=GLANCE_PASS: It sets the Glance user’s password in order to authenticate to Keystone.

To finalize the configuration, the database configured in the configuration file has been populated using the following
command:
su -s /bin/sh -c "glance-manage db_sync" glance
Once the database is populated both glance-api and glance-registry services have to be restarted in order to enable all
the configured settings.
In addition, the file SQLite file stored in /var/lib/glance/glance.db is removed as it is being used the database server.

6.4 Verify operation

To verify that the Image service has been configured properly and it can executes the commands correctly, it has been
created an image into Glance with a downloaded image and then it has been listed to verify that the image was created
properly.
All these operations have to be made with admin credentials in order to be able to execute admin-only commands by
sourcing the admin openrc file.
The downloaded image with be stored in a temporal directory that is created mkdir command. In this case the
directory created is /tmp/images and the downloaded image is cirros-0.3.3-x86_64-disk.img.
With the following glance command is created the image in the Image service:
$ glance image-create --name "cirros-0.3.3-x86_64" --file
/tmp/images/cirros-0.3.3-x86_64-disk.img --disk-format qcow2
--container-format bare --is-public True --progress
To verify that the creation has worked properly the images created are listed and it should appear the created previously:
$ glance image-list

Figure 6.2: image list

As it can be seen in figure 6.2, the cirros-0.3.3-x86_64 image has been created correctly. Therefore, the temporal
directory can be erased.

72
Chapter 7

Compute service: Nova

7.1 Compute database

For OpenStack Compute, there is the need of a database to store available instance types, instances in use, available
networks and projects.
Once connected to the database server as root user, the process is the same followed when configuring the Glance
database but only changing the name and password of the created database.
The database’s name will be nova and all the privileges will be granted to access to this database from the localhost or
any other host. The password to access to the keystone’s database will be ’NOVAKEY’. The commands used were the
following:
CREATE DATABASE nova;
GRANT ALL PRIVILEGES ON nova.* TO ’nova@’localhost’ IDENTIFIED BY ’NOVAKEY’;
GRANT ALL PRIVILEGES ON nova.* TO ’nova@’%’ IDENTIFIED BY ’NOVAKEY’;

7.2 Nova service configuration

7.2.1 Credentials creation

In order to interact with the other OpenStack services there is the need to create admin role credentials under the
service tenant for Compute. To create these credentials it is needed to source the admin-openrc file to gain access to
admin-only commands.
The following command is executed to create the Nova service user:
$ keystone user-create --name nova --password NOVA_PASS
The figure 7.1 shows the output of the command:
To finalize the setting of the credentials the user is added to the admin role and to tenant service:
$ keystone user-role-add --user nova --tenant service --role admin

73
 Figure 7.1: nova user

7.2.2 Nova service entity and API Endpoint creation

The Nova service entity has to be created to be located by the other services running in the OpenStack environment.
It has to be the same way with its API endpoint in order to enable the running services to know how to communicate
with it.
The commands used are the following:
$ keystone service-create --name nova --type compute --description “OpenStack Compute”
$ keystone endpoint-create --service-id $(keystone service-list | awk ’/
compute / print $2’) --publicurl http://controller:8774/v2/%(tenant_id)s
--internalurl http://controller:8774/v2/%(tenant_id)s --adminurl
http://controller:8774/v2/%(tenant_id)s --region regionOne

7.3 Service installation and configuration for Controller node

7.3.1 Required modules

To run Compute the following modules have to be installed in Controller node:

• nova-api service: This service accepts and responds to end user compute API calls. The service supports
the OpenStack Compute API, the Amazon EC2 API, and a special Admin API for privileged users to perform
administrative actions. It enforces some policies and initiates most orchestration activities, such as running an
instance.

• nova-cert service: This service accepts metadata requests from instances. The nova-api-metadata service is
generally used when you run in multi-host mode with nova-network installations.

• nova-conductor service: This service mediates interactions between the nova-compute service and the database.
It eliminates direct accesses to the cloud database made by the nova-compute service.

• nova-consoleauth daemon: This daemon authorizes tokens for users that console proxies provide.

• nova-novncproxy daemon: This daemon provides a proxy for accessing running instances through a VNC
connection. Supports browser-based novnc clients.

• nova-scheduler service: This service takes a virtual machine instance request from the queue and determines
on which compute server host it runs.

• python-novaclient: This module contains the client library for the OpenStack Compute API.

74
To download and install these modules the next command has to be executed:
# apt-get install nova-api nova-cert nova-conductor nova-consoleauth
nova-novncproxy nova-scheduler python-novaclient

7.3.2 Service configuration

The file that configures the behaviour of Nova is the nova.conf which is stored in /etc/nova/nova.conf. This file is
distributed in several sections [nameofthesection] in order to sort the configuration options.
In the DEFAULT section there have been defined these options:

• verbose=True: It enables the verbose logging which gives more information about a possible error.
• rpc_backend=rabbit: It sets the RabbitMQ as messaging system.
• rabbit_host=controller: It sets controller as the RabbitMQ broker host
• rabbit_password=RABBITMQKEY: It sets the configured rabbitMQ password in order to grant authentica-
tion to the service.
• auth_strategy = keystone: It sets Keystone service as the authentication strategy.
• my_ip=192.168.122.11: It forces to use the management interface of the controller node.
• vncserver_listen=192.168.122.11: It sets the management interface as the IP address on which instance vnc-
servers should listen.
• vncserver_proxyclient_address=192.168.122.11: It sets the management interface as the address to which
proxy clients should connect.

In the database section there have been defined these options:

• connection=mysql://nova:NOVAKEY@controller/nova: This option sets the database access to the Nova

service database created previously.

In the keystone_authtoken section there have been defined these options:

• auth_uri=http://controller:5000/v2.0: It sets the complete public Identity API endpoint.

• identity_uri=http://controller:35357: It sets the complete admin Identity API endpoint.
• admin_tenant_name=service: It sets service as the Nova admin tenant.
• admin_user=nova: It sets nova as the Nova admin user.
• admin_password=NOVA_PASS: It sets the Nova user’s password in order to authenticate to Keystone.

In the glance section there have been defined these options:

• host=controller: It sets controller node as the location of Glance service.

To finalize the configuration, the database has been populated using the following command:
# su -s /bin/sh -c "nova-manage db sync" nova
Once the database is populated, all the services installed have to be restarted in order to enable all the configured
settings. These services are:

75
 • nova-api
• nova-cert
• nova-consoleauth
• nova-scheduler
• nova-conductor
• nova-novncproxy

In addition, the file SQLite file stored in /var/lib/nova/nova.db is removed as it is being used the database server.

7.4 Service installation and configuration for Compute node

7.4.1 Required modules

To run Compute the following modules have to be installed in Compute node:

• nova-compute service: This service is a worker daemon that creates and terminates virtual machine instances
through hypervisor APIs. The compatible hypervisors APIs are XenAPIfor XenServer/XCP, libvirt for QEMU
or KVM which is the default option and VMwareAPI for VMware.
• sysfsutils: This module sysfs query tool and boot-time setup.

To download and install these modules the next command has to be executed:
# apt-get install nova-compute sysfsutils

7.4.2 Service configuration

For the compute node, the file that configures the behaviour of Nova is also named nova.conf and it is also stored in
/etc/nova/nova.conf. This file is distributed in several sections [nameofthesection] in order to sort the configuration
options.
In the DEFAULT section there have been defined these options:

• verbose=True: It enables the verbose logging which gives more information about a possible error.
• rpc_backend=rabbit: It sets the RabbitMQ as messaging system.
• rabbit_host=controller: It sets controller as the RabbitMQ broker host
• rabbit_password=RABBITMQKEY: It sets the configured rabbitMQ password in order to grant authentica-
tion to the service.
• auth_strategy = keystone: It sets Keystone service as the authentication strategy.
• my_ip=192.168.122.31: It forces to use the management interface of the compute node in order to match the
network configured in this option for Nova service in controller node.
• vnc_enabled=True: This options enable VNC related features.
• vncserver_listen=0.0.0.0: It sets the vnc server component to listen to all IP addresses.

76
 • vncserver_proxyclient_address=192.168.122.31: It sets the management interface as the address to which
proxy component should listen.
• novncproxy_base_url=http://controller:6080/vnc_auto.html: This option sets the URL that indicates the lo-
cation where it can be used a web browser to access remote consoles of instances on the compute node.

In the keystone_authtoken section there have been defined these options:

• auth_uri=http://controller:5000/v2.0: It sets the complete public Identity API endpoint.

• identity_uri=http://controller:35357: It sets the complete admin Identity API endpoint.
• admin_tenant_name=service: It sets service as the Nova admin tenant.
• admin_user=nova: It sets nova as the Nova admin user.

• admin_password=NOVA_PASS: It sets the Nova user’s password in order to authenticate to Keystone.

In the glance section there have been defined these options:

• host=controller: It sets controller node as the location of Glance service.

To determine which hypervisor is supported in the compute node, it is needed to determine if hardware acceleration
for virtual machines is supported:
$ egrep -c ’(vmx|svm)’ /proc/cpuinfo
As the output is 0, it means that hardware acceleration is not supported, therefore it has to be specified in /etc/nova/nova-
compute.conf the type of hypervisor that have to be used:

• virt_type=qemu

Once the database is populated, the nova-compute service has to be restarted in order to enable all the configured
settings.
For compute node, the SQLite file stored in /var/lib/nova/nova.db is also removed as it is being used the database
server in controller node.

7.5 Verify operation

To verify that the Nova service has been configured properly and it can executes the commands correctly, it has been
listed the nova service components in order to verify that every process is launched successfully.
This operation has to be launched from the controller node with admin credentials by sourcing the admin openrc file.
The following Nova command lists the launched service components:
$ nova service-list
While verifying the launching of the services, there were several error that prevented their starting. The solving of
these errors are explained in the section 13.2.

77
Figure 7.2: service list

78
Chapter 8

Networking component: Neutron

8.1 Neutron database

For OpenStack Compute, there is the need of a database to store available networks subnetworks and networking
devices.
Once connected to the database server as root user, the process is the same followed when configuring the previous
databases but only changing the name and password of the created database.
The database’s name will be neutron and all the privileges will be granted to access to this database from the localhost
or any other host. The password to access to the keystone’s database will be ’NEUTRONKEY’. The commands used
were the following:
CREATE DATABASE neutron;
GRANT ALL PRIVILEGES ON neutron.* TO ’neutron@’localhost’ IDENTIFIED BY ’NEUTRONKEY’;
GRANT ALL PRIVILEGES ON neutron.* TO ’neutron@’%’ IDENTIFIED BY ’NEUTRONKEY’;

8.2 Neutron service configuration

8.2.1 Credentials creation

In order to interact with the other OpenStack services there is the need to create admin role credentials under the
service tenant for Neutron. To create these credentials it is needed to source the admin-openrc file to gain access to
admin-only commands.
The following command is executed to create the Neutron service user:
$ keystone user-create --name neutron --password NEUTRON_PASS
The figure 8.1 is the output of the command:
To finalize the setting of the credentials the user is added to the admin role and to tenant service:
$ keystone user-role-add --user neutron --tenant service --role admin

79
 Figure 8.1: neutron user

8.2.2 Neutron service entity and API Endpoint creation

The Neutron service entity has to be created to be located by the other services running in the OpenStack environment.
It has to be the same way with its API endpoint in order to enable the running services to know how to communicate
with it.
The commands used are the following:
$ keystone service-create --name neutron --type network --description “OpenStack
Network”
$ keystone endpoint-create --service-id $(keystone service-list | awk ’/
network / print $2’) --publicurl http://controller:9696 --internalurl
http://controller:9696 --adminurl http://controller:9696 --region regionOne

8.3 Service installation and configuration for Controller node

8.3.1 Required modules

To run Neutron, the following modules have to be installed in Controller node:

• neutron-server: This module accepts and routes API requests to the appropriate OpenStack Networking plug-in
for action.

• neutron-plugin-ml2: This plugin is a framework allowing OpenStack Networking to simultaneously utilize the
variety of layer 2 networking technologies found in complex real-world data centers.

• python-neutronclient: This module contains the client library for the OpenStack Neutron API.

To download and install these modules the next command has to be executed:
# apt-get install neutron-server neutron-plugin-ml2 python-neutronclient

8.3.2 Service configuration

The behaviour of Neutron is configured by two file named neutron.conf stored in /etc/neutron/ and ml2_conf.ini stored
in ml2_conf.ini. As all the OpenStack configuration files, these are also distributed in several sections [nameofthe-
section] in order to sort the configuration options.

80
neutron.conf

The first configured file is neutron.conf which is responsible for the configuration of the database, authentication
mechanism, message broker, topology change notifications, and plug-in.
In the DEFAULT section there have been defined these options:

• verbose=True: It enables the verbose logging which gives more information about a possible error.
• rpc_backend=rabbit: It sets the RabbitMQ as messaging system.
• rabbit_host=controller: It sets controller as the RabbitMQ broker host
• rabbit_password=RABBITMQKEY: It sets the configured rabbitMQ password in order to grant authentica-
tion to the service.
• auth_strategy = keystone: It sets Keystone service as the authentication strategy.
• core_plugin=ml2: It sets ml2 framework as the Neutron core plugin entrypoint to be loaded from the neu-
tron.core_plugins namespace.
• service_plugins=router: It sets router as the service plugin entrypoint to be loaded from the neutron.service_plugins
namespace.
• allow_overlapping_ips=True: It enables the capability to overlap IP’s for different subnets.
• notify_nova_on_port_status_changes=True: It enables to send notification to nova when port status changes.
• notify_nova_on_port_data_changes=True: It enables to send notification to nova when port data, such as
fixed ips or floating ip, changes so nova can update its cache.
• nova_url=http://controller:8774/v2: It sets the URL for connection to nova service.
• nova_admin_auth_url=http://controller:35357/v2.0: It sets the authorization URL for connecting to nova in
admin context.
• nova_region_name=regionOne: It sets the name of nova region to use. As the configuration in Keystone only
consider one region it is not very important.
• nova_admin_username=nova: It sets the username for connecting to nova in admin context.
• nova_admin_tenant_id=0d7e4b78d65244028fa88750828a3ee4: It sets the uuid of the admin nova tenant.
• nova_admin_password=NOVA_PASS: It sets the password for connection to nova in admin context.

In the database section there have been defined these options:

• connection=mysql://nova:NEUTRONKEY@controller/neutron: This option sets the database access to the

Neutron service database created previously.

In the keystone_authtoken section there have been defined these options:

• auth_uri=http://controller:5000/v2.0: It sets the complete public Identity API endpoint.

• identity_uri=http://controller:35357: It sets the complete admin Identity API endpoint.
• admin_tenant_name=service: It sets service as the Neutron admin tenant.
• admin_user=neutron: It sets nova as the Neutron admin user.
• admin_password=NEUTRON_PASS: It sets the Neutron user’s password in order to authenticate to Keystone.

81
ml2_conf.ini

The second configured file is ml2_conf.ini which configures the Open vSwitch (OVS) mechanism to build the virtual
networking framework for instances. In controller node there aren’t OVS components because it doesn’t handle
instance network traffic.
In the ml2 section there have been defined these options:

• type_drivers=flat,gre: It sets flat and GRE as type driver entrypoints to be loaded from the neutron.ml2.type_drivers
namespace.
• tenant_network_types=gre: It sets GRE network type as the network type to allocate as tenant networks.
• mechanism_drivers=openvswitch: It sets openvswitch as the networking mechanism driver entrypoint to be
loaded from the neutron.ml2.mechanism_drivers namespace.

In the ml2_type_gre section there have been defined these options:

• tunnel_id_ranges=1:1000: It sets from 1 to 1000 the possible GRE tunnel ID’s that are available for tenant
network allocation.

In the securitygroup section there have been defined these options:

• enable_security_group=True: It enables neutron security group

• enable_ipset=True: It enables to speed-up the iptables security groups.
• firewall_driver=neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriv e r: It sets the OVS
iptables firewall driver.

8.3.3 Networking Compute configuration

By default, Compute use legacy networking so it has to be configured in order to be able to manage networks through
Networking. To do so, it has been added neutron options to Compute’s nova.conf file.
In the DEFAULT section there have been added these options:

• network_api_class=nova.network.neutronv2.api.API: It sets the full class name of the network API class to
use.
• security_group_api=neutron: It sets the full class name of the security API class.
• linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver: It sets driver to create eth-
ernet devices. In this case it is used the openvswitch driver.
• firewall_driver=nova.virt.firewall.NoopFirewallDriver: It sets the default hypervisor firewall driver.

In the neutron section there have been added these options:

• url=http://controller:9696: It sets the URL for connecting to neutron.

• auth_strategy=keystone: It sets Keystone as authorization strategy for connecting to neutron in admin context.
• admin_auth_url=http://controller:35357/v2.0: It sets the authorization URL for connecting to neutron in
admin context.

82
 • admin_tenant_name=service: It sets “service” as the tenant name for connecting to neutron in admin context.

• admin_username=neutron: It sets “neutron” as username for connecting to neutron in admin context.

• admin_password=NEUTRON_PASS: It sets the password for connecting to neutron in admin context.

8.3.4 Finish the configuration

To finalize the configuration, the database has been populated in Controller node using the following command:
# su -s /bin/sh -c "neutron-db-manage --config-file /etc/neutron/neutron.conf
--config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade juno" neutron
Once the database is populated, all the services have to be restarted in order to enable all the configured settings. For
Controller node the service to restart is neutron-server, and for Compute node, as the nova configuration file has been
edited the services to restart are nova-api, nova-conductor and nova-scheduler.

8.3.5 Verify operation

To verify that the Neutron service has been configured properly and it can executes the commands correctly, it has
been listed neutron extensions in order to verify that neutron-server process is launched succesfully.
This operation has to be launched from the controller node with admin credentials by sourcing the admin openrc file.
The following Neutron command lists the available service extensions showed in the figure 8.2:
$ neutron ext-list

Figure 8.2: neutron extensions list

8.4 Service installation and configuration for Network node

The network node is responsible for handling internal and external routing and DHCP services for virtual networks.

83
8.4.1 Prerequisites configuration
As network node will be handling routing and DHCP service there is the need to configure some kernel networking
parameters:

• net.ipv4.ip_forward=1 This options enables IP forwarding which is needed as the Network node’s function is
routing.

• net.ipv4.conf.all.rp_filter=0 This option disables IP address checking in order to avoid invalid IP addresses.
The option is disabled because Network node performs routing functions, and for this cases this option has to
be disabled.

• net.ipv4.conf.default.rp_filter=0 This option has the same function as the previous option, so it also has to be
disabled.

To implement the changes the following command has to be executed:

# sysctl -p

8.4.2 Required modules

The modules that have to be installed in Network node are the following:

• neutron-plugin-openvswitch-agent: This agent installs the plugin for integrating openvswitch with neutron.

• neutron-plugin-ml2: This plugin is a framework allowing OpenStack Networking to simultaneously utilize the
variety of layer 2 networking technologies found in complex real-world data centers.

• neutron-l3-agent: This agent creates network devices that connect to l2 networks.

• neutron-dhcp-agent: This agent distributes IP addresses to the VMs using dnsmasq driver.

To download and install these modules the next command has to be executed:
# apt-get install neutron-plugin-ml2 neutron-plugin-openvswitch-agent
neutron-l3-agent neutron-dhcp-agent

8.4.3 Service configuration

The behaviour of Neutron in Network node is configured by these five files:

• neutron.conf stored in /etc/neutron/

• ml2_conf.ini stored in /etc/neutron/plugins/ml2/

• l3_agent.ini stored in /etc/neutron/

• dhcp_agent.ini stored in /etc/neutron/

• metadata_agent.ini stored in /etc/neutron/

As all the OpenStack configuration files, these are also distributed in several sections [nameofthesection] in order to
sort the configuration options.

84
neutron.conf

The first configured file is neutron.conf which is responsible for the configuration of the database, authentication
mechanism, message broker and plug-in.
In the DEFAULT section there have been defined these options:

• verbose=True: It enables the verbose logging which gives more information about a possible error.

• rpc_backend=rabbit: It sets the RabbitMQ as messaging system.

• rabbit_host=controller: It sets controller as the RabbitMQ broker host

• rabbit_password=RABBITMQKEY: It sets the configured rabbitMQ password in order to grant authentica-

tion to the service.

• auth_strategy = keystone: It sets Keystone service as the authentication strategy.

• core_plugin=ml2: It sets ml2 framework as the Neutron core plugin entrypoint to be loaded from the neu-
tron.core_plugins namespace.

• service_plugins=router: It sets router as the service plugin entrypoint to be loaded from the neutron.service_plugins
namespace.

• allow_overlapping_ips=True: It enables the capability to overlap IP’s for different subnets.

In the keystone_authtoken section there have been defined these options:

• auth_uri=http://controller:5000/v2.0: It sets the complete public Identity API endpoint.

• identity_uri=http://controller:35357: It sets the complete admin Identity API endpoint.

• admin_tenant_name=service: It sets service as the Neutron admin tenant.

• admin_user=neutron: It sets nova as the Neutron admin user.

• admin_password=NEUTRON_PASS: It sets the Neutron user’s password in order to authenticate to Keystone.

ml2_conf.ini

The second configured file is ml2_conf.ini which configures the Open vSwitch (OVS) mechanism to build the virtual
networking framework for instances. In this node there is the same configuration as in the Controller node but it adds
the OVS components are configured.
In the ml2 section there have been defined these options:

• type_drivers=flat,gre: It sets flat and GRE as type driver entrypoints to be loaded from the neutron.ml2.type_drivers
namespace.

• tenant_network_types=gre: It sets GRE network type as the network type to allocate as tenant networks.

• mechanism_drivers=openvswitch: It sets openvswitch as the networking mechanism driver entrypoint to be

loaded from the neutron.ml2.mechanism_drivers namespace.

In the ml2_type_flat section there have been defined these options:

85
 • flat_networks=external: It sets external as the list of physical network names with which flat networks can be
created.

In the ml2_type_gre section there have been defined these options:

• tunnel_id_ranges=1:1000: It sets from 1 to 1000 the possible GRE tunnel ID’s that are available for tenant
network allocation.

In the securitygroup section there have been defined these options:

• enable_security_group=True: It enables neutron security group

• enable_ipset=True: It enables to speed-up the iptables security groups.

• firewall_driver=neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriv e r: It sets the OVS

iptables firewall driver.

In the ovs section there have been defined these options:

• local_ip=192.168.123.21: It sets the internal network IP of the Network node as the address of the instance
tunnels.

• enable_tunneling=True: This options enable the tunnel support for Neutron system.

• bridge_mappings=external:br-ex It maps a the external physical network into the br-ex virtual bridge.

In the agent section there have been defined these options:

• tunnel_types=gre: It sets GRE (Generic Routing Encapsulation) as the tunnel type used.

l3_agent.ini

The third configured file is l3_agent.ini which provides routing services for virtual networks.
In the DEFAULT section there have been defined these options:

• verbose=True: It enables the verbose logging which gives more information about a possible error.

• interface_driver=neutron.agent.linux.interface.OVSInterfaceDriver: It specifies the OVS driver as it is the

plugins used in this configuration.

• use_namespaces=True: This options allows overlapping IP.

• external_network_bridge=br-ex: It sets the name of the bridge used for external network traffic.

• router_delete_namespaces=True: It enables to delete namespaces when a router is destroyed.

86
dhcp_agent.ini

The forth configured file is dhcp_agent.ini which provides DHCP services for virtual networks.
In the DEFAULT section there have been defined these options:

• verbose=True: It enables the verbose logging which gives more information about a possible error.

• interface_driver=neutron.agent.linux.interface.OVSInterfaceDriver: It specifies the OVS driver as it is the

plugins used in this configuration.

• dhcp_driver=neutron.agent.linux.dhcp.Dnsmasq: This options sets Dnsmasq as the DHCP server.

• use_namespaces=True: It sets the name of the bridge used for external network traffic.

• dhcp_delete_namespaces=True: It enables to delete namespaces when DHCP server is disabled.

metadata_agent.ini

The fifth configured file is dhcp_agent.ini which provides DHCP services for virtual networks.
In the DEFAULT section there have been defined these options:

• verbose=True: It enables the verbose logging which gives more information about a possible error.

• auth_url=http://controller:5000/v2.0: It sets the complete public Identity API endpoint for the authentication.

• auth_region=regionOne: It sets the region for accessing to metadata server.

• admin_tenant_name=service: It sets the service tenant for accessing to metadata server.

• admin_user=neutron: It sets the neutron user for accessing to metadata server.

• admin_password=NEUTRON_PASS: It sets the password for neutron user for accessing to metadata server.

• nova_metadata_ip=controller: It sets the node in which there is the metadata server installed which is Con-
troller node.

• metadata_proxy_shared_secret=METADATA_SECRET: It sets a shared secret to sign instance-id request.

This shared secret has to be configured in metadata server.

8.4.4 Metadata agent Controller configuration

To complete the configuration for the metada agent the nova.conf file in Controller node have to be reconfigured in
order to enable metadata proxy and configure the secret. The options have to be configured in neutron section:

• service_metadata_proxy=True: It enables the metadata proxy.

• metadata_proxy_shared_secret=METADATA_SECRET: It configures the shared secret set in Network node

to sing instance-id requiest.

To enable these settings the nova-api service has to be restarted in Controller node.

87
8.4.5 Open vSwitch service configuration
As openvswitch service is the backend used by Neutron to configure the network, the bridge that handles the external
instance network traffic has been configured to provide connectivity between the virtual and the external networks of
the environment. To do so, a port of the physical network has to be added to the bridge in order to provide instances
with external network access:
First the bridge is added to openvswitch:
# ovs-vsctl add-br br-ex
The interface eth2 is added as a port to br-ex bridge, as it is the interface that physically provides access with the
external network:
# ovs-vsctl add-port br-ex eth2

8.4.6 Finish the configuration

To implement all the configured settings, all the services installed in Network node have to be restarted:

• neutron-plugin-openvswitch-agent

• neutron-l3-agent

• neutron-dhcp-agent

• neutron-metadata-agent

8.4.7 Verify operation

For the verification that the Neutron service has been configured properly in Network node and it can executes the
commands correctly, it has been listed neutron agents in order to verify that neutron agents are launched successfully.
This operation has to be launched from the controller node with admin credentials by sourcing the admin openrc file.
The following Neutron command lists the available service extensions:
$ neutron agent-list
During the verification of the service configuration, the agents weren’t launched properly. The solution of this error is
described in section 13.3.

8.5 Service installation and configuration for Compute node

The compute node is responsible for handling the connectivity and security group for instances.

8.5.1 Prerequisites configuration

To grant the connectivity for instances there is the need to configure some kernel networking parameters:

• net.ipv4.conf.all.rp_filter=0 This option disables IP address checking in order to avoid invalid IP addresses.
The option is disabled because Compute node performs routing functions between instances, and for this cases
this option has to be disabled.

88
 • net.ipv4.conf.default.rp_filter=0 This option has the same function as the previous option, so it also has to be
disabled.

To implement the changes the following command has to be executed:

# sysctl -p

8.5.2 Required modules

The modules that have to be installed in Network node are the following:

• neutron-plugin-openvswitch-agent: This agent installs the plugin for integrating openvswitch with neutron.
• neutron-plugin-ml2: This plugin is a framework allowing OpenStack Networking to simultaneously utilize the
variety of layer 2 networking technologies found in complex real-world data centers.

To download and install these modules the next command has to be executed:
# apt-get install neutron-plugin-ml2 neutron-plugin-openvswitch-agent

8.5.3 Service configuration

The behaviour of Neutron in Compute node is configured by these two files:

• neutron.conf stored in /etc/neutron/

• ml2_conf.ini stored in /etc/neutron/plugins/ml2/

As all the OpenStack configuration files, these are also distributed in several sections [nameofthesection] in order to
sort the configuration options.

neutron.conf

The first configured file is neutron.conf which is responsible for the configuration of the database, authentication
mechanism, message broker and plug-in.
In the DEFAULT section there have been defined these options:

• verbose=True: It enables the verbose logging which gives more information about a possible error.
• rpc_backend=rabbit: It sets the RabbitMQ as messaging system.
• rabbit_host=controller: It sets controller as the RabbitMQ broker host
• rabbit_password=RABBITMQKEY: It sets the configured rabbitMQ password in order to grant authentica-
tion to the service.
• auth_strategy = keystone: It sets Keystone service as the authentication strategy.
• core_plugin=ml2: It sets ml2 framework as the Neutron core plugin entrypoint to be loaded from the neu-
tron.core_plugins namespace.
• service_plugins=router: It sets router as the service plugin entrypoint to be loaded from the neutron.service_plugins
namespace.

89
 • allow_overlapping_ips=True: It enables the capability to overlap IP’s for different subnets.

In the keystone_authtoken section there have been defined these options:

• auth_uri=http://controller:5000/v2.0: It sets the complete public Identity API endpoint.

• identity_uri=http://controller:35357: It sets the complete admin Identity API endpoint.
• admin_tenant_name=service: It sets service as the Neutron admin tenant.
• admin_user=neutron: It sets nova as the Neutron admin user.
• admin_password=NEUTRON_PASS: It sets the Neutron user’s password in order to authenticate to Keystone.

ml2_conf.ini

The second configured file is ml2_conf.ini which configures the Open vSwitch (OVS) mechanism to build the virtual
networking framework for instances. In this node there is added the OVS component in order to configure the virtual
networking framework for instances. The configuration is similar to Network node, but it isn’t configured the external
network options as Compute node only provides the internal networking between instances.
In the ml2 section there have been defined these options:

• type_drivers=flat,gre: It sets flat and GRE as type driver entrypoints to be loaded from the neutron.ml2.type_drivers
namespace.
• tenant_network_types=gre: It sets GRE network type as the network type to allocate as tenant networks.
• mechanism_drivers=openvswitch: It sets openvswitch as the networking mechanism driver entrypoint to be
loaded from the neutron.ml2.mechanism_drivers namespace.

In the ml2_type_gre section there have been defined these options:

• tunnel_id_ranges=1:1000: It sets from 1 to 1000 the possible GRE tunnel ID’s that are available for tenant
network allocation.

In the securitygroup section there have been defined these options:

• enable_security_group=True: It enables neutron security group

• enable_ipset=True: It enables to speed-up the iptables security groups.
• firewall_driver=neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriv e r: It sets the OVS
iptables firewall driver.

In the ovs section there have been defined these options:

• local_ip=192.168.123.31: It sets the internal network IP of the Network node as the address of the instance
tunnels.
• enable_tunneling=True: This options enable the tunnel support for Neutron system.

In the agent section there have been defined these options:

• tunnel_types=gre: It sets GRE (Generic Routing Encapsulation) as the tunnel type used.

90
8.5.4 Finish the configuration
To implement all the configured settings, the following services installed in Compute node have to be restarted:

• openvswitch-switch
• nova-compute

• neutron-plugin-openvswitch-agent

8.5.5 Verify operation

For the verification that the Neutron service has been configured properly in Compute node and it can executes the
commands correctly, it has been listed neutron agents in order to verify that neutron agents are launched successfully.
This operation has to be launched from the Controller node with admin credentials by sourcing the admin openrc file.
The following Neutron command lists the available service extensions. In this case there are listed the Network and
Compute agents as it can be seen in the figure 8.3:
$ neutron agent-list

Figure 8.3: neutron agents list

During the verification of the service configuration, the agents weren’t launched properly. The solution of this error is
described in section 13.3.

91
92
Chapter 9

Dashboard: Horizon

9.1 Dashboard Configuration

9.1.1 Required modules

To use the dashboard, the following modules have to be installed in Controller node:

• openstack-dashboard: This module installs the django web interface to Openstack.

• apache2: This module installs the apache HTTP server.
• libapache2-mod-wsgi: This module installs the python WSGI adapter module for Apache.
• memcached: This module installs a high-performance memory object caching system.
• python-memcache: This module the python memcached client.

To download and install these modules the next command has to be executed:
# apt-get install openstack-dashboard apache2 libapache2-mod-wsgi memcached
python-memcache

9.1.2 Service configuration

To configure the Dashboard, the local_settings.py file stored in /etc/openstack-dashboard/ has been configured with
following options:

• OPENSTACK_HOST = "controller“: This option configures the Controller node as the host that will used
Openstack services.
• ALLOWED_HOSTS = [’*’]: This option configures that all hosts can access the Dashboard.
• TIME_ZONE = "UTC": This option sets UTC as the time zone identifier.

For the default memcached session storage service the following options have been configured:

• ’BACKEND’: ’django.core.cache.backends.memcached.MemcachedCache’: This option sets the Mem-

cached backend as the session storage service.
• ’LOCATION’: ’127.0.0.1:11211’: This option sets the localhost as the location for the Memcached backend.

93
9.1.3 Finish the configuration
To finalize the configuration and implement the settings the web server and storage session service have been restarted.
# service apache2 restart
# service memcached restart

9.1.4 Verify operation

To verify that the Dashboard has been configured properly, the access to the dashboard has been checked through the
firefox web browser by the following URL:
http://controller/horizon
To authenticate it has been used the demo credentials configured in the Keystone service:

• user: upcuser
• password: UPCUSER

94
Chapter 10

Network configuration

10.1 Network configuration

Once the Neutron service is properly working, it has been configured a virtual network infrastructure to which instances
will connect and be able to exchange data among them.
To do so, it has been created two type networks, the external and the internals which will be part of a single tenant.

10.2 External network

The external network is responsible for providing Internet access to the instances by using NAT(Network Address
Translation). To enable the Internet access to the instances, it has been used a floating IP address and security group
rules for each instance.
Due to the fact that the external network can provide external access to several tenants, this network will be owned by
the admin tenant.
To create the network it has been used the following Neutron service command executed with admin access:
$ neutron net-create ext-net --router:external True --provider:physical_network
external --provider:network_type flat

Figure 10.1: External network ext-net

As it can be seen in figure 10.1, the created external network, ext-net, is configured as a flat network

95
Once the external network is configured, it has been assigned a subnet to it in order to work as a physical one.
$ neutron subnet-create ext-net --name ext-subnet --allocation-pool
start=203.0.113.101,end=203.0.113.200 --disable-dhcp --gateway 203.0.113.1
203.0.113.0/24

Figure 10.2: External subnetwork ext-subnet

The subnet created showed in figure 10.2, ext-subnet, is the 203.0.113.0/24 with the 203.0.113.1 IP address as the
gateway and with an floating IP range from 203.0.113.101 to 203.0.113.200. These subnet has the DHCP service
disabled because the instances won’t be connected automatically to the internet but they will have the floating IP
addresses assigned manually.

10.3 Tenant network

The tenant network is responsible for providing the internal access for instances. This network is isolated from other
tenant networks as it only gives internal access to the instances attached to it, so this network will be owned by the
UPC tenant.
To create the three UPC tenant network it has been used the following Neutron service commands executed with
upc-network access:
$ neutron net-create upc-net
$ neutron net-create upc-net2
$ neutron net-create upc-net3
As for the external network, it has been needed to attach a subnet for each tenant network which will be isolated from
other tenant networks.
$ neutron subnet-create upc-net --name upc-subnet --gateway 192.168.2.1
192.168.2.0/24
$ neutron subnet-create upc-net2 --name upc-subnet2 --gateway 192.168.3.1
192.168.3.0/24
$ neutron subnet-create upc-net3 --name upc-subnet3 --gateway 192.168.4.1
192.168.4.0/24
The subnets created, upc-subnet, upc-subnet2 and upc-subnet3, are the 192.168.2.0/24, 192.168.3.0/24 and 192.168.4.0/24
respectively. Each subnet has the DHCP service enabled, so the instances attached to them will obtain IP addresses
from this network automatically.

96
10.4 Tenant router
In order to pass network traffic between the tenant and the external network, it is has been created a router which will
have attached the tenant and external networks.
To create the router it has been used the following Neutron service command executed with upc-network access:
$ neutron router-create upc-router
$ neutron router-create upc-router2
$ neutron router-create upc-router3
Once the routers are created, the upc-subnet has been attached to it as a new interface:

• upc-router:
$ neutron router-interface-add upc-router upc-subnet
The external network is attached to the route as the gateway:
$ neutron router-gateway-set upc-router ext-net
• upc-router2:
$ neutron router-interface-add upc-router2 upc-subnet
$ neutron router-interface-add upc-router2 upc-subnet2
• upc-router3:
$ neutron router-interface-add upc-router3 upc-subnet
$ neutron router-interface-add upc-router3 upc-subnet3

10.5 Final virtual network infrastructure

The final network configuration is showed in the figure 10.3:

97
Figure 10.3: Network topology

98
Chapter 11

Instance creation

11.1 Instance launching

Once the networking configuration is properly set and properly working an instance can be created.

11.1.1 Previous requirements

Key pair

For public key authentication it must be generated a public/private key pair using ssh-keygen and add the public key
to your OpenStack environment. The key has been created with the upc tenant credentials in order to make the tenant
own it. To do so the following commands have been used:
$ ssh-keygen
$ nova keypair-add --pub-key /.ssh/id_rsa.pub demo-key
To be sure that the key pair has been added successfully the keypairs added have been listed:
$ nova keypair-list

Figure 11.1: UPC keypair

Available flavors

A flavor specifies a virtual resource allocation profile which includes processor, memory, and storage. There are only
the following available flavors:
$ nova flavor-list

99
 Figure 11.2: Flavors

Available images

The instance will boot using an image stored in the image repository as it has been described in section 6.1. The
available images are the following:
$ nova image-list

Figure 11.3: Images

Available networks

The instance needs a network in order to have assigned an IP address for providing connectivity. The available
networks for the UPC tenant are the following:
$ neutron net-list

Figure 11.4: Nets

11.1.2 Instance creation

When the requirements for launching an instance are achieved, the following command can be executed:
$ nova boot --flavor m1.tiny --image cirros-0.3.3-x86_64 --nic
net-id=14cee21a-a4e7-483c-b71c-c38880610e5d --security-group default
--key-name upc-key upc
As it can be seen in figure 11.5, the instance called upc, has been created with the m1.tiny flavor which is the smallest
one and has used the available image (cirros-0.3.3-x86_64). The network selected for associating the instance is the
upc-net as it is an upc’s tenant instance.
To check is the instance has been already started, the instance is listed in order to see its state:

100
 Figure 11.5: Instance boot

$ nova list

Figure 11.6: Instance activated

In figure 11.6, the state of the launched instance has to have changed from BUILD to ACTIVE as the instance has
been fully initiated.
While performing these instances some problems came at the instance activation. These are described in the section
13.4.

101
102
Chapter 12

Policy configuration

For providing security to an environment, there is the need to configure an access system in order to differentiate the
use given to the user and to prevent user from performing not authorized actions.

12.1 Identity configuration

The identity configuration for providing security to the environment is based on separating each service administration.
This will prevent a user performing an action on a service which is not administrating.

12.1.1 Roles
For each service administration it has been created an specific role:

• netadmin: This role is used for administrating the network service of every tenant.
• acsadmin: This role is used for administrating the identity service of every tenant.

12.1.2 Users
For each role it has been created a user and it has been associated to it. For providing redundancy it is possible to
create a pool of users for each role. The following table shows the identity system:

UserId Password Tenant Role Description

netadm NETADMIN admin netadmin Network service administrator
acsadm ACSADMIN admin acsadmin Identity service administrator

12.2 Policy configuration

Once the the role configuration is set, they have to be mapped in the policy.json file of each service.
The netadmin role is mapped in the /etc/neutron/policy.json which configures the policies of the network service. To
do so the netadmin role has been configured as an administrator of this service implementing the following rule:
“context_is_admin”: “role:admin or role:netadmin”

103
With this rule any action that can only be performed by the rule context_is_admin will be able for any user with the
netadmin role.
The acsadmin role is mapped in the /etc/keystone/policy.json where the policies of the identity service are configured.
The rule for implementing that role as an administrator of the service is the following:
“admin_required”: “role:admin or is_admin:1 or role:acsadmin”
With this rule any action that can only be performed by the rule admin_required will be able for any user with the
acsadmin role.

104
Chapter 13

Troubleshooting

In this chapter there are described the issues that have happened during the configuration of the environment.

13.1 Keystone troubleshooting

During the verification of the proper behaviour of the Identity service, there was an error that prevented the execution
of any keystone command successfully. The error described in the Identity service log file named keystone.log was
the following:
TRACE keystone ImportError: No module named persistence.backends.sql
This error ocurred due to the version of the packages installed weren’t the latest version and the driver for Keystone
that was installed didn’t support the " persistence.backend.sql" module.
To solve the problem the driver was changed to "backends.sql" as it is indicated in the section 5.2.2.

13.2 Nova troubleshooting

During the verification of the proper behaviour of the Identity service, there were two errors that prevented the launch-
ing of the nova service components and when they where listed with the command in the 7.5 section, the output didn’t
show any component.The errors were described in the Nova service log file named nova-compute.log. The first error
found was the following:
TRACE nova.virt.driver ImportError: Class LibvirtDriver cannot be found
This error ocurred because the compute-driver option was configured in nova.conf file but to be recognized, it had to
be defined in nova-compute.conf. Once the option in nova.conf file was commented the error was solved.
The second error appeared once the first was solved:
TRACE nova.openstack.common.threadgroup libvirtError: XML error: Missing CPU
model name
In this case, the error ocurred because the controller, compute and network node were running using qemu as hyper-
visor and the CPU host flags weren’t copied to the guest node. To solve the problem, the virtualization driver was
enabled in the BIOS of the host and the hypervisor used was changed to KVM. With this change the flags the CPU
host flags were properly copied, the nodes performance was improved and the error was definetly solved.

105
13.3 Neutron troubleshooting

During the verification of the proper behaviour of Neutron service, there where three errors that prevented the launch-
ing of Neutron agents of both Network and Compute nodes when they were listed during their verification.

13.3.1 Rabbit module error

The errors from Network and Compute node where described in the l3-agent.log stored in /var/log/neutron. This error
reported showed that neutron didn’t recognize the rabbit module used as the message service:
TRACE neutron.openstack.common.service ImportError: No module named rabbit
This error ocurred because in the configuration file there was enabled the option rpc_backend. This option made the
Neutron service to look for the rabbit module which in the version used didn’t have to be used.
To solve the issue, the rpc_backend option was disabled in the configuration file (neutron.conf) and all the services
were restarted in order to implement the change.

13.3.2 Controller node unreachable error

After solving the error, in Network node appeared another error also reported in l3-agent.log file. The error showed
that the Controller node was unreachable by the message service:
ERROR neutron.openstack.common.rpc.common AMPQ server on controller:5672 is
unreachable: [Errno 113] EHOSTUNREACH
In order to find the origin of the error, there were made the following tests:

• Ping to 192.168.122.11: It was send a ping to the IP address of Controller node in order to check if there was
connectivity. The result of the test showed that there wasn’t connectivity.

• Ping to 192.168.122.31: It was send a ping to the IP address of Compute node to check if the problem only con-
cerned Controller node or it affected all the system. The result of the test showed that there wasn’t connectivity,
so the error afected all the system.

As the only modification with the Network node interfaces was made when the openvswitch service was configured
and the interface eth2 was added to the br-ex bridge.
To check which interface was actually added to the br-ex bridge there were listed all the ports added to this bridge:
# ovs-vsctl list-ports br-ex
The result showed that there was a misconfiguration and the interface eth0 was added to the bridge instead of the eth2
and for this reason the connection through this interface wasn’t available.
To change the added interface, the port corresponding to the eth0 had to be deleted from the bridge and then the eth2
had to be added:
# ovs-vsctl del-port br-ex eth0
# ovs-vsctl add-port br-ex eth2
The openvswitch service was restarted in order to implement the changes and the neutron agent were listed again from
the Controller node, this time being launched successfully.

106
13.4 Instance creation troubleshooting
During the launching of the first instance, a long time after creating it, its state didn’t change from BUILDING to
ACTIVE. To achieve the final solution, several errors had to be resolved.

13.4.1 Low RAM

To know what problem could be originating the error, the log from the nova-scheduler service was checked. The
problem was found in the nova-compute.log file stored in /var/log/nova of the compute node. The problem that was
found is the following:

Figure 13.1: Nova-compute error

The problem in figure 13.1 showed that the available free RAM was 481 MB when the instance that was being created
needed a minimum RAM of 512 MB. To solve this issue it was added to compute node more RAM in order to have
available the needed amount of memory.

13.4.2 RPC version not supported

After solving the problem with the compute RAM, the following problem arised in the nova-compute.log file:
ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling:
Endpoint does not support RPC version 3.33
While searching a solution[18] for supporting the RPC version, it was found that the version installed in compute and
network node of nova and neutron services wasn’t actualized. This problem came because in those node the OpenStack
Juno repository wasn’t configured. To solve that problem, the repository was actualized and the configuration files of
the affected services (nova and neutron) were reconfigured. After solving this issue the instance was ACTIVATED
properly.

107
108
Bibliography

[1] Amqp and rabbitmq. http://ilearnstack.com/2013/04/16/

introduction-to-amqp-messaging-with-rabbitmq/.

[2] Amqp protocol. https://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol.

[3] Ceilometer service documentation. http://docs.openstack.org/developer/ceilometer/

architecture.html.

[4] Designate service documentation. http://docs.openstack.org/developer/designate/

architecture.html.

[5] Glance service documentation. http://docs.openstack.org/developer/glance/

architecture.html.

[6] Hadoop documentation. https://en.wikipedia.org/wiki/Apache\_Hadoop.

[7] Heat service documentation. http://docs.openstack.org/developer/heat/architecture.

html#heat-services.

[8] Horizon service documentation. http://docs.openstack.org/developer/horizon/intro.

html.

[9] Ironic service documentation. http://docs.openstack.org/developer/ironic/deploy/

user-guide.html.

[10] Keystone service documentation. http://es.slideshare.net/openstackindia/

openstack-keystone-identity-service?related=1.

[11] Magnum service documentation. http://docs.openstack.org/developer/magnum/.

[12] Manila service documentation. http://docs.openstack.org/developer/manila/adminref/

intro.html.

[13] Neutron service documentation. http://docs.openstack.org/admin-guide-cloud/

networking_arch.html.

[14] Nova service documentation. http://docs.openstack.org/developer/nova/project_scope.

html.

[15] Openstack basic architecture. http://docs.openstack.org/juno/install-guide/install/

apt/content/ch_basic_environment.html.

[16] Openstack messaging. http://ilearnstack.com/2013/04/24/

messaging-in-openstack-using-rabbitmq/.

109
[17] Policy documentation. http://docs.openstack.org/kilo/config-reference/content/
policy-json-file.html.
[18] Rpc openstack support. https://ask.openstack.org/en/question/51800/
endpoint-does-not-support-rpc-version-333/.

[19] Sahara service documentation. http://docs.openstack.org/developer/sahara/

architecture.html.
[20] Swift service documentation. http://docs.openstack.org/developer/swift/overview_
architecture.html#proxy-server.
[21] Trove service documentation. http://docs.openstack.org/developer/trove/dev/design.
html.
[22] Zaqar service documentation. http://docs.openstack.org/developer/zaqar/.
[23] Timothy Grance Peter Mell. The NIST definition of Cloud Computing. National Institute of Standards and
Technology, 2011.

110