Python For Network Engineers
Python For Network Engineers
Khelil SATOR
JUNIPER NETWORKS
May 2016
Version 3.8
AGENDA
INTRODUCTION TO NETWORK AUTOMATION
INTRODUCTION TO PYTHON PROGRAMMING
BUILDING DOCUMENTS WITH JINJA2 TEMPLATES
YAML (NON-PROGRAMMERS TO PROGRAMS)
JUNOS AUTOMATION WITH PYEZ PYTHON LIBRARY
JSON DATA FORMAT (EXCHANGE DATA BETWEEN APPLICATIONS)
PROGRAMMATIC ACCESS WITH REST APIs
VERSION CONTROL (GIT WITH GITHUB)
CONTINIOUS INTEGRATION (TRAVIS CI AND COVERALLS)
ONLINE NETWORK AUTOMATION QUIZ
INTRODUCTION TO NETWORK AUTOMATION
SOME USE CASES
A collection of items
Items are ordered
Items separated by commas
Items are enclosed within square brackets [ ]
A list is iterable: a for loop iterates over its items
LISTS
For each iteration of the iterable, the for loop executes the
statements in the loop body
FOR LOOPS WITH A LIST
If we use a for loop with a list, it iterates over its items.
>>> my_devices_list
['172.30.108.11', '172.30.108.133', '172.30.108.133', '172.30.108.14',
'172.30.108.176', '172.30.108.254']
>>> for device in my_devices_list:
print ("the current device is: " + device)
Module:
A file with Python code. A python file.
The file name is the module name with the suffix .py appended
(module.py).
A module can define functions, classes, variables ...
Function:
A function returns a value. Call a function passing arguments.
There are many built-in functions. You can also create your own
functions.
A function is defined once and can be called multiple times.
PYTHON BUILDING BLOCKS
Class:
Classes define objects.
Call a class passing arguments. The returned value is an object.
So each instance of a class is an object.
Method:
A class defines functions available for this object (in a class,
these functions are called methods)
A method is a function defined in a class.
To call a method, we first need to create an instance of the class.
Once you have an instance of a class, you can call a method for
this object.
MODULES FOR NETWORK ENGINEERS
Python allows you to import modules to reuse code.
Good programmers write good code; great programmers reuse/steal
code
Importing a module is done without using the .py extension
Anyone can create modules for private uses or to share with
community
Some very nice Python modules/packages for network engineers:
netaddr: a Python library for representing and manipulating network
addresses
re: regular expressions
requests: rest api manipulation
jinja2: generate documents based on templates
Yaml: users to programs communication (to define variables)
PyEZ: Python library to interact with Junos devices
MANIPULATE IP ADDRESSES
WITH PYTHON
PYTHON NETADDR PACKAGE
There are many Python modules to manipulate IP
addresses: ipaddr (google contribution), ipaddress (easy
but requires python 3), IPy, netaddr,
netaddr is a Python package to manipulate IP addresses
and subnets.
IPAddress is a class in module netaddr.ip. An IPAddress instance
is an individual IPv4 or IPv6 address object (without net mask)
IPNetwork is a class in module netaddr.ip. An IPNetwork
instance is an IPv4 or IPv6 network or subnet object
THE CLASS IPADDRESS
Import the class IPAddress
>>> from netaddr import IPAddress
Then you can easily play with the created objects using
methods and properties.
THE CLASS IPADDRESS
Some methods and properties:
>>> ip
IPAddress('192.0.2.1')
>>> ip.version
4
>>> ip.is_private()
False
>>> ip.is_unicast()
True
>>> ip.is_multicast()
False
>>> ip.bits()
'11000000.00000000.00000010.00000001
>>>
>>>
>>> ip
IPAddress('192.0.2.1')
>>> ip+1
IPAddress('192.0.2.2')
>>> ip+255
IPAddress('192.0.3.0')
THE CLASS IPNEWORK
The class IPNetwork is define in the package
netaddr.ip
Each instance of the class IPNetwork is an object (a
subnet)
Once you have created an instance of the class
IPNetwork, you can use the methods defined in the
class IPNetwork with this subnet.
THE CLASS IPNEWORK
Import the class IPNetwork
>>> from netaddr import IPNetwork
192.0.2.0
192.0.2.1
192.0.2.2
192.0.2.3
192.0.2.4
192.0.2.5
192.0.2.6
192.0.2.7
MANIPULATE IP ADDRESSES
The method iter_hosts provides all the IP addresses
that can be assigned to hosts within a subnet: for IPv4,
the network and broadcast addresses are always
excluded.
>>> f=open("python_basics/list_of_ip.txt","r")
>>> f
<open file 'python_basics/list_of_ip.txt', mode 'r' at 0x000000000317AE40>
>>> help(f.read)
>>> s=f.read()
>>> type(s)
<type 'str'>
>>> s
'172.30.179.101\n172.30.179.102\n172.30.179.103\n172.30.179.104\n172.30.179.105\n'
>>> print s
172.30.179.101
172.30.179.102
172.30.179.103
172.30.179.104
172.30.179.105
>>> f.close()
WRITE CONTENT ON A FILE
To open a file with write mode, use "w".
If the file doesnt exist, python will create it.
If the file already exists, python will overwrite its content.
To open a file with append mode, use "a".
The file pointer is at the end of the file if the file exists. That is,
the file is in the append mode. You can write content without
overwriting the file content.
If the file does not exist, it creates a new file for writing.
>>> f=open("python_basics/list_of_ip.txt","a")
>>> f
<open file 'python_basics/list_of_ip.txt', mode 'a' at 0x000000000317AD20>
>>> help(f.write)
>>> f.write("172.30.179.106\n")
>>> f.write("172.30.179.107\n")
>>> f.close()
JINJA2 TEMPLATES
JINJA2 PACKAGE
vlan_name: v14
DEMO:
CREATE JUNOS CONFIGURATION FILES
WITH PYTHON, JINJA2 AND YAML
JINJA2 AND YAML
Lets use a jinja2 template and a yaml file to build the initial junos
configuration files. We can use with a ZTP setup to configure new
devices (build phase).
We need to provide to each new device (factory default configuration)
at least the following:
-a root password (otherwise we can not commit the conf).
-a management ip @ and subnet, and a route (to be able to reach remotely the new
device).
-allow ssh connection (in case we want to ssh it).
-enable netconf over ssh (to be able then to use PyEZ in the run and audit phases).
-an hostname.
Only the hostname and the management ip @ are unique per device.
So only these 2 details are define as variables in the jinja2 template.
The yaml file define their values for each device.
YAML
configuration_builder/variables_build.yml is a yaml file.
This is a yaml list. With 3 items. Each item is a device.
Each item of the list is a dictionary with the device hostname and
management ip @.
It is extremely easy to add other devices.
You can use another yaml structure (i.e instead of a list of dictionaries) but
in that case youll need to parse it differently from the jinja2 and python
files.
pytraining@py-automation-master:~$ more configuration_builder/variables_build.yml
JINJA2
configuration_builder/template_build.j2 is a jinja2 template.
this is the template to build the initial junos configuration file.
It uses the variables defined in the yaml file.
pytraining@py-automation-master:~$ more configuration_builder/template_build.j2
PYTHON
configuration_builder/configuration_builder.py is a python script.
It uses the jinja2 template and the yaml file to create the initial junos
configuration file for each device defined in the yaml file.
You can use these files with a ZTP setup to configure automatically new devices
(build phase).
pytraining@py-automation-master:~$ more configuration_builder/configuration_builder.py
BUILD A DOC BASED ON A JINJA2 TEMPLATE
Use the python to generate the junos configuration file.
pytraining@py-automation-master:~$ python configuration_builder/configuration_builder.py
Start configuration building
generate config file for device qfx5100-10 : conf_file_build_phase_qfx5100-10.conf
generate config file for device qfx5100-6 : conf_file_build_phase_qfx5100-6.conf
generate config file for device qfx5100-8 : conf_file_build_phase_qfx5100-8.conf
done
call the dir function without argument to get the list of the
names defined in the current scope. The class Config is
now in the current scope.
>>>dir()
METHODS DEFINED IN THE CLASS CONFIG
>>> # confjunos.conf is a file with junos commands with the format set that define vlan 911
>>> cfg.load(path="configuration_management/confjunos.conf", format='set')
<Element load-configuration-results at 0x7f77c84317a0>
COMPARE CONFIGURATIONS
Compare the candidate configuration and the active
configuration (or a provided rollback) with the method
pdiff. Examples:
>>> cfg.pdiff()
[edit interfaces]
+ ge-0/0/23 {
+ description PyEZ;
+ }
[edit vlans]
+ vlan-911 {
+ description "created with python";
+ vlan-id 911;
+ }
+ vlan-927 {
+ description "created with python";
+ vlan-id 927;
+ }
>>> cfg.pdiff(rb_id=1)
ROLLBACK THE CANDIDATE CONFIGURATION
ge-0//0/0
192.168.0.4/31 ge-0//0/1
192.168.0.1/31
ge-0//0/1
192.168.0.3/31 ge-0//0/0
192.168.0.0/31
ge-0//0/1 EX-4300-4
192.168.0.2/31
ASN 104
DEMO JINJA2 AND YAML AND PYEZ
pytraining@py-automation-master:~$ python configuration_builder/configuration_builder_2.py
Start configuration building
generate config file for device qfx5100-10 : conf_file_run_phase_qfx5100-10.conf
generate config file for device qfx5100-6 : conf_file_run_phase_qfx5100-6.conf
generate config file for device qfx5100-8 : conf_file_run_phase_qfx5100-8.conf
done
applying the conf to the devices ...
configuration commited on qfx5100-10
configuration commited on qfx5100-6
configuration commited on qfx5100-8
done
pytraining@py-automation-master:~$ ls | grep run
conf_file_run_phase_qfx5100-8.conf
conf_file_run_phase_qfx5100-10.conf
conf_file_run_phase_qfx5100-6.conf
pytraining@py-automation-master:~$ more conf_file_run_phase_qfx5100-8.conf
pytraining@py-automation-master:~$ more conf_file_run_phase_qfx5100-10.conf
pytraining@py-automation-master:~$ more conf_file_run_phase_qfx5100-6.conf
DEMO JINJA2 AND YAML AND PYEZ
Lets connect on the devices and double check if everything is
correct
We will see later how to audit the network with automation instead of
manually
python tables_and_views/search_an_lldp_neighbor.py
name of the neighbor you are looking for:qfx5100-10
this neighbor is connected to the interface ge-0/0/1 of the device qfx5100-6
this neighbor is connected to the interface ge-0/0/1 of the device qfx5100-8
Done
TABLES AND VIEWS: DEMO WITH BGP
For each device in a device list, this program prints:
The list of its BGP neighbors
The status of its BGP connections
python tables_and_views/bgp_states.py
Let's use Python to send the request and parse the response
ksator@ubuntu:~/pytraining$ python rest_basics/google_map_api.py
which address: 9 HaMenofim Street Herzliya
latitude is 32.1602492
longitude is 34.8081319
3- Pull Request
Clone
https://desktop.github.com/
Git Directories: Three states of files