Managing Unix Systems With Ansible - J P Mens
Managing Unix Systems With Ansible - J P Mens
EuroBSDCon
created by Jan-Piet Mens
@jpmens
1 © JPMens
Ansible
Created by Michael DeHaan
Automation
Configuration management and deployment
Orchestration
Rolling updates
Ad-hoc tasks
Goals
Simplicity and ease of use
Reliability
Readability (YAML Ain't Markup Language)
Minimal dependencies
2 © JPMens
More than just shell scripts
Idempotence
No daemons, no agents on nodes
Not another PKI
No special "management" server
No additional open firewall ports
Push-based; pull is possible
3 © JPMens
How Ansible works
4 © JPMens
No root (maybe)
Ansible doesn't require root access on the nodes, but some modules do
Ansible can login as any user and escalate privileges ( sudo , doas , su , ...)
SSH keys (SSH agent)
5 © JPMens
Installing Ansible: Requirements
Control machine
Python
OpenBSD, FreeBSD, NetBSD, *BSD, RHEL, CentOS, Debian, OS/X, macOS, etc.
Unix/Linux with Python 2.6+ or 3.5+, and SSH / SFTP (SCP possible)
Windows with WinRM and Powershell 3
6 © JPMens
Installing Ansible
From source
Via Pip
7 © JPMens
Configuration file: ansible.cfg
Know about it; Tune / change defaults
Disable annoying features ;-)
__________________________________
< PLAY [JP doesn't much like cows] >
----------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
[defaults]
nocows = 1
$ export ANSIBLE_NOCOWS=true
8 © JPMens
ansible.cfg precedence
These are searched for in the following order; first found wins (no merge):
$ANSIBLE_CONFIG
./ansible.cfg
~/.ansible.cfg
/etc/ansible/ansible.cfg
9 © JPMens
Modules
10 © JPMens
Modules
Modules do the actual work in Ansible
Modules are executed in playbook tasks, and can be run ad-hoc
Many modules are idempotent
11 © JPMens
Modules in playbooks
Within a playbook, modules are executed similarly (choose a syntax):
12 © JPMens
Built-in documentation
Modules contain documentation:
$ ansible-doc -l
apt Manages apt-packages
apt_key Add or remove an apt key
$ ansible-doc copy
> COPY
The [copy] module copies a file on the local box to remote locations. Use the
[fetch] module to copy files from remote locations to the local box. If you
need variable interpolation in copied files, use the [template] module.
Options (= is mandatory):
- backup
Create a backup file including the timestamp information so you can get
the original file back if you somehow clobbered it incorrectly.
(Choices: yes, no) [Default: no]
...
13 © JPMens
Inventory
14 © JPMens
Inventory
Defaults to /etc/ansible/hosts in INI-format
(don't confuse with /etc/hosts )
Possible to override default in ansible.cfg ( inventory = )
Can be executable, have multiple sources / directory
[web]
www01
www02 tz=Europe/Berlin
[dbservers]
psql.example.com
p2 ansible_host=192.0.2.42 ansible_port=333 ansible_user=jane
[dbservers:vars]
spooldir=/var
logging=yes
[openbsd:vars]
ansible_become_method=doas
15 © JPMens
Inventory patterns
How to specify inventory hosts when using Ansible utilities:
What Example
A host or group myhost , webservers
All hosts all or *
Intersection (all Web in staging) staging:&webservers
Exclusion webservers:!www01
Wildcard *.example.net
Range numbered www[5:10] and www[05:10]
Regexp ~www\d\.example\.net
16 © JPMens
Hostvars & Groupvars
Variables for a particular host in their own file ( host_vars/www02.yaml ):
---
database_port: 3106
ansible_user: jane
ansible_port: 444
---
spooldir: /var
logging: yes
ansible_python_interpreter: /usr/local/bin/python2.7
17 © JPMens
Variables
Variables can be defined in a play:
- hosts: dbservers
vars:
spooldir: /var
logging: true
- hosts: localhost
vars_files:
- myvars.yml
18 © JPMens
Registered variables
- shell: /bin/pwd; echo Oops >&2; whoami; echo "Hello"
register: out
- debug: var=out
19 © JPMens
Variables from the command-line
Pay attention to white space and quoting:
20 © JPMens
Information about other hosts
hostvars , group_names , groups are maintained by Ansible.
{{ hostvars['www02']['ansible_distribution'] }}
21 © JPMens
Variable precedence
From lower to higher:
role defaults
inventory vars
inventory group_vars
inventory host_vars
playbook group_vars
playbook host_vars
host facts
play vars
play vars_prompt
play vars_files
registered vars
set_facts
role and include vars
block vars (only for tasks in block)
extra vars (always win precedence)
22 © JPMens
Lab
Create an inventory file hosts
[all:vars]
ansible_connection=local
; ansible_python_interpreter=/usr/local/bin/python3 ; openbsd
ansible_python_interpreter=/usr/local/bin/python3.6 ; freebsd
$ export ANSIBLE_INVENTORY=./hosts
$ ansible localhost -m ping
23 © JPMens
Playbooks
24 © JPMens
YAML
Start of document --- , Comments start with #
Booleans: True , Yes , On , Strings: hello world vs. "hello world"
Lists
- uno
- dos
Dictionaries
25 © JPMens
YAML
26 © JPMens
Playbooks
A playbook is an Ansible configuration management recipe
It contains a list of plays and is written in YAML
YAML (YAML Ain't Markup Language) <-> JSON
A play must have a set of hosts to configure and a list of tasks.
A task is the most granular thing you do:
install an authorized key
copy a file
create a user
Tasks are linear
Tasks can loop
The name attributes are optional but highly recommended
27 © JPMens
Playbooks
---
- name: Deploy fortune generator
hosts: alice
become: true
tasks:
- name: Ensure pip package is available
package: name=python-pip state=present
- name: Ensure required Python modules are available
pip: name="{{ item }}"
loop:
- paho-mqtt
- fortune
- name: Ensure fortune data file is installed
copy: src=../files/fortunes/fortunes dest=/etc/fortunes mode=0444
28 © JPMens
Multiple plays
Playbooks can contain more than one play:
29 © JPMens
Hosts and users (1)
Each play describes the hosts it should target and as which user it should do so
---
- hosts:
- webservers
- dbs
remote_user: ansible
---
- hosts: webservers
remote_user: root
tasks:
- name: Test connection
ping:
remote_user: jane
30 © JPMens
Hosts and users (2)
Run modules as another user (sudo, su, pbrun, pfexec, doas, dzdo, ksu)
---
- hosts: webservers
remote_user: yourname
become: yes
- hosts: webservers
remote_user: yourname
tasks:
- service: name=httpd state=started
become: yes
become_method: sudo
become_user: root
31 © JPMens
Handlers
- hosts: all
become: yes
tasks:
- name: Install nginx
yum: name=nginx state=latest
notify:
- kick_nginx
- name: Configure nginx
template: src=nginx.j2 dest=/etc/nginx/nginx.conf
notify:
- kick_nginx
handlers:
- name: kick_nginx
service: name=nginx state=restarted
32 © JPMens
Blocks for logical grouping of tasks
tasks:
- block:
- package: name={{ item }} state=present
loop:
- httpd
- memcached
- template: src=templates/src.j2 dest=/etc/foo.conf
- service: name=example state=started enabled=True
33 © JPMens
Tags
Plays and tasks supports a "tags:" attribute:
- openbsd_pkg:
name: [ mosquitto, rsync, ttyd, jo ]
state: present
tags:
- pkgs
- copy:
src: mosquitto.conf
dest: /etc/mosquitto/mosquitto.conf
tags:
- config
34 © JPMens
Conditionals
When
tasks:
- name: "shut down Debian flavored systems"
command: /sbin/shutdown -t now
when: ansible_os_family == "Debian"
when:
- ansible_distribution == "CentOS"
- ansible_distribution_major_version == "6"
when: number > 4
35 © JPMens
Loops with with and with loop (1)
- group: name="{{ item }}" state=present
loop:
- "ops"
- "dev"
36 © JPMens
Loops with with and with loop (2)
- copy: src={{ item.src }} dest={{ item.dest }}
loop:
- { src: httpd.conf, dest: /etc/httpd.conf }
- { src: master.cf, dest: /etc/postfix/master.cf }
37 © JPMens
Facts and information
38 © JPMens
Facts
Facts are bits of information obtained by speaking with your remote nodes. For example IPv4
address, configured swap space, etc.
39 © JPMens
Facts: networking
IPv4 and IPv6 addresses, interfaces
"ansible_default_ipv4": {
"address": "10.0.2.15",
"broadcast": "10.0.2.255",
"device": "em0",
"flags": [
"UP",
"BROADCAST",
"RUNNING",
"SIMPLEX",
"MULTICAST"
],
"gateway": "10.0.2.2",
"interface": "em0",
"macaddress": "08:00:27:a9:28:81",
"media": "Ethernet",
"media_options": [
"full-duplex"
],
"media_select": "autoselect",
"media_type": "1000baseT",
"metric": "0",
"mtu": "1500",
"netmask": "255.255.255.0",
"network": "10.0.2.0",
"options": [
"PERFORMNUD",
"IFDISABLED",
"AUTO_LINKLOCAL"
],
"status": "active",
"type": "ether"
},
40 © JPMens
Facts: sundry
"ansible_distribution": "FreeBSD",
"ansible_distribution_major_version": "12",
"ansible_distribution_release": "12.0-RELEASE",
"ansible_distribution_version": "12.0",
"ansible_dns": {
"nameservers": [
"192.168.1.82",
"192.168.1.113"
],
"search": [
"ww.mens.de"
]
},
"ansible_os_family": "FreeBSD",
"ansible_pkg_mgr": "pkgng",
41 © JPMens
Using facts
IP address: {{ ansible_default_ipv4.address }}
Hostname as reported by system: {{ ansible_nodename }}
Fully qualified name: {{ ansible_fqdn }}
Configured swap size: {{ ansible_swaptotal_mb }}
Disable facts
- hosts: webservers
gather_facts: no
42 © JPMens
Fact caching
Ansible can cache facts in Redis or JSON files.
[defaults]
gathering = smart
fact_caching = jsonfile
fact_caching_connection = /tmp/factcache
# fact_caching = redis
# fact_caching_connection = localhost:6379:0
fact_caching_timeout = 3600
Caching in Redis requires the Python redis library be installed via pip.
Gathering
smart - gather by default, but don't regather if already cached
implicit - gather by default; disable with gather_facts: False
explicit - do not gather by default; must say gather_facts: True
43 © JPMens
Fact caching example
$ ansible-playbook t.yml
$ jq -r .ansible_date_time.iso8601 < /tmp/factcache/localhost
2018-10-29T15:01:01Z
$ ansible-playbook t.yml
$ jq -r .ansible_date_time.iso8601 < /tmp/factcache/localhost
2018-10-29T15:01:01Z
$ rm /tmp/factcache/localhost
$ ansible-playbook t.yml
$ jq -r .ansible_date_time.iso8601 < /tmp/factcache/localhost
2018-10-29T15:03:42Z
44 © JPMens
Local facts ( facts.d )
Obtained from remote /etc/ansible/facts.d
Files ending in .fact (JSON, INI, or executables emitting JSON)
Example:
$ cat /etc/ansible/facts.d/beverage.fact
[favorite]
tonic=yes
Fact gathering
"ansible_local": {
"beverage": {
"favorite": {
"tonic": "yes"
}
}
},
"ansible_machine": "x86_64",
...
45 © JPMens
Local facts: executable
$ cat /etc/ansible/facts.d/serial.fact
#!/bin/sh
jo currybeer=true dish=vindaloo epoch=$(date +%s)
{"currybeer":true,"dish":"vindaloo","epoch":1512401355}
46 © JPMens
Lab
Create a playbook fact.yml with which you print out, using debug , the operating system
of your workstation
$ export ANSIBLE_INVENTORY=./hosts
$ ansible-playbook fact.yml
PLAY [My First Playbook] ******************
TASK [Gathering Facts] ********************
ok: [localhost]
TASK [Which OS am I on?] ******************
ok: [localhost] => {
"ansible_distribution": "MacOSX"
}
47 © JPMens
{{ Templates }}
48 © JPMens
{{ Templates }}
Templates are processed by Jinja2 (on the control machine)
49 © JPMens
{% if %}
# sshd_config
50 © JPMens
Managing users (1)
vars_files:
- users.yml
tasks:
- name: Add users to system
user: name={{ item.username }} shell=/bin/bash createhome=yes
loop: '{{ users }}'
users.yml
users:
- username: jane
sudo: true
- username: john
sudo: false
51 © JPMens
Managing users (2)
sudoers.in
# Individual users
{% if users is iterable %}
{% for u in users %}
{% if u.sudo == True %}
{{ u.username }} ALL=(ALL) NOPASSWD: ALL
{% endif %}
{% endfor %}
{% endif %}
or
sudoers
# Individual users
jane ALL=(ALL) NOPASSWD: ALL
52 © JPMens
Validation of file content is important!
- hosts: bindservers
user: root
vars:
checkconf: /usr/sbin/named-checkconf
tasks:
- name: Install and validate named.conf
template:
src=named.conf.j2
dest=/etc/named.conf
validate='{{ checkconf }} %s'
53 © JPMens
Filters
Filters transform template expressions (think Unix pipe)
Jinja2 has many built-in filters; Ansible adds more
Filters can be used within {{ }} in playbooks
{{ dbserver | default('127.0.0.1') }}
54 © JPMens
Filter examples
{% set mylist = [ "one", "two", "three" ] -%}
{{ 'secret' | password_hash('sha256') }} $5$rounds=535000...vM0HGLdrmZ1
{{ [3, 2, 5] | min }} 2
{{ 59|random }} * * * * /program/in/cron 14 * * * * /program/in/cron
{{ '192.0.2.1/24' | ipaddr('address') }} 192.0.2.1
{{ 'test1'|hash('md5') }} 5a105e8b9d40e1329780d62ea2265d8a
{{ mylist | join(" | ") }} one | two | three
{{ "/etc/profile" | basename }} profile
{{ "/etc/profile" | dirname }} /etc
{{ "~jpm" | expanduser }} /Users/jpm
{{ "httpd.conf" | splitext }} ('httpd', '.conf')
{{ "hello world" | b64encode }} aGVsbG8gd29ybGQ=
{{ "aGVsbG8gd29ybGQ=" | b64decode }} hello world
{{ "ansible" | regex_replace('a', 'A') }} Ansible
55 © JPMens
Creating a custom filter
{{ "Hello world" | stars() }}
def star_this(var):
return "*** %s ***" % (var)
class FilterModule(object):
def filters(self):
return { 'stars' : star_this }
56 © JPMens
$LOOKUP
Access data from foreign sources
Evaluated on control machine
Results available in templating engine
file
password
csv, ini
credstash (AWS's KMS and DynamoDB)
DNS ( dig )
env
passwordstore
pipe
redis, mongodb
template
shelvefile
etcd
url
57 © JPMens
file lookup
- hosts: localhost
vars:
- sentence: "{{ lookup('file', 'data') }}"
tasks:
- name: Show data as read via file lookup
debug: var=sentence
58 © JPMens
CSV lookup
Code,Country
NL,Netherlands
DE,Germany
ES,Spain
FR,France
- hosts: localhost
vars:
- cc: FR
- s: "{{ lookup('csvfile', cc + ' file=europe.csv delimiter=,') }}"
tasks:
- debug: msg="country code {{ cc }} == {{ s }}"
59 © JPMens
INI lookup
[params]
curry = Vindaloo
drink = lassi
- hosts: localhost
vars:
- food: "{{ lookup('ini', 'curry section=params file=info.ini') }}"
tasks:
- debug: msg="a favorite dish is {{ food }}"
60 © JPMens
password lookup (1)
The password lookup generates a random password and stores it in a file
- hosts: localhost
vars:
password: "{{ lookup('password', 'pw.file length=20 chars=xxx') }}"
tasks:
- debug: msg="{{ password }}"
Output:
61 © JPMens
password lookup (2)
ascii_letters='abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'
ascii_lowercase='abcdefghijklmnopqrstuvwxyz'
ascii_uppercase='ABCDEFGHIJKLMNOPQRSTUVWXYZ'
digits='0123456789'
hexdigits='0123456789abcdefABCDEF'
letters='abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'
lowercase='abcdefghijklmnopqrstuvwxyz'
octdigits='01234567'
printable='0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!\
"#$%&()*+,-./:;<=>?@[\]^_`{|}~ \t\n\r\x0b\x0c'
punctuation='!"#$%&()*+,-./:;<=>?@[\]^_`{|}~'
uppercase='ABCDEFGHIJKLMNOPQRSTUVWXYZ'
whitespace='\t\n\x0b\x0c\r '
62 © JPMens
dns lookup
The dig lookup performs all sorts of DNS queries; it can return flat results or lists:
Output:
63 © JPMens
custom lookup plugins (1)
vars:
userlist: [ jjolie, ev00 ]
tasks:
- name: Create user if required
user:
name: "{{ item }}"
comment: "{{ lookup('ldapget', item) }}"
home: "/home/{{ item }}"
create_home: true
password: "{{ '123456' | password_hash('sha512') }}"
update_password: on_create
loop: "{{ userlist }}"
$ tail -2 /etc/passwd
jjolie:x:1003:1003:Jane Jolie:/home/jjolie:/bin/bash
ev00:x:1004:1004:E. Valleri:/home/ev00:/bin/bash
64 © JPMens
custom lookup plugins (2)
from ansible.plugins.lookup import LookupBase
import ldap
class LookupModule(LookupBase):
def run(self, terms, variables, **kwargs):
ld = ldap.initialize("ldap://192.168.33.101")
ld.bind_s(None, None)
ret = []
for term in terms:
filter = "(uid={uid})".format(uid=term)
cn = "unknown"
res = ld.search_s("dc=example,dc=net",
ldap.SCOPE_SUBTREE, filter, None)
if len(res) > 0:
dn, entry = res[0]
if "cn" in entry:
cn = entry["cn"][0].decode("utf-8")
ret.append(cn)
return ret
65 © JPMens
Lab
Create a template source with the following content:
Create a playbook conference.yml with which you template out that template to a file on
your local workstation
Run the playbook and verify content and permissions of the target file.
66 © JPMens
Packaging
67 © JPMens
Module categories
Cloud, Clustering, Commands, Database, Files, Identity, Inventory, Messaging, Monitoring,
Network, Notification, Packaging, Remote Management, Source Control, Storage, System,
Univention, Utilities, Web Infrastructure, Windows
68 © JPMens
Code reuse
69 © JPMens
Import Playbooks
Includes a file with a list of plays
Can only be included at the top level; you cannot use this action inside a play.
---
- import_playbook: one.yml
- import_playbook: two.yml
70 © JPMens
Import / include tasks
vars:
username: jane
keys:
- "{{ lookup('file', 'xxx') }}"
tasks:
- include_tasks: something.yml
- copy: src=file.in dest=file.out
- import_tasks: db.yml dbtype=psql
Includes are dynamic which allows loops etc. as well as constructs like include_tasks: "{{
hostname }}.yml"
71 © JPMens
Roles (1)
Modularization of playbooks
Simplification, reuse and share
Roles have name ( webserver ) and associated files
Element Directory
Tasks roles/webserver/tasks/main.yml
Files roles/webserver/files/
Templates roles/webserver/templates/
Handlers roles/webserver/handlers/main.yml
Variables roles/webserver/vars/main.yml
Defaults roles/webserver/defaults/main.yml
Dependency info roles/webserver/meta/main.yml
Tests roles/webserver/tests/{inventory,test.yml}
Library roles/webserver/library/
[defaults]
roles_path = /etc/ansible/roles
72 © JPMens
Roles (2)
Invoking roles
- hosts: webservers
roles:
- ntp
- webserver
- { role: appli, dir: "/var/apps/1", port: 6000 }
73 © JPMens
Roles (3)
We can instruct Ansible to run certain tasks before and/or after roles
- hosts: webservers
pre_tasks:
- name: Disable the frobnicator
command: frob --disable
roles:
- ntp
post_tasks:
- name: Startup the frobnicator
command: frob --launch
74 © JPMens
Role example (1)
$ tree
.
├── roles
│ └── tiny
│ ├── tasks
│ │ └── main.yml
│ ├── templates
│ │ └── hosts.in
│ └── vars
│ └── main.yml
└── roletest.yml
75 © JPMens
Role example (2)
roles/tiny/tasks/main.yml
roles/tiny/templates/hosts.in
127.0.0.1 localhost
{{ ansible_default_ipv4.address }} box
roles/tiny/vars/main.yml
directory: /tmp
roletest.yml
- hosts: localhost
roles:
- tiny
76 © JPMens
Boilerplate roles
$ ansible-galaxy init rolename
$ tree
.
└── rolename
├── README.md
├── defaults
│ └── main.yml
├── handlers
│ └── main.yml
├── meta
│ └── main.yml
├── tasks
│ └── main.yml
├── tests
│ ├── inventory
│ └── test.yml
└── vars
└── main.yml
77 © JPMens
Ansible Galaxy
Browse and search for roles at https://galaxy.ansible.com.
---
- hosts: lampservers
roles:
- geerlingguy.mysql
- geerlingguy.apache
- geerlingguy.php
78 © JPMens
Delegation
79 © JPMens
Delegation
- hosts: www[01:10]
tasks:
- package: name=httpd state=latest
- copy: src=httpd.conf dest=/etc/somewhere/httpd.conf
80 © JPMens
Local action
Basically a delegate_to: localhost
- hosts: 127.0.0.1
connection: local
81 © JPMens
Further study
82 © JPMens
Ansible features to study
Pull mode
Vault
Creating custom modules
Module facts
83 © JPMens
Books
84 © JPMens
Ansible Up & Running
85 © JPMens
Ansible for DevOps
86 © JPMens
Dynamic inventory
87 © JPMens
Inventory scripts
Inventory scripts
Cobbler, Foreman, Spacewalk
AWS EC2, Azure, Cloudstack, Consul, OpenVZ
OpenStack, docker, VMware
Digital Ocean, Linode, ...
Nagios, Zabbix
https://github.com/ansible/ansible/tree/devel/contrib/inventory
Build your own
LDAP
CMDB
...
88 © JPMens
Your own dynamic inventory
Program in any language
produce JSON
Configuration:
--list
--host host1
--host host2
--host alice
--host bob
--host db1
89 © JPMens
Dynamic inventory with _meta
{
"webservers": {
"hosts": [ "127.0.0.1", "www01" ],
"vars": { "http_port": 80, "spooldir" : "/dump" }
},
"_meta": {
"hostvars": {
"www01": { "regno": "A001", "location" : "Spain" },
"127.0.0.1": { "regno": "X094" }
}
}
}
90 © JPMens
91 © JPMens