By:

By: Ravindra
Ravindra Mittal
Mittal

Puppet Session #1

12/10/2015
Agenda:

 What is puppet?
 How Puppet Works?
 Installing and Configuring puppet master and
agent
 Configuration file : Puppet.conf
 Certificate Management in Puppet
What is puppet?
 A Configuration Management Tool

 A framework for Systems Automation

 A Declarative Domain Specific Language (DSL)

 An OpenSource software in written Ruby

 Works on Linux, Unix (Solaris, AIX), MacOS, Windows

 Developed by Puppet Labs

 Two Operational modes: Master less - puppet apply, Master / Client - puppet agent
Puppet Master/Agent Mode

Pritiman Panda 4
Puppet Standalone Mode

Pritiman Panda 5
What is puppet?
 Automate
 Move Faster
 Increase Reliability
 You define the state of your IT infrastructure, and Puppet automatically enforces the desired state. Puppet
automates every step of the software delivery process, from provisioning of physical and virtual machines to
orchestration and reporting; from early-stage code development through testing, production release and
updates
 Puppet is a solution that lets you control what installed/configured on multiple machines from a single central
machine. This single central machine is referred to as the “Puppet Master”, and the machines it controls are
called “Puppet Agents” (aka nodes or slaves).

How Puppet Works?

 Puppet agent nodes and Puppet masters communicate via HTTPS with client-verification.
 On puppet master define a desired state for each of your nodes.
 If node doesn’t resemble a desired state , we can say drift has occurred. A drift is identified every 30 minutes. If
drift occurred following happens.
 Node sends it’s current setup info ( knows as “FACTS” ) to the puppet master.
 Puppet master uses the facts to compile a catalog. This catalog contains detailed data about how the node
should be configured.
 Puppet master sends the catalog back to node.
 Node enforces the changes as described in catalog.
 Node sends report back to the puppet master.
Installing and Configuring puppet master and agent
 Pre-requisites:

1. At least two machines for client/server architecture. First machine will be setup as puppet master and another
for puppet agent.
2. Ensure all machines internal clocks are synced.
3. Firewalls: The Puppet master server must allow incoming connections on port 8140, and agent nodes must be
able to connect to the master on that port. Puppet agent allow connections on 8139.
4. Set Hostname and domain names for each machine.

 Setting up the Puppet Master

1. Install “puppetlabs” in my case RHEL 7 “rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-

7.noarch.rpm” , “yum install puppet-server”
2. Verify puppet installation : rpm -qa | grep puppet-server
3. Update hosts file: ip-172-31-48-108.ec2.internal
4. Create default site.pp file ( Leave empty at this time)
5. Start the puppet master service : service puppetmaster start , service puppetmaster status
6. After starting of puppet master first time, various files will be generated at /var/lib/puppet/ssl/
Installing and Configuring puppet master and agent
 Setting up the Puppet Agent

1. Install “puppetlabs” in my case RHEL 7 “rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-

7.noarch.rpm” , “yum install puppet”

2. Verify puppet installation : rpm -qa | grep puppet

3. Update hosts file: ip-172-31-48-108.ec2.internal

4. Define the “server” parameter with master domain name in the puppet.conf file : cat
/etc/puppet/puppet.conf

5. Start the puppet agent service : service puppet start , service puppet status

6. After starting of puppet agent first time, various files will be generated at /var/lib/puppet/ssl/
Configuration file : Puppet.conf
• There are many config files such as auth.conf, autosign.conf, hiera.yaml but main config file is
puppet.conf
• Present in /etc/puppet/puppet.conf

• Important options under [main] section:

• vardir: Path where Puppet stores dynamic data.

• ssldir: Path where SSL certifications are stored.
• server: Host name of the PuppetMaster. (Default: puppet)
• certname: Certificate name used by the client. (Default is its fqdn)
• runinterval: Number of minutes between Puppet runs, when running as service. (Default: 30)

• Under [master] section:
• autosign: If new clients certificates are automatically signed. (Default: false)
• reports: How to manage clients' reports (Default: store)
• dns_alt_names : Mention all dns names which resolves to puppet master
• report: If to send Puppet runs' reports to the **report_server. (Default: true)
Certificate Management in Puppet
 Puppet agent nodes and Puppet masters communicate via HTTPS with client-verification.

 Each Master or Agent have an identifying SSL certificate, and will examine their counterpart
certificate to exchange information.

 Puppet includes a built-in certificate authority (CA) for managing certificates. Agents can
automatically request certificates via the master’s HTTP API, users can use the puppet
cert command to inspect requests and sign new certificates, and agents can then download the
signed certificates.
 There are two ways for certificate management in Puppet :
• Manually signed approach : Here CSR’s must be manually signed by an admin user, by puppet
cert commands.

• Auto signing Certificate Requests: Here, we can configure Puppet master to automatically sign
certificate request to speed up process.
Certificate Management in Puppet
 Generate and Sign Certificates: Requesting certificate from agent and Sign on master ( Manually
signed Approach)

1. Run the puppet agent to generate a certificate for the puppet master to sign: puppet agent –t . It will give error
as no certificate found because certificate needs to be approved by Puppet master.

2. Log into to your Puppet master and list the certifications that need approval: puppet cert list , output -
"ip-172-31-48-109.ec2.internal“ (SHA256)
D9:8B:5C:0B:36:9A:7F:7C:F0:29:7B:03:BF:27:16:45:BC:FD:FC:74:16:5D:2E:B9:CC:7A:F0:FE:69:34:F4:23

3. Approve the certificate, replacing hostname.example.com with your node’s name (needs to be done on

puppetmaster: puppet cert sign ip-172-31-48-109.ec2.internal

4. See if the agent can finally communicate with the master (task performed on the puppet agent) , puppet agent –
t
5. If no errors, Puppet agent has been successfully authenticated with puppet master.
6. Remember if we do not give any configuration on puppet agent for server parameter , it will take by default
server name as puppet. If /etc/hosts file or DNS entry has puppet entry mapped with puppet master hostname,
puppet agent will communicate to puppet master by default without giving server name in puppet.conf file.
Certificate Management in Puppet
 Generate and Sign Certificates: Auto signing Certificate Requests

1. Naive Auto signing : It causes the Certificate authority (CA) to auto sign all Certificate signing requests (CSR’s).
2. Enabling Naive Auto signing : Set autosign = true in the [master] section of the CA Puppet master’s
puppet.conf
3. Never used above approach in production deployment. There may be security implications to use this approach.
4. Basic Auto signing ( autosign.conf ) : To enable basic autosigning, set autosign = <whitelist file> in
the [master] section of the CA Puppet master’s puppet.conf .
5. autosign.conf whitelist file is a list of certnames or domain name globs (one per line) like *.sapient.com, *.local
whose certificate requests will automatically be signed.
6. Above approach also have security implications as attacker able to guess an unused certname allowed by
autosign.conf would be able to obtain a signed agent certifcate from puppet master.
7. Policy-Based Auto signing : In policy-based auto signing, the CA will run an external policy executable every time it
receives a CSR. This executable will examine the CSR and tell the CA whether the certificate is approved for auto
signing. If the executable approves, the certificate is auto signed; if not, it is left for manual review
8. Enabling Policy-Based Autosigning: set autosign = <policy executable file> in the [master]section of the CA Puppet
master’s puppet.conf. The policy executable file must be executable by the same user as the Puppet master. Two keys
that can be used in this file can be “custom_attributes” key and/or an “extension_requests” key. Each key is an OID
(object identifier).
9. In this approach these attributes are passed in CSR’s from agent to master like puppetagentuid = value and master
will check this in executable file whether to sign certificate or not. To achieve this, we create csr_Attributes file on
agent and executable file like autosign.conf on puppet master.
Certificate Management in Puppet
 Revoking and Regenerating Certificates

1. TO REVOKE AND COMPLETELY REMOVE ALL PUPPET CERTIFICATES FOR AGENT: On Master, puppet cert
clean ip-172-31-48-109.ec2.internal
2. For Regenerating , remove /var/lib/puppet/ssl from puppet agent. Note: Make a backup of this directory .
3. Restart puppet agent service. This will again create new ssl directory for puppet agent.
4. Generate and sign the certificate again.
5. TO REVOKE AND MOVE THE CERTIFICATE TO A CERTIFICATE REVOCATION LIST : On Master, puppet cert revoke
ip-172-31-48-109.ec2.internal . Certificate move in carl “ Certification revoke list” “crl.pem”
6. In above revoke command , puppet agent will not able to regenerate the certificate unless it gets cleaned
from puppet master.

 Other Useful Commands

1. To list all of the puppet certificates : puppet cert list –all

2. To show the whole details of puppet certificate of agent : puppet cert print ip-172-31-48-109.ec2.internal
3. To clean all puppet certificates , puppet cert clean –all -- ( Use this command carefully )
 By:
By: Ravindra
Ravindra Mittal
Mittal

Puppet Session #2

27/10/2015
Agenda:

 Certificate Management - Policy Based auto

signing
 Managing Systems with Puppet Agent
 Introduction to Puppet language and terms
 Puppet- Resources and Types
 Manifests and variables
 Puppet Classes
 Testing Yourself-Exercise #1
Certificate Management – Policy based auto signing
 Policy based auto sign

1. In policy-based auto signing, the CA will run an external policy executable every time it receives a
CSR. This executable will examine the CSR and tell the CA whether the certificate is approved for
auto signing. If the executable approves, the certificate is auto signed; if not, it is left for manual
review
2. This is more secured approach because it will not allow every node to be signed and no person
can guess any pattern written in whitelist file for auto signing. In this approach csr attributes like
pp_uuid can be embedded in the Certifcate signing requests .
 Enabling Policy-Based Autosigning :

a) Create the file /etc/puppet/csr_attributes.yaml and add extension_requests:

pp_uuid: 960bc1c8f7afb8e180c6795ae1e6d413 on puppet agent. We can maintain different
pre shared key while provisioning the nodes.
b) Go to puppet master server and create a directory /etc/puppet/autosign/psk. Create the
file ip-172-31-48-109.ec2.internal with 960bc1c8f7afb8e180c6795ae1e6d413 as content.
c) Create a utility autosign_cert.pl. Once master receives CSR this policy executable file can
extract information from it and decided whether to approve the certificate for auto signing.
d) Call this utility in [master] section of puppet.conf file at puppet master.
e) Restart puppet master and request the certificate from agent to master. puppet agent -t
Managing Systems with Puppet Agent

 How puppet agents do configuration runs ?

• Every puppet agent should periodically do configuration runs, to revert unwanted changes and to
pick up recent updates. There are three main ways to do this:
• Run Puppet agent as a service. The easiest method. The Puppet agent daemon will do
configuration runs at a set interval, which can be configured.
• Make a cron job that runs Puppet agent. Requires more manual configuration, but a good choice
if you want to reduce the number of persistent processes on your systems.
• Only run Puppet agent on demand. To trigger runs on groups of systems, you can use Puppet
Enterprise’s built-in orchestration features or Mcollective.

 Run Puppet agent as a service:

• For starting either start by service as service puppet start or directly execute puppet agent.
• The Puppet agent service defaults to doing a configuration run every 30 minutes. You can
configure this with the run interval setting in puppet.conf

• This is not a good approach as if you have more puppet agents configured in the system , at every
2 hours every node will try to communicate with master at the same time which can impact the
performance of master.
Managing Systems with Puppet Agent

 Running Puppet Agent as a Cron Job

• Set simple crontab on each and every node. By running puppet agent using cron we can enable
all nodes to run periodically.
• create class crontab_test in site.pp on master.
• Send pull request from puppet agent to master at one time. /usr/bin/puppet agent --onetime
--no-daemonize
• Check , crontab –e on puppet agent : */1 * * * * touch /etc/puppet/crontab_test1.sh
There is a limitation in above type of crontab set. Let say, if we want to run periodic scripts which are
using common resources like taking a database dump , In this case we can end up being configured to
run at the same time for multiple puppet agents which results puppet runs or backups can take longer
and can cause an impact for that common resource. For avoiding such situations one way is to give
each cron task unique time manually or can use below approach.
 Random Start times for cron jobs with puppet.
• For generating random execution time we can use some external utilities or can use fqdn_rand
function in puppet.
• fqdn_rand function creates a random number between 0 and whatever we passes in based on
the fqdn of the host.
•
Managing Systems with Puppet Agent

 Random Start times for cron jobs with puppet.

• On puppet master : (/etc/puppet/manifests/site.pp)

• On puppet agent : crontab -l

• This gives a cron job which runs twice an hour but would reduce the chance of 2 running at the
same time.
• There are multiple utilities present on Internet or you can write your own to give random start
time for cron. One more way to use some math function in perl like taking ip address of the host,
sum up the number , and after dividing by some random value take its remainder . Eg:- 172-31-48-
109=360 , Rem 360/19(random value)= 18. Therefore; minutes = 18,48
Managing Systems with Puppet Agent

 Run Puppet agent on demand

• Some Infrastructure requirement is to run puppet agent on demand basis. Puppet agent can be
started and pull the configuration locally or remotely by using some orchestration tool.
• Locally:
• Run in the foreground, with verbose logging to the terminal or in background: puppet agent –
test, puppet agent --onetime
• Remotely:
• Use ssh or parallel ssh for many puppet agents or use some orchestration tool.
ssh [email protected] puppet agent –test
• For puppet Enterprise we have build in tools but for open source, most flexible tools we can use
is Mcollective .
• MCollective is a framework built for server orchestration or parallel job execution systems. In
other terms it programmatically executes actions on remote servers.
• MCollective is something unique and doesn’t depend on any central inventory or tools like SSH. It
operates on top of a pub-sub middleware tool and uses technologies like realtime discovery of
network resources using metadata.
• There will be a separate session for Mcollective where we will cover the orchestration part.
Introduction to Puppet language and terms

 Puppet Language

• A Declarative Domain Specific Language (DSL)

• It defines STATES (Not procedures)
• Puppet code is written in manifests (files with .pp extension)
• In the code we declare resources that affect elements of the system (files, packages, services ...)
• Resources are grouped in classes which may expose parameters that affect their behavior.
• Classes and configuration files are organized in modules.

 Nodes Classification
• When clients connect, the Puppet Master generates a catalog with the list of of the resources that clients have
to apply locally.
• The Puppet Master has to classify nodes and define for each of them:
• node 'ip-172-31-48-109.ec2.internal‘ {
}
• The classes to include
• The parameters to pass
• The Puppet environment to use
• The catalog is generated by the Master according to the logic of our Puppet code and data.
Puppet- Resources and Types

 Puppet Resources
• Resources are the fundamental unit for modelling your system configurations.
• Each resource describe a particular thing, e.g. a service that must be running, or a package that must be
installed.
• Resource Types are single units of configuration composed by:
• A type (package, service, file, user, mount, exec ...)
• A title (how is called and referred)
• Zero or more arguments
• type { 'title':   
argument  => value,   
other_arg => value,
}
• Example for a file resource type:
• file {'test.txt':
• ensure => present,
• path => '/tmp/test.txt',
• mode => 0777,
• content => "Hello ALL agents \n",
• }
• The resource’s title and the attribute names should always be lowercase.
Puppet- Resources and Types

 Puppet Resources

• There are various types of resources, sample are given below, rest you can find out puppet describe --list
command.
• Installation of OpenSSH package
• package { 'openssh':   
ensure => present,
}
• Creation of /etc/motd file
• file { 'motd':  
 path => '/etc/motd',
}
• Start of httpd service
• service { 'httpd':
  ensure => running,
  enable => true,
}
Resource Abstraction Layer (RAL)

 RAL

• Abstraction – Resources are described independently from the target operating system, i.e. a resource gives
enough info to tell you what needs to exist on a puppet agent, regardless of whether the puppet agent is a
windows or linux machine. Puppet does the abstraction by providing RAL layer in between platform and
description of configuration.

• It explains how actually puppet works after writing manifests. For puppet to run , it needs to compile a
catalog, send to puppet agent and then apply the catalog. When catalog is being applied all the actual work is
done by interacting to RAL.

• Puppet does abstraction using providers. Eg: If, we want to create resource type user on many platforms ,
puppet considers definition of that user to be identical. Puppet detects the platform of the node, and then
appropriate provider of that platform is called to create the resource user on that platform

• Transaction is the class which provides interface between catalog and RAL.

• To view list of providers, use puppet describe resource --providers { puppet describe user --providers}
Manifests and variables

 Manifests
• In Puppet resources are written into files using resource declaration syntax. These files are called manifests
which end with .pp extension
• The puppet master always reads one special manifest, called the “site manifest” or
/etc/puppet/manifests/site.pp. It always reads this to compile a catalog, which it sends back to the agent. It
resides only on puppet master.
 Variables
• Variables store values so they can be accessed later. Two types of variables , Built in variables and
user variables.
• Built in variables:
• $servername, $serverip – Puppet master FQDN and IP
• $serverversion — the current version of Puppet on the Puppet master.
• User Variables:
• $content = "some content\n“
• [$a, $b, $c] = [1,2,3] # $a = 1, $b = 2, $c = 3
• file {'/tmp/testing':
ensure => file,
content => $content,
}
Classes

 Classes
• Classes are containers of different resources. Classes generally configure large or medium-sized chunks of
functionality, such as all of the packages, config files, and services needed to run an application.
 Defining Classes:
• Defining a class makes it available for later use.
• Example of a class definition with parameters:
Classes

 Classes
• In Puppet we use “classes” to reduce code repetition.
• For example: cat site.pp

• Here we are using same code for two different nodes, i.e. we are repeating the code.
• Rewriting the resource by classes and declaring it.
• Here we are declaring classes by class (xxxx:) to add the class with node definition , it
is one way of declaring a class. Will touch on declaring classes in coming slides.
Classes- Parameters and Variables

 Class Parameters and Variables

• Parameters allow a class to request external data. Each class parameter can be used as a normal variable
 inside the class definition.
• For Example: In previous example for first agent we want user homer and second agent to have user bart,
then we cant use the homer_simpson class in it’s current form. We need to make it more versatile by accepting
parameters or class parameters.
• We can also pass default value for a class paramater:
Puppet Classes

 Declaring class :
• Declaring a class means adding a class to nodes for applying configuration in the class on each node defined.
• Include-Like Syntax:
• node 'PuppetAgent1' {
include user_account
}
• There is a limitation with this, which is that the include syntax doesn’t let you specify any class parameters.

• Resource-Like Syntax:

• There is limitation with the above as can’t declared the class more than once, even with different class-
parameters. Solution for this, Rewrite your class into very own customized resource type, which in puppet is
referred to as a “defined type”
Puppet Classes

 Defined resource types

• Defined resource types are blocks of Puppet code that can be evaluated multiple times with different
parameters within same node configuration.
• Definition:
•

• Declaration:

• Note: $title should be different for each manifests otherwise catalog will fail.
Testing Yourself-Exercise #1

 Exercise #1:

• Setup one master and at least 1 puppet agent.

• Enable policy based auto signing .

• Create a new manifest from the file /tmp/testfile.txt.

• Change the file mode to be group writeable. (either 0664 or ug=rw,o=r)

• Change the content to say something that amuses you.

• Retrieve above configuration on puppet agent and verify that file is present with proper permissions and
content.
Thank You!!!

32
Class Inheritance----incomplete

 Class Inheritance
• Classes can be derived from other classes using the inherits keyword.
• Inheritance is used when subclass inherits resources from the parent class.
• For example: We need to create a module that let’s you set up any of the following scenarios on
each of your nodes:
• Creates a user (called ‘homer’) only
• Creates a user (called ‘homer’) and textfile.
• Creates a user (called ‘homer’) and ensures a service is running
• Here, user homer must always exist.
• There are 4 main ways to achieve this in your module’s design:
• create 3 classes, one for each resource. In this scenario, you would have 3 manifests, e.g., init.pp
(which contains the user_account class), ensure_file.pp, and ensure_service.pp.
• have all resources in a single class in the init.pp file, and control weather file or service resource
using class parameters.
• Write a class for each resource, assume the “user” resource is the main class. Also the file and
service class also includes a user class.
• Use inheritance