Chef Essentials

John Ewart









Chapter No.1
"Installing Chef"
In this package, you will find:
The authors biography
A preview chapter from the book, Chapter no.1 "Installing Chef"
A synopsis of the books content
Information on where to buy this book















About the Author
John Ewart is a system architect, software developer, and lecturer. He has designed and
taught courses at a variety of institutions, including the University of California, The
California State University, and local community colleges. These courses cover a wide
range of computer science topics, including Java, data structures and algorithms,
operating systems fundamentals, Unix and Linux system administration, and web
application development. In addition to working and teaching, he maintains and
contributes to a number of open source projects. He currently resides in Redmond,
Washington, with his wife, Mary, and their two children.


For More Information:
www.packtpub.com/application-development/chef-essentials


Chef Essentials
Chef is an indispensable tool to manage your infrastructure. It consists of a set of tools
that are designed to work together to enable you to model and manage your systems. This
is a large space to fill, and Chef provides you with the tools to do this in a very flexible
and powerful way. It achieves this through a combination of services, end host agents, a
web interface, and command-line tools that work in unison to deliver an incredible suite
of tools.
Chef's services are responsible for storing, managing, and distributing data about your
infrastructure through an API. Endhost software agents that run on nodes (managed
systems) are responsible for performing updates to systems, and the web interface, along
with command-line tools, allows an administrator to edit and consume information that is
vended by the API service.
One of the most attractive features of Chef is that you can leverage its API to easily
integrate existing tools, or you can develop new tools to meet specific needs. Any
organization with a moderate number of developer resources can harness the power of
Chef to manage their systems. For example, one can easily build software to import data
from Chef into a reporting tool of some form and dynamically reconfigure infrastructure
based on a third-party tool's outputthe sky's the limit. This is incredibly valuable to
anybody who has an existing infrastructure because it provides a convenient path to
integrate Chef into their environment.
There are a number of ways to access Chef. The quickest way for a single user to manage
his/her infrastructure (virtual machines, a handful of hosts, and so on) is to use Chef-solo,
a product geared towards single-user environments. In a small environment, setting up a
hosted server is a good way to manage infrastructure automation among team members.
If you need to, you can configure the hosted environment as a highly available system
using load balancers and other technologies. Alternatively, if hosting the service yourself
is not an option, you can use hosted Chef, a software-as-a-service (SaaS) model, thus
paying for access to a hosted service.
Configuration management software was created to fill a needmanaging infrastructure
is a challenging task. Regardless of the scale you operate on, keeping track of software
versions, upgrading systems, and generating consistent configuration data is a lot of
work. It is tempting to update a configuration file on one system, only to forget to commit
those changes somewhere, or to apply them to existing or future hosts. This is very
convenient, but it quickly leads to inconsistency between hosts. When you are working
with only one or two hosts, this may be acceptable. As such, a system grows from a few
servers to dozens, hundreds, or possibly thousandsthis type of system management
does not scale due to time requirements and configuration errors that result from size
and complexity.


For More Information:
www.packtpub.com/application-development/chef-essentials

Consider a scenario where you are migrating a database server to a new host. This would
involve: bringing up a new host, installing all of the required software on your list to
ensure it has parity with the old server (you did keep a list, right?), ensuring that your
database server was configured with the same options, putting the correct firewall rules in
place, tuning the filesystem, setting up monitoring tools, updating DNS records or
changing web application configurations to point to the new host, and so on. Now
imagine that, instead of one server in one data center, you have 10 database servers in 10
data centers, each with their own IP ranges, hardware configurations, and networking
rules. Situations such as this are exactly why system configuration management software
packages were developed: to make the lives of system administrators and engineers
much easier.
This scenario, and many others like it, is where Chef is indispensable. Having the ability
to describe your hosts, configuration data, and roles, and then apply that across as many
hosts as you like means that you can manage large fleets of hosts just as easily as you can
manage one or two.
What This Book Covers
Chapter 1, Installing Chef, introduces you to the architecture of Chef, various installation
methods, and a guide to setting up Chef (solo and self-hosted). It includes information on
using hosted Chef (and what that means for your team) and Vagrant with Chef-solo.
Chapter 2, Modeling Your Infrastructure, introduces how to model your infrastructure
with Chef using your newly installed system. This chapter will cover modeling
environments, small and large, as well as how to integrate with cloud technologies using
Chef (AWS, Rackspace Cloud, and so on).
Chapter 3, Integrating with the Cloud, covers how Chef helps you scale your
infrastructure using any combination of physical, virtual, and cloud-hosted systems. This
chapter discusses how to use Chef to provision and manage hosts using cloud providers
as easily as your local systems including AWS and Rackspace Cloud.
Chapter 4, Working with Cookbooks, covers how every Chef needs cookbooks once
your systems are part of your Chef-managed fleet, you can begin collecting, developing,
and applying recipes to your hosts. It includes in-depth explanations of the structure and
development of cookbooks and recipes, as well as how to test, publish, and share them.
Chapter 5, Testing Your Recipes, focuses on one compelling reason to use Chef to
configure your infrastructure, that is, recipes are written in Ruby code and can be tested
as any program would be tested. Here, you will learn how to test your recipes through a
variety of testing mechanisms.



For More Information:
www.packtpub.com/application-development/chef-essentials


Chapter 6, From Development to Deployment, covers how to take a custom application
from development to a production deployment with Chef. It contains a complete example
that includes provisioning a web server, database server, and users as well as deploying
code from source control.
Chapter 7, Beyond Basic Recipes and Cookbooks, delves into developing extensions to
Chef through advanced concepts, including custom providers and resource types, using
the Chef search engine, advanced scripting, and more.
Chapter 8, Extras You Need to Know, expands your knowledge of how to leverage Chef
for infrastructure automation, complex systems integration, and securely storing and
distributing sensitive data with Chef.


For More Information:
www.packtpub.com/application-development/chef-essentials

Installing Chef
Before you can start using Chef, you will need to install it. Here you will nd a guide
to install Chef, and because Chef requires Ruby, some Ruby concepts as well. This
chapter discusses the following:
Key terminology and concepts related to Chef
An overview of Chef's architecture
Working with Ruby gems
Installing chef-solo (a local-only engine to use Chef)
A brief example on using chef-solo
Installing the Chef server on your own host
Verifying your Chef installation
Terminology
As with any other technology, Chef has its own terminology. As you will see,
Chef's nomenclature is a mix of technological terms (nodes, workstations, servers,
roles, and so on) and cooking terms (cookbooks, recipes, and so on). There are three
primary actors that we are concerned with at this point: nodes, the Chef service,
and workstations.
Node: A node is a client that applies roles and recipes, as described by the
administrator in the Chef service (that is, a server in your environment that is
being congured via Chef). These are the consumers of the conguration, the
elements of your infrastructure. They can be physical or virtual machines and
can run on Linux, Windows, or technically any other system that is capable of
running Ruby (some systems may not be supported out of the box by Chef).


For More Information:
www.packtpub.com/application-development/chef-essentials

Installing Chef
[ 8 ]
Chef service: The Chef service is a multicomponent system that combines
several services to provide its functionality. The primary functional
components are an API service, full-text searching via Solr, persistent
storage using PostgreSQL, and RabbitMQ for interservice communication.
Additionally, there is a web interface that provides a graphical tool to
manage system data. Clients (nodes) use the API service to determine which
roles and recipes to apply, and knife (a command-line tool) uses the API to
allow an administrator to edit and manage their Chef conguration.
Workstation: A workstation is a host that is used to issue commands. A
workstation can be a separate host outside of your Chef service installation, a
managed node, or the server that the Chef components are running on. There
are a variety of command-line tools that are provided to interact with the
service, which will be installed onto your workstation(s).
Recipe: A recipe is a script that describes a set of steps to take to achieve a
specic goal. As an example, a recipe might describe how to deploy your
custom software, provision a database, or add a host to a load balancer.
Cookbook: A cookbook is a collection of recipes that are used to collectively
describe how to install, congure, and manage various aspects of your
infrastructure. For example, a cookbook might describe how to provision
MySQL, PostgreSQL or Apache, manage users, install printers, or perform
any other system tasks.
Working with Chef
For single user setups, chef-solo is a version of the chef-client that allows you to use
Chef without requiring access to a server. Chef-solo runs locally and requires that a
cookbook and any of its dependencies be present on the node being managed. As a
result, chef-solo provides a limited subset of the full chef server mode of operation.
Most of the features that chef-solo is missing revolve around search and centralized
data management, which are not critical for managing virtual machines or a small
collection of nodes. The installation and maintenance is simple, but the feature set
is smaller.
Installing the Chef server will give you access to the full set of Chef functionality.
This mode requires access to a Linux-based host that is network-accessible by the
nodes and workstations that will interact with the system. Thanks to the recent effort
from the folks at Chef (formerly Opscode), the process of installing Chef has been
greatly simplied. The benets of this installation model are that you get centralized
management, search, user authentication, and such, but at the cost of managing your
own service.


For More Information:
www.packtpub.com/application-development/chef-essentials

Chapter 1
[ 9 ]
If you need the features of Chef but do not want to maintain your own server,
hosted Chef is a great option for you. Hosted Chef (https://manage.opscode.com/
signup) gives you all the features of a self-hosted Chef but without having to worry
about upgrades, extra hardware, or system availability. For a small infrastructure
(up to ve hosts), hosted Chef is free and a great way to get started. Beyond this,
plans have a monthly fee, and the price will vary according to the number of hosts
you want to manage.
Installing chef-solo
Chef-solo is designed for individuals who do not need a hosted installation
for a large-scale infrastructure management. Typical use cases of chef-solo
include developers managing virtual machines, test installations, or small-scale
infrastructure management. The installation of chef-solo is as simple as installing
a single Ruby gem.
The Ruby gem
For those who are not intimately familiar with Ruby, a Ruby gem is a mechanism
to package, deliver, and manage Ruby code. These packages may be libraries that
provide functionality for developers, or they may be composed only of scripts and
tools. Chef-solo is, like many things in life, somewhere in the middle. The gem
contains a set of libraries that make up the core functionality as well as a suite of
scripts that are used by end users. Before you install Chef, you should consider
installing Ruby Version Manager (RVM), rbenv, chruby, or another Ruby manager
of your choice to keep your gem collections isolated.
Managing gems
A great tool to manage your gems is RVM. The simplest way to install RVM is to
use the installation script provided by the development team on the RVM website
(http://rvm.io). The following command will download the script and pipe it
through bash:
curl -sSL https://get.rvm.io | bash -s stable
Once it is installed, you will initially need to include RVM's functionality in your shell:
source ~/.rvm/scripts/rvm


For More Information:
www.packtpub.com/application-development/chef-essentials

Installing Chef
[ 10 ]
Additionally, you might need to add the previous command line to your shell's
startup scripts (such as ~/.bashrc or ~/.zshrc, depending on which shell you use).
Once RVM is installed, you will want to install a recent version of Ruby, for example,
Ruby 1.9.3:
rvm install 1.9.3
Once Ruby 1.9 is installed, you will want to create a gemset. A gemset is RVM's way
of isolating gems inside a container, and it will provide you with a place to install
gems in such a way that they will not conict with other gems. This has the benet
of allowing you to install anything you want, without requiring administrative
privileges and keeping gems from conicting with each other. A gemset can be
created using the following command:
rvm use 1.9.3@chef --create
The previous command will simultaneously create the gemset named chef (if it does
not exist) for your installation of Ruby 1.9.3 and then set it as the active gemset. Once
you start using this new gemset, you will want to install the Chef gemthis contains
chef-solo and all the command-line tools you need to work with Chefusing the gem
command-line tool:
gem install chef
Verifying that chef-solo works
Now that the Chef gem is installed, it is time to verify that everything is working
ne. In order to use chef-solo, you need to give the following information to it:
What recipes to apply by providing a run list in a le named node.json
What the recipes arethese are stored in cookbooks that are found in
the cookbooks directory
How to nd the cookbooks and the run list via a le named solo.rb
For simplicity, we will store all of these les inside of the chef directory in your
home directory. You are free to put things where you see t as you become more
comfortable working with Chef.
In order to exercise our new tool, we will do something simple: we'll write a recipe
that will create an example.txt le in your home directory. The recipe we create
will be called create_file, and we'll put that recipe inside a cookbook, which will
be named demo.


For More Information:
www.packtpub.com/application-development/chef-essentials

Chapter 1
[ 11 ]
First, create the directory that will contain the demo cookbook's recipes (and any
in between):
user@host:~ $ mkdir -p ~/chef/cookbooks/demo/recipes
Next, add the following code to a le, create_file.rb, located in the demo
cookbook directory you created at ~/chef/cookbooks/demo/recipes:
file "#{ENV['HOME']}/example.txt" do
action :create
content "Greetings #{ENV['USER']}!"
end
This tells Chef that we want to create a le, $HOME/example.txt. Its contents
should be Greetings $USER, where $USER will be replaced with the value of
$USER, typically the login name of whoever is executing the recipe.
For those unfamiliar, UNIX (and Windows as well) uses
environment variables as a mechanism to exchange data between
processes. Some environment variables are set when the user logs in
to the system such as HOME, USER, and a variety of others. These
variables are available in Ruby using the ENV hash, where the keys
are the variable names. In a UNIX shell, these are accessed using the
$ prex. So, the user's home is referred to as $HOME in the shell and
ENV['HOME'] inside Ruby.
Now we will need to create a JSON document that describes what chef-solo should
execute. JSON is an acronym for JavaScript Object Notation, and Chef uses JSON
extensively because it is easy to parse, human readable, and easy to generate from all
sorts of tools and languages. Create a le, node.json, located in our work directory
(~/chef/ in this case) and add the following content in order to tell Chef that we
want to execute the newly created create_file recipe in the demo cookbook:
{
"run_list": [
"recipe[demo::create_file]"
]
}
Here, we are dening the node as having a run list, which is just an array of things
to do, and that the run list contains one recipe, create_file, which it can nd in the
demo cookbook (the general form of a recipe being cookbook::recipe).


For More Information:
www.packtpub.com/application-development/chef-essentials

Installing Chef
[ 12 ]
Finally, we'll tell Chef where to nd the les we just created using a solo.rb le that
we will store in our working directory (~/chef in our case):
CHEF_ROOT="#{ENV['HOME']}/chef"
file_cache_path "#{CHEF_ROOT}"
cookbook_path "#{CHEF_ROOT}/cookbooks"
json_attribs "#{CHEF_ROOT}/node.json"
Now that you have populated the required conguration les, you can run
chef-solo and execute the run list specied. In our case, the run list is dened as
only one recipe, create_file, but can be as simple or as complex as needed. The
previous conguration tells Chef to load the node conguration from the le node.
json to look for cookbooks in ~/chef/cookbooks/ and to store any state data in
~/chef/. In order to execute these commands, you will want to run chef-solo:
chef-solo -c ~/chef/solo.rb
The -c option tells chef-solo which script contains the conguration. Once you do
this, you will see the actions that your recipe is performing:
Starting Chef Client, version 11.8.2
Compiling Cookbooks...
Converging 1 resources
Recipe: demo::create_file
* file[/home/user/example.txt] action create
- create new file /home/user/example.txt
- update content in file /home/user/example.txt from none to b4a3cc
--- /home/user/example.txt 2014-01-20 23:59:54.692819000
-0500
+++ /tmp/.example.txt20140122-13411-1vxtg7v 2014-01-20
23:59:54.692819000 -0500
@@ -1 +1,2 @@
+Greetings user!
Chef Client finished, 1 resources updated
Once it is completed, you will see that ~/example.txt contains the greeting that you
dened in the recipe. Now that you have successfully used chef-solo, let's move on to
the Chef service.


For More Information:
www.packtpub.com/application-development/chef-essentials

Chapter 1
[ 13 ]
Installing a Chef server
If your team needs to have centralized infrastructure management and does not
want to use a hosted platform, then a self-installed Chef server is a perfect t. This
installation guide assumes that you will be running the Chef server on a supported
Linux-based system.
The Chef service components can be installed on a single machine without any issue.
Installing it on a single host will limit your ability to scale or be highly available, but
will provide a very simple path to getting started with the Chef service.
Requirements and recent changes
Since the Chef service is designed to be a multiuser platform and provides
functionalities that chef-solo does not offer, the installation is more complex and
involves more software to achieve this functionality. Services such as Solr for full-text
indexing and PostgreSQL for data storage can be a signicant resource for consumers,
so you will want to install Chef on a host with sufcient memory and disk space.
A system with 2 GB of memory and 5-10 GB of disk space available will be plenty
for a small to medium sized installation. You will need more resources as your
requirements for data storage and indexing increase over time, so plan accordingly.
Additionally, for those who have installed the Chef server before, the installation path
has been greatly simplied. In addition to replacing CouchDB with PostgreSQL as the
primary data storage engine, there is now a single omnibus installation package for
Chef that installs all of the requirements for Chef at a single location so that it operates
in isolation and does not require dependencies to be installed separately.
Installation requirements
In order to install the Chef service, you will need to have the following:
A system running a supported Linux variant (64 bit Ubuntu Linux 10.04
through 12.10 or 64 bit Red Hat Enterprise Linux 5 or 6)this can be physical
or virtual. If you do not have the local resources for this, AWS or RackSpace
cloud servers are good options.
A network connection to the host in order to download the installer.
Administrative privileges (using sudo or direct root access) on the host
where you are installing the services.
Enough free space on the host to perform the download and installation
(minimum 500 MB, including the download, but 1 GB to 2 GB is preferred).


For More Information:
www.packtpub.com/application-development/chef-essentials

Installing Chef
[ 14 ]
What you will be installing
At the end of this section, you will have a fully functional Chef service installed and
ready to work with. Before you get started, let's look at what you will be installing on
your system so that you know what to expect. The components that make up a Chef
service are as follows:
The Chef API service
Message queue (AMQP)
Data storage
Search service
Web-based management console
The Chef API service is responsible for delivering run lists and receiving information
from nodes as well as providing a way for a system administrator to congure
recipes, run lists, data bags, and the like. In order to generate this data, the API
service relies on its persistent data storage engine, in this case PostgreSQL, to store
its data. The option to search for data is provided by the Solr search engine, and
RabbitMQ is responsible for gluing them all together. Together, these components
provide Chef with the ability to distribute, store, index, and manage your
infrastructure's conguration data.


For More Information:
www.packtpub.com/application-development/chef-essentials

Chapter 1
[ 15 ]
Getting the installer
The easiest way to install Chef is through a single downloadable package, which is
provided for Ubuntu 10.04 through 12.10 and Red Hat Enterprise Linux Versions 5
and 6. This package, referred to as the omnibus installer, contains everything you
need to get a server up and running. You can nd it on http://www.getchef.com/
chef/install/.
At the time of writing this, 11.0.10 is the latest version and is the one that will be used
for this book. The newer version of the 11.x series of Chef should have a very similar,
if not identical, conguration. Note that these installers are somewhat large, being
that they contain all of the dependencies needed. For example, the Ubuntu 12.10
package for Chef 11.0.10 is approximately 200 MB in size.
Although these are the ofcially supported distributions and
releases, it is entirely possible that these installers will work on
different but compatible distributions. It may be possible, for
example, to use CentOS instead of Red Hat Enterprise Linux or
Debian instead of Ubuntu. However, these will most likely require
some manual dependency resolutions and may not work without a
lot of effort (and even then, possibly not at all).
Installation outline
Installation on all supported platforms is relatively similar. The only key differences
are the names of the package les that you will download and the commands you
will use to install Chef.
The high-level steps you will take are as follows:
1. Downloading the Chef installer for your platform.
2. Installing the package as an administrative user.
3. Conguring the Chef service.
4. Testing the server using command-line tools.
Because steps 3 and 4 will be the same for both Ubuntu and Red Hat installation
procedures, the instructions will be in a section following the Red Hat installation guide.
Installing on Ubuntu
The following are instructions for an Ubuntu-based system; they were performed on
an Ubuntu 12.04 host, but should be identical for all supported Ubuntu distributions.
For Red Hat-based installation instructions, see the next section.


For More Information:
www.packtpub.com/application-development/chef-essentials

Installing Chef
[ 16 ]
Downloading the package
You can download the package by returning to the download page referenced
previously (http://www.getchef.com/chef/install/), or you can download
Version 11.0.10 directly from https://opscode-omnibus-packages.
s3.amazonaws.com/ubuntu/12.04/x86_64/chef-server_11.0.10-1.
ubuntu.12.04_amd64.deb.
Installing the package
In order to perform the installation, open a terminal on your Ubuntu host (either
locally or connect via SSH) as a user who has administrative privileges. This can be
done directly either as the root or any user who has permission to execute arbitrary
commands via sudo.
Once you log in to the host, navigate to where you want to store the package
(remember it's quite large, approximately 200 MB) and download the le using curl:
user@ubuntu:~ $ curl -O https://opscode-omnibus-packages.s3.amazonaws.
com/ubuntu/12.04/x86_64/chef-server_11.0.10-1.ubuntu.12.04_amd64.deb
Once the le is downloaded, the dpkg tool will be used to perform the package
installation:
user@ubuntu:~ $ sudo dpkg -i chef-server_11.0.10-1.ubuntu.12.04_amd64.deb
Once this is nished, the Ubuntu-specic portion of the setup is complete, and you
will need to congure Chef using the chef-server-ctl command, which we will
discuss in the Conguring Chef Server section, following the Installing on Red Hat
Enterprise Linux section.
Installing on Red Hat Enterprise Linux
Installation on a Red Hat Enterprise Linux distribution is as straightforward as
installing any other package. You download the package to the local disk and
install it using RPM tools.
Downloading the package
You can download the latest version of the package by returning to the download
page referenced previously (http://www.getchef.com/chef/install/), or you
can download Version 11.0.10 directly from https://opscode-omnibus-packages.
s3.amazonaws.com/el/6/x86_64/chef-server-11.0.10-1.el6.x86_64.rpm.


For More Information:
www.packtpub.com/application-development/chef-essentials

Chapter 1
[ 17 ]
In order to perform the installation, open a terminal on your Red Hat host (either
locally or connect via SSH) as a user who has administrative privileges. This can be
done directly either as the root or any user who has permission to execute arbitrary
commands via sudo.
Once you log in to the host, navigate to where you want to store the package
(remember it's quite large, approximately 200 MB) and download the le using curl:
user@rhel:~ $ curl -O https://opscode-omnibus-packages.s3.amazonaws.com/
el/6/x86_64/chef-server-11.0.10-1.el6.x86_64.rpm
How long this takes will vary according to the available bandwidth but should take
somewhere between 5 and 20 minutes on a reasonably fast connection.
Once the le is downloaded, the rpm tool will be used to perform the package
installation:
user@rhel:~ $ sudo rpm -ivh chef-server-11.0.10-1.el6.x86_64.rpm
Once this is nished, the Red Hat-specic portion of the setup is complete, and you
will need to congure Chef using the chef-server-ctl command, which we will
discuss in the following section.
Conguring a Chef server
Historically, installing Chef requires manual editing of conguration les, choosing
RabbitMQ credentials, installing CouchDB, and a handful of other tasks. Now, with
the omnibus installer, all of this is taken care of for you. If you have been following
along, your system has the Chef server and all of its dependencies installed on the
system in the /opt/chef-server directory.
Included with the installation of the package is a shell script, chef-server-ctl
(located at /opt/chef-server/bin), which is responsible for conguring your
newly installed Chef server. In order to congure your services, you will need to
run it as root because the scripts will need to modify your system in ways that your
regular account may not be able to. Initializing the conguration tool is as simple as
issuing the following command:
sudo chef-server-ctl reconfigure
Running this script may take a few minutes, and it will produce a lot of output while
it is doing its work. While it is running, let's take a few minutes to discuss how it
works and what it is doing.


For More Information:
www.packtpub.com/application-development/chef-essentials

Installing Chef
[ 18 ]
Understanding how chef-server-ctl works
Earlier in this chapter, you were briey introduced to the chef-solo tool. You
saw how it can be used to manage your local server using on-disk recipes and
conguration data. The Chef team has leveraged this ability to do just that with
the Chef server using chef-solo to bootstrap the server conguration. If you were
to look at the code for the /opt/chef-server/bin/chef-server-ctl script, you
would see that the last line in the script executes the following command:
/opt/chef-server/embedded/bin/omnibus-ctl chef-server /opt/chef-server/
embedded/service/omnibus-ctl $@
If you follow the trail and dig into the omnibus-ctl script, you will nd that it is just
a wrapper around the omnibus-ctl Ruby gem. Digging into the omnibus-ctl gem,
you will see that in the end, the reconfigure command you pass on the command
line is a Ruby method that makes the following call:
run_command("chef-solo -c #{base_path}/embedded/cookbooks/solo.rb -j
#{base_path}/embedded/cookbooks/dna.json")
This tells us that the Chef omnibus package uses chef-solo to congure itselfa
pretty clever trick indeed! You can see just how powerful a tool chef-solo can be,
being able to congure and recongure the Chef service.
What's happening on my server?
What you will probably notice right away is that a lot of text is being scrolled past
in your terminal window. If you were to look at the contents, you would see that
it shows you the actions that are being taken by chef-solo to provision your new
services. As there is a lot of information going past (thousands of lines), here is a
high-level overview of what is happening on your host:
1. A new user, chef_server, and its corresponding group are being provisioned.
2. Chef services are being set up, and startup scripts for upstart are being
placed in the appropriate system directories. The Run scripts for Chef
services are located at /opt/chef-server/sv.
3. Chef state directories are being created in /var including /var/opt/chef-
server and /var/log/chef-server.
4. RabbitMQ is being congured to store data in /var/opt/chef-server and
log the output to /var/log/chef-server as well as its startup scripts in /
opt/chef-server/sv/rabbitmq/run.


For More Information:
www.packtpub.com/application-development/chef-essentials

Chapter 1
[ 19 ]
5. PostgreSQL is being congured with its data in /var/opt/chef-server/
postgresql/data along with a user, opscode-pgsql, to run the service.
Some system-level changes to share memory sizes are being set via sysctl
to make PostgreSQL work as well as persisted in systctl.conf.
6. Solr is being set up to work with the conguration and data rooted in
/var/opt/chef-server/chef-solr/, with the run script being placed
in /opt/chef-server/sv/chef-solr/run.
7. Chef-expander (the data-indexing service) is being congured for /var/
opt/chef-server/chef-expander as its working directory with Solr and
RabbitMQ endpoints on the localhost. The run script is located at /opt/
chef-server/sv/chef-expander/run.
8. The Chef bookshelf metadata service is readied in /var/opt/chef-server/
bookshelf/ with its run script at /opt/chef-server/sv/bookshelf/run.
9. Erchef, the Erlang Chef service, is installed and pointed at the local Solr,
RabbitMQ, bookshelf, and PostgreSQL services.
10. The system is then bootstrapped using the bootstrap recipe. This recipe
veries that the system is running (by checking that the http://
localhost:8000/_status returns an HTTP 200 response) and installs the SSL
certicate for the web-based UI in /etc/chef-server/chef-webui.pem.
11. The web-based UI conguration les are generated and placed in /var/opt/
chef-server/chef-server-webui/.
12. A copy of nginx to host the web UI is placed in /var/opt/chef-server/
nginx, and the initial self-signed SSL certicates as well as the static assets
are installed in /var/opt/chef-server/nginx/html.
13. The Chef API testing framework, chef-pedant, is installed.
14. Finally, /etc/chef-server/chef-server-running.json is generated with
the current conguration settings for your Chef services.
Clearly, there is a lot happening here; if you have any outstanding concerns about
what is being done, be sure to read through the output. One of the great things
about Chef is that the recipes are just a set of scripts that you can open and view
the contents of, and the output shows you what is happening during the execution.
Everything it does is transparent and manageable by you.


For More Information:
www.packtpub.com/application-development/chef-essentials

Installing Chef
[ 20 ]
Verifying that the services are running
Once the conguration of your services is complete, you will want to validate
that the required services are running. Again, the chef-server-ctl script will
be used, but we will invoke the status subcommand instead of the reconfigure
subcommand, as shown in the following code:
user@host:~ $ sudo chef-server-ctl status
run: bookshelf: (pid 3901) 3123s; run: log: (pid 3900) 3123s
run: chef-expander: (pid 3861) 3129s; run: log: (pid 3860) 3129s
run: chef-server-webui: (pid 4053) 3095s; run: log: (pid 4052) 3095s
run: chef-solr: (pid 3819) 3135s; run: log: (pid 3818) 3135s
run: erchef: (pid 4230) 3062s; run: log: (pid 3937) 3117s
run: nginx: (pid 4214) 3064s; run: log: (pid 4213) 3064s
run: postgresql: (pid 3729) 3146s; run: log: (pid 3728) 3146s
run: rabbitmq: (pid 3423) 3172s; run: log: (pid 3422) 3172s
The status subcommand will show you the process ID of each component, how long
it has been running for, the PID of the logging process associated with that service,
and how long the logging service has been running. For example, we can see that
chef-server-webui has a PID of 4053 and has been running for close to an hour,
and the logger has a PID of 4052, having been running for just as long as the service.
As you can see, the installation of Chef yields a number of components that will
need to be up and running in order to successfully use Chef. You should have the
following components running and listening on the following network ports:
Component What to look for in the process list Port(s) Public?
Chef API server Erchef and nginx 80, 443 Yes
Web management
console
chef-server-webui and nginx 80, 443 Yes
Data indexer chef-expander N/A N/A
Solr java (running start.jar in the Chef
directory)
8,983 No
PostgreSQL postmaster 5,432 No
RabbitMQ beam.smp running rabbit 5,672 No


For More Information:
www.packtpub.com/application-development/chef-essentials

Chapter 1
[ 21 ]
Public components need to be made available to any clients, nodes, or end users that
expect to use the Chef service over the network. Conguring your infrastructure to
ensure that your services are available via the network is outside of the scope of this
book as there are a near-innite number of possible network congurations.
At a higher level, make sure that any rewall devices or packet-ltering systems are
not preventing trafc from reaching these services if you see that they are running,
but are having difculties in connecting to them. If any of these services are not
running, you will need to consult the logles generated by the service to determine
what might be preventing them from starting up.
Validating that your service is working
In order to work with Chef, you will need a way to interact with it. Fortunately,
Chef provides a suite of command-line utilities, which we will discuss at length as
the book progresses. There is one primary tool, knife, that allows an administrator
to interact with the service in the command line. The knife tool is run from a
workstation and provides many commands to view, search, and modify data
maintained by the Chef service. Once you have installed and veried that all the
services are running, we can move on to setting up knife.
You will see that the standard place to store your Chef conguration
data is in $HOME/.chef (on a UNIX-like system.) This is not
mandatory, and these les can be stored anywhere you like.
The knife tool communicates with the Chef server via HTTP and uses certicates
for authentication between the workstation and the server. In order to get started
with knife, we will need to do two things: gain access to the certicates that were
generated during the installation of Chef and then use those credentials to set up a
new user in the system.
In the following examples, we will be using the host that the Chef services were
installed on as our workstation (where we will use knife). If you want to use a
different host, you will need to get the required certicate (.pem) les to your local
machine using scp or some other mechanism. By using the following commands,
we can get the required authentication materials into our work directory:
mkdir $HOME/.chef
sudo cp /etc/chef-server/admin.pem $HOME/.chef
sudo cp /etc/chef-server/chef-validator.pem $HOME/.chef
sudo cp /etc/chef-server/chef-webui.pem $HOME/.chef
sudo chown R $UID $HOME/.chef


For More Information:
www.packtpub.com/application-development/chef-essentials

Installing Chef
[ 22 ]
Chef uses a signed header authentication for requests to the API, which
means there must be a shared key that is present on both the client and
the server. Chef-server will generate the chef-validator.pem le
when it is congured. New nodes or clients use the chef-validator.
pem le to sign the requests used to register themselves with the system.
Once you have these les copied into your Chef work directory, it is time to
congure knife itself. Fortunately, knife has an interactive conguration mode that
will walk you through the process of generating a conguration le. First, ensure
that you are using your Chef gemset (if you are using RVM as we discussed earlier)
and then run knife on your workstation (again, in this example, we are using our
Chef service host for both purposes):
user@chef:~$ rvm use 1.9.3@chef
user@chef:~$ knife configure -i
When you run knife with the -i ag, you will be prompted by the following
questions, which you can answer with the defaults for almost everything
(non-default answers are in bold):
WARNING: No knife configuration file found
Where should I put the config file? [/home/user/.chef/knife.rb]
Please enter the chef server URL: [https://localhost:443]
Please enter a name for the new user: [user]
Please enter the existing admin name: [admin]
Please enter the location of the existing admin's private key: [/etc/
chef-server/admin.pem] ~/.chef/admin.pem
Please enter the validation clientname: [chef-validator]
Please enter the location of the validation key: [/etc/chef-server/chef-
validator.pem] ~/.chef/chef-validator.pem
Please enter the path to a chef repository (or leave blank):
Creating initial API user...
Please enter a password for the new user:
Created user[user]
Configuration file written to /home/user/.chef/knife.rb
user@chef:~$


For More Information:
www.packtpub.com/application-development/chef-essentials

Chapter 1
[ 23 ]
As mentioned earlier, this does two things:
First, it uses the validation key and client name specied at the prompts to
contact the API service and register a new client (user) with the service
Secondly, it generates a conguration le for knife that has the settings
needed to connect to the service from now on
Since Chef and its components are written in Ruby, the resulting conguration le is
a Ruby script, which contains some code that congures knife so that it knows what
API server to connect to, which key les to use, what client name to use, and so on.
An inspection of the conguration le that was generated by the previous command
will look like the following:
log_level :info
log_location STDOUT
node_name 'user'
client_key '/home/user/.chef/user.pem'
validation_client_name 'chef-validator'
validation_key '/home/user/.chef/chef-validator.pem'
chef_server_url 'https://localhost:443'
syntax_check_cache_path '/home/user/.chef/syntax_check_cache'
Because we are using the service host as our workstation, the Chef server URL
points to the localhost. If your workstation were to be a different system such as
your laptop, then this URL would be the IP or hostname of the host running the
Chef service.
Ensuring that your knife conguration works
After setting up knife, we can use it to validate that it was congured correctly
by querying the Chef server using some simple commands. The knife commands
follow the format knife <command> <subcommand>, where command is either a
client, conguration, cookbook, cookbook site, data bag, environment, exec, help,
index, node, recipe, role, search, ssh, status, or tag. Subcommands will vary with
the command, but they typically include things such as show, create, list, and delete
(among others).
As there will initially be no nodes, cookbooks, recipes, roles, data bags, and such,
we will query the list of clients that the server knows about. This should be a list of
two clients: chef-webui (as it is a consumer of the API itself) and chef-validator
(without it, it wouldn't be possible to register a new client).


For More Information:
www.packtpub.com/application-development/chef-essentials

Installing Chef
[ 24 ]
The client command, with the list subcommand, prints a list of clients that the
server knows about. At this point, running the command would look like this:
user@chef:~$ knife client list
chef-validator
chef-webui
user@chef:~$
If you do not get the previous output, but get an error instead,
you will need to go back and make sure that all the previous
steps are completed and veried.
Once you know that it works, you can use knife to interact with the API.
Unfortunately, we do not have much data in the system just yet, but we can use the
show subcommand in conjunction with the client command and a client name to
display more detailed information about a client:
user@chef:~$ knife client show chef-webui
admin: true
chef_type: client
json_class: Chef::ApiClient
name: chef-webui
public_key: -----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAos5cQ1NxP7zKf1zRM33g
YeVyHNOO5NcICjSIvqQ5A37wwLfgtPLJQqboW7ZcNL3xYcKOlfYSEK7xha3ss8tT
A+XMifaFp3JsdheyPeIJir2bc9iltUUcbpw9PJ2aQKTBlFNx23A7ag+zBfxcDjbY
7RkdcziwB74ynd6e/K8c0JTRnA5NxoHkFc6v8a/itwujGwugWJXDQunWfCmAvjws
JgDOUu2aHOCVIVkc8it51Sc7Anx0YnCjNmdhz1xIo0MOVNOEmC9ypP0Z7mVv1C69
WWBOEvS9zimjXo4rxBwFmWkPEIG6yPQjhuNmFd69K14vZQtAsH07AZFRSS7HLWnZ
WQIDAQAB
-----END PUBLIC KEY-----
validator: false
user@chef:~$


For More Information:
www.packtpub.com/application-development/chef-essentials

Chapter 1
[ 25 ]
Summary
Congratulations! If you have gotten this far, you now have a fully functional Chef
service and a copy of the command-line utilities, including chef-solo. You now have
covered the following:
Using RVM
Installing chef-solo
Creating a simple recipe
Running recipes with chef-solo
Installing the Chef service
Getting started with the knife utility
Verifying that your Chef service is operating correctly
Now that you are able to use your Chef service, we can begin to investigate how to
model our environment with Chef and see what it can do for us.


For More Information:
www.packtpub.com/application-development/chef-essentials

Where to buy this book
You can buy Chef Essentials from the Packt Publishing website:
.
Free shipping to the US, UK, Europe and selected Asian countries. For more information, please
read our shipping policy.
Alternatively, you can buy the book from Amazon, BN.com, Computer Manuals and
most internet book retailers.



















www.PacktPub.com


For More Information:
www.packtpub.com/application-development/chef-essentials