A quick intro to

Docker,
Terraform, and TERRAFORM

Amazon ECS
Amazon ECS
In this talk, we’ll show how to
deploy two apps:
A Rails Frontend and a
Sinatra Backend
Slides and code from this talk:
ybrikman.com/speaking
require 'sinatra'

get "/" do
"Hello, World!"
end

The sinatra backend just returns

“Hello, World”.
class ApplicationController < ActionController::Base
def index
url = URI.parse(backend_addr)
req = Net::HTTP::Get.new(url.to_s)
res = Net::HTTP.start(url.host, url.port) {|http|
http.request(req)
}
@text = res.body
end
end

The rails frontend calls the sinatra

backend…
<h1>Rails Frontend</h1>
<p>
Response from the backend: <strong><%= @text %></strong>
</p>

And renders the response as

HTML.
We’ll package the two apps as
Docker containers…
 Amazon ECS

Deploy those Docker containers

using Amazon ECS…
 TERRAFORM

And define our infrastructure-as-

code using Terraform.
I’m
Yevgeniy
Brikman

ybrikman.com
 Co-founder of
Gruntwork

gruntwork.io
 We offer DevOps
as a Service

gruntwork.io
 And DevOps
as a Library

gruntwork.io
PAST LIVES
Author of
Hello,
Startup

hello-startup.net
 And
Terraform:
Up & Running

terraformupandrunning.com
Outline
1. Docker
2. Terraform
3. ECS
4. Recap
Outline
1. Docker
2. Terraform
3. ECS
4. Recap
Docker allows you to build and
run code in containers
Containers are like lightweight
Virtual Machines (VMs)
Like an isolated process that
happens to be an entire OS
> docker run –it ubuntu bash

root@12345:/# echo "I'm in $(cat /etc/issue)”

I'm in Ubuntu 14.04.4 LTS

Running an Ubuntu image in a

Docker container
> time docker run ubuntu echo "Hello, World"
Hello, World

real 0m0.183s
user 0m0.009s
sys 0m0.014s

Containers boot quickly, with

minimal CPU/memory overhead
You can define a Docker image
as code in a Dockerfile
FROM gliderlabs/alpine:3.3

RUN apk --no-cache add ruby ruby-dev

RUN gem install sinatra --no-ri --no-rdoc

RUN mkdir -p /usr/src/app

COPY . /usr/src/app
WORKDIR /usr/src/app

EXPOSE 4567
CMD ["ruby", "app.rb"]

Here is the Dockerfile for the

Sinatra backend
FROM gliderlabs/alpine:3.3

RUN apk --no-cache add ruby ruby-dev

RUN gem install sinatra --no-ri --no-rdoc

RUN mkdir -p /usr/src/app

COPY . /usr/src/app
WORKDIR /usr/src/app

EXPOSE 4567
CMD ["ruby", "app.rb"]

It specifies dependencies, code,

config, and how to run the app
> docker build -t gruntwork/sinatra-backend .
Step 0 : FROM gliderlabs/alpine:3.3
---> 0a7e169bce21

(...)

Step 8 : CMD ruby app.rb

---> 2e243eba30ed

Successfully built 2e243eba30ed

Build the Docker image

> docker run -it -p 4567:4567 gruntwork/sinatra-backend
INFO WEBrick 1.3.1
INFO ruby 2.2.4 (2015-12-16) [x86_64-linux-musl]
== Sinatra (v1.4.7) has taken the stage on 4567 for
development with backup from WEBrick
INFO WEBrick::HTTPServer#start: pid=1 port=4567

Run the Docker image

> docker push gruntwork/sinatra-backend
The push refers to a repository [docker.io/gruntwork/sinatra-
backend] (len: 1)
2e243eba30ed: Image successfully pushed
7e2e0c53e246: Image successfully pushed
919d9a73b500: Image successfully pushed

(...)

v1: digest: sha256:09f48ed773966ec7fe4558 size: 14319

You can share your images by

pushing them to Docker Hub
Now you can reuse the same
image in dev, stg, prod, etc
> docker pull rails:4.2.6

And you can reuse images created

by others.
FROM rails:4.2.6

RUN mkdir -p /usr/src/app

COPY . /usr/src/app
WORKDIR /usr/src/app

RUN bundle install

EXPOSE 3000
CMD ["rails", "start"]

The rails-frontend is built on top of

the official rails Docker image
rails_frontend:
image: gruntwork/rails-frontend
ports:
- "3000:3000"
links:
- sinatra_backend

sinatra_backend:
image: gruntwork/sinatra-backend
ports:
- "4567:4567"

Define your entire dev stack as

code with docker-compose
rails_frontend:
image: gruntwork/rails-frontend
ports:
- "3000:3000"
links:
- sinatra_backend

sinatra_backend:
image: gruntwork/sinatra-backend
ports:
- "4567:4567"

Docker links provide a simple

service discovery mechanism
> docker-compose up
Starting infrastructureascodetalk_sinatra_backend_1
Recreating infrastructureascodetalk_rails_frontend_1

sinatra_backend_1 | INFO WEBrick 1.3.1

sinatra_backend_1 | INFO ruby 2.2.4 (2015-12-16)
sinatra_backend_1 | Sinatra has taken the stage on 4567

rails_frontend_1 | INFO WEBrick 1.3.1

rails_frontend_1 | INFO ruby 2.3.0 (2015-12-25)
rails_frontend_1 | INFO WEBrick::HTTPServer#start: port=3000

Run your entire dev stack with one

command
Outline
1. Docker
2. Terraform
3. ECS
4. Recap
 Terraform is a tool for
provisioning infrastructure
Terraform supports many
providers (cloud agnostic)
And many resources for each
provider
You define infrastructure as code
in Terraform templates
provider "aws" {
region = "us-east-1"
}

resource "aws_instance" "example" {

ami = "ami-408c7f28"
instance_type = "t2.micro"
}

This template creates a single EC2

instance in AWS
> terraform plan
+ aws_instance.example
ami: "" => "ami-408c7f28"
instance_type: "" => "t2.micro"
key_name: "" => "<computed>"
private_ip: "" => "<computed>"
public_ip: "" => "<computed>"

Plan: 1 to add, 0 to change, 0 to destroy.

Use the plan command to see

what you’re about to deploy
> terraform apply
aws_instance.example: Creating...
ami: "" => "ami-408c7f28"
instance_type: "" => "t2.micro"
key_name: "" => "<computed>"
private_ip: "" => "<computed>"
public_ip: "" => "<computed>”
aws_instance.example: Creation complete

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Use the apply command to apply

the changes
Now our EC2 instance is running!
resource "aws_instance" "example" {
ami = "ami-408c7f28"
instance_type = "t2.micro"
tags {
Name = "terraform-example"
}
}

Let’s give the EC2 instance a tag

with a readable name
> terraform plan
~ aws_instance.example
tags.#: "0" => "1"
tags.Name: "" => "terraform-example"

Plan: 0 to add, 1 to change, 0 to destroy.

Use the plan command again to

verify your changes
> terraform apply
aws_instance.example: Refreshing state...
aws_instance.example: Modifying...
tags.#: "0" => "1"
tags.Name: "" => "terraform-example"
aws_instance.example: Modifications complete

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

Use the apply command again to

deploy those changes
Now our EC2 instance has a tag!
resource "aws_elb" "example" {
name = "example"
availability_zones = ["us-east-1a", "us-east-1b"]
instances = ["${aws_instance.example.id}"]
listener {
lb_port = 80
lb_protocol = "http"
instance_port = "${var.instance_port}"
instance_protocol = "http”
}
}

Let’s add an Elastic Load Balancer

(ELB).
resource "aws_elb" "example" {
name = "example"
availability_zones = ["us-east-1a", "us-east-1b"]
instances = ["${aws_instance.example.id}"]
listener {
lb_port = 80
lb_protocol = "http"
instance_port = "${var.instance_port}"
instance_protocol = "http”
}
}

Terraform supports variables,

such as var.instance_port
resource "aws_elb" "example" {
name = "example"
availability_zones = ["us-east-1a", "us-east-1b"]
instances = ["${aws_instance.example.id}"]
listener {
lb_port = 80
lb_protocol = "http"
instance_port = "${var.instance_port}"
instance_protocol = "http"
}
}

As well as dependencies like

aws_instance.example.id
resource "aws_elb" "example" {
name = "example"
availability_zones = ["us-east-1a", "us-east-1b"]
instances = ["${aws_instance.example.id}"]
listener {
lb_port = 80
lb_protocol = "http"
instance_port = "${var.instance_port}"
instance_protocol = "http"
}
}

It builds a dependency graph and

applies it in parallel.
After running apply, we have an ELB!
> terraform destroy
aws_instance.example: Refreshing state... (ID: i-f3d58c70)
aws_elb.example: Refreshing state... (ID: example)
aws_elb.example: Destroying...
aws_elb.example: Destruction complete
aws_instance.example: Destroying...
aws_instance.example: Destruction complete

Apply complete! Resources: 0 added, 0 changed, 2 destroyed.

Use the destroy command to

delete all your resources
For more info, check out The Comprehensive Guide
to Terraform
Outline
1. Docker
2. Terraform
3. ECS
4. Recap
EC2 Container Service (ECS) is a
way to run Docker on AWS
 {
"name": "example", {
"image": "foo/example", "cluster": "example",
"cpu": 1024, "serviceName": ”foo",
"memory": 2048, "taskDefinition": "",
"essential": true, "desiredCount": 2
}
ECS Agent
}

ECS Tasks

ECS Task Definition ECS Service Definition ECS Scheduler

EC2 Instance

ECS Cluster

ECS Overview
 EC2 Instance

ECS Cluster

ECS Cluster: several servers

managed by ECS
 EC2 Instance

Auto Scaling Group

Typically, the servers are in an

Auto Scaling Group
 ECS Agent

EC2 Instance

ECS Cluster

Each server must run the ECS

Agent
 {
"name": "example",
"image": "foo/example",
"cpu": 1024,
"memory": 2048,
"essential": true, ECS Agent
}

ECS Task Definition

EC2 Instance

ECS Cluster

ECS Task: Docker container(s)

to run, resources they need
 {
"name": "example", {
"image": "foo/example", "cluster": "example",
"cpu": 1024, "serviceName": ”foo",
"memory": 2048, "taskDefinition": "",
"essential": true, "desiredCount": 2
}
ECS Agent
}

ECS Task Definition ECS Service Definition

EC2 Instance

ECS Cluster

ECS Service: long-running ECS

Task & ELB settings
 {
"name": "example", {
"image": "foo/example", "cluster": "example",
"cpu": 1024, "serviceName": ”foo",
"memory": 2048, "taskDefinition": "",
"essential": true, "desiredCount": 2
}
ECS Agent
}

ECS Tasks

ECS Task Definition ECS Service Definition ECS Scheduler

EC2 Instance

ECS Cluster

ECS Scheduler: Deploys Tasks

across the ECS Cluster
 {
"name": "example", {
"image": "foo/example", "cluster": "example",
"cpu": 1024, "serviceName": ”foo",
"memory": 2048, "taskDefinition": "",
"essential": true, "desiredCount": 2
}
ECS Agent
}

ECS Tasks

ECS Task Definition ECS Service Definition

EC2 Instance

ECS Cluster

You can associate an ALB or

ELB with each ECS service
 {
"name": "example", {
"image": "foo/example", "cluster": "example",
"cpu": 1024, "serviceName": ”foo",
"memory": 2048, "taskDefinition": "",
"essential": true, "desiredCount": 2
}
ECS Agent
}

ECS Tasks

ECS Task Definition ECS Service Definition

EC2 Instance

ECS Cluster

This allows you to distribute

load across your ECS Tasks
 {
"name": "example", {
"image": "foo/example", "cluster": "example",
"cpu": 1024, "serviceName": ”foo",
"memory": 2048, "taskDefinition": "",
"essential": true, "desiredCount": 2
}
ECS Agent
}

ECS Tasks

ECS Task Definition ECS Service Definition

EC2 Instance

ECS Cluster

You can also use it as a simple

form of service discovery
Let’s deploy our apps on ECS
using Terraform
 EC2 Instance

ECS Cluster

Define the ECS Cluster as an

Auto Scaling Group (ASG)
resource "aws_ecs_cluster" "example_cluster" {
name = "example-cluster"
}

resource "aws_autoscaling_group" "ecs_cluster_instances" {

name = "ecs-cluster-instances"
min_size = 5
max_size = 5
launch_configuration =
"${aws_launch_configuration.ecs_instance.name}"
}
 ECS Agent

EC2 Instance

ECS Cluster

Ensure each server in the ASG

runs the ECS Agent
 # The launch config defines what runs on each EC2 instance
resource "aws_launch_configuration" "ecs_instance" {
name_prefix = "ecs-instance-"
instance_type = "t2.micro"

# This is an Amazon ECS AMI, which has an ECS Agent

# installed that lets it talk to the ECS cluster
image_id = "ami-a98cb2c3”
}

The launch config runs AWS ECS

Linux on each server in the ASG
 {
"name": "example",
"image": "foo/example",
"cpu": 1024,
"memory": 2048,
"essential": true, ECS Agent
}

ECS Task Definition

EC2 Instance

ECS Cluster

Define an ECS Task for each

microservice
resource "aws_ecs_task_definition" "rails_frontend" {
family = "rails-frontend"
container_definitions = <<EOF
[{
"name": "rails-frontend",
"image": "gruntwork/rails-frontend:v1",
"cpu": 1024,
"memory": 768,
"essential": true,
"portMappings": [{"containerPort": 3000, "hostPort": 3000}]
}]
EOF
}

Rails frontend ECS Task

resource "aws_ecs_task_definition" "sinatra_backend" {
family = "sinatra-backend"
container_definitions = <<EOF
[{
"name": "sinatra-backend",
"image": "gruntwork/sinatra-backend:v1",
"cpu": 1024,
"memory": 768,
"essential": true,
"portMappings": [{"containerPort": 4567, "hostPort": 4567}]
}]
EOF
}

Sinatra Backend ECS Task

 {
"name": "example", {
"image": "foo/example", "cluster": "example",
"cpu": 1024, "serviceName": ”foo",
"memory": 2048, "taskDefinition": "",
"essential": true, "desiredCount": 2
}
ECS Agent
}

ECS Task Definition ECS Service Definition

EC2 Instance

ECS Cluster

Define an ECS Service for each

ECS Task
resource "aws_ecs_service" "rails_frontend" {
family = "rails-frontend"
cluster = "${aws_ecs_cluster.example_cluster.id}"
task_definition =
"${aws_ecs_task_definition.rails-fronted.arn}"
desired_count = 2
}

Rails Frontend ECS Service

resource "aws_ecs_service" "sinatra_backend" {
family = "sinatra-backend"
cluster = "${aws_ecs_cluster.example_cluster.id}"
task_definition =
"${aws_ecs_task_definition.sinatra_backend.arn}"
desired_count = 2
}

Sinatra Backend ECS Service

 {
"name": "example", {
"image": "foo/example", "cluster": "example",
"cpu": 1024, "serviceName": ”foo",
"memory": 2048, "taskDefinition": "",
"essential": true, "desiredCount": 2
}
ECS Agent
}

ECS Tasks

ECS Task Definition ECS Service Definition

EC2 Instance

ECS Cluster

Associate an ELB with each

ECS Service
resource "aws_elb" "rails_frontend" {
name = "rails-frontend"
listener {
lb_port = 80
lb_protocol = "http"
instance_port = 3000
instance_protocol = "http"
}
}

Rails Frontend ELB

resource "aws_ecs_service" "rails_frontend" {

(...)

load_balancer {
elb_name = "${aws_elb.rails_frontend.id}"
container_name = "rails-frontend"
container_port = 3000
}
}

Associate the ELB with the Rails

Frontend ECS Service
resource "aws_elb" "sinatra_backend" {
name = "sinatra-backend"
listener {
lb_port = 4567
lb_protocol = "http"
instance_port = 4567
instance_protocol = "http"
}
}

Sinatra Backend ELB

resource "aws_ecs_service" "sinatra_backend" {

(...)

load_balancer {
elb_name = "${aws_elb.sinatra_backend.id}"
container_name = "sinatra-backend"
container_port = 4567
}
}

Associate the ELB with the Sinatra

Backend ECS Service
 {
"name": "example", {
"image": "foo/example", "cluster": "example",
"cpu": 1024, "serviceName": ”foo",
"memory": 2048, "taskDefinition": "",
"essential": true, "desiredCount": 2
}
ECS Agent
}

ECS Tasks

ECS Task Definition ECS Service Definition

EC2 Instance

ECS Cluster

Set up service discovery

between the ECS Services
resource "aws_ecs_task_definition" "rails_frontend" {
family = "rails-frontend"
container_definitions = <<EOF
[{
...
"environment": [{
"name": "SINATRA_BACKEND_PORT",
"value": "tcp://${aws_elb.sinatra_backend.dns_name}:4567"
}]
}]
EOF
}

Pass the Sinatra Bckend ELB URL

as env var to Rails Frontend
 {
"name": "example", {
"image": "foo/example", "cluster": "example",
"cpu": 1024, "serviceName": ”foo",
"memory": 2048, "taskDefinition": "",
"essential": true, "desiredCount": 2
}
ECS Agent
}

ECS Tasks

ECS Task Definition ECS Service Definition ECS Scheduler

EC2 Instance

ECS Cluster

It’s time to deploy!

> terraform apply
aws_ecs_cluster.example_cluster: Creating...
name: "" => "example-cluster"
aws_ecs_task_definition.sinatra_backend: Creating...
...

Apply complete! Resources: 17 added, 0 changed, 0 destroyed.

Use the apply command to deploy

the ECS Cluster & Tasks
See the cluster in the ECS console
Track events for each Service
As well as basic metrics
Test the rails-frontend
Outline
1. Docker
2. Terraform
3. ECS
4. Recap
Slides and code from this talk:
ybrikman.com/speaking
For more
info, see
Hello,
Startup
hello-startup.net
 And
Terraform:
Up & Running

terraformupandrunning.com
 For DevOps help, see
Gruntwork

gruntwork.io
Questions?