Running Docker in Production

Using Apache Brooklyn, Clocker and Project Calico

Andrew Donald Kennedy

 Agenda

•  Introduction
•  Application Management
•  Networking with Calico
•  Demonstration
•  Roadmap
•  Questions
Copyright 2015 by Cloudsoft Corporation Limited @grkvlt
Clocker  Introduc.on  
 Clocker Developers

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

 Clocker Statistics
•  Open Source
•  1+ Years Old
•  725 Commits
•  153 Pull Requests
•  11 Contributors
•  2 External

•  15 KLOC
•  26 Releases
http://www.redotheweb.com/CodeFlower/  

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

 Apache Brooklyn

•  Application Management Platform

•  Deploy, Manage and Monitor Blueprints
•  Provisioning, Installation and Customization
•  Management
•  AutoScaling, Resilience, Performance, Security

@grkvlt
 Apache jclouds
•  Java Cloud Library
•  API Agnostic
•  SoftLayer, OpenStack, AWS EC2, GCE…
•  Create Virtual Machines
•  Return SSH Endpoint
•  Create Containers
•  Docker REST API

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

 Docker

•  Containers
•  Isolation
•  Performance
•  Composable
•  Huge Ecosystem
•  Compute Abstraction
•  Process Wrapper
@grkvlt
 Docker Extras

•  So0ware-­‐Defined  Networking  
•  Calico  
•  Weave  
•  Storage  and  Volume  Management  
•  Flocker  
•  More  Being  Developed…  
•  Na.ve  Plugins  

@grkvlt
 Apache Mesos ...
•  Distributed Systems Kernel
•  Cluster Management
•  Resource Sharing and Placement
•  Calico SDN Support
•  Frameworks
•  Aurora and Marathon
•  Riak, Spark, Hadoop, Storm et al
•  Brooklyn Scheduler
Copyright 2015 by Cloudsoft Corporation Limited @grkvlt
 What does it do?

1.  Spins up and Manages Docker

Clusters in the Clouds
2.  Serves up Containers on Demand
3.  Manages Composite Application
Deployments
Copyright 2015 by Cloudsoft Corporation Limited @grkvlt
 What does it provide?
•  Infrastructure Management
•  Docker Hosts
•  Swarm Controller
•  Multi Host and Multi Container Applications
•  Seamless Networking
•  Communication Between Services
•  Orchestration and Clustering
•  Control of Containers
•  Container Management
Copyright 2015 by Cloudsoft Corporation Limited @grkvlt
 Clocker and Brooklyn Summary
•  What is it?
•  Brooklyn Application
•  Brooklyn Location
•  What does it provide?
•  First Class Docker Support in Brooklyn
•  Optimized Brooklyn Blueprints for Docker
•  Container Orchestration

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

Applica.on  Management  
 Brooklyn Blueprints
•  Describe Applications
•  OASIS CAMP Standard
•  TOSCA and Compose in Development
•  List of Services
•  NoSQL Database Clusters
•  Web Servers and Load Balancers
•  Shell or Python Scripts
•  Targeting Multiple Destinations
•  VM, Container, Bare Metal
•  Sensors, Effectors and Policies
Copyright 2015 by Cloudsoft Corporation Limited @grkvlt
 Application Management
•  Sensors
•  Data from Services
•  Effectors
•  Brooklyn Policies
•  Attached to Entities in Application
•  Nothing Docker Specific
•  Elastic Scaling and Cluster Resizing
•  Service Resilience and Replacement

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

 Blueprint Example 1

id:  redis-­‐service  
name:  "Docker  Hub  Redis  Service"  
origin:  "https://registry.hub.docker.com/_/redis/"  
 
locations:  
-­‐  my-­‐docker-­‐cloud  
 
services:  
-­‐  type:  docker:redis:3  
   id:  redis  
   openPorts:  
   -­‐  6379  

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

 Blueprint Example 2

id:  redis-­‐cluster  
name:  "Redis  Cluster"  
 
locations:  
-­‐  jclouds:aws-­‐ec2:  
       region:  eu-­‐central-­‐1  
 
services:  
-­‐  type:  org.apache.brooklyn.entity.nosql.redis.RedisCluster  
   initialSize:  3  

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

 Blueprint Example 3
id:  appserver-­‐with-­‐policy  
name:  "Tomcat  Scaling  Webapp  Server"  
location:  jclouds:aws-­‐ec2:eu-­‐west-­‐1  
services:  
-­‐  type:  org.apache.brooklyn.entity.webapp.ControlledDynamicWebAppCluster  
   initialSize:  3  
   memberSpec:  
       $brooklyn:entitySpec:  
           type:  org.apache.brooklyn.entity.webapp.tomcat.Tomcat8Server  
           brooklyn.config:  
               wars.root:  
                   https://  s3-­‐eu-­‐west-­‐1.amazonaws.com/brooklyn-­‐clocker/brooklyn-­‐example-­‐hello-­‐world-­‐sql-­‐webapp-­‐0.6.0.war  
               http.port:  8080+  
               java.sysprops:    
                   brooklyn.example.db.url:  $brooklyn:formatString("jdbc:%s%s?user=%s\\&password=%s",  
                           component("db").attributeWhenReady("datastore.url"),  "visitors",  "brooklyn",  "br00k11n")  
   brooklyn.policies:  
   -­‐  policyType:  org.apache.brooklyn.policy.autoscaling.AutoScalerPolicy  
       brooklyn.config:  
           metric:  $brooklyn:sensor("org.apache.brooklyn.entity.webapp.DynamicWebAppCluster",  "webapp.reqs.perSec.windowed.perNode")  
           metricLowerBound:  10  
           metricUpperBound:  100  
           minPoolSize:  1  
           maxPoolSize:  5  
-­‐  type:  org.apache.brooklyn.entity.database.mysql.MySqlNode  
   id:  db  
   name:  DB  HelloWorld  Visitors  
   brooklyn.config:  
       datastore.creation.script.url:  
           https://  s3-­‐eu-­‐west-­‐1.amazonaws.com/brooklyn-­‐clocker/visitors-­‐creation-­‐script.sql  

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

 Clocker Blueprints

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

 Docker Cloud
•  Brooklyn Blueprint for Docker Cluster
•  Docker Engine on Cloud VM or Bare Metal
•  Manage TLS Certificates and access credentials
•  Create and Mount Volumes
•  Setup Logging and Monitoring
•  Install SDN and other Agents
•  Configure local or remote Docker Registry

•  Manage Capacity or Headroom

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt
 Headroom
•  Ensure resources available
•  Based on MaxContainers strategy limit
•  Or Percentage Utilisation
•  Or CPU and RAM allocation
•  Scale Docker Host Cluster Automatically
•  Add new Docker hosts
•  Remove empty Docker hosts

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

 Clocker 1.x Architecture

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

 Docker Cloud

1.  On-demand
2.  Multi-Tenant
3.  Hardware Independent
4.  Application Level
Copyright 2015 by Cloudsoft Corporation Limited @grkvlt
 Clocker Features

•  Docker Extensions to Brooklyn

– Docker Image as First-Class Service Type
– Placement Strategies for Containers
– Create Docker Images and Networks
•  Manages Docker Engine and Swarm
– Deployment and Management
– Installation and Configuration
– Software-Defined Networking
Copyright 2015 by Cloudsoft Corporation Limited @grkvlt
 Container Management
•  Sources
•  Docker Image Definition
•  Docker Hub or Registry
•  Dockerfile
•  Brooklyn Entity Definition
•  Create Image Automatically
•  Commit or Push for Reuse

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

 Clocker Orchestration

Cloud Virtual
Brooklyn
Provider Machine

Docker
Container
Engine

SDN Network
Clocker
Provider Segment

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

Clocker  Networking  
 Software-Defined Networking
•  Needed for Seamless Provisioning
•  Host to Host Communication
•  Same LAN Segment
•  No Port Forwarding
•  Natural Application Configuration
•  Initial Driver was EPMD Applications
•  Useful for any opinionated applications

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

 Networking Providers
•  Implementation Agnostic
•  L2 overlay, L3 routing etc.
•  Similar to Hypervisor in Clouds
•  Generic Interfaces
•  Host Component
•  Service Component (or Endpoint)
•  Same idea as Docker Network Plugins
Copyright 2015 by Cloudsoft Corporation Limited @grkvlt
 Networking Capabilities

•  Attach Containers to Networks  

•  Create Networks as Required

•  Also Attach to VMs and Metal
•  Provide Multiple Networks
•  Per-Application or Shared
•  Segmented Private Address Space
•  Docker Port Forwarding for Ingress
Copyright 2015 by Cloudsoft Corporation Limited @grkvlt
 Network Provisioning
•  Minimal (Zero!) Configuration
•  Use Sensible Defaults
•  Allows SDN or Cloud Specific Configuration
•  Allocate Address Space on Demand
•  IP Pool Controlled by Clocker
-­‐  type:  brooklyn.networking.VirtualNetwork  
   networkId:  database-­‐net  
   cidr:  192.168.34.0/24  
   gateway:  192.168.34.1  
   dnsServers:  
   -­‐  $brooklyn:entity("bind-­‐server").attributeWhenReady("host.address")  
   addIptablesRules:  true  
-­‐  type:  brooklyn.networking.OpenStackVirtualNetwork  
   networkId:  couchbase-­‐net  
Copyright 2015 by Cloudsoft Corporation Limited @grkvlt
 Clocker Networking

SDN SDN
Internet Gateway Bridge

Host Container Container

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

 Metaswitch Project Calico
•  SDN for Bare Metal, VMs and Containers
•  Layer 3
•  Uses OS IP routing and forwarding

•  Configuration in an etcd Cluster

•  Version 0.4.9 in Clocker
•  0.6.0 with libnetwork when Docker stable

•  Spans VMs and Containers

•  OpenStack Neutron network driver

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

 Cross-Target Deployment

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

 Networking Capabilities
•  Wide Area and Multi Region SDN
•  VPN or IPIP and NAT configuration

•  Cross Platform SDN

•  Both VMs and Containers on one VLAN

•  Name Resolution and Service Discovery

•  Contributing to Weave DNS for orchestration
•  Use traditional external BIND service entity
•  Brooklyn can inject correct endpoint address
Copyright 2015 by Cloudsoft Corporation Limited @grkvlt
Demonstra.on  
 Application Blueprint 1
id:  my-­‐application  
location:  my-­‐docker-­‐cloud  
services:  
-­‐  type:  docker:redis:3  
   id:  redis  
   openPorts:  6379  
-­‐  type:  docker:amouat/dnmonster:1.0  
   id:  dnmonster  
   openPorts:  8080  
-­‐  type:  docker:amouat/identidock:1.0  
   id:  identidock  
   portBindings:  
       80:  9090  
   links:  
   -­‐  $brooklyn:component("redis")  
   -­‐  $brooklyn:component("dnmonster")  

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

 Application Blueprint 2
id:  my-­‐application  
location:  my-­‐docker-­‐cloud  
services:  
-­‐  type:  org.apache.brooklyn.entity.nosql.redis.RedisStore  
   id:  redis  
   install.version:  3.0.0  
-­‐  type:  docker:amouat/dnmonster:1.0  
   id:  dnmonster  
   openPorts:  8080  
-­‐  type:  docker:amouat/identidock:1.0  
   id:  identidock  
   portBindings:  
       80:  9090  
   links:  
   -­‐  $brooklyn:component("redis")  
   -­‐  $brooklyn:component("dnmonster")  

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

 Application Blueprint 3
id:  my-­‐application  
location:  my-­‐docker-­‐cloud  
services:  
-­‐  type:  org.apache.brooklyn.entity.nosql.redis.RedisStore  
   location:  jclouds:softlayer:ams01  
   id:  redis  
   install.version:  3.0.0  
-­‐  type:  docker:amouat/dnmonster:1.0  
   id:  dnmonster  
   openPorts:  8080  
-­‐  type:  docker:amouat/identidock:1.0  
   id:  identidock  
   portBindings:  
       80:  9090  
   links:  
   -­‐  $brooklyn:component("redis")  
   -­‐  $brooklyn:component("dnmonster")  

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

 Application Blueprint 4
id:  my-­‐application  
location:  jclouds:softlayer:ams01  
services:  
-­‐  type:  org.apache.brooklyn.entity.nosql.redis.RedisStore  
   id:  redis  
   install.version:  3.0.0  
-­‐  type:  docker:amouat/dnmonster:1.0  
   id:  dnmonster  
   openPorts:  8080  
-­‐  type:  docker:amouat/identidock:1.0  
   id:  identidock  
   portBindings:  
       80:  9090  
   links:  
   -­‐  $brooklyn:component("redis")  
   -­‐  $brooklyn:component("dnmonster")  

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

 Application Blueprint 5
id:  my-­‐application  
services:  
-­‐  type:  org.apache.brooklyn.entity.nosql.redis.RedisStore  
   location:  jclouds:softlayer:ams01  
   id:  redis  
   install.version:  3.0.0  
-­‐  type:  marathon:amouat/dnmonster:1.0  
   location:  my-­‐mesos-­‐cluster  
   id:  dnmonster  
   openPorts:  8080  
-­‐  type:  docker:amouat/identidock:1.0  
   location:  my-­‐docker-­‐cloud  
   id:  identidock  
   portBindings:  
       80:  9090  
   links:  
   -­‐  $brooklyn:component("redis")  
   -­‐  $brooklyn:component("dnmonster")  

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

 Features
•  Orchestrated Docker deployment and
configuration, with Project Calico SDN
•  Brooklyn application blueprints deployed with
network topology linked to OpenStack using
Project Calico
•  Automated attachment of containers to multiple
dynamic networks
•  Zero Config Multi-Target Deployment
Copyright 2015 by Cloudsoft Corporation Limited @grkvlt
Clocker  1.x  
 Roadmap 1
•  Mesos Integration
•  View and manage existing Mesos clusters
•  Provide Mesos as another Brooklyn endpoint
•  Run Marathon tasks for Brooklyn entities
•  Future Possibilities...
•  Brooklyn as a Mesos framework or scheduler
•  Integrate with other frameworks like Riak BDP

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

 Roadmap 2

Copyright 2015 by Cloudsoft Corporation Limited @grkvlt

 Summary  

Clocker Solves:
– Docker Cloud Networking
– Container Placement and Provisioning
– Composite Application Management
Ques.ons?  
Nyan Whale
 Resources

http://clocker.io/  
http://brooklyn.io/  
https://github.com/brooklyncentral/clocker/  
https://github.com/apache/incubator-­‐brooklyn/  
https://github.com/Metaswitch/calico-­‐docker/  
https://github.com/weaveworks/weave/  
http://blog.abstractvisitorpattern.co.uk/  
@grkvlt