Source

https://www.youtube.com/watch?v=ZNdc4-yFTeA
https://github.com/docker/labs/blob/master/slides/docker-java-dockercon-2017.pdf
1
 What is Docker?
▪Open source project and company

▪Used to create containers for software applications

3
2
Containers & Images
Running Your First Container

3
What Are Linux Containers?
Linux containers are a way to create isolated environments that can run code while sharing a
single operating system.

Each container is
Container 1 Container 2 Container 3
completely isolated
from the others
App App App

Bin/lib Bin/lib Bin/lib

Physical Server + OS
A computer somewhere - could even be the laptop
or desktop computer you’re using right now!
4
Why Docker?
Managing Linux containers is hard.
Docker is a tool that makes it much easier to manage Linux containers.

Container 1 Container 2 Container 3

App App App

Application that manages
Bin/lib Bin/lib Bin/lib containers behind the scenes

Docker Engine

Physical Server + OS

5
 6
6
How Can Docker Help Me?
There are many different ways people can use Docker.

Developers

Create contained, controlled dev environment

Share identical dev environment across team
Bug reporting This is what

IT Ops we’ll focus on

in this course
Testing

Deployment

7
Installing Docker
The simplest way to install Docker is to download one of the official Docker applications.

Applications
Docker for Mac - Community Edition
Docker for Windows - Community Edition
Installation Instructions
Linux AWS

Azure Windows Server

8
 Containers & Images
An image is a blueprint for creating a container.

Image Container

Pre-built images
available in Docker
Store (and Docker
Hub)

9
 Dockerfiles
Automating the Creation of Custom Images

10
The Problem: Creating Containers Is Clunky
Creating containers from the command line works, but it quickly gets a little clunky the more
customization that you need to do.

Image
1 open port 80
Container
2 update package manager

3 download a package

4 copy web server config

Each step modifies Dockerfiles help make

the container a this process slightly
little bit less manual
11
Dockerfiles Help You Create Images
A Dockerfile is a specially formatted text file where you can add a list of instructions that will
run and result in a new image that can be used to make a container.

Image Container
Dockerfile

1 open port 80

The steps in a FROM httpd:2.4

2 update package manager
Dockerfile are run EXPOSE 80
3 download a package and turned into a RUN apt-get update
single image COPY ./my-httpd.conf /usr/local/
4 copy web server config apache2/conf/httpd.conf

12
FROM ubuntu

CMD echo “Hello world”

FROM openjdk

COPY target/hello.jar /usr/src/hello.jar

CMD java -cp /usr/src/hello.jar org.example.App

7
19
 Volumes
Working With Data in Containers

13
Getting Data Into Containers
If the image you’re building a container with doesn’t already contain application files, you’ll
need an extra step to get them into your container.

Copy a file into a container from

the command line

Copy a file into an image with

instructions in a Dockerfile

14
The Problem: Containers Don’t Persist Data
Our containers aren’t really doing much right now because we don’t have a way to get data in them.

Modified data
is gone!

Stop
Start Modify
container
container files in
container

15
The Solution: Data Volumes
Data volumes expose files on your host machine to the container.

Host Container

Volume

Data is
still there!

Start Modify files in Stop

container data volume container
16
 {J,W,E}AR

Mac OS X Windows Ubuntu CentOS

WORA = Write Once Run Anywhere

Image

Mac OS X Windows Ubuntu CentOS

PODA = Package Once Deploy Anywhere 4

17
 Docker Mission

Build Ship Run

Distributed/ Applications

Anywhere

5
18
 8
20

https://docs.docker.com/engine/reference/builder/
Docker Workflow

9
21
 Image Layers - OpenJDK
~ > docker image ls openjdk
REPOSITORY TAG IMAGE ID CREATED SIZE
openjdk latest d23bdf5b1b1b 5 days ago 643 MB
~ > docker image history openjdk
IMAGE CREATED CREATED BY SIZE
COMMENT
d23bdf5b1b1b 5 days ago /bin/sh -c /var/lib/dpkg/info/ca-certifica... 419 kB
<missing> 5 days ago /bin/sh -c set -x && apt-get update && a... 352 MB
<missing> 5 days ago /bin/sh -c #(nop) ENV CA_CERTIFICATES_JAV... 0 B
<missing> 5 days ago /bin/sh -c #(nop) ENV JAVA_DEBIAN_VERSION... 0 B
<missing> 5 days ago /bin/sh -c #(nop) ENV JAVA_VERSION=8u111 0 B
<missing> 5 days ago /bin/sh -c #(nop) ENV JAVA_HOME=/usr/lib/... 0 B
<missing> 5 days ago /bin/sh -c { echo '#!/bin/sh'; echo 's... 87 B
<missing> 5 days ago /bin/sh -c #(nop) ENV LANG=C.UTF-8 0 B
<missing> 5 days ago /bin/sh -c echo 'deb http://deb.debian.org... 55 B
<missing> 5 days ago /bin/sh -c apt-get update && apt-get insta... 1.29 MB
<missing> 5 days ago /bin/sh -c apt-get update && apt-get insta... 123 MB
<missing> 5 days ago /bin/sh -c apt-get update && apt-get insta... 44.3 MB
<missing> 6 days ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0 B
<missing> 6 days ago /bin/sh -c #(nop) ADD file:89ecb642d662ee7... 123 MB

10
22
 Docker for AWS/Azure
▪Amazon Web Services
– EKS
– Integrated with AutoScaling, ELB, and EBS.
▪Azure
– Integrated with VM Scale Sets for autoscaling, Azure Load
Balancer, Azure Storage
▪docker.com

11
23
 Docker for Mac/Windows
▪Native application and UI
▪Auto update capability
▪No additional software required, e.g. VirtualBox
– OSX: xhyve VM using Hypervisor.framework
– Windows: Hyper-V VM
▪Download: docker.com/getdocker
▪Requires Yosemite 10.10+ or Windows 10 64-bit

12
24
25
26
27
28
29
30
FROM python:3.6-alpine Docker default runs as root!
RUN adduser -D microblog You should alway create user for your
container!
WORKDIR /home/microblog
COPY requirements.txt requirements.txt
RUN apk add --no-cache --update gcc musl-dev libffi-dev openssl-dev
RUN python3 -m venv venv
RUN venv/bin/pip3 install --upgrade pip
RUN venv/bin/pip3 install -r requirements.txt
RUN venv/bin/pip3 install gunicorn
COPY app app
COPY migrations migrations
COPY microblog.py config.py run.py boot.sh ./
RUN chmod +x boot.sh
ENV FLASK_APP run.py
RUN chown -R microblog:microblog ./
USER microblog
EXPOSE 5000
ENTRYPOINT ["./boot.sh"] 31
 Docker Compose
▪Defining and running multi-container applications
▪Configuration defined in one or more files
– docker-compose.yml (default)
– docker-compose.override.yml (default)
– Multiple files specified using -f
▪Deployed as Docker Stack
▪Great for dev, staging, and CI

13
32
Docker Compose - One Service
version: “3”
services:
db:
image: couchbase
volumes:
- ~/couchbase:/opt/couchbase/var
ports:
- 8091:8091
- 8092:8092
- 8093:8093
- 11210:11210

docker-compose up -d

docker stack deploy \

--compose-file=docker-compose.yml \
couchbase 33
14
Docker Compose - Two Services

GET
POST
PUT
DELETE
CRUD
using
N1QL

15
34
Docker Compose - Two Services
version: “3”
services:
db:
image: arungupta/couchbase:travel
ports:
- 8091:8091
- 8092:8092
- 8093:8093
- 11210:11210
web:
image: arungupta/couchbase-wildfly-javaee:travel
environment:
- COUCHBASE_URI=db
ports:
- 8080:8080
- 9990:9990

docker stack deploy \

--compose-file=docker-compose.yml \
16
couchbase 35
Overriding Services in Docker Compose
web:
image: jboss/wildfly
ports:
- 8080:8080

docker-compose.yml

web:
ports:
- 9080:8080

docker-compose.override.yml

17
36
 Dev/Prod with Compose
db-dev: docker-compose.yml
image: arungupta/couchbase
ports:
- . . .
web:
image: arungupta/wildfly
environment:
- COUCHBASE_URI=db-dev:8093
ports:
- 8080:8080 docker-compose up -d

web: production.yml
environment:
- COUCHBASE_URI=db-prod:8093
ports: docker-compose up
- 80:8080 -f docker-compose.yml
db-prod: -f production.yml
image: . . . -d 18
37
Docker Compose Common Use Cases
Use Case Command
Dev Setup docker-compose up
Local/remote host DOCKER_HOST, DOCKER_TLS_VERIFY,
DOCKER_CERT_PATH
Single/multiple hosts Integrated with Swarm
Multiple isolated environments docker-compose up -p <project>
Automated test setup docker-compose up
mvn test
docker-compose down
Dev/Prod Impedance mismatch docker-compose up -f docker-compose.yml -f
production.yml

19
38
 Docker 1.13
▪Deploy Compose services to Swarm
▪CLI restructured
▪Clean-up commands
▪Monitoring commands
▪Build improvements
▪Improved CLI backwards compatibility
▪Docker for AWS/Azure for Production

20
39
 Docker 1.13 - Compose v3

▪docker stack deploy now supports Compose file

– Number of desired instances of each service
– Rolling update
– Server constraints

21
40
 Docker 1.13 - CLI Restructured
Management Commands:
checkpoint Manage checkpoints
container Manage containers
image Manage images
network Manage networks
node Manage Swarm nodes
plugin Manage plugins
secret Manage Docker secrets
service Manage services
stack Manage Docker stacks
swarm Manage Swarm
system Manage Docker
volume Manage volumes
22
41
 Docker 1.13 - Cleanup Commands
▪docker system df and docker system cleanup

docker system df
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 15 1 5.081 GB 4.498 GB (88%)
Containers 1 0 130.1 kB 130.1 kB (100%)
Local Volumes 7 0 110.1 MB 110.1 MB (100%)

23
42
Docker 1.13 - Monitoring Commands
▪docker service logs and Prometheus endpoint

24
43
 Docker 1.13 - Build Improvements

▪docker build --squash: Squash newly built layers into a single

layer
▪docker build --compress: Compress the build context using
gzip

25
44
 Swarm Mode
▪New in 1.12
▪Natively managing a cluster of Docker Engines called a Swarm
▪Docker CLI to create a swarm, deploy apps, and manage swarm
– Optional feature, need to be explicitly enabled
▪No Single Point of Failure (SPOF)
▪Declarative state model
▪Self-organizing, self-healing
▪Service discovery, load balancing and scaling
▪Rolling updates
26
45
 Swarm Mode: Initialize

docker swarm init --listen-addr <ip>:2377

27
46
 Swarm Mode: Add Worker

docker swarm join --token <worker_token> <manager>:2377

28
47
 Swarm Mode: Add More Workers

docker swarm join --token <worker_token> <manager>:2377

29
48
 Swarm Mode: Primary/Secondary Master

docker swarm join --manager --token <manager_token> --listen-

addr <master2>:2377 <master1>:2377
©2016 Couchbase Inc. 30
49
Swarm Mode using Docker Machine
Task Command
Create manger docker-machine create -d virtualbox managerX
Create worker docker-machine create -d virtualbox workerX
Initialize Swarm mode docker swarm init --listen-addr <ip1> --advertise-addr <ip1>
Manager token docker swarm join-token manager -q
Worker token docker swarm join-token worker -q
Manager X join docker swarm join --token manager_token --listen-addr <ipX>
--advertise-addr <ipX> <ip1>
Worker X join docker swarm join --token worker_token --listen-addr <ipX>
--advertise-add <ipX> <ip1>

31
50

https://github.com/docker/labs/blob/master/swarm-mode/quickstart/buildswarm-node-vbox-setup.sh
 Swarm Mode: Protocols
Raft Consensus Group
primary secondary secondary Container

Swarm Manager Swarm Manager Swarm Manager

Swarm Worker Swarm Worker Swarm Worker Swarm Worker Swarm Worker
Gossip Network

Strongly consistent
Replicated (Raft based)
Extremely fast (in-memory reads) 51
 Swarm Mode in Production

33
52

https://blog.online.net/2016/07/29/docker-swarm-an-analysis-of-a-very-large-scale-container-system/
Secure by Default

▪Cryptographic node identity

▪Automatic encryption and
mutual authentication (TLS)
▪Automatic cert rotation (90
days, can be up to 30 mins)
▪External CA integration

34
53
 Swarm Mode: Replicated Service

docker service create --replicas 3 --name web jboss/wildfly

35
54
 Swarm Mode - Routing Mesh

▪Load balancers are host-aware, not container-aware

▪Swarm mode introduces container-aware routing mesh
▪Reroutes traffic from any host to a container
– Reserves a Swarm-wide ingress port
– Uses DNS-based service discovery

36
55
 Swarm Mode: Routing Mesh

Load
Balancer

docker service create --replicas 3 --name web -p 8080:8080 jboss/

wildfly 37
56
Swarm Mode: Node Failure

X
38
57
Swarm Mode: Desired != Actual

39
58
Swarm Mode: Reconcile

40
59
Swarm Mode: Container Failure

X
41
60
Swarm Mode: Desired != Actual

42
61
Swarm Mode: Reconcile

43
62
 Swarm Mode: Scale

docker service scale web=6

44
63
 Swarm Mode: Global Service

docker service create --mode=global --name=prom prom/prometheus

45
64
 Swarm Mode: Pause Node

X For Debugging

docker node update --availability pause <nodename>

46
65
 Swarm Mode: Active Node

docker node update --availability active <nodename>

47
66
 Swarm Mode: Drain Node

X OS Update.

docker node update --availability drain <nodename>

48
67
 Swarm Mode: Rolling Updates

docker service update web --image wildfly:2 --update-parallelism

2 --update-delay 10s 49
©2016 Couchbase Inc.
68
 Swarm Mode: Label

“docker daemon --label

=wildfly.storage=ssd”

“docker daemon --label

=wildfly.storage=ssd”

DOCKER_OPTS="--label=wildfly.storage=ssd"
50
69
 Swarm Mode: Constraints

“docker daemon --label

=wildfly.storage=ssd”

“docker daemon --label

=wildfly.storage=ssd”

docker service create --replicas=3 --name=web --constraint

engine.labels.wildfly.storage==ssd jboss/wildfly
©2016 Couchbase Inc. 51
70
 Swarm Mode: Constraints

“docker daemon --label

=com.example.storage=ssd”

“docker daemon --label

=com.example.storage=ssd”

docker service scale web=6

52
71
 Swarm Mode: Constraints

“docker daemon --label

=com.example.storage=ssd”

“docker daemon --label

=com.example.storage=ssd”

docker service create --replicas=3 --name=db couchbase

53
72
 Scheduling Backends using Filters
▪Label: Metadata attached to Docker Daemon
▪Filters: Used by Docker Swarm scheduler to create and run container

Node
Constraint Default or custom tags node, operatinsystem, kernelversion, …

Health Schedule containers on healthy nodes only

Container Slots Maximum number of containers on a node --labels containerslots=3

Container
Affinity “Attraction” between containers -e affinity:container=<name>/<id>, image, …

Dependency Dependent containers on same node --volumes-from=<id>, --net=container:<id>, …

Port Port availability on the host -p <host>:<container>

©2016 Couchbase Inc. 54

73
Couchbase Multi Dimensional Scaling

55
74
 Optimal Utilization of Resources
couchbase.mds=query
couchbase.mds=query couchbase.mds=index couchbase.mds=data
couchbase.mds=query couchbase.mds=index
couchbase.mds=index couchbase.mds=data
couchbase.mds=data
couchbase.mds=index couchbase.mds=data
couchbase.mds=data
couchbase.mds=data

CPU intensive Memory +

Disk intensive
Fast read/write

• Attach labels: DOCKER_OPTS=“--label.couchbase.mds=data”

• Run Containers: docker service create --constraint
engine.labels.couchbase.mds==index couchbase
56
75
Couchbase Cluster using Docker Services
Couchbase
Image

replicas=1 replicas=3

“Master” Couchbase “Worker”

Service Cluster Service

57

http://blog.couchbase.com/2016/september/docker-service-swarm-mode-couchbase-cluster 76
 Docker Lifecycle
Dockerfile
FROM …

COPY … docker docker CONTAINER

Image CONTAINER
CONTAINER
build run Container
RUN …

CMD …

Single service Single service Single service

image distributable runtime
description image format
format

58
77
 Distributed Application Bundle
docker-compose.yml
version: "2"
services:
Distributed
db: docker-compose docker CONTAINER
CONTAINER
Application CONTAINER
… build deploy Stack
Bundle
web:
…

Multiservice Multiservice Multiservice

image distributable runtime
description image format
format

59

NOTE: Docker Engine and Registry do not support distribution of DABs 78

 Monitoring Docker Containers
▪docker stats command
– LogEntries
▪Service logs: docker service logs <service>
▪Prometheus endpoint - New in Docker 1.13
▪Docker Remote API: /container/{container-name|cid}/stats
▪Docker Universal Control Plane
▪cAdvisor
– Prometheus
– InfluxDB
60
79
 CI/CD with Docker + Jenkins
2
1. Developer updates workspace
Git web hook/poll 2. Jenkins receives the notification
Server 3. Clones the workspace
4. Creates an image
5. Runs the test
6. Pushes the image to Docker Hub
1

git push 3

clone
Create Run Test Docker Hub
Image

4 5 6

61

http://blog.couchbase.com/2016/september/deployment-pipeline-docker-jenkins-java-couchbase 80
 References

▪Slides: github.com/docker/labs/tree/master/slides
▪Workshop: github.com/docker/labs/tree/master/java
▪Docs: docs.docker.com

64
82