Docker for Reproducible Environment

What do we need?
• Hardware Platform
• Operating System
• OS Library
• Process Dependencies

When do we need it?

• Development time
• Testing time
• Production
Support for Containment (in a nutshell)

Docker is organized as:

A repository from which to download all components
A client that commands the entire deployment
A target of nodes on which you run your application
Docker

Docker makes possible to specify simple applications and their need,

independently from hardware and software support details

In case you are interested in several containers,

you must refer to a manager suited to it to handle several containers,
several nodes and their management
An orchestrator can automatize the most common operations of
containers to simplify management

There are many on the market

Orchestrator for Dynamic Handling (in a nutshell)

Docker needs some tools to coordinate the static and dynamic configuration
with Orchestrators → we will go deeper into some of them in the next module
Docker Swarm, Kubernetes, MESOS, …
Containers (& Orchestration…) for Scalability

Large targets
Under the hood: Docker Engine

Docker is a container engine that enables configuration, deployment,

and Lifecycle Management of containers.

Client-server architecture
• Docker daemon

• API REST/Docker CLI

Docker Architecture

▪ Docker daemon – The Docker daemon listens for Docker API requests and manages
Docker components such as images, containers, networks, and volumes.
▪ Docker client – The Docker client (docker) is the primary way that many Docker users
interact with Docker. When you use commands such as docker run, the client sends these
commands to dockerd, which carries them out.
▪ Docker registries – A Docker registry stores Docker images. Docker Hub and Docker Cloud
are public registries that anyone can use, and Docker is configured to look for images on
Docker Hub by default. You can even run your own private registry. Docker registries are
the distribution component of Docker.
Docker main components

Docker images
A Docker image is a read-only template. For example, an image could contain an Ubuntu
operating system with Apache and your web application installed. Images are used to create
Docker containers. Docker provides a simple way to build new images or update existing images,
or you can download Docker images that other people have already created. Docker images
are the build component of Docker.

Docker containers
Docker containers are similar to a directory. A Docker container holds everything that is needed
for an application to run. Each container is created from a Docker image. Docker containers can
be run, started, stopped, moved, and deleted. Each container is an isolated and secure
application platform. Docker containers are the run component of Docker.
Main Docker components revised
LOCAL HOST

DOCKER CLI DOCKER DAEMON

DOCKER HOST
Main Docker components revised
LOCAL HOST DOCKER HUB

IMAGE 1 IMAGE 2

REPOSITORY

IMAGE N IMAGE M

DOCKER CLI DOCKER DAEMON REPOSITORY

DOCKER HOST DOCKER REGISTRY

Main Docker components revised
LOCAL HOST DOCKER HUB

IMAGE 1 IMAGE 2

REPOSITORY

IMAGE 1 IMAGE 2 IMAGE N

IMAGE N IMAGE M

DOCKER CLI DOCKER DAEMON REPOSITORY

DOCKER HOST DOCKER REGISTRY

Main Docker components revised
LOCALHOST DOCKER HUB

CONTAINER A1 CONTAINER A2
IMAGE 1 IMAGE 2

CONTAINER B CONTAINER C REPOSITORY

IMAGE 1 IMAGE 2 IMAGE N

IMAGE N IMAGE M

DOCKER CLI DOCKER DAEMON REPOSITORY

DOCKER HOST DOCKER REGISTRY

Main Docker components revised
LOCAL HOST DOCKER HUB

CONTAINER A1 CONTAINER A2
IMAGE 1 IMAGE 2

CONTAINER B CONTAINER C REPOSITORY

IMAGE 1 IMAGE 2 IMAGE N

IMAGE N IMAGE M

DOCKER CLI DOCKER DAEMON REPOSITORY

DOCKER HOST DOCKER REGISTRY

REMOTE HOST

CONTAINER X

CONTAINER Y

IMAGE 6 IMAGE 7

DOCKER DAEMON

DOCKER HOST
Main Docker components revised
LOCAL HOST DOCKER HUB

CONTAINER A1 CONTAINER A2
IMAGE 1 IMAGE 2

CONTAINER B CONTAINER C REPOSITORY

IMAGE 1 IMAGE 2 IMAGE N

IMAGE N IMAGE M

DOCKER CLI DOCKER DAEMON REPOSITORY

DOCKER HOST DOCKER REGISTRY

PRIVATE REGISTRY

IMAGE 6 IMAGE 7

REPOSITORY

DOCKER REGISTRY
Docker Container Lifecycle
Docker images
holds all container-specific CONTAINER
Random UUID
writes and deletes LAYER (R/W)

91e54dfb1179 0B

Cryptograph d74508fb6632 1.895 KB IMAGE

ic content- LAYERS (R/O)
based
hashes c22013c84729 194.5 KB
(sha-256)
d3a1f33e8a5a 188.1 MB

ubuntu:15.04
Docker images

Docker images are read-only stacks of layers → copy-on-write approach

each layer is uniquely identified by a cryptographic content-based hash (>=v.1.10)
– collision detection mitigation
– strong and efficient content comparison mechanism

This approach is hugely beneficial

– efficient disk usage
• each new layer keeps only differences from preceding layers
• layers can be shared among images, e.g. “base” layers such as OS layers (fedora:latest, ubuntu:latest)
– ease of modification
• new images may be built by simply stacking new layers on top of preceding ones, leaving the below layers
unmodified

39
Docker Images - Naming convention

[hostname[:port]]/[username]/reponame[:tag]

Hostname/port of registry holding the image. If missing,

defaults to Docker Hub public registry (docker.io).

Username. If missing, defaults to library username on

Docker Hub, which hosts official, curated images.

Reponame. Actual image repository.

Tag. Optional image specification (e.g., version number). If

missing, defaults to latest.

40
 Dockerfile

Starting Image
Configuration
NO Layer creation

Modify fs
Multi-Stage Build Layer creation
Docker container lifecycle (and related events)
Docker plugins

Docker is a rich ecosystem and there are plugins to extend Docker basic functionalities
(we do not have time to go into details, but just report some hints for those who are interested…)

Volume

Network

Authorization
 Networks

• Bridge: uses a software bridge which allows containers connected to the same bridge
network to communicate, while providing isolation from containers which are not
connected to that bridge network

• Host: Remove network isolation between the container and the Docker host, and use the
host’s networking directly

• Overlay: software network abstraction that can be used to run multiple separate,
discrete virtualized network layers on top of the physical network ( We will go deeper….)

• Macvlan: assign a MAC address to a container, making it appear as a physical device

on your network. The Docker daemon routes traffic to containers by their MAC addresses

• None: disable all networking

• External Plugin: use third-party network plugins

Persistence
(Advanced) Configuration

Configure container execution

• Dockerfile
• Docker CLI
• Docker-Compose

Share configurations with the containerized process

• Docker secrets
• Docker Environment variables
• Docker Config files
 Docker-Compose

To organize internally your application, e.g., a data container and an application logic container
Example: containerization of a 2-tier Web/REST application

For those interested… source code link: https://github.com/davidMonnuar/projectActivity

3-tier web application for books collection management

Client Tier Application Tier Database Tier

REST APIs JPA

• HTML5 • Java • Relational DB

• Javascript • Spring Boot • MySQL
• CSS • Tomcat
Example: Spring Boot Application Tier 1

Spring Boot is an approach to develop spring application with minimal or zero

configuration.

Tomcat used as an Embedded Server

Think about what you would need to be able to deploy your application (typically) on a virtual
machine.

Step 1: Install Java

Step 2: Install the Web/Application Server (Tomcat/Websphere/Weblogic etc)
Step 3: Deploy the application war

What if we want to simplify this? This idea is the genesis for Embedded Servers.

For example, for a Spring Boot Application, you can generate an application jar which contains
Embedded Tomcat. You can run a web application as a normal Java application!
Example: Spring Boot Application Tier 2
Entities
- Author.java
Thanks to Spring deep integration with JPA we don’t need to
- Book.java
spend too much time in configuring the persistence layer
since most part of the setup is done automatically (i.e. db
creation) by exploiting Java annotations. Repositories
- AuthorRepository.java
Application Tier Database Tier - BookRepository.java

Controller
- MainController.java
JPA

The only thing we have to write down is a configuration file called

View
application.yml in which we specify
- CustomBookSerializer.java
-the URL to access the database
-username Resources
-password - application.yml
-plus other parameters
Example: Sping Boot application.yml - resources package

We use global variables to avoid the database information hard-coding.

This let us to re-use this web app with any database we want.

spring:
profiles: container
datasource:
driverClassName: ${DATABASE_DRIVER}
url: jdbc:mysql://${DATABASE_HOST}:${DATABASE_PORT}/${DATABASE_NAME}?useSSL=false&allowPublicKeyRetrieval=true
username: ${DATABASE_USER}
password: ${DATABASE_PASSWORD}
tomcat:
test-while-idle: true
time-between-eviction-runs-millis: 60000
validation-query: SELECT 1
jpa:
hibernate.ddl-auto: create-drop
properties.hibernate.dialect: org.hibernate.dialect.MySQL5Dialect
Example: Application containerization

App organization
One container for Spring
One container for MySQL + Volume

The first step is to define an image for each container by writing the Dockerfile.
Thanks to a simple sintax a Dockerfile let us to define all the steps necessary to create an
image and to execute it.
 Spring container Dockerfile
Image creation and upload in the DockerHub
1 FROM openjdk:8-jre-alpine $ docker build –t librarydemo
2 MAINTAINER Dmitrij David Padalino Montenero
3 $ docker login
5 VOLUME /tmp
4 EXPOSE 8080 $ docker tag librarydemo davidmonnuar/springbootlibrarydemo:1.0
6
7 ARG JAR_FILE $ docker push davidmonnuar/springbootlibrarydemo:1.0
8 ADD target/${JAR_FILE} app.jar
9 ADD wrapper.sh wrapper.sh
10
11
12
RUN apk add --update bash && rm -rf /var/cache/apk/*
RUN bash -c 'chmod +x /wrapper.sh'
Image size only 116 MB
13 RUN bash -c 'touch /app.jar'
14
15 ENTRYPOINT ["/bin/bash", "/wrapper.sh"]
MySQL container Dockerfile
No need to create a new Docker image since the
Script entrypoint official MySQL image available in the DockerHub is
1 #!/bin/bash enough for our purposes.
2 while ! exec 6<>/dev/tcp/${DATABASE_HOST}/${DATABASE_PORT}; do
3 echo "Trying to connect to MySQL at ${DATABASE_PORT}..."
4 sleep 10
5 done
6
7 java -Djava.security.egd=file:/dev/./urandom
-Dspring.profiles.active=container -jar /app.jar
Cluster creation and configuration

A swarm is a group of machines, physical or virtual, that are running Docker and joined into a
cluster. The cluster we will use is composed by two nodes, a master and one worker.
Virtual machines creation
$ docker-machine create –driver virtualbox myvm1

$ docker-machine create –driver virtualbox myvm2

$ docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
myvm1 - virtualbox Running tcp://192.168.99.100:2376 v18.06.1-ce
myvm2 - virtualbox Running tcp://192.168.99.101:2376 v18.06.1-ce

Cluster initialization
$ docker-machine ssh myvm1 "docker swarm init --advertise-addr 192.168.99.100:2377
Swarm initialized: current node 0unmutpbeeeytxpquhiy1b6ok is now a manager.
To add a worker to this swarm, run the following command:
docker swarm join \
--token SWMTKN-1-4y8bpxyrbno89dapkkwyjdw3268qfzp128tpf5hecjdti5hb1k-00oatopl9dzw3jejbietxa9vp 192.168.99.100:2377

$ docker-machine ssh myvm2 "docker swarm join --token SWMTKN-1-4y8bpxyrbno89dapkkwyjdw3268qfzp128tpf5hecjdti5hb1k-

00oatopl9dzw3jejbietxa9vp 192.168.99.100:2377"

$ docker-machine ssh myvm1 "docker node ls"

ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
0unmutpbeeeytxpquhiy1b6ok * myvm1 Ready Active Leader 18.06.1-ce
ktvyi0nypz31l645iej8torkx myvm2 Ready Active 18.06.1-ce
Deployment - 1
In a distributed application, different pieces of the app are called “services”.

Services in Docker are really just “containers in production.” A service only runs one image, but it
codifies the way that image runs - what ports it should use, how many replicas of the container
should run so the service has the capacity it needs, and so on. Scaling a service changes the
number of container instances running that piece of software, assigning more computing resources
to the service in the process.

In order to connect services, Docker uses the concept of Stack. A stack is a group of interrelated
services that share dependencies, and can be orchestrated and scaled together.

In the docker-compose.yml file are defined

• A service “ mysqldb”, port 3306
• A service “ spring” , port 80
• A service “ visualizer”, port 8080
• A volume “ my-datavolume” used by “mysqldb”
• A network “ webnet” with the default settings (which is a load-balanced overlay network)

Application deployment
$ docker-machine ssh myvm1 "docker stack deploy -c docker-compose.yml librarywebapp"
Deployment - 2
1 version: "3" 31 environment:
2 services: 32 - DATABASE_DRIVER=com.mysql.jdbc.Driver
3 mysqldb: 33 - DATABASE_HOST=mysqldb
4 image: mysql:latest 34 - DATABASE_PORT=3306
5 volumes: 35 - DATABASE_NAME=db_example
6 - my-datavolume:/var/lib/mysql 36 - DATABASE_USER=springuser
7 deploy: 37 - DATABASE_PASSWORD=mysql2019
8 placement: 38 networks:
9 constraints: [node.role == manager] 39 - webnet
10 environment: 40 visualizer:
11 - MYSQL_ROOT_PASSWORD=password 41 image: dockersamples/visualizer:stable
12 - MYSQL_DATABASE=db_example 42 ports:
13 - MYSQL_USER=springuser 43 - "8080:8080"
14 - MYSQL_PASSWORD=mysql2019 44 volumes:
15 networks: 45 - "/var/run/docker.sock:/var/run/docker.sock"
16 - webnet 46 deploy:
17 spring: 47 placement:
18 image: davidmonnuar/springbootlibrarydemo:1.0 48 constraints: [node.role == manager]
19 depends_on: 49 networks:
20 - mysqldb 50 - webnet
21 deploy: 51 volumes:
22 replicas: 5 52 my-datavolume:
23 restart_policy: 53 networks:
24 condition: on-failure 54 webnet:
25 resources:
26 limits:
27 cpus: "0.15"
28 memory: 500M
29 ports:
30 - "80:8080"
31
Deployment - 3

Swarm
vm1 (manager) 192.168.99.100 vm2 (worker) 192.168.99.101
Node

Service
MySQL (port 3306)

Visualizer (port 8080)

Spring (5 replicas == 5 containers) (port 80)

Deployment - 4
By inserting one of the two cluster node
URLs in the browser it is possible to have
access to the application
• http://192.168.99.100
• http://192.168.99.101

For the Visualizer service

• http://192.168.99.100 :8080
• http://192.168.99.101:8080