Docker Swarm

Docker Container
 Agenda Swarm
● What is Docker Swarm ?
Instructions / Notes
● Creating Swarm ● Starts: 19 Apr, 7:30am PDT/PST or 10 Apr, 8pm IST
● Deploy Service on Swarm ● 2 Hours session with 10 minutes Break
● Service scaling ● Except Instructor, everyone is on mute
● Please use the Q/A Window to Ask Questions
● Applying rolling update
● The recording & slides will be shared after session
● Managing Swarm
● More Info: CloudxLab.com
● Draining node

Ashok Singh
Docker Container
 What is Swarm?

Docker Container
 Docker Swarm
● Cluster management

● Declarative service management

● Scaling

● Desired state configuration

● Network overlay creation

● Service discovery

● Load balancing

● Secure communication on control plane

Docker Container
 Docker Jargon
● Cluster :Group of Machine working in coordination to host multiple container

services for high availability and optimized resource utilisation

● Swarm :- Manager and Worker host implementing Docker’s orchestration

● Nodes : Hosts that are part of cluster running services.

● Services : Definition of set of tasks to be executed.
● Tasks : All Information to run a docker container.

Docker Container
 Docker Swarm Cluster
● Manager Node
● Worker Node
● Distributed database
● Raft consensus group

Docker Container
 Communication Ports
Port Protocol Purpose

2377 TCP Cluster Control plane

communications

7946 TCP and UDP Communication

between nodes

4789 UDP Overlay network traffic

Docker Container
 Manager Node
● Manager node functions:

○ Maintains cluster control plane

○ Maintain desired state by scheduling services

○ serves as HTTP API response server for swarm node

For High Availability and Fault tolerance, at least 3 master are recommended

Docker Container
 Manager Node Failure Scenario

● A three-manager swarm tolerates a maximum loss of one manager.

● A five-manager swarm tolerates a maximum simultaneous loss of two
manager nodes.
● An N manager cluster tolerates the loss of at most (N-1)/2 managers.

Manager nodes implement the Raft Consensus Algorithm to manage the global
cluster state.

Docker Container
 Raft Consensus Algorithm
Visual explanation of Raft Consensus Algorithm

http://thesecretlivesofdata.com/raft/

● Multiple Master Node can have split network scenario.

Manager Manager Manager

Network 1 Manager Manager Network 2

Docker Container
 Creating Swarm

Docker Container
 Docker Swarm Setup

Worker Node 1

Manager Worker Node 2

Worker Node 3

Docker Container
 Node Setup
● Prerequisite:

○ Install docker

○ Set-up hostname
○ sudo hostnamectl set-hostname --static <HOSTNAME>

○ echo "preserve_hostname: true" >> /etc/cloud/cloud.cfg

○ sudo reboot

Docker Container
 Create Swarm
● Connect to manager node and initialize the swarm

ssh -i <aws-key.pem> centos@<public_ip>

use private IP of
sudo su - EC2

docker swarm init --advertise-addr <MANAGER-IP>

[root@manager ~]# docker swarm init --advertise-addr 172.31.18.36
Swarm initialized: current node (do2t685v0hqxnj6pflzoupiyx) is now a manager.

To add a worker to this swarm, run the following command:

docker swarm join --token

SWMTKN-1-4kbmy9oy74ak16lhuiqcbb5mkodiaxw452ggnax8x3nahsafyz-5a1o2hwicwx258cvgut0bae9l
172.31.18.36:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

[root@manager ~]#

Docker Container
 Add node to Swarm
● Connect to worker-1 node and join it to to swarm

ashoksingh@Ashoks-Air ~/Devops-Course/AWS ssh -i aws-key.pem [email protected]

The authenticity of host '35.178.147.122 (35.178.147.122)' can't be established.
ECDSA key fingerprint is SHA256:WKPmKrWFhCeb2Hyms8PQVCOzZzrTYFiy2eABmVRLsqk.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '35.178.147.122' (ECDSA) to the list of known hosts.
[centos@worker-1 ~]$ sudo su -
[root@worker-1 ~]# docker swarm join --token
SWMTKN-1-4kbmy9oy74ak16lhuiqcbb5mkodiaxw452ggnax8x3nahsafyz-5a1o2hwicwx258cvgut0bae9l
172.31.18.36:2377
This node joined a swarm as a worker.
[root@worker-1 ~]#

Docker Container
 Check Swarm nodes
● Connect back to manager node and check node is added

[root@manager ~]# docker node ls

ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
mg6cim0tx3wlmgprzz518eq2i worker-1 Ready Active 19.03.8
do2t685v0hqxnj6pflzoupiyx manager Ready Active Leader 19.03.8
[root@manager ~]#

Docker Container
 Add 2 more nodes to Swarm
● Join command can be obtained from master node
[root@manager ~]# docker swarm join-token worker
To add a worker to this swarm, run the following command:
docker swarm join --token
SWMTKN-1-4kbmy9oy74ak16lhuiqcbb5mkodiaxw452ggnax8x3nahsafyz-5a1o2hwicwx258cvgut0bae9l
172.31.18.36:2377
[root@manager ~]#

● Connect to worker-2 node and run join command

[centos@worker-2 ~]$ sudo su -
[root@worker-2 ~]# docker swarm join --token
SWMTKN-1-4kbmy9oy74ak16lhuiqcbb5mkodiaxw452ggnax8x3nahsafyz-5a1o2hwicwx258cvgut0bae9l
172.31.18.36:2377
This node joined a swarm as a worker.
[root@worker-2 ~]#

Docker Container
 Swarm Cluster Ready !!
● Connect to worker-3 node and run join command
[centos@worker-3 ~]$ sudo su -
[root@worker-3 ~]# docker swarm join --token
SWMTKN-1-4kbmy9oy74ak16lhuiqcbb5mkodiaxw452ggnax8x3nahsafyz-5a1o2hwicwx258cvgut0bae9l
172.31.18.36:2377
This node joined a swarm as a worker.
[root@worker-3 ~]#

● Connect to manager node and check swarm

[root@manager ~]# docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
mg6cim0tx3wlmgprzz518eq2i worker-1 Ready Active 19.03.8
do2t685v0hqxnj6pflzoupiyx master Ready Active Leader 19.03.8
1telhywrq06ap3mwq00i18c8n worker-2 Ready Active 19.03.8
vz5pd0nu7qohkims55xsb3g25 worker-3 Ready Active 19.03.8
[root@manager ~]#

Docker Container
 Lab01: Create your own swarm cluster
● Create a 2 node docker swarm cluster

● Step 1: Launch 2 VM on cloud

● Step 2: Connect to VM1 and set hostname as manager

● sudo hostnamectl set-hostname --static manager

● sudo echo “preserve_hostname: true” >> /etc/cloud/cloud.cfg

● sudo reboot

Docker Container
 Lab01
● Step 3: Connect to VM1 and set hostname as worker-1

● sudo hostnamectl set-hostname --static worker-1

● sudo echo “preserve_hostname: true” >> /etc/cloud/cloud.cfg

● sudo reboot

● Step 4: Connect to manager node and initialize swarm

● docker swarm init --advertise-addr <Private_IP>

docker swarm join --token

SWMTKN-1-4kbmy9oy74ak16lhuiqcbb5mkodiaxw452ggnax8x3nahsafyz-5a1o2hwicwx258cvgut0bae9l
172.31.18.36:2377

● Step 5: Copy the output of above command to join worked node

Docker Container
 Lab01
● Step 6: Connect to worker-1 node and join it to swarm

● sudo su -

● docker swarm join --token <Token from output of swarm init>

<Private_IP_of_manager_node>:2377

● Step 7: Connect to manager node and check swarm

● docker node ls

This command should show 2 nodes in cluster with 1 as Manager role and other as Worker

Docker Container
 Scenario 1
● Create a swarm which has 2 manager . Remove one master from

swarm and re-join it . Rejoining fails and healthy manager

stop working

Manager2 node which left : Error response from daemon: rpc error: code = Unknown desc = The
swarm does not have a leader. It's possible that too few managers are online. Make sure more
than half of the managers are online.

Healthy manager: Error response from daemon: rpc error: code = DeadlineExceeded desc =
context deadline exceeded

Docker Container
 Scenario 1
You can use the --force option on a manager to remove it from the swarm. However, this

does not reconfigure the swarm to ensure that there are enough managers to maintain a

quorum in the swarm. The safe way to remove a manager from a swarm is to demote it to a

worker and then direct it to leave the quorum without using --force. Only use --force in

situations where the swarm will no longer be used after the manager leaves, such as in a

single-node swarm.

Docker Container
 Automate Docker Swarm on AWS

Docker Container
 AWS CLI Installation
● Mac users

● Ubuntu
○ ( Window user with Virtual box)

● Bastion host on Amazon EC2

○ (Windows user who cannot run Virtual box)

https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html

Docker Container
 AWS CLI Installation for MAC
ashoksingh@Ashoks-Air# ~ curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o
"AWSCLIV2.pkg"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 20.5M 100 20.5M 0 0 3123k 0 0:00:06 0:00:06 --:--:-- 3892k
ashoksingh@Ashoks-Air# ~ sudo installer -pkg AWSCLIV2.pkg -target /
Password:
installer: Package name is AWS Command Line Interface
installer: Installing at base path /
installer: The install was successful.
ashoksingh@Ashoks-Air# ~ aws --v
aws-cli/1.18.40 Python/3.7.7 Darwin/18.7.0 botocore/1.15.40
ashoksingh@Ashoks-Air# ~

Docker Container
 AWS CLI Installation on Ubuntu
root@ip-172-31-23-164:~# curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 31.2M 100 31.2M 0 0 72.6M 0 --:--:-- --:--:-- --:--:-- 72.4M
rooot@ip-172-31-23-164:~# apt install unzip
Reading package lists... Done
…….(more line)
Setting up unzip (6.0-25ubuntu1) ...
Processing triggers for mime-support (3.64ubuntu1) ...
Processing triggers for man-db (2.9.1-1) ...
root@ip-172-31-23-164:~# unzip awscliv2.zip
Archive: awscliv2.zip
creating: aws/
creating: aws/dist/
inflating: aws/README.md
…..(many lines)
inflating: aws/dist/botocore/data/comprehendmedical/2018-10-30/service-2.json
inflating: aws/dist/botocore/data/comprehendmedical/2018-10-30/paginators-1.json
root@ip-172-31-23-164:~# sudo ./aws/install
You can now run: /usr/local/bin/aws --version
root@ip-172-31-23-164:~# aws --v
aws-cli/2.0.9 Python/3.7.3 Linux/5.4.0-1009-aws botocore/2.0.0dev13
root@ip-172-31-23-164:~#

Docker Container
 AWS CLI is pre installed on EC2
ashoksingh@Ashoks-Air ~/Devops-Course/AWS ssh -i "aws-key.pem"
[email protected]
The authenticity of host 'ec2-35-178-10-233.eu-west-2.compute.amazonaws.com (35.178.10.233)'
can't be established.
ECDSA key fingerprint is SHA256:Fs3trcKJJ5ZRYqLHMpphsFqwKclwcciE5fk6RvKvE8M.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ec2-35-178-10-233.eu-west-2.compute.amazonaws.com,35.178.10.233'
(ECDSA) to the list of known hosts.

__| __|_ )
_| ( / Amazon Linux 2 AMI
___|\___|___|

https://aws.amazon.com/amazon-linux-2/
No packages needed for security; 4 packages available
Run "sudo yum update" to apply all updates.
[ec2-user@ip-172-31-28-149 ~]$ aws --v
aws-cli/1.16.300 Python/2.7.16 Linux/4.14.173-137.229.amzn2.x86_64 botocore/1.13.36
[ec2-user@ip-172-31-28-149 ~]$

Docker Container
 Bastion host preparation (Amazon Linux)
Step 1:Become root
sudo su -

Step 2: Install Git

yum install -y git

Step 3: Clone Repo https://github.com/singh-ashok25/aws-swarm-docker.git

git clone https://github.com/singh-ashok25/aws-swarm-docker.git

Step 4: Change Directory to “aws-swarm-docker”

cd aws-swarm-docker

Step 5: Install prerequisite package before launching docker swarm

./package.sh

Step 6: Configure aws cli

aws configure
Docker Container
 Configure AWSCLI
root@ip-172-31-23-164:~# aws configure
AWS Access Key ID [None]:

To get value of AWS Access Key ID -

Step 1: Login to AWS Console and Select IAM service as shown below then click Users

Docker Container
 Configure AWSCLI
Step 2: click on the user in IAM
service.

Step 3: Select tab security_credentials

Step 4: Click Create access key

Docker Container
 Configure AWSCLI
Step 5: click on show secret
key and Download .csv file

Step 6: Fill the value of

AWS Access Key ID and AWS
Secret Access Key

root@ip-172-31-23-164:~# aws configure

AWS Access Key ID [None]: AKIAXP6SPYSGTLWBK4FK
AWS Secret Access Key [None]: 1x4UcKGzPGGrraPGxpCRMaxStLLESmLgiN5AU6Nn

Docker Container
 Configure AWSCLI
Step 7: Click on EC2 and then select Region to display
region code .Fill your region in AWS configure command

root@ip-172-31-23-164:~# aws configure

AWS Access Key ID [None]: AKIAXP6SPYSGTLWBK4FK
AWS Secret Access Key [None]: 1x4UcKGzPGGrraPGxpCRMaxStLLESmLgiN5AU6Nn
Default region name [None]: eu-west-2

Docker Container
 Configure AWSCLI
Step 8: Enter “json” as default output format

root@ip-172-31-23-164:~# aws configure

AWS Access Key ID [None]: AKIAXP6SPYSGTLWBK4FK
AWS Secret Access Key [None]: 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
Default region name [None]: eu-west-2
Default output format [None]: json
root@ip-172-31-23-164:~#

Step 9: Check aws command by entering a command below “aws s3 ls“

(aws) ashoksingh@Ashoks-Air ~/Devops-Course/AWS aws s3 ls

2020-04-25 20:56:57 docker-swarm.terraform
2020-04-25 20:11:42 my-docker-swarm.terraform

Docker Container
 Create Swarm
Step 1: Launch docker swarm script
./create-swarm.sh
Worker Node 1
[root@ip-172-31-19-169 aws-swarm-docker]#
./create-swarm.sh
Running pre-create checks... Manager
Creating machine...
(leader1) Launching instance...
Waiting for machine to be running, this may take a few
minutes...
Worker Node 2
Detecting operating system of created instance...

.
…
…
..

This node joined a swarm as a worker.

This node joined a swarm as a worker.
[root@ip-172-31-19-169 aws-swarm-docker]#

Docker Container
 Check swarm
Step 1: Run docker-machine ls to check worker and manager status in cluster
docker-machine ls

[root@ip-172-31-19-169 aws-swarm-docker]# docker-machine ls

NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
leader1 - amazonec2 Running tcp://18.130.251.210:2376 v19.03.8
worker1 - amazonec2 Running tcp://18.132.38.214:2376 v19.03.8
worker2 - amazonec2 Running tcp://3.10.56.53:2376 v19.03.8

Step 2: Run docker node ls to check worker and manager status in cluster
docker-machine ls
root@leader1:~# docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
i74qp219dh7onziyo86gt3dxm * leader1 Ready Active Leader 19.03.8
d1exem5t3fscge02pwgw9hhjj worker1 Ready Active 19.03.8
6hffq9ro1xhmpqt57w07i1ofx worker2 Ready Active 19.03.8

Docker Container
 Docker Machine

Docker Container
 What is Docker Machine
● Tool that installs and manages docker engine on remote host.
● Install and run Docker on Mac or Windows
● Provision and manage multiple remote Docker hosts
● Provision Swarm clusters

Source: docker.com
Docker Container
 Install Docker Machine
● If you are running macOS:

$ base=https://github.com/docker/machine/releases/download/v0.16.0 &&
curl -L $base/docker-machine-$(uname -s)-$(uname -m) >/usr/local/bin/docker-machine &&
chmod +x /usr/local/bin/docker-machine

● If you are running Linux :

$ base=https://github.com/docker/machine/releases/download/v0.16.0 &&
curl -L $base/docker-machine-$(uname -s)-$(uname -m) >/tmp/docker-machine &&
sudo mv /tmp/docker-machine /usr/local/bin/docker-machine &&
chmod +x /usr/local/bin/docker-machine

Docker Container
 Install Docker Machine
● Create docker host :

docker-machine create --driver amazonec2 --amazonec2-access-key <Access_key>

--amazonec2-secret-key <Secret-Key> --amazonec2-region eu-west-2 <name of machine>

● if aws configure is already set as on host machine above command can be

simplified to

docker-machine create --driver amazonec2 <name of machine>

Docker Container
 Docker Machine ssh and ls
● List all machines

ashoksingh@Ashoks-Air ~/ docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM DOCKER
ERRORS
leader1 - amazonec2 Running tcp://35.176.108.247:2376 v19.03.8
worker1 - amazonec2 Running tcp://35.179.96.104:2376 v19.03.8
worker2 - amazonec2 Running tcp://3.10.171.252:2376 v19.03.8

● Connect to machine

ashoksingh@Ashoks-Air ~/ docker-machine ssh leader1

Welcome to Ubuntu 16.04.4 LTS (GNU/Linux 4.4.0-1052-aws x86_64)

New release '18.04.4 LTS' available.

Run 'do-release-upgrade' to upgrade to it.
ubuntu@leader1:~$

Docker Container
 Docker Machine env and eval
● Get local environment variable which allow to run docker command on remote host

ashoksingh@Ashoks-Air ~/ docker-machine env leader1

export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://35.176.108.247:2376"
export DOCKER_CERT_PATH="/Users/ashoksingh/.docker/machine/machines/leader1"
export DOCKER_MACHINE_NAME="leader1"
# Run this command to configure your shell:
# eval $(docker-machine env leader1)
ashoksingh@Ashoks-Air ~/

● Set shell env for leader1 machine and run command to get worker token

ashoksingh@Ashoks-Air ~/ eval $(docker-machine env leader1)

ashoksingh@Ashoks-Air ~/ docker swarm join-token worker
To add a worker to this swarm, run the following command:

docker swarm join --token

SWMTKN-1-2mdznjlzxzcz6at1vsn0drqd6xptkhukdu75ltdmuwkqj46f05-7bkj28z8z2wr28bambie436ln
172.31.30.22:2377

Docker Container
 Docker Machine inspect
● Inspect list details of machine
ashoksingh@Ashoks-Air ~/ docker-machine inspect leader1
[
{
"ID": "0jfq0elhf2ngpg1ekvf3vz7io", "Status": {
"Version": { "State": "ready",
"Index": 9 "Addr": "172.31.30.22"
},
"CreatedAt": "2020-04-26T12:01:15.671957839Z", },
"UpdatedAt": "2020-04-26T12:01:15.791712262Z", "ManagerStatus": {
"Spec": { "Leader": true,
…..
…. "Reachability": "reachable",
"Addr": "172.31.30.22:2377"
}
}
● Extract machine IP from output of docker inspect ]

ashoksingh@Ashoks-Air ~ docker-machine inspect --format='{{ .ManagerStatus.Addr}}' leader1

172.31.30.22:2377
ashoksingh@Ashoks-Air ~

Docker Container
 Service Deployment on Swarm

Docker Container
 Deploy a service
● Step 1: ssh to manager-node and run service create command

[root@manager ~]sudo su -
[root@manager ~]docker service create --replicas 1 --name devops-webapp -p 80:4080
ashok25/flaskapp:v1.0
ohmhdju5qgbgouzsvc1jwqwpw
overall progress: 1 out of 1 tasks
1/1: running [==================================================>]
verify: Service converged

● Step 2: Run below command to check the service running

[root@manager ~] docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
ohmhdju5qgbg devops-webapp replicated 1/1 ashok25/flaskapp:v1.0 *:80->4080/tcp

Docker Container
 Access the service
● Step 3: Locate public IP address of manager node

● Step 4: Goto http://<Public_IP_of_Manager_node>:80

Docker Container
 Why Docker service ?

Docker service leverages docker swarm functionality to be

fault tolerant without any need of manual intervention.

Docker Container
 Scenario 1 : Service dies and host alive
● Step 1: Run docker service ps <service id> to find the node

which is running service container

[root@master ~]# docker service ps devops-webapp
ID NAME IMAGE NODE DESIRED
STATE CURRENT STATE ERROR PORTS
vf6cio4lphp7 devops-webapp.1 ashok25/flaskapp:v1.0 master Running
Running 3 minutes ago

● Step 2: Master node can run service container. Connect to

Master node and kill running container

Docker Container
 Scenario 1 : Service dies and host alive

● Step 3: Kill the docker container using below command

[root@master ~]# docker kill bc941cdcc863

bc941cdcc863
[root@master ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS
PORTS NAMES
[root@master ~]#

Docker Container
 Scenario 1 : Service dies and host alive
● Step 4: Goto http://<Public_IP_of_Manager_node>:80

○ Page not found error

● Step 5: Refresh page after 1 min

○ Webapp is running without taking any action.

○ Manager node has detected a failure and restarted

container on a next healthy node.

Docker Container
 Scenario 1 : Service dies and host alive
● Step 6: Check service status while container is killed
[root@master ~]# docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
ohmhdju5qgbg devops-webapp replicated 0/1 ashok25/flaskapp:v1.0 *:80->4080/tcp

● Step 7: Check when service is restored

[root@manager ~] docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
ohmhdju5qgbg devops-webapp replicated 1/1 ashok25/flaskapp:v1.0 *:80->4080/tcp

Docker Container
 Scenario 2 : Host/Node dies
● Prerequisite: Worker node are label to apply constraints

● Add Label “type=worker” to worker1 and worker2 node

root@leader1:~# docker node update --label-add type=worker worker2 worker1

worker2

● Check label is correctly applied

root@leader1:~# docker inspect worker2 | grep node_type
"node_type": "t2.micro"
root@leader1:~#

Docker Container
 Scenario 2 : Host/Node dies
● Step 1: Start the service on worker2 node
root@leader1:~# docker service create --replicas=1 --constraint node.labels.type==worker
--name devops-webapp -p 80:4080 ashok25/flaskapp:v1.0
ba9umbrr3vi2s6rzqbucbjvcz
overall progress: 1 out of 1 tasks
1/1: running [==================================================>]
verify: Service converged
root@leader1:~#

● Step 2: Check service is running on any worker node

root@leader1:~# docker service ps devops-webapp
ID NAME IMAGE NODE DESIRED STATE
CURRENT STATE ERROR PORTS
zrpcq60bx1dt devops-webapp.1 ashok25/flaskapp:v1.0 worker2 Running
Running 28 seconds ago
root@leader1:~#

Docker Container
 Scenario 2 : Host/Node dies
● Step 3: Goto http://<Public_IP_of_Manager_node>:80

● Step 4: Login to AWS console and terminate

worker2 node

Docker Container
 Scenario 2 : Host/Node dies
● Step 5 : Goto http://<Public_IP_of_Manager_node>:80

○ Service down - Page not found error

○ Retry after 30 sec - Webpage is up and running

● Step 6: Check when service is restored

root@leader1:~# docker service ps devops-webapp

ID NAME IMAGE NODE
DESIRED STATE CURRENT STATE ERROR PORTS
fjdexi9712l9 devops-webapp.1 ashok25/flaskapp:v1.0 worker1
Running Running 2 seconds ago
l9dsfyeap6fv \_ devops-webapp.1 ashok25/flaskapp:v1.0 worker2
Shutdown Running 30 seconds ago

Docker Container
 Inspecting Docker Service
● docker service inspect --pretty devops-webapp
[root@master ~]# docker inspect devops-webapp
[
{
"ID": "ohmhdju5qgbgouzsvc1jwqwpw",
"Version": {
"Index": 27
},
"CreatedAt": "2020-04-24T22:38:54.853369294Z",
"UpdatedAt": "2020-04-24T22:38:54.854303892Z",
"Spec": {
"Name": "devops-webapp",
"Labels": {},
"TaskTemplate": {
"ContainerSpec": {
"Image":
"ashok25/flaskapp:v1.0@sha256:9fae5a965772ba27f52538fd57a150f91e2e506a1a2dfbf404f649232d5c8fb0",
"Init": false,

Docker Container
 Service Scaling on Swarm

Docker Container
 Why Service scaling?

Service scaling archives high availability of service in case

of any failure. when service run multiple replicas of same

container, swarm cluster distributes it over worker nodes.

Docker Container
 Service Scaling
● Scale devops-webapp service to 4

[root@master ~]# docker service scale devops-webapp=4

devops-webapp scaled to 4
overall progress: 4 out of 4 tasks
1/4: running [==================================================>]
2/4: running [==================================================>]
3/4: running [==================================================>]
4/4: running [==================================================>]
verify: Service converged
[root@master ~]#

Containers running as part of service are called “tasks.”

Docker Container
 Service Scaling Distribution
● Verify service is distributed across all 4 node in swarm
[root@master ~]# docker service ps devops-webapp
ID NAME IMAGE NODE
DESIRED STATE CURRENT STATE ERROR PORTS
0n6n05m6g3bj devops-webapp.1 ashok25/flaskapp:v1.0 worker-3
Running Running 10 minutes ago
1sm5tn9lyn3b devops-webapp.2 ashok25/flaskapp:v1.0 master
Running Running about a minute ago
i2k5ln391i9k devops-webapp.3 ashok25/flaskapp:v1.0 worker-1
Running Running about a minute ago
kt9itys6znvs devops-webapp.4 ashok25/flaskapp:v1.0 worker-2
Running Running 10 seconds ago
[root@master ~]#

Docker Container
 Scaling Service
● Scale devops-webapp service down to 2 replicas
[root@master ~]# docker service scale devops-webapp=2
devops-webapp scaled to 2
overall progress: 2 out of 2 tasks
1/2: running [==================================================>]
2/2: running [==================================================>]
verify: Service converged

● Delete devops-webapp service

[root@master ~]# docker service rm devops-webapp
devops-webapp
[root@master ~]# docker service ls
ID NAME MODE REPLICAS IMAGE
PORTS
[root@master ~]#

Docker Container
 Lab 02: Deploy flaskapp as Highly available service
● Pre-requiste: Working swarm cluster from Lab-01

● Step 1: Connect to master node as root user

● Step 2: Check cluster status

○ docker node ls
[root@manager ~]# docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
mg6cim0tx3wlmgprzz518eq2i worker-1 Ready Active 19.03.8
do2t685v0hqxnj6pflzoupiyx master Ready Active Leader 19.03.8
1telhywrq06ap3mwq00i18c8n worker-2 Ready Active 19.03.8
vz5pd0nu7qohkims55xsb3g25 worker-3 Ready Active 19.03.8
[root@manager ~]#

Docker Container
 Lab 02: Deploy flaskapp as Highly available service
● Step 3:Check that no service is running

[root@master ~]# docker service ls

ID NAME MODE REPLICAS IMAGE PORTS
[root@master ~]#

● Step 4:Launch web service with 4 replicas

[root@master ~]# docker service create --replicas=4 --name webapp -p 80:4080 ashok25/flaskapp:v1.0
rxv6zzeua90pg54rwy6a2v37v
overall progress: 4 out of 4 tasks
1/4: running [==================================================>]
2/4: running [==================================================>]
3/4: running [==================================================>]
4/4: running [==================================================>]
verify: Service converged
[root@master ~]#

Docker Container
 Lab 02: Deploy flaskapp as Highly available service
● Step 5:Access the service page and keep refreshing:

○ http://<Public_IP_of_master_node>:80

● Step 6:Terminate 3 worker node and keep refreshing browser

to look for page not error

● Step 7: Webpage is accessible all the time. Since 1

container still available for service the request .

HA : Web Service was not impacted by 3 server going down

Docker Container
 Rolling update

Docker Container
 Rolling update
● Rolling update : On every node which is running service task, docker container

with old image will be stopped and new container is started with updated

image.

● By default , update is performed in sequential order, node by node.if task is

failed rolling update is paused

Docker Container
 Rolling update
● Perform rolling update of devops-webapp to use image v2.0

[root@master webserver]# docker service update --image=ashok25/flaskapp:v2.0

devops-webapp
devops-webapp
overall progress: 4 out of 4 tasks
1/4: running [==================================================>]
2/4: running [==================================================>]
3/4: running [==================================================>]
4/4: running [==================================================>]
verify: Service converged
[root@master webserver]#

Docker Container
 Rolling update
● Connect to http://<Master_Public_IP>:80

○ update webpage should be shown in browser

Docker Container
 Lab 03: Apply Rolling update for flask app
● Step 1: Run below command to clean up environment
○ docker service rm devops-webapp

○ docker system prune

● Step 2: Launch service with v1.0 of docker image

[root@master ~]# docker service create --replicas=4 --name devops-webapp -p 80:4080

ashok25/flaskapp:v1.0
rxv6zzeua90pg54rwy6a2v37v
overall progress: 4 out of 4 tasks
1/4: running [==================================================>]
2/4: running [==================================================>]
3/4: running [==================================================>]
4/4: running [==================================================>]
verify: Service converged
[root@master ~]#

Docker Container
 Lab 03: Apply Rolling update for flask app
● Step 3: Access service: http://<public_IP_of_master>:80

● Step 4: Apply rolling update for image version 2.0

[root@master ~]# docker service update --image=ashok25/flaskapp:v2.0 devops-webapp

devops-webapp
overall progress: 4 out of 4 tasks
1/4: running [==================================================>]
2/4: running [==================================================>]
3/4: running [==================================================>]
4/4: running [==================================================>]
verify: Service converged
[root@master ~]#

Docker Container
 Lab 03: Apply Rolling update for flask app
● Step 5: Refresh service URL and webpage is updated to v2.0

Docker Container
 Lab-04 Voting App Visualisation

Docker Container
 Lab 04: Deploy Voting app in Docker Swarm
● Step 1: Create 3 node Swarm with 1 manager and 2 worker

○ Use steps from “Docker Swarm automation” Slide 24

ashoksingh@Ashoks-Air ~/Devops-Course/git/aws-swarm-docker master ● ./create-swarm.sh

● Step 2: Connect to leader node and become root

ashoksingh@Ashoks-Air ~/● docker-machine ssh leader1

ubuntu@leader1:~$ sudo su -
root@leader1:~#

Docker Container
 Lab 04: Deploy Voting app in Docker Swarm
● Step 3: Clone example vote repo on leader node

root@leader1:~# git clone https://github.com/singh-ashok/example-voting-app.git

Cloning into 'example-voting-app'...
remote: Enumerating objects: 860, done.
remote: Total 860 (delta 0), reused 0 (delta 0), pack-reused 860
Receiving objects: 100% (860/860), 953.95 KiB | 0 bytes/s, done.
Resolving deltas: 100% (311/311), done.
Checking connectivity... done.
root@leader1:~#

● Step 4: Change Directory to “example-voting-app”

root@leader1:~# cd example-voting-app/

Docker Container
 Lab 04: Deploy Voting app in Docker Swarm
● Step 5: Deploy Example voting app stack

root@leader1:~/example-voting-app# docker stack deploy --compose-file docker-stack.yml vote

Creating network vote_backend
Creating network vote_default
Creating network vote_frontend
Creating service vote_worker
Creating service vote_visualizer
Creating service vote_redis
Creating service vote_db
Creating service vote_vote
Creating service vote_result
root@leader1:~/example-voting-app#

Docker Container
 Lab 04: Deploy Voting app in Docker Swarm
● Step 6: Open service access:

● Voting webpage

○ http://<public_ip_of_swarm>:5000

● Result Webpage

○ http://<public_ip_of_swarm>:5001

● Visualizer

○ http://<public_ip_of_swarm>:8080

Docker Container
 Lab 04: Deploy Voting app in Docker Swarm
● Step 5: Kill random container on different nodes and watch it reschedule on visualiser

.
root@leader1:~/example-voting-app# docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
yewxznvpq404 vote_db replicated 1/1 postgres:9.4
loqo2sdu479o vote_redis replicated 1/1 redis:alpine
lj2ryg8525lv vote_result replicated 2/2 dockersamples/result:latest *:5001->80/tcp
gxx51pihgz0o vote_visualizer replicated 1/1 dockersamples/visualizer:stable
*:8080->8080/tcp
233unxenq6e3 vote_vote replicated 2/2 dockersamples/vote:latest *:5000->80/tcp
d3kjqfb1a7x4 vote_worker replicated 1/1 dockersamples/worker:latest
root@leader1:~/example-voting-app#

● Step 6: Identify Single point of failure .

● Database - Use distributed database cluster

● Redis - user redis cluster - master and slave nodes

● worker - user at least 2 replicas on schedule it on different nodes.

Docker Container
 Managing Swarm

Docker Container
 Drain a Node
● Taking a node offline for maintenance

root@leader1:~/# docker node update --availability drain worker1

worker1
root@leader1:~/# docker node inspect --pretty worker1
ID: jvi42gnyilq5de6v4hn4wfg99
Hostname: worker1
Joined at: 2020-05-02 00:25:15.25411799 +0000 utc
Status:
State: Ready
Availability: Drain
Address: 172.31.31.1
Platform:
…
..

Docker Container
 Make Node Active after draining
● Bringing back node after completion of maintenance

root@leader1:~/# docker node update --availability active worker1

worker1
root@leader1:~/# docker node inspect --pretty worker1
ID: jvi42gnyilq5de6v4hn4wfg99
Hostname: worker1
Joined at: 2020-05-02 00:25:15.25411799 +0000 utc
Status:
State: Ready
Availability: Active
Address: 172.31.31.1
Platform:
…
..

Docker Container
 Add Label to Node
● Label Allow to manage node and create services policies

○ docker node update --label-add <key><value> <node-id>

●
root@leader1:~# docker node update --label-add node_type=t2.micro worker2
worker2
root@leader1:~# docker node update --label-add type=worker worker2 worker1
worker2

● Check label is correctly applied

root@leader1:~# docker inspect worker2 | grep node_type

"node_type": "t2.micro"
root@leader1:~#

Docker Container
 Switch node - Manager to worker
● Promote worker node to manager node
root@leader1:~/example-voting-app# docker node promote worker1
Node worker1 promoted to a manager in the swarm.
root@leader1:~/example-voting-app#

● Demote manager node to worker node

root@leader1:~/example-voting-app# docker node demote worker1

Manager worker1 demoted in the swarm.

Docker Container
 Monitor Swarm
● Check manager node Reachability

root@leader1:~# docker node inspect leader1 --format "{{ .ManagerStatus.Reachability }}"

reachable

● Check worker1 node Status

root@leader1:~# docker node inspect worker1 --format "{{ .Status.State }}"
ready

● From Bastion host stop the worker1 node and check status

root@bastion:~#docker-machine stop worker1

Stopping "worker1"...
Machine "worker1" was stopped.

Docker Container
 Monitor Swarm
● Connect to leader 1 node as root
root@bastion:~#docker-machine ssh leader1
Welcome to Ubuntu 16.04.4 LTS (GNU/Linux 4.4.0-1052-aws x86_64)
…
ubuntu@leader1:~$ sudo su -
root@leader1:~#

● Check worker1 node Status and now node is reported as Down

root@leader1:~# docker node inspect worker1 --format "{{ .Status.State }}"

down

Docker Container
 Thank you

Docker Container