Containers, Dockers,

and Kubernetes Overview

1. What is a Container and Why?

2. How Docker helps using containers
3. Docker Commands
Raj Jain 4. Orchestration: Swarms and Kubernetes
Washington University in Saint Louis
Saint Louis, MO 63130 5. Docker Networking and Security
[email protected]
These slides and audio/video recordings of this class lecture are at:
http://www.cse.wustl.edu/~jain/cse570-18/ Key Reference: N. Poulton, "Docker Deep Dive," Oct 2017, ISBN: 9781521822807 (Not a Safari Book)

Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain

21-1 21-2

Advantages of Virtualization Problems of Virtualization

‰ Minimize hardware costs (CapEx)
VM VM VM
Multiple virtual servers on one physical hardware
‰ Easily move VMs to other data centers App App App
¾ Provide disaster recovery. Hardware maintenance.
OS OS OS
¾ Follow the sun (active users) or follow the moon (cheap power)
‰ Consolidate idle workloads. Usage is bursty and asynchronous. Hypervisor

Increase device utilization Physical Hardware

‰ Conserve power
Free up unused physical resources ‰ Each VM requires an operating system (OS)
‰ Easier automation (Lower OpEx) ¾ Each OS requires a license Ÿ CapEx
Simplified provisioning/administration of hardware and software ¾ Each OS has its own compute and storage overhead
‰ Scalability and Flexibility: Multiple operating systems ¾ Needs maintenance, updates Ÿ OpEx

¾ VM Tax = added CapEx + OpEx

Ref: http://en.wikipedia.org/wiki/Platform_virtualization
Ref: K. Hess, A. Newman, "Practical Virtualization Solutions: Virtualization from the Trenches," Prentice Hall, 2009,
ISBN:0137142978
Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain

21-3 21-4
 Solution: Containers Containers
VM
‰ Run many apps in the same virtual machine
App 1 App 2 App 3 App 4 App 5 App 6
¾ These apps share the OS and its overhead
Container
¾ But these apps can’t interfere with each other
Shim Shim
¾ Can’t access each other’s resources Operating System Operating System
without explicit permission
¾ Like apartments in a complex Hypervisor
Ÿ Containers
‰ Multiple containers run on one operating system on a
virtual/physical machine
‰ All containers share the operating system Ÿ CapEx and OpEx
‰ Containers are isolated Ÿ cannot interfere with each other
¾ Own file system/data, own networking Ÿ Portable
Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain

21-5 21-6

Containers (Cont) VM vs. Containers

‰ Containers have all the good properties of VMs Criteria VM Containers
¾ Come complete with all files and data that you need to run
Image Size 3X X
¾ Multiple copies can be run on the same machine or different
machine Ÿ Scalable Boot Time >10s ~1s
¾ Same image can run on a personal machine, in a data center Computer Overhead >10% <5%
or in a cloud Disk I/O Overhead >50% Negligible
¾ Operating system resources can be restricted or unrestricted
Isolation Good Fair
as designed at container build time
¾ Isolation: For example, “Show Process” (ps on Linux) Security Low-Medium Medium-High
command in a container will show only the processes in the OS Flexibility Excellent Poor
container
Management Excellent Evolving
¾ Can be stopped. Saved and moved to another machine or for
later run Impact on Legacy application Low-Medium High
Ref: M. K. Weldon "The Future X Network: A Bell Labs Perspective," CRC Press, 2016, 476 pp., ISBN:9781498779142
Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain

21-7 21-8
 Docker Docker
‰ Provides the isolation among containers ‰ Docker Engine: Runtime
‰ Helps them share the OS ‰ Two Editions:
‰ Docker = Dock worker Ÿ Manage containers ¾ Community Edition (CE): Free for experimentation

‰ Developed initially by Docker.com ¾ Enterprise Edition (EE): For deployment with paid support

‰ Downloadable for Linux, Windows, and Mac from ‰ Written in “Go” programming language from Google
Docker.com ‰ Now open source project under mobyproject.org
‰ Customizable with replacement modules from others https://github.com/moby/moby
‰ Download the community edition and explore
App 1 App 2 App 3

Docker
Operating System
Ref: https://golang.org/

Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain

21-9 21-10

Docker Engine Components Image Registries

‰ daemon: API and other features ‰ Containers are built from images and can be saves as images
‰ containderd: Execution logic. Responsible for container ‰ Images are stored in registries
lifecycle. Start, stop, pause, unpause, delete containers.
‰ runc: A lightweight runtime CLI ¾ Local registry on the same host

‰ shim: runc exists after creating the container. ¾ Docker Hub Registry: Globally shared
shim keeps the container running. Keep stdin/stdout open. ¾ Private registry on Docker.com
Docker Client daemon Docker Engine Receives instructions ‰ Any component not found in the local registry is downloaded
>_
containerd Gives image to runc from specified location
‰ Official Docker Registry: Images vetted by Docker
shim shim shim Enables daemon-less
containers ‰ Unofficial Registry: Images not vetted (Use with care)
runc runc runc
‰ Each image has several tags, e.g., v2, latest, …
Container Container Container ‰ Each image is identified by its 256-bit hash
Ref: N. Poulton, "Docker Deep Dive," Oct 2017, ISBN: 9781521822807 (Not a Safari Book)
Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain

21-11 21-12
 Layers Building Container Images
‰ Each image has many layers ‰ Create a Dockerfile that describes the application, its
‰ Image is built layer by layer dependencies, and how to run it
‰ Layers in an image can be inspected by Docker commands FROM Alpine Start with Alpine Linux
‰ Each layer has its own 256-bit hash LABEL maintainer=“[email protected]” Who wrote this container
‰ For example: RUN apk add –update nodejs nodejs –npm Use apk package to install nodejs
¾ Ubuntu OS is installed, then
COPY . /src Copy the app files from build context
WORKDIR /src Set working directory
¾ Python package is installed, then
RUN nmp install Install application dependencies
¾ a security patch to the Python is installed EXPOSE 8080 Open TCP Port 8080
‰ Layers can be shared among many containers ENTRYPOINT [“node”, “./app.js”] Main application to run
RUN nmp install Layer 4
Image
Layer 3 Copy . /src Layer 3
Patch Layer 2 RUN apk add … Layer 2
Python Layer 1
Ubuntu FROM Alpine Layer 1

Note: WORKDIR, EXPOSE, ENTRYPOINT result in tags. Others in Layers.

Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain

21-13 21-14

Docker Commands Open Container Initiative (OCI)

‰ docker container run: Run the specified image ‰ A company called CoreOS defined alternative image format
‰ docker container ls: list running containers and container runtime API’s
‰ docker container exec: run a new process inside a container ‰ Led to formation of OCI under Linux Foundation to govern
‰ docker container stop: Stop a container container standards
¾ OCI Image spec
‰ docker container start: Start a stopped container
¾ OCI Runtime spec
‰ docker container rm: Delete a container
‰ docker container inspect: Show information about a container ‰ Everyone including Docker is now moving to OCI

Ref: https://www.opencontainers.org/
Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain

21-15 21-16
 Swarm Swarms (Cont)
‰ Orchestrating thousands of containers ‰ The managers select a leader, who really keeps track of the
‰ Swarm: A group of nodes collaborating over a network swarm
‰ Two modes for Docker hosts: ‰ Assigns tasks, re-assigns failed worker’s tasks, …
¾ Single Engine Mode: Not participating in a swarm ‰ Other mangers just monitor passively and re-elect a leader if
¾ Swarm Mode: Participating in a Swarm leader fails
‰ A service may run on a swarm ‰ Services can be scaled up or down as needed
‰ Each swarm has a few managers that dispatch tasks to workers. ‰ Several Docker commands:
Managers are also workers (i.e., execute tasks)
¾ docker service : Manage services

Swarm ¾ docker swarm: Manage swarms

Swarm Node Swarm Node
¾ docker node: Manage nodes
Single-Engine Node

Swarm Node Swarm Node

Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain

21-17 21-18

Docker Swarm Commands Docker Overlay Networking

‰ docker swarm init ‰ Nodes in a swarm may not be in the same LAN
‰ docker swarm join-token ‰ VXLAN is used to provide virtual overlay networking
‰ docker node ls ‰ VXLAN was discussed in another module of this course
‰ docker service create
‰ docker service ls 172.116.56.67 172.118.56.67 192.168.0.1 192.168.0.2
‰ docker service ps Node 1 Node 2 Node 1 Node 2
‰ docker service inspect
‰ docker service scale
‰ docker service update Physical Virtual
‰ docker service rm

Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain

21-19 21-20
 Docker Security Kubernetes
‰ All built-in security mechanisms in Linux are used and more ‰ Open Source Container Orchestration alternative
‰ Cryptographic node IDs ‰ Original source released by Google
‰ Mutual Authentication ‰ Cloud Native Computing Foundation (CNCF) project in Linux
‰ Automatic Certificate Authority configuration Foundation
‰ Automatic Certificate Renewal on expiration ‰ Pre-cursor to Swarms
‰ Encrypted Cluster Store ‰ Facilities similar to Swarms
‰ Encrypted Network traffic
‰ A set of related containers is called a “Pod”
‰ Signed images in Docker Content Trust (DCT) A Pod runs on a single host.
‰ Docker Security Scanning detects vulnerabilities
‰ Swarm is called a “Cluster”
‰ Docker secrets are stored in encrypted cluster store, encrypted
transmission over network, and stored in in-memory file
system when in use

Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain

21-21 21-22

Hyper-V Containers Intel Clear Containers

‰ Microsoft allows two kinds of containers: ‰ Started 2015 to address security concerns (Dirty COW) in
¾ Windows Server Containers: Multiple containers on a single containers
VM (like Docker containers) ‰ Idea: Allow lightweight VMs using Intel Virtualization
¾ Hyper-V containers: Each container runs on its own VM Technology
Ÿ No need for a Linux ¾ Own lightweight OS and a dedicated kernel
Ÿ Isolation of network, memory, and I/O
Container
Container
Container

¾ Help by hardware enforced isolation

VM
VM
VM

¾ No need for full VMs for containers

HyperV HyperV ‰ Merged with HyperV to form Kata containers on Dec 5, 2017
Hardware Hardware

Ref: https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/hyperv-container
Ref: https://clearlinux.org/containers
Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain

21-23 21-24
 Kata Containers Summary
‰ Lightweight virtual machines
‰ Dedicated VMs to run one and only one container
‰ Combines “Intel Clear Containers” and “HyperV runV”
‰ Open source project under OpenStack Foundation
‰ Compatible with the OCI specs for Docker containers
‰ Compatible with CRI for Kubernetes
‰ Performance like containers, isolation and security like VMs ‰ Virtual Machines provide scalability, mobility, and cost
‰ Six Components: Agent, Runtime, Proxy, Shim, Kernel and QEMU 2.9
reduction but need OS which increase resource requirements
‰ Kubernetes will be extended to provision VMs (Kata Containers)
‰ OpenStack’s VM orchestration engine (Nova) will be extended to handle ‰ Containers provide isolation on a single OS and are lightweight
containers ‰ Docker allows managing containers
‰ Package once and run anywhere
‰ Docker Swarm and Kubernetes allow orchestrating a large
¾ VMware, Google, and Amazon are all moving towards this approach

‰ No installable distribution of Kata containers yet (April 22, 2018) number of containers
Ref: https://katacontainers.io/
https://www.forbes.com/sites/janakirammsv/2017/12/11/why-kata-containers-is-good-for-the-industry-and-customers/2/#3d8cc2e9404f
‰ Docker provides overlay networking and security
Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain

21-25 21-26

Acronyms References
‰ API Application Programming Interface ‰ N. Poulton, "Docker Deep Dive," Oct 2017, ISBN: 9781521822807 (Not a
‰ CapEx Capital Expenditure Safari Book) Highly Recommended.
‰ CE Community Edition ‰ Parminder Singh Kocher, "Microservices and Containers, First edition,"
‰ CLI Command Line Interface Addison-Wesley Professional, April 2018, 304 pp., ISBN:978-0-13-459838-
‰ CNCF Native Computing Foundation 3 (Safari Book).
‰ DCT Docker Content Trust ‰ Russ McKendrick; Pethuru Raj; Jeeva S. Chelladhurai; Vinod Singh,
‰ EE Enterprise Edition "Docker Bootcamp," Packt Publishing, April 2017, 196 pp., ISBN:978-1-
‰ ID Identifier 78728-698-6 (Safari Book).
‰ ISBN International Standard Book Number ‰ Russ McKendrick; Scott Gallagher, "Mastering Docker - Second Edition,"
‰ LAN Local Area Network Packt Publishing, July 2017, 392 pp., ISBN:978-1-78728-024-3 (Safari
‰ OpEx Operational Expenses Book).
‰ OS Operating System ‰ Jeeva S. Chelladhurai; Vinod Singh; Pethuru Raj, "Learning Docker -
‰ TCP Transmission Control Protocol Second Edition," Packt Publishing, May 2017, 300 pp., ISBN:978-1-78646-
‰ VM Virtual Machine 292-3 (Safari Book).

Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain

21-27 21-28
 Wikipedia Links Scan This to Download These Slides
‰ https://en.wikipedia.org/wiki/Docker_(software)
‰ https://en.wikipedia.org/wiki/Operating-system-
level_virtualization
‰ https://en.wikipedia.org/wiki/Kubernetes
‰ https://en.wikipedia.org/wiki/Microservices
‰ https://en.wikipedia.org/wiki/DevOps
‰ https://en.wikipedia.org/wiki/OpenShift Raj Jain
http://rajjain.com
‰ https://en.wikipedia.org/wiki/LXC

Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain

21-29 21-30

Related Modules
CSE567M: Computer Systems Analysis (Spring 2013),
https://www.youtube.com/playlist?list=PLjGG94etKypJEKjNAa1n_1X0bWWNyZcof

CSE473S: Introduction to Computer Networks (Fall 2011),

https://www.youtube.com/playlist?list=PLjGG94etKypJWOSPMh8Azcgy5e_10TiDw

Wireless and Mobile Networking (Spring 2016),

https://www.youtube.com/playlist?list=PLjGG94etKypKeb0nzyN9tSs_HCd5c4wXF

CSE571S: Network Security (Fall 2011),

https://www.youtube.com/playlist?list=PLjGG94etKypKvzfVtutHcPFJXumyyg93u

Video Podcasts of Prof. Raj Jain's Lectures,

https://www.youtube.com/channel/UCN4-5wzNP9-ruOzQMs-8NUw
Washington University in St. Louis http://www.cse.wustl.edu/~jain/cse570-18/ ©2018 Raj Jain

21-31