Notes for the textbook - Docker in action

Chapter 1 - Welcome to docker

Docker simplifies your experience installing, running, publishing and removing

software.

Containers - a unix technology to isolate a process from all resources except where
explicitly allowed.
A program running in an os can see all resources, but one running in a container
can only see the resources allocated to the container. This is implemented using
linux namespaces.
They are challenging to build manually. Docker runs all software inside a
container, using existing container engines to provide consistent containers built
with best practices.

Virtualization - provides virtual hardware and require a whole os to be installed

before installing your software. Docker is not virtualization. Programs running in
docker containers interface directly with the host’s linux kernel, which is more
efficient.

Users interact with docker using the docker cli. All containers run as child
processes of the docker daemon, wrapped with a container.
Containers built by docker are isolated in 8 aspects:
* PID
* UTS (host and domain name)
* MNT
* IPC
* NET
* USR
* chroot() - controls fs root location
* cgroups - resource protection (cpu, memory, disk io etc)

A docker image is a bundled snapshot of all the files that should be available to a
program running inside a container.
A container is created from an image.
Registries and indexes are infrastructure components that simplify distributing
docker images.

The more software you use, the more difficult it is to manage, due to things like
shared application dependencies. Docker helps to focus on using the software,
instead of spending time installing/upgrading/publishing it.

What are containers made from?

cgroups - control groups, allow limiting resources like cpu, memory etc. each
resource has a tree structure of processes that use it, a process will have a node
in each tree for the different resources it needs.

memory cgroup
-hard limits - if container exceeds hard limit, it will be killed. so only put one
service in a container, so in case of oom only one will be killed.
-soft limits - in case system is running out of memory, it will take memory pages
away from processes that are above their soft limits.

cpu cgroup
-track user/system cpu time & usage per cpu
-can’t set cpu limits
cpuset cgroup
-lets you pin groups of processes to specific cpu - so you can dedicate cpus to
specific tasks. this prevents processes from bouncing between cpus.

blkio cgroup
keeps track of io for each group; reads and writes, sync vs async operations; per
block device

net_cls and net_prio cgroup

-automatically set traffic class or priority for traffic generated by processes in
the group.

devices cgroup
-controls what the group can do on device nodes like network
interfaces(/dev/net/tun), filesystems in user space (/dev/fuse), etc

freezer cgroup
-lets you freeze a group of processes

New processes start in their parent’s cgroups. groups are created by mkdir in the
pseudo-fs:
mkdir /sys/fs/cgroup/memory/somegroup/subcgroup
to move a process to a cgroup:
echo $PID > /sys/fs/cgroup/../tasks

cgroups limits how much a process can use

namespaces limits what a process can view

-provides processes with their own system view.
-multiple namespaces are available: pid, net, mnt, uts, ips, user
-each process is in one namespace of each type

pid namespace
-processes in a pid namespace can only see processes in the same pid namespace
-a process has multiple pids, one per namespace in which it’s nested
so in a container you can only see those processes, but from system you can see
processes inside the containers on that system
-inside the container, that process could be pid=1, but outside the container
the pid will be different.

network namespace
lets each container have its own network sources.
processes in a namespace get their own private network stack, including network
interface, routing tables, sockets etc.
you can move network interfaces across containers.

mnt namespace
lets a container mount something that isn’t visible in other containers.
processes can have their own root fs
mounts can be private or shared

uts namespace
lets a container have it’s own hostname

ipc namespace
allows a process (or group of processes) to have own
-ipc semaphores
-ipc message queues
-ipc shared memory

user namespace
lets you map uids. in a container you can be uid 0, but outside you are uid 1234.
so it maps container uids to host uids.
user namespace is about usable security. lets you be root inside the container but
not outside the container.

namespace manipulation
-can create a ns by passing flags when creating a new process
-so to manually create a container, start bash with such flags to keep it in
separate namespaces.
-ns are materialized by pseudo files, /prod/<pid>/ns
-when last process in ns exits, ns is destroyed (can be persisted if needed)

copy-on-write storage
-cow is a standard unix pattern that provides a single shared copy of data until
the data is modified. this data is in the docker image. if a write is performed in
the running container, a copy of the file to be modified is first placed in the
writeable layer of the container, and then the write operation takes place on that
copy.
-lets you create a new container instantly (startup time can be less than 0.1 s),
since it doesn’t have to create a copy of the full filesystem.
-lets containers have a small footprint (can occupy less than 1mb on disk)
unlike vms, which take minutes to startup and occupy gbs of disk space.
cow is implemented using the union file system(ufs) (other options may be
available) which allows layered file systems. a docker image is composed of
multiple read only layers. when you create a container, a writable layer is created
on top of the layers. when a file is needed, the layers are searched top down, and
the first file to be found is returned.

orthogonality:
all above features can be used independently
e.g put a debugger in a containers namespace but not its cgroups, so it doesn’t
steal resources.

above features can be used by different container runtimes, not just docker.
they are hard and error prone to use to get a container running, docker and other
container runtimes simplify the process and take care of using them under the hood.

A docker image is a bundled snapshot of all the files that should be available to a
program running inside the container. images are the shippable units in docker.
registries and indexes simplify distributing docker images.

Docker runs natively on linux, and uses a single virtual machine on windows and os
x in which to run containers - vm has a constant overhead while the number of
containers can scale up. This improves portability.

Containers limit the scope of impact a program has on other programs, the data it
can access and system resources. The scope of any security threat is therefore
limited to this scope of impact.

docker is important because

* it makes containers available to everyone. lets you focus on the software you are
installing, and when removed the software leaves no lingering artifacts
* strong push in the community to adopt docker
* makes software cross platform
* brings about better adoption of advanced isolation features of operating systems
that were already available.

Docker can only run applications that can run on a linux system. Helps protect your
system from attack by running software in containers and as a user with reduced
privileges. This won’t work for software that needs elevated privileges.
It’s a bad idea to blindly run third party containers in a collocated environment.

docker runs as the root user of the system.

"docker run” command checks if the image is present on the computer. if not, it
downloads the image from the docker registry. then it starts the container from the
image.
the state of the container is directly tied to the state of the running program on
the container. when the program is stopped, the container is stopped.
each time docker run is called, it creates a new container from the same
repository.

Chapter 2 - Running software in containers

Docker hub is the public registry provided by docker.

docker run - -detach - -name web nginx:latest

starts the container in detached mode. -d is the short form of the detach argument.
calling docker run starts the container, and the command returns a blob of
characters. this is the unique identifier for the container.

docker run - - interactive - - tty - - name web_test busybox:latest /bin/sh

the - - interactive argument (or -i for short) tells docker to keep the stdin
input stream for the container open even if there’s no terminal attached
the - - tty argument (-t for short) tells docker to allocate a virtual terminal
to the container
once started, the sh command will run in the container.
docker run -it is a short form to start an interactive terminal. after working in
an interactive terminal, you can detach from it without stopping the container by
holding ctrl, pressing P and then Q.

To list running containers, use the “docker ps” command. This returns the following
details for each running container:
* container id
* the image used
* the command executed inside the container
* the time since the container was created
* the duration that the container has been running
* the network prots exposed by the container
* the name of the container

to restart a container, type “docker restart container_name”.

to see container logs, type “docker logs container_name”. this records anything the
program writes to stdout or stderr. This is never rotated or truncated, which can
be a problem for long lived processes. its preferable to use volumes for log data,
discussed later.

to stop a container, type “docker stop container_name”. This tells the program with
pid #1 in the container to halt.

Linux has tools to create multiple pid namespacese.

run “docker exec container_name ps” to run a command in a container, in this case
the ps command. This shows that the command passed to the docker run command runs
in the container with a pid of 1, and subsequent commands run e.g by using docker
exec have other pids.
Most docker isolation features can be optionally disabled, e.g to disable pid
isolation, run
docker run —pid host busybox:latest ps
this would list all processes running on the computer.
using isolation tools like namespaces, docker avoids software conflicts like
* binding to the same network port,
* trying to access the same file(but encountering file locks),
* needing two versions of the same globally installed library,
* modifying an environment variable that another program uses, etc.

metaconflicts - conflicts between containers in the docker layer.

"docker run -d - -name newcont imgname” starts a container with the name newcont.
One example of a conflict is if you run this command again - docker doesn’t allow
repeated container names.
if - -name is omitted, docker creates a name by default
(personalAdjective_famousLastName e.g distracted_turing). the flag overrides this
provided name.
“docker rename oldname newname” can be used to change the name.
In addition to the name, docker assigns a unique identifier, a 1024-bit hex encoded
number. this identifier can be used in place of the name in commands, like “docker
exec hexIdentifier ps”. Even the first 12 characters of this identifier are unique,
so they can be used in place of the whole value. e.g “docker exec 7cb36f7vj7ea ps”
wais to get this container id:
* docker create - lets you create a container in the stopped state. CID=$(docker
create nginx:latest)
* container id (cid) file - “docker create - -cidfile /tmp/web.cid nginx” - if this
file exists, the container won’t be created
* one file should be created per container
* docker ps can get the id of a container.
* CID=$(docker ps - -latest - -quiet) or $(docker ps -l -q) returns the
truncated ids.
* - -no-trunc option for docker ps will return the full container id.

docker container states:

* docker ps only shows running containers by default. to see all containers, use
“docker ps -a”.
* "docker run” creates and starts a container, making it enter the running state.
* “docker pause” pauses a container, and “docker unpause” takes it back to running
* “docker stop” or “docker kill” takes it to the exited state, and “docker restart”
or “docker start” takes it to running
* “docker create” creates a container in the exited state, and “docker remove”
removes the container.
if you use - -link option when creating a container (link is deprecated now), it
links to another container. if you start a container that depends on another
container that’s not started yet, it returns an error.
whether using “docker run” or “docker create”, the resulting containers need to be
started in reverse order of their dependency chain. this means that circular
dependencies are impossible to build.