Docker networking Tutorial 101
Download as PPTX, PDF66 likes21,672 views
The document provides an overview of Docker networking options and access control. It discusses the default Linux bridge networking (Docker0), port mapping to access containers externally, using the host's network, and connecting containers via their networks. It also covers more advanced options like Open vSwitch for encapsulation and programmable networking. The document recommends using iptables and the --icc and --link flags for access control between containers and only allowing connected containers to communicate.
![Copyright Reserved
Docker
• Excellent way to track
application dependencies and
configuration in a portable
format.
• For instance, the Dockerfile
on the right can be used to
spawn a container with nginx
LB and accessed at a host port
$ docker build XYZ
$ docker images
$ docker run --name=nginx1
-d –i nginx
4
# Pull base image.
FROM dockerfile/ubuntu
# Install Nginx.
RUN
add-apt-repository -y ppa:nginx/stable &&
apt-get update &&
apt-get install -y nginx &&
rm -rf /var/lib/apt/lists/* &&
echo "ndaemon off;" >> /etc/nginx/nginx.conf &&
chown -R www-data:www-data /var/lib/nginx
# Define mountable directories.
VOLUME ["/etc/nginx/sites-enabled", "/etc/nginx/certs",
"/etc/nginx/conf.d", "/var/log/nginx"]
# Define working directory.
WORKDIR /etc/nginx
# Define default command.
CMD ["nginx"]
# Expose ports.
EXPOSE 80
EXPOSE 443](https://image.slidesharecdn.com/dockernetworking101-141116172405-conversion-gate02/85/Docker-networking-Tutorial-101-4-320.jpg)
![Copyright Reserved
"NetworkSettings": {
"Bridge": "docker0",
"Gateway": "172.17.42.1",
"IPAddress": "172.17.0.19",
"IPPrefixLen": 16,
"MacAddress": "02:42:ac:11:00:0f",
"PortMapping": null,
"Ports": {
"443/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "49157"
}
],
"80/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "49158"
}
]
}
Networking Still in Early Stages
• Today Docker usage is
predominantly within
a single laptop or host.
• The default network
assigned is a port on
Linux bridge docker0
• The network on right
is allocated to the
nginx container we
spawned.
8](https://image.slidesharecdn.com/dockernetworking101-141116172405-conversion-gate02/85/Docker-networking-Tutorial-101-8-320.jpg)
![Copyright Reserved
For the new nginx2 container, we
show network settings below
# docker inspect nginx2
"NetworkSettings": {
"Bridge": "docker0",
"Gateway": "172.17.42.1",
"IPAddress": "172.17.0.19",
"IPPrefixLen": 16,
"MacAddress": "02:42:ac:11:00:0f",
"PortMapping": null,
"Ports": {
"443/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "49157"
}],
"80/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "49158"
}]
}}
Option 2: Port mapping
Advanced:
‒ It is possible to restrict the port
mapping to listen on specific host IP
address and/or a specific host port
number
‒ Use -p option as follows
# docker run
–p host_IP:host_port:container_port
–d –i –t nginx
13](https://image.slidesharecdn.com/dockernetworking101-141116172405-conversion-gate02/85/Docker-networking-Tutorial-101-13-320.jpg)
Docker networking Tutorial 101
- 1. Docker Networking Tutorial – Basic Options and Access Control Srini Seetharaman [email protected] November, 2014
- 2. Key Takeaways 1. Docker networking is in early stage and diverse 2. Applications must choose what networking is right for their needs. It is possible to use same principles as VMs 3. Open vSwitch brings powerful networking capabilities 4. LorisPack is an easy way to add pod-level isolation for Docker containers 5. User space vs Kernel space packet processing is an important design choice
- 3. Copyright Reserved • Over the past few years, LXC came up as an alternative to VM for running workload on hosts • Each container is a clone of the host OS • Docker brought Linux containers to prominence ‒ Tracks application configuration and possibly archives to DockerHub Linux Containers 3 Container 1 App X Container 2 Container 3 Host OS Guest root App Y Guest root App Z Guest root
- 4. Copyright Reserved Docker • Excellent way to track application dependencies and configuration in a portable format. • For instance, the Dockerfile on the right can be used to spawn a container with nginx LB and accessed at a host port $ docker build XYZ $ docker images $ docker run --name=nginx1 -d –i nginx 4 # Pull base image. FROM dockerfile/ubuntu # Install Nginx. RUN add-apt-repository -y ppa:nginx/stable && apt-get update && apt-get install -y nginx && rm -rf /var/lib/apt/lists/* && echo "ndaemon off;" >> /etc/nginx/nginx.conf && chown -R www-data:www-data /var/lib/nginx # Define mountable directories. VOLUME ["/etc/nginx/sites-enabled", "/etc/nginx/certs", "/etc/nginx/conf.d", "/var/log/nginx"] # Define working directory. WORKDIR /etc/nginx # Define default command. CMD ["nginx"] # Expose ports. EXPOSE 80 EXPOSE 443
- 5. Copyright Reserved Setup: 3 containers on 1 host Spawn 1 nginx and 2 bash containers in detached mode $ docker run –i –d --name=nginx1 nginx $ docker run -i -d --name=c1 base /bin/bash $ docker run -i -d --name=c2 base /bin/bash List running containers Linux bridge ports $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 4846cacc4a1e nginx:latest "nginx -g" 4 days ago Up 4 days 443/tcp, 80/tcp nginx1 0ad1a5eeff69 base:latest "/bin/bash" 4 days ago Up 4 days c1 a51357948bfd base:latest "/bin/bash" 4 days ago Up 4 days c2 5
- 6. Copyright Reserved High-level Concepts Namespace Containerized networking at the process level managed at /proc Linux Bridge L2/MAC learning switch built into the Kernel to use for forwarding OpenvSwitch Advanced bridge that is programmable and supports tunneling NAT Network address translators are intermediate entities that translate IP address + Ports (Types: SNAT, DNAT) iptables Policy engine in kernel that is used for managing packet forwarding, firewall, NAT features Unix domain sockets File descriptor based communication that is restricted to a single host. Works like a FIFO pipe. User-space vs Kernel-space Application domain that regulates access to resources and performance possible. • Container applications run in user-space • Typically network forwarding runs in kernel space
- 7. Options for Docker Networking 7
- 8. Copyright Reserved "NetworkSettings": { "Bridge": "docker0", "Gateway": "172.17.42.1", "IPAddress": "172.17.0.19", "IPPrefixLen": 16, "MacAddress": "02:42:ac:11:00:0f", "PortMapping": null, "Ports": { "443/tcp": [ { "HostIp": "0.0.0.0", "HostPort": "49157" } ], "80/tcp": [ { "HostIp": "0.0.0.0", "HostPort": "49158" } ] } Networking Still in Early Stages • Today Docker usage is predominantly within a single laptop or host. • The default network assigned is a port on Linux bridge docker0 • The network on right is allocated to the nginx container we spawned. 8
- 9. Copyright Reserved Many ways to network in Docker • Many of these are similar to what we can do with VM (except the Unix-domain socket method of direct access) 9 Host Container C Container D Container E Container FContainer A Container B Direct Host network Unix-domain sockets and other IPC Docker0 Linux bridge Docker proxy (using iptables) Open vSwitch Port mapping
- 10. Copyright Reserved Option 1: Docker0 bridge • Default network automatically created when no additional options “- -net“ or “-P” are specified • Each container is addressed by a static IP address assigned by Docker • Similar to what we have as default with KVM or VirtualBox • Host can reach container with IP on the bridge • But, outside traffic cannot reach the container 10 Host Nginx1 172.17.0.18 C1 172.17.0.19 C2 172.17.0.20 172.17.42.1 Docker0 bridge eth0 eth0 eth0 veth002aa7a veth6df8377 veth7b0e4c6 eth0192.168.50.16
- 11. Copyright Reserved Option 1: Docker0 bridge Check Linux bridge ports and NAT rules under the hood # iptables –L –t nat -n ... Chain POSTROUTING (policy ACCEPT) target prot opt source destination MASQUERADE all -- 172.17.0.0/16 anywhere # sudo brctl show bridge name bridge id STP enabled interfaces docker0 8000.56847afe9799 no veth002aa7a veth6df8377 veth7b0e4c6 # docker inspect --format='{{.NetworkSettings}}' nginx1 (See for yourself) 11
- 12. Copyright Reserved Option 2: Port mapping • Provide access to the container from outside by allocating a DNAT port in the range 49153-65535 • Still uses Linux bridge docker0, but adds iptables rules for the DNAT • In our example, nginx2 container is reachable by accessing 192.168.50.16:49155 # docker run -P -d -i --name=nginx2 -t nginx # iptables –L –t nat -n ... Chain DOCKER (2 references) target prot opt source destination DNAT tcp -- anywhere anywhere tcp dpt:49155 to:172.17.0.19:80 DNAT tcp -- anywhere anywhere tcp dpt:49156 to:172.17.0.19:443 ... 12 Host nginx2 c1 172.17.42.1 Docker0 bridge eth0 eth0 veth79ed06d veth6df8377 eth0192.168.50.16
- 13. Copyright Reserved For the new nginx2 container, we show network settings below # docker inspect nginx2 "NetworkSettings": { "Bridge": "docker0", "Gateway": "172.17.42.1", "IPAddress": "172.17.0.19", "IPPrefixLen": 16, "MacAddress": "02:42:ac:11:00:0f", "PortMapping": null, "Ports": { "443/tcp": [ { "HostIp": "0.0.0.0", "HostPort": "49157" }], "80/tcp": [ { "HostIp": "0.0.0.0", "HostPort": "49158" }] }} Option 2: Port mapping Advanced: ‒ It is possible to restrict the port mapping to listen on specific host IP address and/or a specific host port number ‒ Use -p option as follows # docker run –p host_IP:host_port:container_port –d –i –t nginx 13
- 14. Copyright Reserved Option 3: Host Option 4: Container 14 Give full access of the host network to the container using --net=host option # docker run --net=host --name=c3 -i –d –t base /bin/bash Check network within container using ifconfig command through exec # docker exec c3 ifconfig eth0 eth0 Link encap:Ethernet HWaddr 52:54:00:0d:3c:9f inet addr:192.168.50.16 Bcast:192.168.50.255 Host can talk to container using lo (localhost) interface Containers can listen on privileged ports (i.e., port numbers < 1024) of host Give full access to network of a container XX to the new container YY using --net=container:XX option # docker run --net=container:nginx1 --name=c4 -i –d –t base /bin/bash Check network within container using ifconfig command through exec # docker exec c4 ifconfig eth0 eth0 Link encap:Ethernet HWaddr 02:42:ac:11:00:12 inet addr:172.17.0.18 Bcast:0.0.0.0 Container XX can talk to container YY using lo (localhost) interface
- 15. Copyright Reserved Option 5: Open vSwitch (OVS) • Similar to Linux bridge, but different technology ‒ Today, this is not the default with Docker ‒ Allows programming with OVSDB and OpenFlow protocols • Why? OpenvSwitch has many useful features! ‒ VxLAN, GRE, VLAN based encapsulation and L2 forwarding ‒ Encapsulation allows containers to pick any MAC/IP they want ‒ Also possible to do L3 routing, ARP proxy etc, load-balancing ‒ Access control, traffic rate limiting and prioritization ‒ 10G/s or more packet processing throughput possible ‒ 1) kernel, or 2) userspace, with optionally DPDK acceleration 15
- 16. Copyright Reserved Option 5: Open vSwitch (OVS) Usage steps 1. Create OVS bridge and assign a gateway interface 2. Use --net=none when spawning container 3. Manually connect containers to the OVS bridge using veth pair 16 # ovs-vsctl add-br br0 # ovs-vsctl add-port br0 ovsbr0p1 -- set interface ovsbr0p1 type=internal # ifconfig ovsbr0p1 192.168.50.1 netmask 255.255.255.0 up # iptables -t nat -A POSTROUTING -s 192.168.50.0/24 -j MASQUERADE # docker run --name=c1 --net=none -d -i -t base /bin/bash Docker working towards allowing plugin mechanism for integrating automation agents
- 17. Copyright Reserved Option 5: Open vSwitch (OVS) • Extract process ID for this container, and create network namespace for it • Create veth device pair and move one to the processes’ network namespace • Rename that device to eth0 and assign IP address + route to it • Add pair interface to the OVS bridge 17 # pid=`docker inspect --format '{{ .State.Pid }}' $1` # ln -s /proc/$pid/ns/net /var/run/netns/c1 # ip netns exec c1 ip link set dev peertapc1 name eth0 # ip netns exec c1 ip link set eth0 up # ip netns exec c1 ip addr add 172.27.0.2/24 dev eth0 # ip netns exec c1 ip route add default via 172.27.0.1 # ip link add tapc1 type veth peer name peertapc1 # ip link set peertapc1 netns c1 # ifconfig tapc1 up # ovs-vsctl add-port br0 tapc1
- 18. Copyright Reserved Option 6: Unix-domain socket or pipe • All inter-process communication mechanisms can be used with containers, especially Unix domain sockets or pipes. For instance, • Apps on c1 and c2 can communicate over following Unix socket address 18 # docker run --name=c1 -v /var/run/foo:/var/run/foo -d -i -t base /bin/bash # docker run --name=c2 -v /var/run/foo:/var/run/foo -d -i -t base /bin/bash bind(socket_fd, (struct sockaddr *) &address, sizeof(struct sockaddr_un)); listen(socket_fd, 5); while((connection_fd = accept(socket_fd, (struct sockaddr *) &address, &address_length)) > -1) nbytes = read(connection_fd, buffer, 256); struct sockaddr_un address; address.sun_family = AF_UNIX; snprintf(address.sun_path, UNIX_PATH_MAX, "/var/run/foo/bar"); connect(socket_fd, (struct sockaddr *) &address, sizeof(struct sockaddr_un)); write(socket_fd, buffer, nbytes); C1 : Server.c C2 : Client.c
- 20. Copyright Reserved Other Networking Parameters These are option assigned to the Docker daemon during startup • Change default Linux bridge using --bridge ‒ IP address needs to be set for the bridge using --bip • Change range of fixed IPs assigned to the container ‒ Typical default is the same range as that assigned to bridge with --bip • Change default MTU of the containers using --mtu option ‒ The actual MTU will be the least of the uplink interface, and this mtu • Set DNS server and domain using --dns, --dns-search options • --ip-forward enables proxy forwarding at the bridge • --iptables enable using iptables for access control 20
- 21. Securing containers using iptables … default DROP policy 21
- 22. Copyright Reserved Access Control using icc and links Tighter access control on communication between containers 1. Start docker daemon with --icc=false option. This inserts a default DROP rule in iptables thereby preventing any spawned containers from communicating with each other. 22 $ sudo iptables –L –n ... Chain FORWARD (policy ACCEPT) target prot opt source destination ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 DROP all -- 0.0.0.0/0 0.0.0.0/0
- 23. Copyright Reserved Access Control using icc and links Tighter access control on communication between containers 2. Connect two containers using the --link option in docker run ‒ Some environment variables (e.g., NGINX1_PORT) are set in target container and entry in the /etc/hosts file for easy access to source 23 nginx1c1 Docker0 bridge c2 X $ docker run --name=nginx1 -P -d -i -t nginx $ docker run --name=c1 -d -i -t --link nginx1:nginx1 base /bin/bash $ docker run --name=c2 -d -i -t base /bin/bash $ docker exec c1 cat /etc/hosts 127.0.0.1 localhost ::1 localhost ip6-localhost ip6-loopback 172.17.0.10 nginx1 $ docker exec c1 printenv (See for yourself)
- 24. Copyright Reserved Access Control using icc and links • The --link option allows access to the exposed ports (80, 443) of a source container (nginx1) from a target container (c1) ‒ Implemented using ACCEPT rules in iptables 24 $ sudo iptables –L -n ... Chain FORWARD (policy ACCEPT) target prot opt source destination ACCEPT tcp -- 172.17.0.8 172.17.0.9 tcp spt:80 ACCEPT tcp -- 172.17.0.9 172.17.0.8 tcp dpt:80 ACCEPT tcp -- 172.17.0.8 172.17.0.9 tcp spt:443 ACCEPT tcp -- 172.17.0.9 172.17.0.8 tcp dpt:443 ACCEPT tcp -- 0.0.0.0/0 172.17.0.3 tcp dpt:443 ACCEPT tcp -- 0.0.0.0/0 172.17.0.3 tcp dpt:80 ... Insert by link option Exposed ports in Dockerfile
- 25. Thank you. https://github.com/sdnhub/lorispack © 2015 Copyright Reserved
Editor's Notes
- #2: Netw