Docker Swarm: An overview

Docker adoption, especially in large-scale companies is on the rise (up 40% apparently, by one estimation).

Since v1.12 onwards, Docker added “Swarm mode” to the mix, bringing some very powerful capabilities to it’s container ecosystem.

So what is Docker Swarm? Is it worth exploring? I’ll try to summarize some of it’s unique and interesting features so you can decide.

Docker Swarm adds native support for 4 key aspects of containerization:

Let me reiterate that this is “native support” – Docker Swarm is built in to Docker, which is pretty significant.

Let’s break this down…

1 - Cluster Management:

Docker Swarm mode allows you to connect multiple machines together ( as Manager & Worker nodes) to form a cluster that can host your services – bringing high-availability and elasticity to docker. How so?

  1. The cluster can suffer the loss of several nodes (depending on the configuration) and still continue to work.
  2. The cluster can detect if a service dies or a node leaves the swarm and ensures that the service definition remains satisfied.
  3. Nodes can be added and removed, promoted and demoted on the fly in response to current load with minimal service disruption.

Under the hood, Docker Swarm uses a raft consensus algorithm to save service definitions and uses a gossip protocol for cluster communication.

2 - Native container orchestration:

Docker Swarm allows you to declare service definitions and let the cluster handle task scheduling.

A service definition may include:

Running services this way is referred to as a Docker stack (similar to Docker Compose) with additional Swarm features baked in.

3 - Service discovery and load balancing

Swarm mode has built-in load balancing which is handled at the kernel level (via iptables and ipvs), and also creates an ingress routing mesh. From the docs:

The routing mesh enables each node in the swarm to accept connections on published ports for any service running in the swarm, even if there’s no task running on the node. The routing mesh routes all incoming requests to published ports on available nodes to an active container.

For more details: Docker Swarm Mode Routing

Additionally, services can be attached to user-defined “overlay networks”, allowing other services on the same network to talk to each other even if not on the same node. Service logs are aggregated and accessible to manager nodes no matter on which node the service actually runs:

docker service logs <service-name> #show aggregated logs for <service-name>

4 - Security:

Docker Swarms allow you to define “secrets” – pieces of data that should never be stored or transmitted unencrypted – and explicitly allow access to specific services.

The secret, once defined, is and stored encrypted in the Raft log and replicated to all Manager nodes. When a service is granted access to a secret, it is un-encrypted and mounted on an in-memory filesystem. If the service dies or is stopped, secret is then unmounted and flushed from memory. This method of handling sensitive data is now the recommended approach over passing in environment variables.

Swarm Mode: Summary

I find the native support for clustering and container orchestration in Docker Swarm quite impressive. Please note that Swarm mode, while stable, does have unresolved issues. Experiment, evaluate and use at your own risk!