One of the most hyped up features of Docker 1.13 is the new Managed Plugin System. This was part of the experimental branch in 1.12 but has now been merged as a fully fledged feature.

Why are Managing Plugins Important?

The end goal is to take advantage of a plugin’s extensibility quickly and effortlessly. Docker’s mission is to standardize this process by ensuring all plugins are managed and run as containers and made available from Docker Hub as the central source.

To understand why this is necessary, let’s rewind to take a look at how plugins worked before the Managed Plugin System. Plugins operated as standalone services where each was managed as its own entity and could operate at a host level. These plugins did not have to be in a container. A plugin, whether network or storage related, is activated on-demand against the Docker Engine and requires a .sock, .spec, or .json file in /var/lib/docker/plugins. The Docker engine would then call a plugin when a function was being requested, regardless of its running location. Docker documentation goes as far as calling this the “Legacy” implementation.

A flaw to having a very simple plugin API is the multitude of plugin options on the market using their own discretion for installation. This leads to a detrimental user experience and inconsistency across the board. Most plugins require manually moving binaries, creating hieroglyphic-like docker containers, or even building a go package from source. REX-Ray continuously innovates to maintain a simple three step process of:

  1. curl | sh installation
  2. add a configuration file or environment variables
  3. rexray start to begin the service and start the sock file.

Standardization makes progression easier. It puts up guard rails and minimizes the amount of variability. Furthermore, how do we accomplish placing storage orchestrators in a container?

Deep Dive Into Container Mounts

REX-Ray ran from a container needs to mount volumes and have those mount points be available to the underlying operating system (OS). To make mounts performed in the container available to the host where Docker is running, Linux’s shared bind mounts are utilized. You may have seen this in complex docker run commands such as:
$ docker run -d -v /run:/run:shared ubuntu
The crux of this however, is it only works if the /run path was already a shared mount. This can be inconsistent among operating systems. To mitigate this, Docker implemented a feature called a Propagated Mount. From the documentation: path to be mounted as rshared, so that mounts under that path are visible to docker. A propagated mount is a technique needed so the Docker daemon can see the mounts from within the container.

REX-Ray relies on the libStorage package for mounting so a mounted volume is typically done at /var/lib/libstorage/volumes/<name>. Since REX-Ray is containerized, the mount is actually available from a host level at <containerpath>/rootfs/var/lib/libstorage/volumes. Propagated mounts ensure any volume mounted under this path can be made available to the Docker daemon.

For more details you can use the findmnt utility to view the propagation of mounts before and after plugins initialize with findmnt -R -o SOURCE,TARGET,PROPAGATION /. If it is working correctly you will see a path like /var/lib/docker/plugins/<containerId>/rootfs/var/lib/libstorage/volumes marked as shared (see image below).

REX-Ray with Docker 1.13 Managed Plugin System

We believe containerizing REX-Ray and managing it as a plugin is a great option to simplify how storage extensibility is managed for Docker. The engineering team is busy working on the next version of REX-Ray (0.8) which will include updates relating to this new operational model. We are also happy to see that Docker used REX-Ray as a proof-of-concept for providing end-to-end steps for creating an EBS volume plugin in their documentation. See this if you’re interested in building a custom plugin.

Today we have a tech-preview of REX-Ray using the Managed Plugin System. This process is different from Docker’s example since we are just showing how to run the REX-Ray EBS plugin from Docker Hub. The video below demonstrates the usage of standalone REX-Ray with Docker 1.13 Managed Plugins. If you would like to try for yourself, go to the Experimental Docker Managed Plugin at {code} Labs. Reminder: this is an experimental feature and should not be used in production.

Give it a spin and let us know what you think!