Volume Plugins with Docker Toolbox and Boot2Docker

Have you had a chance to get Docker Toolbox or Volume Plugins running yet? Follow our play by play here to get both running.

Docker Toolbox is a simple set of tools that enable you to get a Docker environment running on your computer.  As of Docker 1.9, the Toolbox is the mainstream method to get a complete container client/server development environment up and running while leveraging Boot2Docker as an ultra-lightweight Tiny Linux distro.  Downloading the Toolbox installs and configures Docker Client, Machine, Compose, Kitematic, and VirtualBox.

Within the Toolbox, Docker Machine is the tool that provisions the virtual machine instances locally to VirtualBox.  The provisioned machine mounts an ISO to boot from containing Boot2Docker (B2D).  On its own, B2D is a super light-weight Tiny Linux ISO that boots in seconds, focuses on running Docker daemon and maintains a minimal footprint.  This should allow you to easily do some good testing on multiple B2D instances that form clusters with minimal resources.

What are Volume Plugins?  Docker has a page here describing them.  Basically they allow us to orchestrate external storage with containers to ensure persistent data is stored outside of a container filesystem.  It is also important to point out that we recently did a post that involved VirtualBox and Volume Plugins here.  The difference in this post is that we will be using Docker Machine with B2D instead of Vagrant with CentOS to deploy and run the environment.  The previous post also expands a bit into application discovery and volume pre-emption, well worth the read!

Interested in Docker Toolbox?  Read further and we will show how you can take advantage of Volume Plugins on Boot2Docker and VirtualBox!  Thank you to @tianon and @akihirosuda for updates to Boot2Docker to support this use case.

We break the rest of the post into three sections: Installing, Deploying, and Using.


The first step is to download and install the Docker Toolbox from here.  This should ensure you have a functioning environment ready to run your containers.

A general requirement for Volume Plugin functionality is that the Boot2Docker instances can communicate to the underlying VirtualBox instance through a HTTP SOAP API that we depict below.

Screen Shot 2016-01-18 at 10.00.03 PM

In order to start the SOAP Web Service the following command must be ran from a Terminal window.  The `vbowebsrv` command can be ran in the background with a `-b` flag.

$ VBoxManage setproperty websrvauthlibrary null
$ /Applications/VirtualBox.app/Contents/MacOS/vboxwebsrv -H -v


Moving on, we are ready to deploy new machines from scratch to ensure we have a repeatable process.  Go ahead and do this by opening a new Terminal window and entering the following.

$ docker-machine create --driver=virtualbox testing12

Optionally you can specify the ‘virtualbox-boot2docker-url’ parameter to use a pre-release or specific boot2docker image. For example, we are using the v.1.10.0-rc4 release below.

$ docker-machine create --driver=virtualbox --virtualbox-boot2docker-url=https://github.com/boot2docker/boot2docker/releases/download/v1.10.0-rc4/boot2docker.iso testing12

Note:  If you reboot any of these instances, you must re-configure the Volume Plugin with steps below.

If you are running Boot2Docker 1.10+, there is a fix that needs to be put in place to ensure the volume functionality works correctly.  Use the following command to install the patch.  As of Boot2Docker 1.10, this will no longer be needed.

$ docker-machine ssh testing12 "wget http://tinycorelinux.net/6.x/x86_64/tcz/udev-extra.tcz && tce-load -i udev-extra.tcz && sudo udevadm trigger"

The next step is to install the Volume Plugin which is responsible for orchestrating volumes with containers.  The following command is a simple curl-bash method of installing REX-Ray.  REX-Ray works very well in Boot2Docker since it is non-persistent, has a simple install, and has a VirtualBox driver.  Get more information about REX-Ray here.

$ docker-machine ssh testing12 "curl -sSL https://dl.bintray.com/emccode/rexray/install | sh -"

Following this there is a simple configuration file that must be set.  The volumePath parameter should be changed to update it for a valid path local to your underlying computer (ie. OS X).  This parameter defines where new volume files (vmdk) that get created by VirtualBox end up.

$ docker-machine ssh testing12 "sudo tee -a /etc/rexray/config.yml << EOF
  - virtualbox
      preempt: false
  tls: false
  volumePath: /Users/clintonkitson/VirtualBox Volumes
  controllerName: SATA

The last step for getting the Volume Plugin running is to start the REX-Ray service.  If the service fails to start, then you can leverage the `sudo rexray start -l debug` to get more information.

$ docker-machine ssh testing12 "sudo rexray start"

Screen Shot 2016-01-18 at 10.30.19 PM.png

That’s it!  You now have a Boot2Docker instance running with a Volume Plugin configured that will leverage VirtualBox and it’s Virtual Media to provide external volume access.  Although a bit beyond this post, the next step could either be to configure Docker Swarm for clustering.  We will configure the Docker CLI to speak with the Docker daemon.


$ eval $(docker-machine env testing12)

Now let’s play with some basic Volume functionality.  Notice below how you can use the `docker volume` sub-command to manage the volumes directly with the Docker CLI.  This is very powerful considering we are also enabling the usage of the `–opt` flag to set things like size.

$ docker volume create --driver=rexray --name=test100 --opt=size=1

The following graphic depicts this process.  In this case the VirtulBox Virtual Media is being used to create individual volumes that can be mounted to containers running on any VirtualBox VM or any B2D instance.

Screen Shot 2016-01-18 at 10.04.43 PM

Now let’s move forward and run a container with a persistent volume.  The image below shows this process.  It starts with the Docker Engine making a request of the Volume Plugin.  From there the VirtualBox REX-Ray driver takes over which leverages the SOAP Web Service.  This interface is used to perform storage orchestration to attach a device to a VM, discover the device in the OS, format the device, and then mount the device to the OS.  The last step in the process is to actually do a bind mount of the mountpath inside of the container.

$ docker run -ti --volume-driver=rexray -v test100:/test100 busybox

Screen Shot 2016-01-18 at 10.07.11 PM

If all was successful, then you should see something similar to the following image.  The last command `df /test100` is used to verify that we are seeing an external volume of `/dev/sdb`.

Screen Shot 2016-01-18 at 10.33.59 PM.png
Another place you can check is within the VirtualBox Manager where you can see the `SATA Port 2` is now populated with the test100 disk that we created.
Screen Shot 2016-01-18 at 10.35.42 PMIf you’re interested in the whole lifecycle, an `exit` from the container will perform the reverse process to unmount the volume from the container and detach the disk from the SATA port.


What’s next?  Rinse, repeat, and provision some more instances that can share volumes in a safe way via storage orchestration!  One of the great things about Docker Toolbox is that you can now easily spin up these lightweight Boot2Docker instances.  With this you can start to take advantage of other Docker platform tools such as Swarm.  Throw Volume Plugins on top as we just showed, and you have some very powerful functionality that expands the use cases for containers into persistent applications.

What do you think?  All done from your laptop just like in production!

3 thoughts on “Volume Plugins with Docker Toolbox and Boot2Docker

  1. Leonid Makarov

    How does rexray volume performance compare to vboxfs shared volumes and native file system performance (no shared volumes/folders)?
    Can the a rexray volume be accessed from the host as well or does it act solely as a storage engine for containers?


    1. clintonskitson Post author

      Great question. The external volume to Virtual Box is attached in an identical manner to the primary disk. Performance should be identical to that acquired by the primary disk. I have not done performance testing comparing the two methods of directly attaching a disk versus vboxfs.

      I need to investigate that, but I am assuming not natively. However the VMDK files are saved to the underlying host OS and should be able to be accessed directly that way.


  2. Pingback: What You Need to Know About Storage in Docker 1.10 | EMC {code} – Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s