Kubernetes is a popular and rapidly growing platform for managing containerized workloads at scale.

Version 1.0 was released on July 21, 2015. Successive releases have kept up a cadence of new features, including many in the storage realm.

This is a quick recap of Kubernetes storage options as of the 1.5 release.

Basic Volumes and External Volume Mounts

Kubernetes started from the original Docker notion of volumes, which had a lifetime aligned to the container that encloses it. The Kubernetes pod aggregation slightly altered this to align with the pod that envelopes the container. The takeaway here is that a basic local volume mount is suited to run classic stateless services – but not a persistent service such as a SQL database – because the volume is lost when a pod is removed or a node failure occurs.

screen-shot-2016-12-14-at-6-25-21-pmStorage need differences: some types of services are stateless, some are not.

Mounting a volume supplied from an external provider (e.g. cloud block volume, Cinder, Dell EMC ScaleIO) means that when a pod is deleted, the volume is merely unmounted. This means that a volume can be pre-populated with data, and simply “handed off” to another pod or container. Compared to the node’s direct attached storage, external volumes deliver improved availability, scale, and performance – essential features when running persistent services in a pod.

Persistent Volumes and Claims

From a governance perspective, external volume mounts are still lacking. When a pod using an external volume is deleted, the volume still remains, leaving no queryable record inside the Kubernetes cluster. In addition, details of the specific source of the volume must be exposed in the consuming pod’s definition. This can be undesirable in a large-scale deployment where self-service deployment of pods takes place.

To support better administrative governance, a PersistentVolume can be created outside a pod. Administrators then utilize PersistentVolumeClaim’s to allow the storage provisioning and lifecycle to be maintained independently of the applications consuming storage. Storage implementation details remain hidden from the consuming application, allowing the potential for portability across platforms and storage providers.

Users of Kubernetes request persistent storage for their pods, referencing only a claim ID in the pod definition.

When a pod is deleted, Kubernetes still maintains PersistentVolume and PersistentVolumeClaim objects, which enables queries and management of the underlying storage inventory.

Volume Provisioning

With basic persistent volumes, an administrator manually pre-creates persistent volumes for later consumption by applications represented by pods.

Dynamic provisioning is a mechanism that allows storage volumes to be created on demand, rather than pre-provisioned by an administrator. It allows volume creation to be deferred and automatically performed when a claim is created.

A storage class abstraction exists to allow a container to use dynamic provisioning that chooses from categories of storage classes; for example “gold” might be high-performance storage, and “bronze” might be a cheaper “capacity” class.

Kubernetes Storage Extensibility

Volume plugins enable the use of external storage with Kubernetes.

Today, Kubernetes offers two approaches to storage integration. The first uses an “in-tree” volume plugin for a platform. The storage interface code is directly embedded into Kubernetes. This is currently used for storage providers such as AWS EBS, GCE, and OpenStack. The downside is that plugin “velocity”, the speed at a plugin can be added, enhanced, or patched, is gated by the Kubernetes release cycle.

The second approach is to leverage a “facade” plugin. A FlexVolume based integration uses externally installed and managed software to handle basic Attach/Detach/Mount/Unmount storage operations.

Kubernetes has a goal to make “onboarding” and patching of plugins for new storage providers easier and quicker. The Kubernetes Storage SIG has been discussing the idea of moving to a new, predominantly “out-of-tree” model for most storage interface code.

Futures

The {code} team is actively working on three projects to advance storage integration options for Kubernetes.

https://www.youtube.com/embed/Zv_uCkSY61M?list=PLbssOJyyvHuWiBQAg9EFWH570timj2fxt