Migrating a Kubernetes PV to a new storage class for applications that don't have tar installed

Kubernetes provides an abstraction layer for persistent data storage. Volumes can be of various storage classes depending on user requirements. Those requirements can evolve with time, meaning that volumes sometimes need to be migrated from one storage class to another. This article introduces a method to do so.

In this article, MinIO will be used as an example. MinIO pods do support the kubectl cp command as the container does not have tar installed. For applications that allow the kubectl cp command, the procedure in this article can be simplified

Here is a manifest for a MinIO pod, where data is stored using a storage class called nfs-csi:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: minio-nfs
spec:
  storageClassName: nfs-csi
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 500Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: minio
spec:
  volumes:
    - name: minio-data
      persistentVolumeClaim:
        claimName: minio-nfs
  containers:
    - name: minio
      image: quay.io/minio/minio
      command:
        - /bin/bash
        - -c
      args:
        - minio server /data --console-address :9001
      volumeMounts:
        - mountPath: /data
          name: minio-data

Now let's imagine we want MinIO to use a new storage class, Longhorn. Thins involves creating a new set of PVC and PV. Assuming the lonhgorn storage class already exists in the cluster: This can be done using the following manifest:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: minio-longhorn
spec:
  storageClassName: longhorn
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 500Gi

The data now needs to be moved from the original PV to the new one. For this purpose, let's start by deleting the MinIO pod:

kubectl delete pod minio

In its place, we will deploy a Ubuntu pod with the PVC mounted to /mnt

apiVersion: v1
kind: Pod
metadata:
  name: ubuntu
spec:
  containers:
  - image: ubuntu
    command:
      - "sleep"
      - "604800"
    name: ubuntu
    volumeMounts:
      - mountPath: /mnt
        name: mnt
  restartPolicy: Always
  volumes:
    - name: mnt
      persistentVolumeClaim:
        claimName: minio-nfs

With the pod deployed, we can leverage kubectl cp to copy data from the PV to our local machine in a temporary folder:

kubectl cp ubuntu:/mnt ./tmp

Once done, we can delete the Ubuntu pod and recreate it by changing the spec.volumes.persistentVolumeClaim.claimName to minio-longhorn. The newly create Ubuntu pod now has the new PVC mounted to /mnt. This allows us to copy the content of the temporary folder into the PV:

microk8s.kubectl cp ./tmp/. ubuntu:/mnt

After the data has been transferred to the new PV, the Ubuntu pod can once again be deleted.

kubectl delete pod minio

The spec.volumes.persistentVolumeClaim.claimName of the MinIO pod can now be changed to longhorn as well, after which the pod can be redeployed.

The MinIO pod should now be using Longhorn as a storage class and have its data from the original nfs-csi PV.