Simplifying Kubernetes Persistent Storage with Rook and Ceph

With the growing adoption of microservices and containerized applications, Kubernetes has become the de facto standard for container orchestration. However, managing persistent storage in Kubernetes can be challenging. One of the tools that makes this easier is the open-source project Rook, which simplifies the deployment and management of Ceph, a highly scalable distributed storage system. In this blog post, we'll explore how to get started with Rook to manage Ceph storage in your Kubernetes cluster.

Why Rook and Ceph?

Rook is an open-source cloud-native storage orchestrator for Kubernetes. Ceph is a distributed storage system that provides object, block, and file storage in a unified system. Together, they offer several benefits:

• Unified Storage: Provides object, block, and file storage in a single solution.
• Scalability: Ceph can scale out to thousands of nodes.
• High Availability: Ensures data redundancy and fault tolerance.
• Reduced Complexity: Rook simplifies the deployment and management of Ceph on Kubernetes.

Setting Up Rook and Ceph in Kubernetes

Step 1: Install Rook

First, you'll need to clone the Rook repository and navigate to the Rook operator directory:

git clone https://github.com/rook/rook.git
cd rook/cluster/examples/kubernetes/ceph

Then, apply the common resources and Rook operator:

kubectl apply -f common.yaml
kubectl apply -f operator.yaml

This will deploy the Rook operator in your Kubernetes cluster, which will manage the Ceph cluster.

Step 2: Configure and Deploy a Ceph Cluster

Create a new cluster.yaml file with the desired configuration for your Ceph cluster. Here is an example configuration:

apiVersion: ceph.rook.io/v1
kind: CephCluster
metadata:
  name: rook-ceph
  namespace: rook-ceph
spec:
  cephVersion:
    image: ceph/ceph:v14.2.5-20190917
    allowUnsupported: false
  dataDirHostPath: /var/lib/rook
  mon:
    count: 3
  dashboard:
    enabled: true
  storage:
    useAllNodes: true
    useAllDevices: true

Apply the cluster.yaml configuration to create the Ceph cluster:

kubectl apply -f cluster.yaml

This deploys the Ceph cluster in your Kubernetes environment.

Step 3: Create Storage Classes

To use the Ceph cluster for persistent storage, you need to create storage classes. Create a storageclass.yaml file:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: rook-ceph-block
provisioner: rook-ceph.rbd.csi.ceph.com
parameters:
  clusterID: rook-ceph
  pool: replicapool
  imageFormat: "2"
  imageFeatures: layering
reclaimPolicy: Retain
allowVolumeExpansion: true

Apply the storage class configuration:

kubectl apply -f storageclass.yaml

You now have a storage class that can be used to provision persistent volumes backed by the Ceph cluster.

Using the Storage Class in Applications

You can now use the created storage class in your application deployments. Here’s an example of how to define a persistent volume claim (PVC) in your application’s YAML file:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: rook-ceph-block

This PVC can then be referenced in a Pod definition:

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  containers:
  - name: my-container
    image: busybox
    volumeMounts:
    - mountPath: "/mnt/data"
      name: my-volume
  volumes:
  - name: my-volume
    persistentVolumeClaim:
      claimName: my-pvc

Conclusion

Rook and Ceph provide powerful capabilities for managing persistent storage in cloud-native environments. By following the steps outlined in this post, you can deploy and configure a Ceph cluster using Rook in your Kubernetes environment. This setup allows you to harness the scalability and reliability of Ceph, while simplifying management through Rook. Experiment with different configurations to optimize storage for your specific use case, and enjoy the benefits of unified, scalable storage in your Kubernetes cluster.

Have you used Rook and Ceph in your projects? Share your experiences and tips in the comments below!