Supported use cases for Kubernetes on Openstack + Ceph storage

juju
#1

I’ve used Juju to deploy an Openstack cloud, and then I’ve used the same controller to deploy Kubernetes inside the Openstack cloud.

Seeing as though Openstack is already making use of Ceph for storage, I’m looking to use the same Ceph cluster as a backend for my K8s persistent volumes.

In my current setup, Openstack lives on it’s own model including Ceph, while K8s lives on a separate model on the same controller.

I’ve noticed that if I offer (share) ceph-mon from the Openstack model, consume it on the K8s model and then relate kubernetes-master to ceph-mon, I’m getting a "Unsupported CMR relation" error on ceph-mon.

Is this error due to a limitation with ceph-mon and Cross Model Relations or is this because I’m trying to “Multitenant” the Ceph cluster between different / unrelated applications?

I’m interested to see how "grown ups" are managing their models, applications and integrations between them.

Shell output of ceph-mon and kuberenetes-master

#2

Hi,

We’re targeting the same kind of setup (KaaS aka Kube as a Service ionside an Openstack tenant) and you have 2 (above others) options :

  • either you use Ceph from Kube directly and for that, the best way might be to build 3 models : one for openstack, one for ceph and one for kube and use ceph-proxy to “plug” your openstack/kube to your ceph
  • or for a better and easier integration inside openstack, you should use the openstack integrator and rely on Cinder (which will rely on Ceph as backend) for your volumes inside Kube : Openstack-integrator charm | Ubuntu

I would personnaly recommend the second option unless you want specific Ceph features that you couldn’t find in Cinder (I don’t see any TBH but you can have your reason).