It seems practical to deploy parts of Openstack into separate models.
In theory, given Juju cross model relations AND given the Openstack charms, deploying Openstack in a cross model context seems reasonable.
Despite what I knew to be true, I have given deploying the Openstack components in a CMR fashion at multiple points in time; 1) Juju 2.3 (the dawn of CMR) - just about everything in the deploy seemed broken, 2) Juju 2.4 - far less brokeness, but many things still borked, 3) Juju 2.5 (congrats on the release today!) - juju status green across all models (first time an Openstack deploy has actually settled using CMR), some functionality is broken/missing.
My hope is that @rr-pdl and myself alongside some insight of others, can help get to the bottom of the issues that remain in CMR and thus also in CMR Openstack.
@rr-pdl and myself have attempted to create a POC of CMR Openstack using 3 separate models:
$ juju models
Controller: maas-controller
Model Cloud/Region Status Machines Cores Access Last connection
ryan/ceph-storage pdl-maas available 12 336 admin 3 hours ago
ryan/pdl-openstack* pdl-maas available 17 424 admin 16 minutes ago
ryan/utilities pdl-maas available 6 104 admin 2 hours ago
$ for model in ryan/ceph-storage ryan/utilities ryan/pdl-openstack; do
juju status -m $model --relations;
done
Although we are applying the same relations between the same components we would be in a non-CMR style deploy, we are seeing missing ceph config where we would expect to see it amongst other things.
Possibly we can use the space below to enumerate the list of issues we are seeing in the cross model context and their associated logs and resources.
@rr-pdl I’m thinking, possibly https://jujucharms.com/ceph-proxy/ might be an answer here. So long as the only relations still giving us grief are the ceph-mon:client and ceph-mon:radosgw relations, possibly we can just sub in the ceph-proxy and be off to the races!
Are these issues that the charms aren’t getting something over a CMR vs a typical relation? I’m not seeing if this is a bug in the Juju CMR code or something that the charms need tweaking to better support the CMR variant (which honestly is still probably a Juju bug as charms shouldn’t need to worry too much about CMR vs not)
The relations that didn’t work in a cross model context were the relations to ceph-mon:client. We were able to get a decoupled storage/control plane working using the ceph-proxy charm.
The ceph-mon charm uses the remote unit names in relation keys when generating responses to broker requests to setup pools and access to the ceph cluster.
In a cross-model relation the remote unit name on the offer side is obfuscated so the ceph-mon charm generates a broker response message with a unit name that does not match the unit names on the consuming side. This means the relation never completes as the consuming side and offer side keys don’t match.
This is a pattern that’s used in a few places in charms - I think MySQL has a similar pattern to indicate when access for a specific remote unit has been setup.
As CMR’s obfuscate the unit names, we need to move towards something unit-name agnostic; I’m leaning towards each consuming unit providing a fixed nonce of data which is then used in the broker responses instead of the unit name but that’s going to need some careful thinking to get right.
To answer Rick’s question, charms don’t need to be CMR aware, but they need to not assume they can rely on knowing the remote unit name as James explains.
Hi, is there any bug assigned to it. I’ve tried today as a test to offer ceph-mon to cinder-ceph in another model and get: “Unsupported CMR relation”.
To answer my own question. It’s all fine now, however, you have to set permit-insecure-cmr in ceph-mon in order to get it to work.
