The 2.8 edge snap contains a new experimental feature to allow k8s charms to be deployed as Kubernetes operators. As such, these charms the will not define a separate OCI image resource for the workload, and will not call k8s-spec-set hook command to have the controller create and manage the workload. Instead, the charm itself will do that using the Python k8s API added to the container which hosts the charm (pip installed to /usr/local/lib/python3.6/dist-packages/kubernetes).
Note: this feature is experimental and intended to allow early access for folks needing to prototype this kind of k8s charm. There’s work to do to handle things like permissions / roles, providing a framework on top of the raw Python API etc.
Since the feature is experimental, you’ll need to turn on a feature flag, either at bootstrap time or any time thereafter.
$ juju controller-config features=[k8s-operators]
Key differences compared to “workload” charms
An “operator” charm’s pod represents the workload as far as Juju is concerned - this pod is what is mapped to the application’s (single) unit in the Juju model.
Since the charm pod is the workload and tje charm is expected to create resources directly in the cluster as needed, the use of k8s-spec-set hook command is not required nor possible. Any resources created by the charm will be annotated with the Juju application name and cleaned up when the application or model is removed (by the time the beta is released, currently WIP).
Operator workloads cannot be scaled - the’re only ever one unit performing the required interactions with the cluster.
Actions run directly on the operator container rather than on a separate workload container as is the case with “workload” charms.
Creating an operator charm
To signify that a charm is to be treated as an operator charm, simply use the mode attribute in the deployment metadata (which defaults to “workload”):
deployment:
mode: operator
Do not define any OCI image resources, as k8s-spec-set is not supported.
Actions and other charm artefacts are defined as normal.
Limitations
One major limitation at the moment is that charm permissions / roles are not modelled. This means that any additional roles other than the basic ones created out of the box will need to be set up manually on the charm pod’s service account using kubectl. Assuming a charm called foo is deployed, the out of the box roles are:
$ kubectl describe role.rbac.authorization.k8s.io/foo-operator
...
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
pods/exec [] [] [create]
pods [] [] [get list]
The relevant service account is called serviceaccount/foo-operator.
Under consideration is for the charm to declare what permissions / roles it needs and for the user to juju trust the charm to activate those declared roles. This is similar to how juju trust works today for various IAAS charms where there’s a need for the charm to be given elevated privileges to interact with the cloud.
Feedback
This feature not complete but is hopefully in a state where it’s ready to be used to spike on implementing k8s operators as charms. We’ll use any feedback to help remove rough edges etc as we keep working on the feature.