Design with multiplicity in mind

charm
,
1

When developing charms, we encounter serveral layers of multiplicity, each layer adding complexity to the previous one.

Level 0: Charms

Charm authors cannot limit the number of app instances a juju admin can deploy of the same charm. For example, deploying two prometheus instances is perfectly valid:

juju deploy prometheus-k8s prom1
juju deploy prometheus-k8s prom2

One of the questions you should be asking yourself is if it makes sense to relate two apps of the same type to one another:

$ juju relate prom1 prom2
ERROR ambiguous relation: "prom1 prom2" could refer to 
  "prom1:metrics-endpoint prom2:self-metrics-endpoint";
  "prom2:metrics-endpoint prom1:self-metrics-endpoint"

In prometheus’s case this could mean: prom1 is my main instance, and prom2 provides self-monitoring by scraping prom1’s /metrics endpoint, in which case we’d

juju relate prom1:metrics-endpoint prom2:self-metrics-endpoint

Level 1: Relations

Charm authors can limit the number of relation a given app has over a given relation. In most cases, a charm can have multiple relations of the same type to other (remote) charms. For example:

  • A single prometheus charm can be related to any number of charms over the metrics-endpoint relation, to scrape each and every one of them.
  • A single traefik charm can be related to any number of charms over the ingress-per-unit relation, providing different ingress address for each. Conversely, however, an ingress requirer (e.g. prometheus) should only allow a single relation to an ingress provider (e.g. traefik). We do this by specifying limit: 1 in metadata.yaml.

Level 2: Units

Multiple units of the same charm may offer redundancy / fault-tolerance / reliability / scalability / high-availability. Charm authors cannot limit the number of units a juju admin can scale up an app to (you could “block” all non-leader units in your charm code though).

juju scale-application prom1 3

For prometheus this means:

  1. All three units of prom1 need to scrape all the related scrape targets over the metrics-endpoint relation.
  2. All remote units related to prom1 over remote-write must be “told” to push metrics not just to the endpoint of the prom1/0 unit, but also to the endpoints of prom1/1 and prom1/2 units as well.
  3. The traefik charm needs to reconfigure itself to provider ingress urls to the two newly joined units of prom1.
  4. prom2 needs to become aware of the additional prom1 units it needs to provide self-monitoring for.
  5. Etc.

Level 3: Charm libraries

When other charm authors use your charm libraries, they may want to create multiple instances of your charm library for the same realtion. For example:

  • Forward logs to Loki from multiple containers in the same pod (loki-k8s/186)
  • Instantiate multiples instances of ingress-per-unit to obtain multiple ingress urls for different ports:
    • IngressPerUnitRequirer(self, relation_name="ingress", port=80)
    • IngressPerUnitRequirer(self, relation_name="ingress", port=81)
  • Or attempt two different relation names but over same relation interface:
    • IngressPerUnitRequirer(self, relation_name="ingress1", port=80)
    • IngressPerUnitRequirer(self, relation_name="ingress2", port=81)

Arguably, this is an antipattern and charm libraries thould be singletons, but currently there is no built-in code construct preventing charm lib users from doing this.

Conclusion

In conclusion, charm authors should be mindful about the semantics, usability and scalability of each level of multiplicity. Opinionated design decisions are key.

2

Great article! Points out the real complexity of what’s going on. Two could-elaborate-more’s:

  • What about cross-model rendundancy?
  • I miss two big words in the article: “distributed system”. Multiplicity per se is trouble, but mix it with ‘nobody is aware of anyone else, unless if Juju kindly decides to fire a hook and let you know that, somewhere else, someone has done something that might be relevant to you’… and you get boom.
3

Agreed. I’m not sure though where it could fit in the context of multiplicity in juju.

charm --(1:N)-- unit --(1:N)-- charm lib --(1:N)-- relation

Seems like “distributed systems” is an overarching concept?

1 Like