Observability Team Updates - Weeks 29-30 (2023)

[sed-i (Leon)]

Hi everyone!

Below are the team’s updates for weeks 29 to 30. First, as always, let me introduce the fantastic team and what we’re building.

The Team

The observability team at Canonical consists of Dylan, Jose, Leon, Luca, Pietro, and Simme. Our goal is to provide you with the best open-source observability stack possible, turning your day-2 operations into smooth sailing.

The Progress

TLS everywhere

End-to-end TLS for COS Lite components is near completion (see tracking issue)! Here are some notable design decision we made along the way:

• Charms communicate their “highest priority URL” over relation data. URL priority is: web_external_url > ingress_url > socket.getfqdn(). The exception is ingress relations with traefik, which are always fqdn. Note that we intend to deprecate web_external_url soon in favor of stripPrefix.
• Utilize the update-ca-certificates machinery, instead of fine-grained approach of a particular ca cert per relation. This obviates the need to map between CAs and relations. This will become especially useful when the cert_transfer relation picks up (currently named mutual_tls but is pending a rename since it’s a misnomer).
• Do not use insecureSkipVerify (curl, traefik) anywhere. For example, if prometheus does not have any CA certs in place, it should fail scraping HTTPS targets.

The next big piece of the puzzle, which is currently underway, is TLS with grafana agent.

Our current design goal is depicted in the diagram below, courtesy of @lucabello.

Charm Tracing

The tracing relation interface draft was published and the tempo charm is ready for testing. An exciting outcome of this is that we can now trace charms themselves! See prometheus for example.

Misc fixes

• The grafana agent snap was bumped to 0.34.3, which means the grafana-agent machine charm has more collectors in place now.
• Various fixes around grafana admin password (grafana-k8s/236).
• Revive karma (karma-k8s/31)
• Cloning private repos with cos-config (cos-config/67)
• We are slowly pivoting towards pydantic for managing relation data (traefik/196).

Random stuff we had fun talking about

• Quentin Tarantino, Wes Andersen movies
• Black Mirror 6th season
• Barbenheimer
• Bubble Bobble
• Kirby’s Dream Land
• (╯°□°）╯︵ ┻━┻

Poll-tergeist

How would you title this image?

• Cattenna
• Offeline
• Purriscope
• Too many pussibilities!

This poll was made possible by @jose and @ppasotti.

Feedback welcome

As always, feedback is very welcome! Feel free to let us know your thoughts, questions, or suggestions either here or on the CharmHub Mattermost.

That’s all for this time! See you again in two weeks!

See also

• Observability Team Updates - Weeks 27-28