Observability Team Update - Weeks 13-14 (2024)

Hi everyone! :wave:

Below are the team’s updates for weeks 13 to 14 of 2024. First, as always, let’s introduce the fantastic team and what we’re building.

:muscle: The Team

The Observability team at Canonical consists of Dylan, Ibrahim, Jose, Leon, Luca, Mateusz, Michael, Pietro, and Simme. Our goal is to provide you with the best open-source observability stack possible, turning your day-2 operations into smooth sailing.

:hammer_and_wrench: The Work

Apart from bug fixes (grafana-agent/81, grafana/314, traefik/317, mimir-coordinator/46) and doc improvements (1, 2), we also had a few enhancements:

New track for cos-lite bundle with pinned revisions

Our default latest track is “floating”: when you juju deploy --channel=latest/stable cos-lite, you get all the charms that happen to be stable at the moment.

With the pinned track, when you juju deploy --channel=pinned/stable cos-lite, you get a combination of charms that have passed matrix bundle tests, so the “stable together” confidence is higher.

The pinned track also helps with collecting artifacts for air-gapped deployments.

Workload tracing for grafana

Grafana itself is already instrumented for tracing, all we needed to do is make a config adjustment.

Better default for alertmanager config

@nobuto discovered that alertmanager accepts a “blackhole” receiver. This is useful when alertmanager is used as a dashboard only (without forwarding to e.g. PagerDuty). With this change, the “AlertmanagerNotificationsFailed” no longer fires for the default placeholder receiver used previously.

The same idea was introduced to the alertmanager rock.

:pray: Feedback welcome

As always, feedback is very welcome! Feel free to let us know your thoughts, questions, or suggestions either here or on our Matrix channel.

That’s all for this time! See you again in two weeks! :partying_face:

See also