Observability Team Update - Weeks 33-34 (2024)

sed-i · 27 August 2024 17:32

Hello, Hola, Ciao, Hej, Hallo, Witam, مرحباً بكم, שלום everyone!

Below are the team’s updates for weeks 33 to 34 of 2024. First, as always, let’s introduce the fantastic team and what we’re building.

The Observability team at Canonical consists of three sub-teams, managed by Simme

Our goal is to provide you with the best open-source observability stack possible, turning your day-2 operations into smooth sailing.

Alert’s source URL now points to the correct ingress address, and incorrect alertmanager config is now better handled.
We added self-monitoring alerts and cleaned up some charm logging in Tempo.
To use our HA charms, you need S3. We fixed a startup issue related to setting up storage.
The remote-write server URL was previously only communicated to a leader unit, and this is now fixed.

Forwarding logs from a charmed workload to loki is now simpler than before, thanks to pebble log forwarding. We added a doc for how to set it up.
The blackbox exporter is very handy for generic endpoint probing. We added a doc doc demonstrating how to configure it.
Grafana makes certain assumptions about how dashboards are used, on of which being the dashboard UID. With COS, dashboards are sourced from the git repo of each charm rather than from the grafana marketplace, so special attention must be given in setting a UID. We updated our docs to reflect this.
If you already use COS Lite, you don’t need to wait for HA to start using Tempo. We added docs explaining how: 1, 2, 3.
We published preliminary results for Tempo’s resource requirements.

We’ve been implementing the “coordinated multiroled workers” pattern for Loki HA, and released edge revisions for the worker and coordinator.
We made progress with waypoints in the istio beacon.

As always, feedback is very welcome! Feel free to let us know your thoughts, questions, or suggestions either here or on our Matrix channel .

That’s all for this time! See you again in two weeks!