[COS] Endpoint split into `charm-tracing` and `workload-tracing`

charm Observability
1

As part of our efforts to make it easier to add tracing capabilities for the workloads, we made a decision to change our recommendations for using tracing protocol in charms. We will split tracing endpoint into two: charm-tracing and workload-tracing in COS charms, using the same tracing protocol. This means that you will be able to integrate the two types of traces separately, giving you more control on what you want to trace.

See more on Tempo: Charmed Tempo HA

What does it mean to you

If you have COS components deployed with Tempo and related using a tracing integration in your Juju model, see the “Breaking change” section in this document to see how to upgrade.

If your charms already implement tracing, please follow the recommendations in the “How to implement the split in your charm”.

Breaking change

Please note that the breaking change only affects you if you have COS Lite integrated with Tempo on production environments. If you haven’t deployed Tempo, you’re not affected.

We are rolling out the split within the COS charms. If you have COS components deployed together with Tempo and related using a tracing integration in your Juju model, please remove the tracing relation before the upgrade and re-relate the applications to tempo after the upgrade.

Revisions where the change was / will be introduced (for revisions marked with TBA, revision will be added once the change is released):

  • prometheus-k8s: rev215
  • loki-k8s: TBA
  • grafana-k8s: TBA
  • traefik-k8s: TBA
  • mimir-coordinator-k8s: rev24
  • loki-coordinator-k8s: rev6
  • tempo-coordinator-k8s: rev35
  • alertmanager-k8s: TBA (once alertmanager starts supporting workload traces)

How to implement the split in your charm

For charms that use charm tracing and would like to add workload tracing, we recommend having two separate endpoints in your charmcraft.yaml (or metadata.yaml), both using tracing interface.

requires:  
(...)
  charm-tracing:  
    interface: tracing  
    limit: 1  
    description: |  
      Used for passing charm traces from the charm to the tracing backend.  
  workload-tracing:  
    interface: tracing  
    limit: 1  
    description: |  
      Used for passing workload traces to the tracing backend.

In charm code, you need to specify the used relation name:

self.charm_tracing = TracingEndpointRequirer(self, relation_name="charm-tracing", protocols=["otlp_http"])  
self.workload_tracing = TracingEndpointRequirer(self, relation_name="workload-tracing", protocols=["otlp_grpc"])  
  
self.charm_tracing_endpoint, self.server_cert = charm_tracing_config(  
    self.charm_tracing, self._ca_cert_path  
)

In order to fetch the workload tracing endpoint to pass it to your workload config, you can use the get_endpoint method:

if self.workload_tracing.is_ready():  
    return self.workload_tracing.get_endpoint("otlp_grpc")
1 Like