Charmed MySQL K8s How-to | Enable Tracing

shayanp · 18 June 2024 18:44

Note: All commands are written for juju >= v3.1

If you’re using juju 2.9, check the juju 3.0 Release Notes.

Enable Tracing

This guide contains the steps to enable tracing with Grafana Tempo for your MySQL K8s application.

To summarize:

Deploy the Tempo charms and their dependencies in a COS K8s environment
Offer interfaces for cross-model integrations
Consume and integrate cross-model integrations
View MySQL K8s traces on Grafana

Warning: This is feature is in development. It is not recommended for production environments.

This feature is available for Charmed MySQL K8s revision 146+ only.

Prerequisites

Enabling tracing with Tempo requires that you:

Have deployed a Charmed MySQL K8s application
- See How to manage units
Have deployed a ‘cos-lite’ bundle from the latest/edge track in a Kubernetes environment
- See Getting started on MicroK8s

Deploy Tempo

First, switch to the Kubernetes controller where the COS model is deployed:

juju switch <k8s_controller_name>:<cos_model_name>

Then, deploy the dependencies of Tempo following this tutorial. In particular, we would want to:

Deploy the minio charm
Deploy the s3 integrator charm
Add a bucket in minio using a python script
Configure s3 integrator with the minio credentials

Finally, deploy and integrate with Tempo HA in a monolithic setup.

Offer interfaces

Next, offer interfaces for cross-model integrations from the model where Charmed MySQL is deployed.

To offer the Tempo integration, run

juju offer <tempo_coordinator_k8s_application_name>:tracing

Then, switch to the Charmed MySQL K8s model, find the offers, and integrate (relate) with them:

juju switch <k8s_controller_name>:<mysql_model_name>

juju find-offers <k8s_controller_name>:

Do not miss the “:” in the command above.

Below is a sample output where k8s is the K8s controller name and cos is the model where cos-lite and tempo-k8s are deployed:

Store  URL                            Access  Interfaces
k8s    admin/cos.tempo                admin   tracing:tracing

Next, consume this offer so that it is reachable from the current model:

juju consume k8s:admin/cos.tempo

Consume interfaces

First, deploy Grafana Agent K8s from the latest/edge channel:

juju deploy grafana-agent-k8s --channel latest/edge

Then, integrate Grafana Agent K8s with the consumed interface from the previous section:

juju integrate grafana-agent-k8s:tracing tempo:tracing

Finally, integrate Charmed MySQL K8s with Grafana Agent K8s:

juju integrate mysql-k8s:tracing grafana-agent-k8s:tracing-provider

Wait until the model settles. The following is an example of the juju status --relations on the Charmed MySQL K8s model:

Model     Controller  Cloud/Region        Version  SLA          Timestamp
database  k8s         microk8s/localhost  3.5.4    unsupported  16:33:26Z

SAAS   Status  Store       URL
tempo  active  k8s         admin/cos.tempo

App                Version                  Status  Scale  Charm              Channel      Rev  Address         Exposed  Message
grafana-agent-k8s  0.40.4                   active      1  grafana-agent-k8s  latest/edge   93  10.152.183.63   no       grafana-dashboards-provider: off, logging-consumer: off, send-remote-write: off
mysql-k8s          8.0.37-0ubuntu0.22.04.3  active      1  mysql-k8s                         0  10.152.183.135  no       Primary

Unit                  Workload  Agent      Address       Ports  Message
grafana-agent-k8s/0*  active    idle       10.1.241.255         grafana-dashboards-provider: off, logging-consumer: off, send-remote-write: off
mysql-k8s/0*          active    executing  10.1.241.253         Primary

Integration provider                Requirer                   Interface              Type     Message
grafana-agent-k8s:peers             grafana-agent-k8s:peers    grafana_agent_replica  peer     
grafana-agent-k8s:tracing-provider  mysql-k8s:tracing          tracing                regular  
mysql-k8s:database-peers            mysql-k8s:database-peers   mysql_peers            peer     
mysql-k8s:restart                   mysql-k8s:restart          rolling_op             peer     
mysql-k8s:upgrade                   mysql-k8s:upgrade          upgrade                peer     
tempo:tracing                       grafana-agent-k8s:tracing  tracing                regular

Note: All traces are exported to Tempo using HTTP. Support for sending traces via HTTPS is an upcoming feature.

View traces

After this is complete, the Tempo traces will be accessible from Grafana under the Explore section with tempo-k8s as the data source. You will be able to select mysql-k8s as the Service Name under the Search tab to view traces belonging to Charmed MySQL.

Below is a screenshot demonstrating a Charmed MySQL trace:

Feel free to read through the Tempo HA documentation at your leisure to explore its deployment and its integrations.