Adding to tracing to charm libraries, starting with `apt`.

Hi charmers,

As part of our push towards having charms traced by default, I’ve merged changes to the apt charm lib to annotate specific operations: every subprocess call to the system apt tool, and file operations where file size or count is not known in advance and the operation could take arbitrarily long.

This is an example of tracing at a coarse granularity, which we feel is a good balance between being minimally sufficient and recording rich information for someone to figure out the details after the fact.

The change also adds the opentelemetry-api dependency to the charm library.

If you are using ops >= 2.21, you have that dependency already via ops, although it’s a good practice to list it explicitly in your charm’s Python project.

If you have pinned an older version of Ops…

  • I would very much like to understand why you could not upgrade.
  • You’ll have to add the new dependency explicitly.

To recap, the division of responsibilities is as follows:

Ops covers hook invocation, observer handler invocations, Juju hook tool calls and Pebble calls.

Charm libs should instrument non-trivial public functions (for example, __init__ and publish_foo) to provide context, as well as operations that may take arbitrarily long (subprocess calls, HTTP or gRPC calls, and workload SDK calls).

Charms, therefore, don’t need much instrumentation; however, it can be helpful if it provides context or if important values can be included. Same as above, charms should instrument subprocess calls, network calls, and workload SDK calls.

Additionally, charms should record important events, especially if the charm makes a decision based information from different sources, which may not be visible post factum. A good example would be failing over from workload primary to a backup: a clear statement that helps someone understand why it happened, when exactly and what may be next.

I’m planning to instrument the snap charm lib next, but the rest I leave to charm library authors and/or charmers who use these libraries.

Dima, on behalf of Charm Tech.

1 Like