`jhack show-relation`'s new little brother: `show-stored`

As we know, ops offers the possibility to use StoredState to persist some data between events, making charms therefore (somewhat) stateful.

This is often not the recommended choice to do things, because statefulness brings with it extra complexity when it comes to making event handlers idempotent, which is considered to be the ‘best pattern’ when charming.

However, sometimes it’s unavoidable to use stored state. In such cases, it can be challenging (or simply tedious) during testing and debugging, to inspect the contents of a live charm’s stored state in a uniform way.

Long story short: the storage could be a charm-local SQLite db, or a remote (juju controller) database. And then there’s the whole k8s/machine thing…

Thus, in the spirit of jhack show-relation, we present jhack show-stored.

How to get

See https://github.com/PietroPasotti/jhack#installation

at the time of writing, the strictly-confined jhack snap is pending approval, and this command was not available in the last devmode snap, so you’ll have to install from sources.

How to use

If you have an application deployed on your current model, say:

j deploy prometheus-k8s --channel beta prom

then you could type:

jhack show-stored prom/0

That will print out something like:

This reveals that prometheus-k8s is using StoredState twice, in two of the charm libs it uses: ingress_per_unit and grafana_dashboard. (It looks like the charm is not holding on to any pointers to GrafanaDashboardProvider however)


The handle name can be useful if you are planning to get your hands dirty and shell into the unit and poke around the database directly. The size (in bytes) of the data is more of a fun fact.


The bottom part of the two table cells contains the ‘blob’ itself. At the moment we only implement ‘pretty-printers’ for python dicts. ops natively serializes only native python datatypes (anything you can `yaml.dump, in fact), but you could be serializing much more complex stuff than that.

For that reason, jhack show-stored exposes a --adapters optional argument, which allows you to inject your custom adapter to deserialize a specific handle. So, for example, if you are not happy with how the ingress StoredData is represented, you could create a file:

from urllib.parse import urlparse

from rich.table import Table

def _deserialize_ingress(raw: dict):
    urls = raw['current_urls']
    table = Table(title='ingress view adapter')

    for unit_name, url in urls.items():
        row = [unit_name]

        p_url = urlparse(url)
        hostname, port = p_url.netloc.split(":")
        row.extend((p_url.scheme, hostname, port, p_url.path))


    return table  # we can return any rich.RenderableType (str, or Rich builtins)

# For this to work, this file needs to declare a global 'adapters' var of the right type.
adapters = {
    "PrometheusCharm/IngressPerUnitRequirer[ingress]/StoredStateData[_stored]": _deserialize_ingress

And then by running jhack show-stored -a /path/to/that/file, you’d magically get:

OF Storage

The Operator Framework uses some internal storage to keep track of meta information such as how many events have been processed so far. You can toggle the visibility of that storage (usually not very interesting) with --of-storage.

Controller storage

If the charm does not store its state locally (i.e. it is run with main(Charm, use_juju_for_storage=True), then you need to pass a --controller-storage flag to the tool.

For now, we can’t detect automatically where the storage is: you need to know (you can check the debug-log to learn which storage backend is used).

Watch and refresh

As you’d expect, there is a --watch flag to keep updating the view every few --refresh-rate (.5) seconds.

Practically, the stored state will only ever change when the charm executes, i.e. when an event occurs. So a good new feature would be: link show-stored with tail (tail already exposes a callback interface for internal use) so that the view is updated only when an event is fired on the unit.


Of course, the view has colors. You can disable them via --color no.