How to peek into a unit's local stored state

The short answer is jhack show-stored yourapp/0. The long answer follows.


A unit’s local stored state is backed by a SQLite3 file located at /var/lib/juju/agents/unit-yourapp-0/charm/.unit-state.db.

After operator/1057 you need sudo to view it. For example:

sudo sqlite3 /var/lib/juju/agents/unit-cos-proxy-0/charm/.unit-state.db .schema
CREATE TABLE snapshot (handle TEXT PRIMARY KEY, data BLOB);
                  sequence INTEGER PRIMARY KEY AUTOINCREMENT,
                  event_path TEXT,
                  observer_path TEXT,
                  method_name TEXT);
CREATE TABLE sqlite_sequence(name,seq);


ops creates a table per Object, so we can find all the interesting names with .dump and convert it into a yaml:

sudo sqlite3 /var/lib/juju/agents/unit-cos-proxy-0/charm/.unit-state.db .dump \
  | grep "INSERT INTO snapshot VALUES" \
  | awk -F"[']" '{print "\""$2"\""":", "\""$4"\""}' \
  | yq

But that would give you pickled-hexlified data, such as:

"COSProxyCharm/StoredStateData[_stored]": "80049576000000000000007d94288c0c686176655f67726166616e6194898c0f686176655f64617368626f6172647394898c0f686176655f70726f6d65746865757394888c0c686176655f7461726765747394898c09686176655f6e72706594888c09686176655f6c6f6b6994898c0d686176655f66696c65626561749489752e"
"StoredStateData[_stored]": "80049515000000000000007d948c0b6576656e745f636f756e74944d4e01732e"

We can unhexlify and unpickle with a short python script:

sudo sqlite3 /var/lib/juju/agents/unit-cos-proxy-0/charm/.unit-state.db .dump \
  | grep "INSERT INTO snapshot VALUES" \
  | awk -F"[']" '{print "\""$2"\""":", "\""$4"\""}' \
  | python3 -c "import sys,yaml,pickle,binascii; stored=yaml.safe_load(; print(yaml.safe_dump({k: pickle.loads(binascii.unhexlify(bytes(v, 'utf8'))) for k, v in stored.items()}))"

Which prints:

  have_dashboards: false
  have_filebeat: false
  have_grafana: false
  have_loki: false
  have_nrpe: true
  have_prometheus: true
  have_targets: false
  event_count: 334

To do this outside of the vm/container, and without pre-installing sqlite3, you could use python all the way:

juju ssh cos-proxy/0 "sudo python3 -c \"import sqlite3,re,pickle,binascii,yaml; db=sqlite3.connect('/var/lib/juju/agents/unit-cos-proxy-0/charm/.unit-state.db', isolation_level=None); matches=filter(None,map(lambda l: re.match('INSERT.+VALUES.\'([^,]+)\',X\'([^\']+)\'',l), db.iterdump())); print(yaml.safe_dump({ pickle.loads(binascii.unhexlify(bytes(, 'utf8'))) for m in matches}))\""


1 Like