The issue
Repository housekeeping can be a tedious task; however, it can be quite frequent when developing charms. I’ve recently introduced some CI changes in our Observability repos to ease that burden, and I’ll explain shortly how that was accomplished and what advantages it creates.
Our CI workflows are held in a separate repository to be referenced by all the others to avoid code duplication. However, there were quite a few repos who were left behind; some had their individual outdated CI, and some weren’t running any. Making sure there was a single clear workflow was a necessary clean starting point for further improvements.
The main problem I had with this process was how charm libraries were handled. Whenever you merge a PR for a change in a charm library, you have to not only manually publish it, but also manually update all of your charms that are using it. If you forget, you can find yourself stuck on issues that are very hard to debug because they are not related to your current code changes, but to outdated libraries (“didn’t we fix this already?”).
Turns out it’s very easy to forget one of these steps, and it gets worse: forgetting to publish a library is not detectable if you’re working on a different charm, as charmcraft fetch-lib will tell you everything’s up-to-date; you would need to individually and manually check all the libraries your charm is using, to make sure they’ve all been published.
The solution
The solution to this problem is, you guessed it, automation; we introduced the following GitHub workflows:
- a new charming action to release-libraries, so that bumped libraries can be published automatically; this makes sure that if something’s in main and passes our quality checks, it will be published;
- a scheduled workflow to update the libraries by periodically running
fetch-libfor all of our charms; however, since we don’t want fully-automatic updates (to avoid pulling breaking changes), the workflow will open a PR for us to review and merge.
These actions are preceded by a good amount of quality checks (i.e., linting, static checking, tests) and accompanied by some further utility actions (i.e., releasing a charm, promoting charms). These new additions to the process, though, will be particularly appealing to a lot of the charmers among you, as it makes those housekeeping chores so much more manageable.
An interesting side effect of these improvements is that it also leads to cool observability benefits. Before, there was no way to know the status of all repositories at a glance: maybe some libraries were not published, maybe some were not pulled; you had to go through each repository individually and manually check. Now, since all those steps are executed as CI, you can produce something like this:
| Charm | Auto-update Libraries | Release Libraries | Release to Edge |
|---|---|---|---|
| grafana-k8s-operator |  |
 |
 |
| prometheus-k8s-operator | |
 |
 |
| … |
By using GitHub badges for the new actions, this allows you to conveniently monitor your repositories and make sure everything’s working smoothly.
Enjoy your new CI!