Matrix tests for the Juju 2-to-3 transition

charm
,
1

Recently we added a matrix to our bundle’s CI for testing on both Juju 2 and 3. Currently our bundle tests are called “integration tests” but the goal is to turn them into end-to-end tests. The idea is to run the entire collection of our bundle tests on some combinations of charms’ channel and juju version:

Juju / Charms -> edge beta candidate stable
Juju 2.9/stable
Juju 3.0/stable
  • Juju 2.9 goes with classically-confined microk8s and Juju 3.0 goes with strictly-confined microk8s.
  • Pinning the Juju agent version in the past was handy for CI, so we do it here as well.

The above can be captured in a GH action’s strategy section:

strategy:
  fail-fast: false
  matrix:
    charm-channel: [ "edge", "beta", "candidate", "stable" ]
    juju-channel: [ "2.9/stable", "3.0/stable" ]
    include:
      - juju-channel: "2.9/stable"
        juju-agent-version: "2.9.34"
        microk8s-channel: "1.25/stable"
      - juju-channel: "3.0/stable"
        juju-agent-version: "3.0.2"
        microk8s-channel: "1.25-strict/stable"

After that we set up the environment using the matrix variables:

- name: Setup operator environment
  uses: charmed-kubernetes/actions-operator@main
  with:
    juju-channel: ${{ matrix.juju-channel }}
    provider: microk8s
    channel: ${{ matrix.microk8s-channel }}
    bootstrap-options: "--agent-version ${{ matrix.juju-agent-version }}"

Some pitfalls

microk8s group name

If your itests make subprocess.run calls to microk8s, then you’re probably doing it with sg. Note that the group name has changed from microk8s for classically-confined microk8s to snap_microk8s for strictly-confined microk8s.

You can use os.environ and grp to figure out the correct command for the environment the test runs in:

if os.environ.get("RUNNER_OS"):
	# Running inside a GitHub runner
	# Need to find the correct group name
	try:
		# Classically confined microk8s
		uk8s_group = grp.getgrnam("microk8s").gr_name
	except KeyError:
		# Strictly confined microk8s
		uk8s_group = "snap_microk8s"
	cmd = ["sg", uk8s_group, "-c", f"microk8s ..."]
else:
	# Running locally
	cmd = ["sudo", "microk8s", "..."]

Juju 3 command names

In Juju 3, run-action was renamed to run. If you subprocess.Popen(["juju", "run-action", ...) then you should probably consider using python-libjuju’s model.applications[app_name].units[0].run_action(...).

python-libjuju support

Only version 3.0.* of python-libjuju supports both Juju 2.9 and Juju 3.

the approach we will follow is to use the latest python-libjuju that matches the juju release (2.9.38.1 -> 2.9.38, 3.1.0.1 -> 3.1.0)

– @tiradojm

Juju API Server maintains compatibility within a major series. So if we release an API method with 3.0, then we will continue to allow you to make that request with Juju 3.1, or 3.2, etc. However, it is fairly clear that a 3.0 candidate won’t know how to do things that we introduced in 3.1 (eg user secrets when they land in 3.2 will only be supported by pylibjuju-3.2+)

Across a major version change, we decided to give you 1 version (python-libjuju-3.0*) that supports both 2.9 and 3.0, but libjuju-3.1 is dropping support for 2.9

For your specific request, python-libjuju 3.0.* should be a bridge version that supports both, and as long as you aren’t trying to do “new things” with 3.1+ should be valid.

– @jameinel

For the transition period you may need to pin python-libjuju:

juju~=3.0.0

Happy testing!

3 Likes