Matrix tests for the Juju 2-to-3 transition

charm
,
#1

Recently we added a matrix to our bundle’s CI for testing on both Juju 2 and 3. Currently our bundle tests are called “integration tests” but the goal is to turn them into end-to-end tests. The idea is to run the entire collection of our bundle tests on some combinations of charms’ channel and juju version:

Juju / Charms -> edge beta candidate stable
Juju 2.9/stable
Juju 3.0/stable
  • Juju 2.9 goes with classically-confined microk8s and Juju 3.0 goes with strictly-confined microk8s.
  • Pinning the Juju agent version in the past was handy for CI, so we do it here as well.

The above can be captured in a GH action’s strategy section:

strategy:
  fail-fast: false
  matrix:
    charm-channel: [ "edge", "beta", "candidate", "stable" ]
    juju-channel: [ "2.9/stable", "3.0/stable" ]
    include:
      - juju-channel: "2.9/stable"
        juju-agent-version: "2.9.34"
        microk8s-channel: "1.25/stable"
      - juju-channel: "3.0/stable"
        juju-agent-version: "3.0.2"
        microk8s-channel: "1.25-strict/stable"

After that we set up the environment using the matrix variables:

- name: Setup operator environment
  uses: charmed-kubernetes/actions-operator@main
  with:
    juju-channel: ${{ matrix.juju-channel }}
    provider: microk8s
    channel: ${{ matrix.microk8s-channel }}
    bootstrap-options: "--agent-version ${{ matrix.juju-agent-version }}"

Some pitfalls

microk8s group name

If your itests make subprocess.run calls to microk8s, then you’re probably doing it with sg. Note that the group name has changed from microk8s for classically-confined microk8s to snap_microk8s for strictly-confined microk8s.

You can use os.environ and grp to figure out the correct command for the environment the test runs in:

if os.environ.get("RUNNER_OS"):
	# Running inside a GitHub runner
	# Need to find the correct group name
	try:
		# Classically confined microk8s
		uk8s_group = grp.getgrnam("microk8s").gr_name
	except KeyError:
		# Strictly confined microk8s
		uk8s_group = "snap_microk8s"
	cmd = ["sg", uk8s_group, "-c", f"microk8s ..."]
else:
	# Running locally
	cmd = ["sudo", "microk8s", "..."]

Juju 3 command names

In Juju 3, run-action was renamed to run. If you subprocess.Popen(["juju", "run-action", ...) then you should probably consider using python-libjuju’s model.applications[app_name].units[0].run_action(...).

Happy testing!

1 Like