Matrix tests for the Juju 2-to-3 transition

Recently we added a matrix to our bundle’s CI for testing on both Juju 2 and 3. Currently our bundle tests are called “integration tests” but the goal is to turn them into end-to-end tests. The idea is to run the entire collection of our bundle tests on some combinations of charms’ channel and juju version:

Juju / Charms -> edge beta candidate stable
Juju 2.9/stable
Juju 3.0/stable
  • Juju 2.9 goes with classically-confined microk8s and Juju 3.0 goes with strictly-confined microk8s.
  • Pinning the Juju agent version in the past was handy for CI, so we do it here as well.

The above can be captured in a GH action’s strategy section:

  fail-fast: false
    charm-channel: [ "edge", "beta", "candidate", "stable" ]
    juju-channel: [ "2.9/stable", "3.0/stable" ]
      - juju-channel: "2.9/stable"
        juju-agent-version: "2.9.34"
        microk8s-channel: "1.25/stable"
      - juju-channel: "3.0/stable"
        juju-agent-version: "3.0.2"
        microk8s-channel: "1.25-strict/stable"

After that we set up the environment using the matrix variables:

- name: Setup operator environment
  uses: charmed-kubernetes/actions-operator@main
    juju-channel: ${{ matrix.juju-channel }}
    provider: microk8s
    channel: ${{ matrix.microk8s-channel }}
    bootstrap-options: "--agent-version ${{ matrix.juju-agent-version }}"

Some pitfalls

microk8s group name

If your itests make calls to microk8s, then you’re probably doing it with sg. Note that the group name has changed from microk8s for classically-confined microk8s to snap_microk8s for strictly-confined microk8s.

You can use os.environ and grp to figure out the correct command for the environment the test runs in:

if os.environ.get("RUNNER_OS"):
	# Running inside a GitHub runner
	# Need to find the correct group name
		# Classically confined microk8s
		uk8s_group = grp.getgrnam("microk8s").gr_name
	except KeyError:
		# Strictly confined microk8s
		uk8s_group = "snap_microk8s"
	cmd = ["sg", uk8s_group, "-c", f"microk8s ..."]
	# Running locally
	cmd = ["sudo", "microk8s", "..."]

Juju 3 command names

In Juju 3, run-action was renamed to run. If you subprocess.Popen(["juju", "run-action", ...) then you should probably consider using python-libjuju’s model.applications[app_name].units[0].run_action(...).

Happy testing!

1 Like