How to test interface tests

doc
1

Also see:

Suppose you have written an interface specification and its corresponding interface tests. You also have a charm, that implements the interface. You now want to verify that your charm would pass the interface tests, and you want to verify that the charm-relation-interfaces machinery correctly picks up and tests your charm.

This poses a chicken-and-egg problem because by default, your charm searches charm-relation-interfaces:main for the tests to run, and it won’t find any because you didn’t merge your PR yet. Conversely, charm-relation-interfaces’ run_matrix.py clones your charm repo’s main branch in order to know what code to run the tests against.

Configure the charm to test against a specific charm-relation-interfaces branch

In order to instruct your charm to gather the tests from a specific branch of a custom repo, you need to configure the interface_tester fixture as follows:

import pytest
from interface_tester import InterfaceTester

@pytest.fixture
def interface_tester(interface_tester: InterfaceTester):
    interface_tester.configure(
        repo="https://example.com/foo.git",  # default = "https://github.com/canonical/charm-relation-interfaces"
        branch="mybranch",  # default = "main",
    )
    yield interface_tester

[note]So long as the repo has the same folder structure as github.com/canonical/charm-relation-interfaces does (interfaces/interface-name/etc...), this will work! The interface testing machinery can be used with any repository containing interface specifications (with a given structure), not just charm-relation-interfaces`. [/note]

Configure your charm-relation-interfaces clone to test a specific charm branch

In order to instruct the matrix tester (or the collector) to clone a specific charm branch for testing, you need to editcharm-relation-interfaces/interfaces/myinterface/charms.yaml as follows:

providers:  # or "requirers"
  - name: mycharm
    url: https://github.com/myorg/mycharm-operator
    branch: mybranch

Do that, and you should be able to run python ./run_matrix.py --include myinterface and test your charm at mybranch.

Configure your charm-relation-interfaces clone to fetch the interface_tester fixture from another place

By default, when charm-relation-interfaces clones your charm repo to obtain its tester configuration, it looks for it in <charm-root>/tests/interfaces/conftest.py, and it assumes it is called interface_tester, overriding the one provided by the pytest-interface-tester plugin.

You can instruct the collector to obtain the fixture from another location, and another identifier, by editing charm-relation-interfaces/interfaces/myinterface/charms.yaml as follows:

providers:  # or "requirers"
  - name: mycharm
    url: https://github.com/myorg/mycharm-operator
    test_setup:
      identifier: mytester  # default: "interface_tester"
      location: "my/custom/interface/tests/folder/file.py"  # default: "tests/interfaces/conftest.py"

This will, essentially, instruct the test runner to do:

from <charm root>.my.custom.interface.tests.folder.file import mytester

Now, running interface-tester discover --include myinterface should show that you have a custom test setup configured:

...
myinterface
  - v0:
    ...
    - charms:
       - mycharm (https://github.com/myorg/mycharm-operator) custom_test_setup=yes

And, assuming the aforementioned import succeeds, you will also be able to run the interface test matrix against your charm:

$ tox -e run-interface-test-matrix

or:

$ python -m run_matrix.py --include myinterface

Contributors: @ppasotti