Integration Testing Cookbook

framework
#1

This is a “cookbook” of useful techniques and solutions to common problems when integration testing a charm that has been built with the Charmed Operator Framework.

Wait for a single unit of the application to finish installing.

ops_test.model.wait_for_idle(apps=['application_name'], status='active', timeout=60, wait_for_exact_units=1)

Temporarily speed up update-status hooks

Useful for making a test run more quickly:

@contextlib.asynccontextmanager
async def fast_forward(ops_test, interval: str = "10s"):
    # temporarily speed up update-status firing rate
    await ops_test.model.set_config({"update-status-hook-interval": interval})
    yield
    await ops_test.model.set_config({"update-status-hook-interval": "60m"})


async def test_deploy(ops_test: OpsTest):
    async with fast_forward(ops_test):
        await ops_test.model.wait_for_idle(...)

Further reading

Writing Integration Tests for Charmed Operators

SDK Reference
How to write an integration test for a charm
Framework Team Updates - Week #16 2022
#2

@tmihoc while responding to comments on the draft of the doc I’m writing about integration testing, I thought it might be useful to have a “cookbook” style post with commonly used techniques. This is that post.

What do you think of the idea?

1 Like
#3
#4

@ppasotti Based on the work that you did visualizing relation data, do you have any suggestions for viewing and verifying relation data in an integration test?

I think that it would be useful to add a section on testing relation data, especially because that is one area where charm authors need to make assumptions in the harness tests.

#5

Huh, interesting question. I’ve always thought of relation data testing as a unittest concern. If AppA and AppB both do “their part” and unittest that to input X follows output Y, even when you have relation data doing some back-and-forth, I wouldn’t expect a ‘real deployment’ to change anything about the dynamics of this interaction, except maybe some concurrency or timing issues that in unittests are implicitly avoided.

So in short: I’ve never integration-tested databag contents. Rather I’d expect any errors in the data to be surfaced by the charm and reflected in the workload status.

Anyhow there are facilities in pytest-operator to read relation data (not sure about writing it).

relation = ops_test.model.get_relation('relation_name')
assert relation.data[relation.app] == {'foo': 'bar'}

@ca-scribner might have some thoughts and useful tips about this?

#6

I’ve taken a similar approach to @ppasotti, interrogating relation data in the unit tests/adding validation in the charm itself to make sure data is as expected, and then relying on the data exchange to work in the integration tests. I think it covers all cases, but now I’m less convinced.

@pengale do you have any examples where interrogating the relation data in integration charms was really helpful?

#7

@pengale see https://github.com/charmed-kubernetes/pytest-operator/pull/79 it looks like fast_forward is getting a promotion!

1 Like