Integration Testing Cookbook

This is a “cookbook” of useful techniques and solutions to common problems when integration testing a charm that has been built with the Charmed Operator Framework.

Wait for a single unit of the application to finish installing.

ops_test.model.wait_for_idle(apps=['application_name'], status='active', timeout=60, wait_for_exact_units=1)

Temporarily speed up update-status hooks

Useful for making a test run more quickly:

async def fast_forward(ops_test, interval: str = "10s"):
    # temporarily speed up update-status firing rate
    await ops_test.model.set_config({"update-status-hook-interval": interval})
    await ops_test.model.set_config({"update-status-hook-interval": "60m"})

async def test_deploy(ops_test: OpsTest):
    async with fast_forward(ops_test):
        await ops_test.model.wait_for_idle(...)

Further reading

Writing Integration Tests for Charmed Operators

@tmihoc while responding to comments on the draft of the doc I’m writing about integration testing, I thought it might be useful to have a “cookbook” style post with commonly used techniques. This is that post.

What do you think of the idea?

1 Like

@ppasotti Based on the work that you did visualizing relation data, do you have any suggestions for viewing and verifying relation data in an integration test?

I think that it would be useful to add a section on testing relation data, especially because that is one area where charm authors need to make assumptions in the harness tests.

Huh, interesting question. I’ve always thought of relation data testing as a unittest concern. If AppA and AppB both do “their part” and unittest that to input X follows output Y, even when you have relation data doing some back-and-forth, I wouldn’t expect a ‘real deployment’ to change anything about the dynamics of this interaction, except maybe some concurrency or timing issues that in unittests are implicitly avoided.

So in short: I’ve never integration-tested databag contents. Rather I’d expect any errors in the data to be surfaced by the charm and reflected in the workload status.

Anyhow there are facilities in pytest-operator to read relation data (not sure about writing it).

relation = ops_test.model.get_relation('relation_name')
assert[] == {'foo': 'bar'}

@ca-scribner might have some thoughts and useful tips about this?

I’ve taken a similar approach to @ppasotti, interrogating relation data in the unit tests/adding validation in the charm itself to make sure data is as expected, and then relying on the data exchange to work in the integration tests. I think it covers all cases, but now I’m less convinced.

@pengale do you have any examples where interrogating the relation data in integration charms was really helpful?

@pengale see it looks like fast_forward is getting a promotion!

1 Like