I’ve done a poor man’s version of this for a while and its really helpful. I might do tox -e integration -- --model my-model once, and if it fails on a test I’d rerun it with tox -e integration -- -k "not build_and_deploy" or tox -e integration -- -k "test_that_failed". But using markers with a standard nomenclature is a much nicer way, especially when setup occurs across several tests. If I could find the time, I’d love to put these markers in all the Kubeflow charms
I’ve wanted to do this too for cases where a bundle can benefit from a charm’s tests. It would be nice for a bundle’s tests to be able to do:
- deploy bundle
- git clone charm-in-bundle
cd charm-in-bundle; tox -e integration -- -k "not setup"# ← test the bundle using a charm’s tests
That could help spot configuration errors that bundles introduce to break charms without literally duplicating tests unnecessarily