I think what you’re trying to run is an integration test (at least we call it that). The CI should set up the infrastructure, but what we usually do is delegate the test running to a specialized framework (usually the stack is tox+pytest+python-libjuju).
There are some pointers about integration testing and how to link them with CI here