Async Charm Events

charm
1

A user “Raymond” asked a good question today in Kubecon around asyncio in the operator framework.

The question is:

What's the underlying IO manager for the juju events? Is it possible to combine asyncio in the python operator framework?

Thanks for the question Raymond and I am going to ask @jameinel to provide the correct answer.

Cheers
tlm

2

Thanks for asking. The Operator framework doesn’t use asyncio for how it handles io and events.

This is primarily because the programming model of charms is a lot of short-lived processes, rather than one long-lived in memory process with lots of asynchronous tasks. (Each event from Juju is actually a separate invocation of your charm script.)
We have talked about designs that would have a longer lived process that exchanges messages with the juju agent. There are two nice properties that we’d like to preserve with any such system:

  • You can write relatively linear code which is easier to rationalize about. The actual cpu/data overhead of operators is generally quite low, so you don’t need to pay the mental multiprocessing tax to squeeze out the last performance of the underlying system.
  • You know that if the process would die or be restarted, its state is appropriately tracked with persistent storage.

If you do have particular interest or concerns, it would be good to hear from you to make sure we are addressing your use case.

4 Likes
3

Thanks for your reply! I’m Raymond that raise this question in the event, thanks for bringing it up.

May I further ask how is the underlying event injected into the python runtime? Is the runtime invoked on every event injected, or is it standing by alongside the charm?

2 Likes
4

A new Python execution is run on every charm hook. Technically Juju runs exec dispatch which then runs python3 src/charm.py with particular environment variables set so that the charm can know what hook is being executed. And then the charm introspects its environment by calling out to various hook tools (config-get, relation-get, etc).

1 Like
5

Thanks for the help.

Are the events guaranteed to be fired up between two python invocation?
Since python(3 specifically) has a impactful startup time compared to other languages(go/rust to my experience), I’m curious what will happen if a juju event is not handled “fast” enough? Will it be re-scheduled if two event is within the same invocation window?

6

The actual event logic is simply a check of ‘current state’ vs ‘actual state’. For example, if you do juju config foo=bar; juju config foo=baz, the unit agent will notice that config has changed, but if it hasn’t sent foo=bar, it skips it and just triggers foo=baz. So events can be coalesced but all types of events will be triggered.

Put a different way, the charm will always be informed if the last state it evaluated is different from the current state, but if the state changes multiple times it will just get informed about the latest state.

Note also that the transactional integrity means that while a charm is evaluating a hook, we won’t give it new data. Imagine that a charm is evaluating relation-changed, and while that is running you run ‘juju config’. The charm won’t be able to see the new config during the relation-changed hook, but will run a config-changed hook and then will see the new data. This is so that hooks don’t have to worry that data they read at the beginning of a hook might be different at the end of the hook. (Think MVCC so you don’t have to worry about reading an inconsistent view during a single hook.)

7

Got it! The MVCC example clear things for me. Considering the fired events won’t be that frequent like a message broker, I think the approach is understandable.