Using juju, totally. The only dependency that really needs to be local to airflow is the spark-submit executable.
Right, use juju for the service discovery and charm code to handle writing out the correct config for each service. This seems dead on.