As discussed in this PR there are some issues with the current pin-set for pip and setuptools in layer-basic. In essence, deploying a charm with dependencies built at deploy time currently fails on Python 3.7 systems.
A solution to the issue is to unpin pip and setuptools which allows a compatible combination of those tools to be distributed with the charm.
However this also mandates a change of the default value for the include_system_packages configuration option in layer-basic from True to False.
Without this change a charm being deployed will end up using incompatible pip or setuptools versions from what is installed on the system and subsequently fail.
Having the charms rely on their distributed dependencies only, without interference from system installed packages, is a safer default. It also prohibits charm authors from inadvertently making boundary violations by having the charm rely on the software it is installing or managing on a system.
Most charms should be OK with this change, and charm authors still have the option to explicitly set include_system_packages to True to restore the old default behaviour.
To clarify, the expected symptom of being affected by this change is an unexpected ModuleNotFoundError or possibly an ImportError and will typically happen if you are relying on Python libraries installed via apt.
To get the old value back, you can just add this into your layer.yaml file:
options:
basic:
include_system_packages: true
Ideally, all charms would move to the safer new default, but currently apt-installed Python packages are one of the ways of working around the limitation of using the snap-packaged charm tool to build charms with wheelhouse entries which require some compilation during installation. (For reference, the build process uses pip download --no-binary to fetch the package sources and all relevant dependencies, but it seems that internally pip implements this with a dummy install rather than a plain download, causing unwanted compilation during build.)
