Layer Basic Errors

Hi folks

Been fiddling with the Bigdata charms today to try and update them as there are some required changes in there.

Anyway, with them updated, I try and kick them off on Xenial I’m seeing a lot of stuff like:

2020-09-29 15:14:03 DEBUG install   Installing build dependencies: finished with status 'error'
2020-09-29 15:14:03 DEBUG install   Complete output from command /var/lib/juju/agents/unit-namenode-5/.venv/bin/python3 -m pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-7q53pdh1 --no-warn-script-location --no-binary :none: --only-binary :none: --no-index --find-links wheelhouse -- setuptools>=42 wheel setuptools_scm[toml]>=3.4.1:
2020-09-29 15:14:03 DEBUG install   Looking in links: wheelhouse
2020-09-29 15:14:03 DEBUG install   Collecting setuptools>=42
2020-09-29 15:14:03 DEBUG install     Could not find a version that satisfies the requirement setuptools>=42 (from versions: scm-1.17.0, 41.6.0)
2020-09-29 15:14:03 DEBUG install   No matching distribution found for setuptools>=42

I’ve seen a few similar topics and also tried to use an older layer-basic and so on, but I’ve not yet found a winning combination. Any ideas?

I believe I’ve just hit the wheelhouse.txt lucky dip jackpot of:

path <=13.0.0
zipp < 2.0.0

Yeah, unfortunately these crop up from time to time because some requirement wants to pull in a newer version of setuptools, sometimes indirectly, which had hard compatibility breaks with older versions of pip which come from apt on older machines series. The latest one in layer-basic was netaddr and because of the way that the wheelhouse.txt files were processed with charm build, pinning it in the base layer didn’t get carried forward. The latest charm snap changed how the wheelhouse was built to address that, so make sure you’re using that, but you might still run into it due to requirements specified in other layers.

The new framework takes the approach of bundling the actual venv into the charm, rather than just the packages. This avoids the need to build the venv at install time but has the risk of making the charm architecture dependent if any binary / compiled packages are included (and also makes the charm itself larger). I’m not sure if there is something done to mitigate that risk, but it might be worth backporting the charmcraft behavior to charm build with a flag to choose which behavior you want.