Wheel building fails during charm deployment

framework
1

Hey,

we’re currently seeing the following error when deploying our charms with Juju 2.6.6:

unit-ams-0: 12:22:57 DEBUG unit.ams/0.install Building wheels for collected packages: setuptools, setuptools-scm
unit-ams-0: 12:22:57 DEBUG unit.ams/0.install   Building wheel for setuptools (setup.py): started
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install   Installing build dependencies: finished with status 'done'
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install   Getting requirements to build wheel: started
unit-ams-0: 12:22:58 DEBUG unit.ams/0.install   Building wheel for setuptools (setup.py): finished with status 'done'
unit-ams-0: 12:22:58 DEBUG unit.ams/0.install   Stored in directory: /root/.cache/pip/wheels/11/12/6a/298db0dc114fea1a87a48e949e0fedb521c5cd3389d0bf0ee1
unit-ams-0: 12:22:58 DEBUG unit.ams/0.install   Building wheel for setuptools-scm (setup.py): started
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install   Getting requirements to build wheel: finished with status 'error'
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install   ERROR: Complete output from command /var/lib/juju/agents/unit-lxd-0/.venv/bin/python3 /var/lib/juju/agents/unit-lxd-0/.venv/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpcxg_gmf5:
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install   ERROR: Traceback (most recent call last):
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install     File "/var/lib/juju/agents/unit-lxd-0/.venv/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 207, in <module>
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install       main()
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install     File "/var/lib/juju/agents/unit-lxd-0/.venv/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 197, in main
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install       json_out['return_val'] = hook(**hook_input['kwargs'])
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install     File "/var/lib/juju/agents/unit-lxd-0/.venv/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 48, in get_requires_for_build_wheel
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install       backend = _build_backend()
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install     File "/var/lib/juju/agents/unit-lxd-0/.venv/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 39, in _build_backend
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install       obj = getattr(obj, path_part)
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install   AttributeError: module 'setuptools.build_meta' has no attribute '__legacy__'
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install   ----------------------------------------
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install ERROR: Command "/var/lib/juju/agents/unit-lxd-0/.venv/bin/python3 /var/lib/juju/agents/unit-lxd-0/.venv/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpcxg_gmf5" failed with error code 1 in /tmp/pip-req-build-wvn4diz9
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install Traceback (most recent call last):
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install   File "/var/lib/juju/agents/unit-lxd-0/charm/hooks/install", line 8, in <module>
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install     basic.bootstrap_charm_deps()
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install   File "lib/charms/layer/basic.py", line 117, in bootstrap_charm_deps
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install     glob('wheelhouse/*'))
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install   File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install     raise CalledProcessError(retcode, cmd)
unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install subprocess.CalledProcessError: Command '['/var/lib/juju/agents/unit-lxd-0/.venv/bin/pip', 'install', '-U', '--no-index', '-f', 'wheelhouse', 'wheelhouse/Tempita-0.5.2.tar.gz', 'wheelhouse/Jinja2-2.10.1.tar.gz', 'wheelhouse/charms.reactive-1.2.1.tar.gz', 'wheelhouse/pip-19.1.1.tar.gz', 'wheelhouse/netaddr-0.7.19.tar.gz', 'wheelhouse/wheel-0.33.6.tar.gz', 'wheelhouse/charmhelpers-0.20.1.tar.gz', 'wheelhouse/setuptools_scm-1.17.0.tar.gz', 'wheelhouse/setuptools-41.1.0.zip', 'wheelhouse/MarkupSafe-1.1.1.tar.gz', 'wheelhouse/six-1.12.0.tar.gz', 'wheelhouse/PyYAML-5.1.2.tar.gz', 'wheelhouse/pyaml-19.4.1.tar.gz']' returned non-zero exit status 1.
unit-lxd-0: 12:22:58 ERROR juju.worker.uniter.operation hook "install" failed: exit status 1
unit-lxd-0: 12:22:58 INFO juju.worker.uniter awaiting error resolution for "install" hook

I haven’t found a bug related to this or I am sure this is a bug in juju/the charm tools and not in our charm.

Has anybody seen this error before?

Thanks!

regards,
Simon

2

We started running into this recently and it’s due to a newer version of pip having a conflict with an older version of setuptools which comes pre-installed on bionic and possibly other series. There was a recent change to layer:basic that changed the default value of include_system_packages to false which I believe should mitigate this bug (and should generally be safer for the charm, if not absolutely required for some reason), however I will also get a PR up to address this for the case where falling back to system packages outside of the venv is required.

3

Thanks @cory_fu!

We try if setting include_system_packages to false fixes the problem.

4

Thanks to https://github.com/juju-solutions/layer-basic/pull/145 things are now working again for us.