Failing to build postgresql charm for xenial

[mthaddon (Tom Haddon)]

Hi,

I’m hoping someone might be able to help me figure out why I can’t build the postgresql charm in a way that makes it deployable on xenial (the charm currently supports xenial, bionic and focal).

To try and simplify the problem, I created a xenial LXC, pulled down the charm, installed the charm snap, and then ran charm build --debug. As part of that I can see

utils: b'created virtual environment CPython3.6.9.final.0-64 in 423ms

In layer-basic we then have:

charmtools.build.tactics:   MarkupSafe<2.0.0;python_version < '3.6'
charmtools.build.tactics:   MarkupSafe<2.1.0;python_version == '3.6' # Just for python 3.6
charmtools.build.tactics:   MarkupSafe;python_version >= '3.7' # newer pythons

As a result, we’re pulling in MarkupSafe-2.0.1.tar.gz.

However, on a xenial unit we have python3.5 installed, and I see the charm install hook failing with:

subprocess.CalledProcessError: Command '['/var/lib/juju/agents/unit-postgresql-xenial-0/.venv/bin/pip', 'install', '-U', '--force-reinstall', '--no-index', '--no-cache-dir', '-f', 'wheelhouse', 'wheel==0.33.6', 'MarkupSafe==2.0.1']' returned non-zero exit status 1

Running that command manually I get:

root@juju-ce1bc5-1:/var/lib/juju/agents/unit-postgresql-xenial-0/charm# /var/lib/juju/agents/unit-postgresql-xenial-0/.venv/bin/pip install -U --force-reinstall --no-index --no-cache-dir -f wheelhouse MarkupSafe==2.0.1 wheel==0.33.6
Looking in links: wheelhouse
Collecting MarkupSafe==2.0.1
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-3a8gt00y/MarkupSafe/setup.py", line 61, in <module>
        run_setup(True)
      File "/tmp/pip-install-3a8gt00y/MarkupSafe/setup.py", line 44, in run_setup
        ext_modules=ext_modules if with_binary else [],
      File "/var/lib/juju/agents/unit-postgresql-xenial-0/.venv/lib/python3.5/site-packages/setuptools/__init__.py", line 145, in setup
        return distutils.core.setup(**attrs)
      File "/usr/lib/python3.5/distutils/core.py", line 121, in setup
        dist.parse_config_files()
      File "/var/lib/juju/agents/unit-postgresql-xenial-0/.venv/lib/python3.5/site-packages/setuptools/dist.py", line 701, in parse_config_files
        ignore_option_errors=ignore_option_errors)
      File "/var/lib/juju/agents/unit-postgresql-xenial-0/.venv/lib/python3.5/site-packages/setuptools/config.py", line 121, in parse_configuration
        meta.parse()
      File "/var/lib/juju/agents/unit-postgresql-xenial-0/.venv/lib/python3.5/site-packages/setuptools/config.py", line 426, in parse
        section_parser_method(section_options)
      File "/var/lib/juju/agents/unit-postgresql-xenial-0/.venv/lib/python3.5/site-packages/setuptools/config.py", line 399, in parse_section
        self[name] = value
      File "/var/lib/juju/agents/unit-postgresql-xenial-0/.venv/lib/python3.5/site-packages/setuptools/config.py", line 184, in __setitem__
        value = parser(value)
      File "/var/lib/juju/agents/unit-postgresql-xenial-0/.venv/lib/python3.5/site-packages/setuptools/config.py", line 514, in _parse_version
        version = self._parse_attr(value, self.package_dir)
      File "/var/lib/juju/agents/unit-postgresql-xenial-0/.venv/lib/python3.5/site-packages/setuptools/config.py", line 349, in _parse_attr
        module = import_module(module_name)
      File "/var/lib/juju/agents/unit-postgresql-xenial-0/.venv/lib/python3.5/importlib/__init__.py", line 126, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
      File "<frozen importlib._bootstrap>", line 986, in _gcd_import
      File "<frozen importlib._bootstrap>", line 969, in _find_and_load
      File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 661, in exec_module
      File "<frozen importlib._bootstrap_external>", line 767, in get_code
      File "<frozen importlib._bootstrap_external>", line 727, in source_to_code
      File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
      File "/tmp/pip-install-3a8gt00y/MarkupSafe/src/markupsafe/__init__.py", line 112
        return f"{self.__class__.__name__}({super().__repr__()})"
                                                                ^
    SyntaxError: invalid syntax

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-3a8gt00y/MarkupSafe/

And looking at https://pypi.org/project/MarkupSafe/2.0.1/ it does say " Requires: Python >=3.6".

Any suggestions for how to resolve this? We’d really rather not have to drop support for a series that’s still covered under ESM.

[emcp]

as I recall this is some sort of SetupTools incompatibility problem… I cannot found the exact post but I think @erik-lonroth was involved at some point

Layer Basic Errors

Yeah, unfortunately these crop up from time to time because some requirement wants to pull in a newer version of setuptools, sometimes indirectly, which had hard compatibility breaks with older versions of pip which come from apt on older machines series. The latest one in layer-basic was netaddr and because of the way that the wheelhouse.txt files were processed with charm build, pinning it in the base layer didn’t get carried forward. The latest charm snap changed how the wheelhouse was buil…

Wheel building fails during charm deployment

Hey, we’re currently seeing the following error when deploying our charms with Juju 2.6.6: unit-ams-0: 12:22:57 DEBUG unit.ams/0.install Building wheels for collected packages: setuptools, setuptools-scm unit-ams-0: 12:22:57 DEBUG unit.ams/0.install Building wheel for setuptools (setup.py): started unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install Installing build dependencies: finished with status 'done' unit-lxd-0: 12:22:58 DEBUG unit.lxd/0.install Getting requirements to build wheel: star…

EDIT: I think it’s this one

https://github.com/canonical/charmcraft/issues/738#issuecomment-1085571383

[erik-lonroth (Erik Lönroth)]

@mthaddon This is what I have done when building blows up because of setuptools. Perhaps it helps:

charmcraft.yaml

type: "charm"
bases:
  - build-on:
    - name: "ubuntu"
      channel: "20.04"
    run-on:
    - name: "ubuntu"
      channel: "20.04"

# This below is needed for charmhelpers 0.20.24 to build 
parts:
  charm:
    charm-python-packages: [setuptools < 58]
    prime:
      - templates/*

Not sure if you need prime.

https://github.com/erik78se/juju-operators-examples/blob/6a86158682b219e991ccc415c49eca4799ebc7b9/haproxy-relate/charmcraft.yaml

[mthaddon (Tom Haddon)]

Thanks for the replies. Just to follow up on this, my understanding from reading the quoted post from Cory about layer basic errors is that charmcraft should be bundling the entire venv to ensure things work when deployed.

As a result I also tried rebuilding the charm in a xenial LXC using the --destructive-mode to charmcraft (because charmcraft won’t let me specify a xenial build series). Doing this the charm built fine, but still failed in the same way on deployment.

I used a charmcraft.yaml as follows (just trying to get a version of the postgresql charm that can deploy on xenial, obviously if I get this working will also need bionic and focal too):

type: charm
bases:
  - build-on:
    - name: "ubuntu"
      channel: "16.04"
    run-on:
    - name: "ubuntu"
      channel: "16.04"
parts:
  charm:
    source: .
    plugin: reactive
    build-snaps: [charm]

@facundo has kindly said he’ll look into this next week, and I’ll post an update here based on what we find. It’s possible we’ll need to pin versions of python-packages as suggested with the charm-python-packages option, but I’d like to confirm with Facundo before making any changes there.

[mthaddon (Tom Haddon)]

@facundo has confirmed that Charmcraft installs all the needed libraries if you use the charm plugin, but since we’re using reactive, the charm tool is the one being used. Currently building a reactive charm with charmcraft is only possible with the reactive plugin, so there’s no difference between running charm build in a xenial LXC and running charmcraft pack --destructive-mode in a xenial LXC with a charmcraft.yaml file that uses the reactive plugin.

So this comes back to why charm build creates a python3.6 environment on to build in on xenial.

[mthaddon (Tom Haddon)]

It looks like charm build is using python3.6 because the snap includes that version.

I’ve tried spinning up a xenial LXC and installing the charm debian package instead, but then get the following when trying to build the postgresql charm:

$ charm build
build: Composing into /home/ubuntu/builds/
build: Destination charm directory: /home/ubuntu/builds/builds/postgresql
build: Unable to locate layer:basic. Do you need to set LAYER_PATH?

Will see if I can find anyone to help figure out what I should be setting that to.

[mthaddon (Tom Haddon)]

I’ve been able to do this here: https://code.launchpad.net/~mthaddon/postgresql-charm/+git/postgresql-charm/+merge/425160. So hopefully we’ll have a new version of the postgresql charm released soon.