Charmcraft release 1.5.0 – Call for testing!

Charmcraft supports the development of operators using the Charmed Operator Framework. It enables developers to build, upload and release operators on Charmhub and collaboration through charm libraries.

This new version was just released to candidate channel, we’d appreciate any tests on this, please report here any issues you find. Once we find this is solid enough, we will release it to stable.

It includes the following changes:

  • Added the --name option in the upload command to override the name declared in the metadata

  • Improved logging with details of the system where it’s being run (operating system, version, desktop, etc) for better debugging

  • Fixed the Framework linter to use the name from metadata in reactive charms

  • Added instructions to install Charmcraft from PyPI.

  • Fixed the release command to allow resources with revision 0

  • Fixed the process execution inside the isolated container so logs can be properly obtained from it

  • Reworked how Craft Store errors are handled to expose failures correctly

  • Backward compatibility notices (check the Charmcraft deprecations documentation for more info):

    • Deprecated the build command, use pack instead (DN06)

    • Enforced DN01: do not support charmhub attributes with underscores anymore

    • Enforced DN02: the charmcraft.yaml is not longer optional for build command

  • Updated Craft Parts to version 1.2.0, relevant change:

    • performance improvement by not refreshing already installed snaps
  • Updated Craft Store to version 2.0.1, relevant changes:

    • raise an error when a keyring is not found to explicit that credentials could not be stored

    • do not allow overwriting existing credentials (to avoid the peril of leaving the system with wrong credentials stored)

    • fail properly when cannot parse correctly the credentials given through an environment variable

  • Fully adopted the Craft CLI library

Use Charmcraft to:

We recommend the Charmed Operator Framework for charm development, but Charmcraft will help you publish charms written in any language. The Charmed Operator Framework enables the development of operators in a simple and straightforward way, using standard Python structures to allow for clean, maintainable, and reusable code.

Happy charming!

2 Likes

I wonder aloud if anyone uses CICD to package and release their charms? I am on gitlab and wondering how to just… test for now… not release or deliver but perhaps one day.

Would love some to chime in if they use charmcraft and how so

I wonder aloud if anyone uses CICD to package and release their charms?

@emcp thanks for the question!

This is a thing that I happen to be working on. It’s not finished yet. My tox.ini is not too terrible thus far:

[testenv:integration]
description = Run integration tests
allowlist_externals = 
    charmcraft
deps =
     juju
     pytest
commands =
    charmcraft pack
    pytest -v --tb native --ignore={[vars]tst_dir}unit --log-cli-level=INFO -s {posargs}

I use pylibjuju to deploy the resulting charm. Here’s an excerpt from a basic “does this deploy?” smoke test:

    # ...
    from juju import jasyncio
    from juju.model import Model
    # ...
    CHARM_FILE="./rolling-ops_ubuntu-20.04-amd64.charm"
    # ...
    class TestSmoke(unittest.TestCase):
        @classmethod
        async def _deploy(cls):
            cls.model = Model()
            await cls.model.connect_current()

            try:
                app = await model.deploy(CHARM_FILE)
                await model.block_until(lambda: app.status == 'active')
            finally:
                await model.disconnect()

The plan is to drive all of this with github actions, which fits with how we currently run tests in the Operator Framework team.

This is all a WIP. Things may change a lot, and I’ll loop back here when it is done!

1 Like

Hey! You should check out the Github Workflows on Grafana K8s.

This repo shows how we lint, unit test, integration test, and then upload/release a charm to the edge channel on Charmhub :slight_smile:

1 Like

is this running for you in a Docker Container? I am on gitlab attempting to get snap to work but… seems quite challenging… perhaps I need to install charmcraft via some other manner?

Here I am trying to start the SNAPD … within a docker container …

`Removing intermediate container 1806b1c55edf` ` ---> 61d15d4462e4` `Step 6/9 : RUN systemctl start snapd.service` ` ---> Running in 6254724ad698` `System has not been booted with systemd as init system (PID 1). Can't operate.` `Failed to connect to bus: Host is down` `The command '/bin/sh -c systemctl start snapd.service' returned a non-zero code: 1 ` `Cleaning up project directory and file based variables  
  00:01
 ` `ERROR: Job failed: exit code 1`

https://github.com/canonical/grafana-k8s-operator/blob/main/.github/workflows/release-edge.yaml#L117

    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2
        with:
          fetch-depth: 0
      - name: Upload to edge channel
        uses: canonical/charmhub-upload-action@0.2.0

is that last uses part … a github “task” or a docker image name or…? thanks Jon

EDIT::

Ah I see I can try this other method according to the github docs

$ sudo apt install -y libffi-dev libapt-pkg-dev libssl-dev
$ python3 -m venv env
$ source env/bin/activate
(env)$ pip install https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/python-apt/2.2.1/python-apt_2.2.1.tar.xz
(env)$ pip install charmcraft

https://github.com/canonical/charmcraft

I have now published a MIT licensed setup with gitlab https://gitlab.com/jrgemcp-public/gitlab-cicd-docker/build-charms-docker

well , after thinking I’d solved my problem I hit the next snag… I will next attempt to see if I can use charmcraft build or pack without LXD … because I am in gitlab with docker based containers and I dont really want to go down the route of having to put LXD in a docker container do I? or is this something others do? Or is everyone just… leveraging custom runners?

output I saw so far when attempting to leverage the docker container build in the previous posts CICD

$ charmcraft version 1.5.0                                                                          ` `
$ charmcraft pack` `LXD is required, but not installed. 
Visit https://snapcraft.io/lxd for instructions on how to install the LXD snap for your distribution                       ` `
Full execution log: '/root/.cache/charmcraft/log/charmcraft-20220313-122414.568475.log'                                                                         ` `
Cleaning up project directory and file based variables  
  00:01
 ` `ERROR: Job failed: exit code 1`

EDIT: I added LXD via apt… no dice

$ charmcraft pack
Failed to query LXD version.
* Command that failed: 'lxd version'
* Command exit code: 1
* Command standard error output: "\nCommand '/usr/bin/lxd' requires the lxd snap to be installed.\nPlease install it with:\n\nsnap install lxd\n\n"   
Full execution log: '/root/.cache/charmcraft/log/charmcraft-20220313-123745.451517.log'                                                                        
Cleaning up project directory and file based variables 00:01
ERROR: Job failed: exit code 1

I tried doing

# Required for Charmcraft Pack
RUN apt install snapd -y

CMD systemctl status snapd.service

but to no avail

$ snap install lxd
error: cannot communicate with server: Post http://localhost/v2/snaps/lxd: dial unix /run/snapd.socket: connect: no such file or directory
Cleaning up project directory and file based variables 00:01
ERROR: Job failed: exit code 1

and

systemctl status snapd.service
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down
Cleaning up project directory and file based variables 00:00
ERROR: Job failed: exit code 1

EDIT:

I guess it’s time @erik-lonroth Deploy gitlab-runner using Charmhub - The Open Operator Collection lol

The “building inside a container” feature from Charmcraft is to provide isolation from your host and exact control of the building environment.

If you already have a container (on which you have control on what is installed), no need to launch another container inside it, you can just use the --destructive-mode option for the pack command so Charmcraft builds directly there.

2 Likes

Big Thank You @facundo ! that was it.

Using docker image sha256:7f4f35cfd9b5d4bdfb155546e29a7140d383b406d28bf862f3a5c0d4af7a3453 for registry.gitlab.com/jrgemcp-public/gitlab-cicd-docker/build-charms-docker:latest with digest registry.gitlab.com/jrgemcp-public/gitlab-cicd-docker/build-charms-docker@sha256:3238dbb9fd181087dac73684d9b02d641736ebe8148b46e138634e33aa072bf0 ...
$ charmcraft version
1.5.0                                                                          
$ charmcraft pack --destructive-mode
Created 'mahrio_ubuntu-20.04-amd64.charm'.                                     
Charms packed:                                                                 
    mahrio_ubuntu-20.04-amd64.charm                                            
Cleaning up project directory and file based variables 00:01
Job succeeded

I have multiuser systems on NFS homes. About 100 users.

Currently, no user can build charms in those systems. Neither can I.

The snapcraft snap does not like NFS.

The multiuser system does not like lxd.snap

How can we build charms?

Hello! Are you able to install Charmcraft from the snap? If no, what is the error you get? If yes, are you able to run charmcraft pack --verbose succesfully? If no, can you provide the logs? Thanks

There is no LXD in the host.

@erik-lonroth

install charmcraft without snap (see this docker container I made for gitlab)

https://gitlab.com/jrgemcp-public/gitlab-cicd-docker/build-charms-docker/-/blob/main/Dockerfile

RUN apt install python3 python3-pip libffi-dev libapt-pkg-dev libssl-dev -y

RUN pip3 install https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/python-apt/2.2.1/python-apt_2.2.1.tar.xz
RUN pip3 install charmcraft

RUN charmcraft version
RUN python3 --version

… then do

charmcraft pack --destructive-mode
1 Like

Two choices. You can install LXD, or run pack with the --destructive-mode option.

I gave things a spin. Everything worked well.

I’ve moved to running charmcraft pack from an Ubuntu Server machine sitting on my desk. Login to the store fails, because I don’t have the ubuntu-desktop packages installed. But the new version of charmcraft fails in a more useful way, which lead me to the docs outlining a fix :slight_smile:

Thank you for all your work on this!

2 Likes

Unfortunately this just add dependency to another container, but I guess I can skip the docker steps and perhaps @heitor can supply information about the systemd-nspawn ?

https://wiki.debian.org/nspawn

It seems this would be a path forward, especially for CI/CD but it would indeed introduce the destructive part of it. Perhaps @heitor is on to something here with systemd-nspawn?

Oh and I saw the PR for the desktop plug in Mattermost, thanx alot for looking at this its really in a dire need to help other communities that hasn’t bought in to snap, to be able to buy in to juju…

There is no problem of using destructive mode in CI/CD, as those containers are ephemeral (you should not use them in your daily computer or places where the installation it’s supposed to be long-lived).

1 Like

Totally, but I’m considering all those trying to build on a non-ubuntu client.

Think community here: “Is juju a Ubuntu community?”

If the answer to that is “yes” then I can understand the logic for just basically don’t give a rats about anything that is not included in the Ubuntu clients.

But if the answer is that is “no” - Juju should stand on its own merits. Then these things matter.

There are consequences of alienating other distros. Picking a technology for building charms that can reach other distros would benefit Juju.

1 Like

I just released 1.5.0 to the stable channel. And, as a novelty, I released it also to the very new track 1.x (which you could install from to not be upgraded from 1.x accidentally).

Current status:

$ snapcraft status charmcraft
Track    Arch     Channel    Version           Revision
latest   amd64    stable     1.5.0             834
                  candidate  1.5.0             834
                  beta       ↑                 ↑
                  edge       1.5.0+7.g51565d8  849
         arm64    stable     1.5.0             836
                  candidate  1.5.0             836
                  beta       ↑                 ↑
                  edge       1.5.0+7.g51565d8  852
         armhf    stable     1.5.0             837
                  candidate  1.5.0             837
                  beta       ↑                 ↑
                  edge       1.5.0+7.g51565d8  853
         ppc64el  stable     1.5.0             838
                  candidate  1.5.0             838
                  beta       ↑                 ↑
                  edge       1.5.0+7.g51565d8  851
         s390x    stable     1.5.0             835
                  candidate  1.5.0             835
                  beta       ↑                 ↑
                  edge       1.5.0+7.g51565d8  850
1.x      amd64    stable     1.5.0             834
                  candidate  ↑                 ↑
                  beta       ↑                 ↑
                  edge       ↑                 ↑
         arm64    stable     1.5.0             836
                  candidate  ↑                 ↑
                  beta       ↑                 ↑
                  edge       ↑                 ↑
         armhf    stable     1.5.0             837
                  candidate  ↑                 ↑
                  beta       ↑                 ↑
                  edge       ↑                 ↑
         ppc64el  stable     1.5.0             838
                  candidate  ↑                 ↑
                  beta       ↑                 ↑
                  edge       ↑                 ↑
         s390x    stable     1.5.0             835
                  candidate  ↑                 ↑
                  beta       ↑                 ↑
                  edge       ↑                 ↑
2 Likes