Using Charmcraft with very high UIDs

charmcraft
1

Problem background

Disclaimer: This may already have been documented somewhere but I haven’t found it. It was tested with Charmcraft 2.1.0 and LXD 5.6.

If your UID/GID is higher than 65535 you’re likely to have issues running charmcraft. This is very typical in enterprise deployments where your machine is using SSSD with an Active Directory backend. Me and @erik-lonroth have seen this many times. In this case, you likely have UID/GID at around 400 million:

$ getent passwd hallback
hallback:*:413906259:413906259:Johan Hallbäck:/home/hallback:/bin/bash

Furthermore, your user isn’t present in /etc/passwd.

Problem 1: charmcraft init can’t find your name in /etc/passwd

This one is simple, you just have to specify your full name using charmcraft init --author:

$ charmcraft init
Unable to automatically determine author's name, specify it with --author                                                                    
Full execution log: '/home/hallback/snap/charmcraft/common/cache/charmcraft/log/charmcraft-20221014-153404.969775.log'

$ charmcraft init --author "Johan Hallbäck"
Charmed operator package file and directory tree initialised.                                                                                
                                                                                                                                             
Now edit the following package files to provide fundamental charm metadata and other information:                                            
                                                                                                                                             
metadata.yaml                                                                                                                                
config.yaml                                                                                                                                  
src/charm.py                                                                                                                                 
README.md

Problem 2: charmcraft pack won’t work due to UID/GID ranges in LXD

Running charmcraft pack will probably end up like this:

$ charmcraft pack
Failed to launch instance 'charmcraft-myfirstcharm-9307267-0-0-amd64'.rchitectures=['amd64'] (may take a while the first time but it's reusable)                                                                                                                                    
* Command that failed: "lxc --project charmcraft launch craft-com.ubuntu.cloud-buildd:22.04 local:charmcraft-myfirstcharm-9307267-0-0-amd64 --config 'raw.idmap=both 413206259 0' --config security.syscalls.intercept.mknod=true"
* Command exit code: 1
* Command output: b'Creating charmcraft-myfirstcharm-9307267-0-0-amd64\n'
* Command standard error output: b'Error: Failed instance creation: Failed creating instance record: Failed initialising instance: Host id is in the range of subids\n'                                                                                                                                                                                                                                                                              
Full execution log: '/home/hallback/snap/charmcraft/common/cache/charmcraft/log/charmcraft-20221013-155608.626706.log'

The problem is that my UID on the host (above 400 million) is within the range of the container UIDs (above 65536). We also need to make sure that the size of the idmap is big enough for my 400 million UID, or else we’ll get charmcraft errors with lxc file push later on.

The fix is to modify the default profile of the charmcraft project in LXD.

This is how it can look by default:

$ lxc --project charmcraft profile list
+---------+---------------------+---------+
|  NAME   |     DESCRIPTION     | USED BY |
+---------+---------------------+---------+
| default | Default LXD profile | 1       |
+---------+---------------------+---------+

$ lxc --project charmcraft profile show default
config:
  boot.autostart: "false"
description: Default LXD profile
devices:
  eth0:
    name: eth0
    nictype: bridged
    parent: lxdbr0
    type: nic
  root:
    path: /
    pool: lxdstoragepool1
    type: disk
name: default
used_by:
- /1.0/instances/charmcraft-myfirstcharm-9307267-0-0-amd64?project=charmcraft

NOTE! If you have no project called charmcraft, run charmcraft pack once and let it fail first.

For all containers in the project charmcraft, let the container idmap start beyond your UID. I chose 500 million here, and made the size 500 million also:

$ lxc --project charmcraft profile set default security.idmap.base=500000000
$ lxc --project charmcraft profile set default security.idmap.size=500000000
$ lxc --project charmcraft profile set default security.idmap.isolated=true

The result should look like this:

$ lxc --project charmcraft profile show default
config:
  boot.autostart: "false"
  security.idmap.base: "500000000"
  security.idmap.isolated: "true"
  security.idmap.size: "500000000"
description: Default LXD profile
devices:
  eth0:
    name: eth0
    nictype: bridged
    parent: lxdbr0
    type: nic
  root:
    path: /
    pool: lxdstoragepool1
    type: disk
name: default
used_by:
- /1.0/instances/charmcraft-myfirstcharm-9307267-0-0-amd64?project=charmcraft

Now charmcraft pack should work just fine for users with high UIDs.

/Johan Hallbäck, Ibeo Automotive Systems GmbH

3 Likes
2

@hallback this is an amazing illustration that highlights the massive issues that anyone using a AD backend will run into trying to develop charms.

A vast amount of businesses which will never find it easy to produce a single charm because of the above.

As Johan describes, this situation is a massive issue for charmcraft or the set of tools used in conjunction with it.

I feel your pain Johan.

3

@hallback, thank you for the pointed and thorough explanation here. Super helpful!

1 Like