How to use LXD clustering with Juju

This page is dedicated to more advanced topics related to using LXD with Juju. The main page is Using LXD with Juju.

The topics presented here are:

  • Add resilience to your models through LXD clustering
  • Registering a remote LXD server as a LXD cloud

Add resilience to your models through LXD clustering

LXD clustering provides the ability for applications to be deployed in a high-availability manner. In a clustered LXD cloud, Juju will deploy units across its nodes.

Background

LXD clustering provides increased resilience in two senses for teams using Juju:

  • first, the LXD cloud itself is not exposed to a single point of failure
  • secondly, your model can be distributed across each node within the cluster. This can add resilience to individual applications that are deployed with multiple units

Forming a LXD cluster

The documentation provided by the LXD project explains the process of forming a LXD cluster.

A helpful tutorial video has also been provided by project lead Stéphane Graber:

Making use of a LXD cluster

From Juju’s point of view, a LXD cluster is a “remote LXD server”. Follow the instructions in the next section to register the cluster with Juju.

Registering a remote LXD server as a LXD cloud

Two commands enable you to register your LXD server with Juju as a cloud:

  • juju add-cloud provides the connectivity details to enable Juju to connect to the LXD server
  • juju add-credential provides the security credentials for Juju to use when connected

Adding the cloud

Option 1: Interactively

To add the remote LXD information to Juju, run juju add-cloud without arguments and follow the prompts:

juju add-cloud

An example session:

Cloud Types

  lxd
  maas
  manual
  openstack
  vsphere

Select cloud type: lxd

Enter a name for your lxd cloud: lxd-remote

Enter the API endpoint url for the remote LXD server: https://10.10.0.1:8443
Auth Types
  certificate

Enter region [default]: 

Enter the API endpoint url for the region [use cloud api url]: 

Enter another region? (y/N): n

Cloud "lxd-remote" successfully added

You will need to add credentials for this cloud (`juju add-credential lxd-remote`)
before creating a controller (`juju bootstrap lxd-remote`).

Option 2: Provide the cloud metadata via a file

Save the contents of the following YAML fragment to a file (/tmp/clouds.yaml), making the appropriate changes.

# clouds.yaml

clouds:
  lxd-remote: # replace lxd-remote with your preferred name
    type: lxd
    auth-types: [interactive, certificate]
    endpoint: https://10.10.0.1:8443/  # replace with the actual endpoint

Run juju add-cloud, specifying the correct cloud name and path to your new clouds.yaml:

juju add-cloud lxd-remote /tmp/clouds.yaml

Adding the security credential

Option 1: Interactively

To add the remote LXD information to Juju, run juju add-credential without arguments and follow the prompts:

juju add-credential

An example session:

Enter credential name: lxd-remote-creds

Auth Types
  certificate
  interactive

Select auth type [interactive]: 

Enter trust-password: *******

Loaded client cert/key from "/home/ubuntu/.local/share/juju/lxd"
Uploaded certificate to LXD server.

Credential "lxd-remote-creds" added locally for cloud "lxd-remote".

Option 2: Provide the credential via a file

Save the contents of the following YAML fragment to a file (/tmp/credentials.yaml), making the appropriate changes.

# credentials.yaml

credentials:
    lxd-remote:  # this name must match the name in clouds.yaml
        admin:
            auth-type: interactive
            trust-password: DL7UXEd8tsTF3Tc # replace with actual password

Run juju add-credential, specifying the correct cloud name and path to your new credentials.yaml:

juju add-credential lxd-remote -f /tmp/credentials.yaml

Next steps

Now that the cloud and credentials have been added the next step is to create a controller. See Creating a controller on the main LXD page.

lxd-profiles:

It is not clear from the documentation, where and how I would apply such an lxd-profile to a charm or bundle. Please provide a more specific example e.g. showing how I would add custom network bridges to certain subnets/network-spaces when trying to deploy cs:~openstack-charmers-next/bundle/ceph-base-bionic-luminous-3, which runs ceph-mon via lxd.

Hi there. Profiles are for charms (so not bundles). I edited the documentation. I admit that it wasn’t clear. Thanks for pointing that out.

Regards the lxd-profile.yaml, the doc suggests that there’s a list of items that are allowed to be in the file, but the list provided doesn’t include the example from neutron-openvswitch. Does this just need an update, or is there something I’m missing here? Clearly adding host specific stuff like NIC parent names etc. would be a bit daft, but I’m wondering if we can add things like security.nesting.

I can’t seem to get this information…

juju show-machine 
model: freeipa
machines:
  "1":
    juju-status:
      current: started
      since: 25 Mar 2021 22:02:01+01:00
      version: 2.8.8
    dns-name: 192.168.2.169
    ip-addresses:
    - 192.168.2.169
    instance-id: juju-321157-1
    machine-status:
      current: running
      message: Running
      since: 25 Mar 2021 22:01:27+01:00
    modification-status:
      current: idle
      since: 25 Mar 2021 22:00:37+01:00
    series: centos7
    network-interfaces:
      eth0:
        ip-addresses:
        - 192.168.2.169
        mac-address: 00:16:3e:f1:39:24
        gateway: 192.168.2.1
        is-up: true
    hardware: arch=amd64 cores=0 mem=0M

I’m trying to get juju to use a custom profile that installs some packages etc. but I can’t figure out the best way to do this except from the model-config “cloud-init-user-data”

Using cloud init user data is your best option right now. Unless you create new versions of all the charms and add an lxd-profile.yaml file. There are other hacky ways to do it, but I wouldn’t recommend them.

There is some minimal stuff needed to get juju to work on centos7 images (possibly also centos8?) which are:

  • install sudo package
  • install openssh-server and start it

The upstream images I’ve tested and modified are from here and perhaps I can propose changes to these images and thereby get closer to a “out of the box” experience with centos7/8 for juju:

I really would like to see at least the agents and instances be able to start without hacks to it all…