Deploying a charm with 2 network interfaces

Hello, I am trying to deploy a machine charm that requires 2 network interfaces using Juju on my local LXD cloud. I’m creating a new lxd network, and deploying the charm with this space:

lxc network create access --type=bridge ipv4.address=192.168.252.1/24 
juju add-space access 192.168.252.0/24
juju deploy sdcore-upf --constraints spaces=access

However I am getting consistent issues from Juju and LXD because both network interfaces “are connected to same network”. I’m not sure I understand what exactly is the issue here and how I should go ahead.

Logs

LXC

INFO   [2024-02-12T16:49:48-05:00] Creating instance                             ephemeral=false instance=juju-e42af0-7 instanceType=container project=default
ERROR  [2024-02-12T16:49:48-05:00] Failed add validation for device, skipping add action  device=eth0 err="Instance DNS name \"juju-e42af0-7\" conflict between \"eth0\" and \"eth3\" because both are connected to same network" instance=juju-e42af0-7 instanceType=container project=default
DEBUG  [2024-02-12T16:49:48-05:00] Adding device                                 device=eth1 instance=juju-e42af0-7 instanceType=container project=default type=nic
DEBUG  [2024-02-12T16:49:48-05:00] Adding device                                 device=eth2 instance=juju-e42af0-7 instanceType=container project=default type=nic

Juju

controller-0: 18:04:37 WARNING juju.worker.provisioner machine 15 failed to start in availability zone potiron: Failed start validation for device "eth0": Instance DNS name "juju-e42af0-15" conflict between "eth0" and "eth2" because both are connected to same network

The error is coming from LXD, when calling the checkAddressConflict() method. The comment for that method says:

// checkAddressConflict checks for conflicting IP/MAC addresses on another NIC connected to same network.
// Can only validate this when the instance is supplied (and not doing profile validation).
// Returns api.StatusError with status code set to http.StatusConflict if conflicting address found.

It seems that the bridge that is being created is conflicting with the NIC on the LXD instance spun up to deploy the charm. I’m wondering why you might be creating a new bridge instead of enrolling the existing subnets with which LXD has been configured into a space. Using the space constraint as you have done should then ensure that the provisioned instance comes up with a NIC in a subnet belonging to that space. There may be some LXD requirement I’m not aware of though, as I’ve not done much with spaces on LXD.

I didn’t know any better, I haven’t found a guide anywhere that explains deploying a charm with multiple network interfaces with LXD. How can I “enroll the existing subnets with which LXD has been configured into a space.”?

@gruyaume what subnets do you see when you list them from juju after adding the network in lxd? you could then move the subnets needed to the space you are going to use as constraint Juju | How to manage subnets

Here’s some doc

https://juju.is/docs/juju/manage-spaces

Juju keeps track of what subnets are available and you can use the subnets command to see them

$ juju subnets
subnets:
  10.0.1.0/24:
    type: ipv4
    provider-id: subnet-lxcbr0-10.0.1.0/24
    provider-network-id: net-lxcbr0
    status: in-use
    space: alpha
    zones:
    - myhost
  10.210.128.0/24:
    type: ipv4
    provider-id: subnet-lxdbr0-10.210.128.0/24
    provider-network-id: net-lxdbr0
    status: in-use
    space: alpha
    zones:
    - myhost
  fd42:110c:48bb:a229::/64:
    type: ipv6
    provider-id: subnet-lxdbr0-fd42:110c:48bb:a229::/64
    provider-network-id: net-lxdbr0
    status: in-use
    space: alpha
    zones:
    - myhost

By default, all subnets are in the default space “alpha”.

You can then add subnets to a space. eg I’ll add the first 2 subnets to a new space:

$ juju add-space myspace 10.0.1.0/24 10.210.128.0/24
added space "myspace" with subnets 10.0.1.0/24, 10.210.128.0/24
$ juju subnets
subnets:
  10.0.1.0/24:
    type: ipv4
    provider-id: subnet-lxcbr0-10.0.1.0/24
    provider-network-id: net-lxcbr0
    status: in-use
    space: myspace
    zones:
    - myhost
  10.210.128.0/24:
    type: ipv4
    provider-id: subnet-lxdbr0-10.210.128.0/24
    provider-network-id: net-lxdbr0
    status: in-use
    space: myspace
    zones:
    - myhost
  fd42:110c:48bb:a229::/64:
    type: ipv6
    provider-id: subnet-lxdbr0-fd42:110c:48bb:a229::/64
    provider-network-id: net-lxdbr0
    status: in-use
    space: alpha
    zones:
    - myhost

@nvinuesa After the network is created in lxd, I have the following networks:

guillaume@potiron:~$ lxc network list
+--------+----------+---------+------------------+---------------------------+------------------------------+---------+---------+
|  NAME  |   TYPE   | MANAGED |       IPV4       |           IPV6            |         DESCRIPTION          | USED BY |  STATE  |
+--------+----------+---------+------------------+---------------------------+------------------------------+---------+---------+
| access | bridge   | YES     | 192.168.252.1/24 | fd42:5e03:3e68:286a::1/64 |                              | 1       | CREATED |
+--------+----------+---------+------------------+---------------------------+------------------------------+---------+---------+
| core   | bridge   | YES     | 192.168.250.1/24 | fd42:419f:54b3:3393::1/64 |                              | 1       | CREATED |
+--------+----------+---------+------------------+---------------------------+------------------------------+---------+---------+
| enp3s0 | physical | NO      |                  |                           |                              | 0       |         |
+--------+----------+---------+------------------+---------------------------+------------------------------+---------+---------+
| lxdbr0 | bridge   | YES     | 10.191.126.1/24  | fd42:4a2:1eba:e511::1/64  |                              | 41      | CREATED |
+--------+----------+---------+------------------+---------------------------+------------------------------+---------+---------+
| mpbr0  | bridge   | YES     | 10.166.2.1/24    | fd42:28ae:6b12:d23e::1/64 | Network bridge for Multipass | 1       | CREATED |
+--------+----------+---------+------------------+---------------------------+------------------------------+---------+---------+
| wlo1   | physical | NO      |                  |                           |                              | 0       |         |
+--------+----------+---------+------------------+---------------------------+------------------------------+---------+---------+

After adding the space, I see the following spaces:

guillaume@potiron:~$ juju spaces
Name    Space ID  Subnets                 
alpha   0         10.166.2.0/24           
                  10.191.126.0/24         
                  10.8.95.0/24            
                  172.17.0.0/16           
                  192.168.250.0/24        
                  fd42:28ae:6b12:d23e::/64
                  fd42:419f:54b3:3393::/64
                  fd42:4a2:1eba:e511::/64 
                  fd42:5e03:3e68:286a::/64
                  fd42:6ea5:9b33:b3cc::/64
                  fd42:754c:cabd:7d57::/64
                  fd42:9c23:2be2:3b2f::/64
                  fd42:9ca5:375e:e41f::/64
                  fd42:a67c:2c1b:68ec::/64
                  fd42:ea96:7382:faa4::/64
                  fd42:f73f:5e64:86ba::/64
access  14        192.168.252.0/24        
            

As well as the following subnets:

guillaume@potiron:~$ juju subnets
subnets:
  1.2.2.0/24:
    type: ipv4
    provider-id: subnet-access-1.2.2.0/24
    provider-network-id: net-access
    status: in-use
    space: alpha
    zones:
    - potiron
  10.8.95.0/24:
    type: ipv4
    provider-id: subnet-mpqemubr0-10.8.95.0/24
    provider-network-id: net-mpqemubr0
    status: in-use
    space: alpha
    zones:
    - potiron
  10.87.159.0/24:
    type: ipv4
    provider-id: subnet-core-10.87.159.0/24
    provider-network-id: net-core
    status: in-use
    space: alpha
    zones:
    - potiron
  10.166.2.0/24:
    type: ipv4
    provider-id: subnet-mpbr0-10.166.2.0/24
    provider-network-id: net-mpbr0
    status: in-use
    space: alpha
    zones:
    - potiron
  10.191.126.0/24:
    type: ipv4
    provider-id: subnet-lxdbr0-10.191.126.0/24
    provider-network-id: net-lxdbr0
    status: in-use
    space: alpha
    zones:
    - potiron
  172.17.0.0/16:
    type: ipv4
    provider-id: subnet-docker0-172.17.0.0/16
    provider-network-id: net-docker0
    status: in-use
    space: alpha
    zones:
    - potiron
  192.168.250.0/24:
    type: ipv4
    provider-id: subnet-core-192.168.250.0/24
    provider-network-id: net-core
    status: in-use
    space: alpha
    zones:
    - potiron
  192.168.251.0/24:
    type: ipv4
    provider-id: subnet-core-192.168.251.0/24
    provider-network-id: net-core
    status: in-use
    space: alpha
    zones:
    - potiron
  192.168.252.0/24:
    type: ipv4
    provider-id: subnet-access-192.168.252.0/24
    provider-network-id: net-access
    status: in-use
    space: access
    zones:
    - potiron
  fd42:4a2:1eba:e511::/64:
    type: ipv6
    provider-id: subnet-lxdbr0-fd42:4a2:1eba:e511::/64
    provider-network-id: net-lxdbr0
    status: in-use
    space: alpha
    zones:
    - potiron
  fd42:5e03:3e68:286a::/64:
    type: ipv6
    provider-id: subnet-access-fd42:5e03:3e68:286a::/64
    provider-network-id: net-access
    status: in-use
    space: alpha
    zones:
    - potiron
  fd42:6ea5:9b33:b3cc::/64:
    type: ipv6
    provider-id: subnet-core-fd42:6ea5:9b33:b3cc::/64
    provider-network-id: net-core
    status: in-use
    space: alpha
    zones:
    - potiron
  fd42:9c23:2be2:3b2f::/64:
    type: ipv6
    provider-id: subnet-access-fd42:9c23:2be2:3b2f::/64
    provider-network-id: net-access
    status: in-use
    space: alpha
    zones:
    - potiron
  fd42:9ca5:375e:e41f::/64:
    type: ipv6
    provider-id: subnet-access-fd42:9ca5:375e:e41f::/64
    provider-network-id: net-access
    status: in-use
    space: alpha
    zones:
    - potiron
  fd42:28ae:6b12:d23e::/64:
    type: ipv6
    provider-id: subnet-mpbr0-fd42:28ae:6b12:d23e::/64
    provider-network-id: net-mpbr0
    status: in-use
    space: alpha
    zones:
    - potiron
  fd42:419f:54b3:3393::/64:
    type: ipv6
    provider-id: subnet-core-fd42:419f:54b3:3393::/64
    provider-network-id: net-core
    status: in-use
    space: alpha
    zones:
    - potiron
  fd42:754c:cabd:7d57::/64:
    type: ipv6
    provider-id: subnet-access-fd42:754c:cabd:7d57::/64
    provider-network-id: net-access
    status: in-use
    space: alpha
    zones:
    - potiron
  fd42:a67c:2c1b:68ec::/64:
    type: ipv6
    provider-id: subnet-core-fd42:a67c:2c1b:68ec::/64
    provider-network-id: net-core
    status: in-use
    space: alpha
    zones:
    - potiron
  fd42:ea96:7382:faa4::/64:
    type: ipv6
    provider-id: subnet-access-fd42:ea96:7382:faa4::/64
    provider-network-id: net-access
    status: in-use
    space: alpha
    zones:
    - potiron
  fd42:f73f:5e64:86ba::/64:
    type: ipv6
    provider-id: subnet-access-fd42:f73f:5e64:86ba::/64
    provider-network-id: net-access
    status: in-use
    space: alpha
    zones:
    - potiron

@nvinuesa I also tried by creating the space and then adding the subnet to it in 2 steps but with the same results

juju add-space access
juju move-to-space access 192.168.250.0/24

@gruyaume just a thought: I see that you have both ipv4 and ipv6 subnets on lxd. Could you try

lxc network set lxdbr0 ipv6.address none 

? At least to check if it serves as a workaround

This looks like it may relate to: