Juju provisioner - failed to aquire node

1

Trying to deploy an openstack bundle that i wrote i am receiving the following error

controller-2: 07:23:52 WARNING juju.worker.provisioner failed to start machine 62 (failed to acquire node: unexpected: ServerError: 400 Bad Request ({"interfaces": ["Invalid label name: '-1' (Must begin with an alphanumeric character, and include only alphanumeric characters, dashes, and underscores.)"]})), retrying in 10s (3 more attempts)

Interfaces do also look ok.

I am selecting the machine using a tag and the name is ok. Is there any chance to increase the logging in order to understand what is wrong ?

###########

juju --version 2.7.6-focal-amd64

juju show-controller xxxxxx: details: uuid: xxxxxxxx controller-uuid: xxxxxxxxxx cloud: maas region: default agent-version: 2.8.7 agent-git-commit: ee2cfeb2c8c716af763a011e184ddea879c0985d controller-model-version: 2.8.7 mongo-version: 3.6.8

###########

2

Hi

Can you attach the bundle that is causing the problem (sanitized as needed). It’s probably just a typo, but it would be good to see it to confirm it.

Thanks

3

There you go

local_overlay_enabled: true series: focal

variables: openstack-origin: &openstack-origin cloud:focal-victoria data-port: &data-port br-ex:enp1s0 worker-multiplier: &worker-multiplier 0.25 osd-devices: &osd-devices /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf expected-osd-count: &expected-osd-count 6 expected-mon-count: &expected-mon-count 6 maas_url: &maas_url http://x.x.x.23:5240/MAAS/api/2.0 maas_credentials: &maas_credentials zone: &zone us-west-1 series: &series focal machines: ‘1’: constraints: arch=amd64 tags=ceph01 zones=us-west-1 series: *series ‘2’: constraints: arch=amd64 tags=ceph02 zones=us-west-1 series: *series ‘3’: constraints: arch=amd64 tags=ceph03 zones=us-west-1 series: *series ‘4’: constraints: arch=amd64 tags=ceph04 zones=us-west-1 series: *series ‘5’: constraints: arch=amd64 tags=ceph05 zones=us-west-1 series: *series ‘6’: constraints: arch=amd64 tags=ceph06 zones=us-west-1 series: *series ‘7’: constraints: arch=amd64 tags=controller01 zones=us-west-1 series: *series ‘8’: constraints: arch=amd64 tags=controller02 zones=us-west-1 series: *series ‘9’: constraints: arch=amd64 tags=controller03 zones=us-west-1 series: *series ‘10’: constraints: arch=amd64 tags=compute01 zones=us-west-1 series: *series ‘11’: constraints: arch=amd64 tags=compute02 zones=us-west-1 series: *series ‘12’: constraints: arch=amd64 tags=compute03 zones=us-west-1 series: *series ‘13’: constraints: arch=amd64 tags=compute04 zones=us-west-1 series: *series

relations:

CEPH

Ceph radosgw

    • ceph-radosgw:mon
    • ceph-mon:radosgw
    • ceph-radosgw:identity-service
    • keystone:identity-service

Ceph mon

    • ceph-mon:client
    • nova-compute:ceph
    • ceph-mon:client
    • cinder-ceph:ceph
    • ceph-mon:client
    • glance:ceph

Ceph osd

    • ceph-osd:mon
    • ceph-mon:osd

Nova compute

    • nova-compute:amqp
    • rabbitmq-server:amqp
    • nova-compute:image-service
    • glance:image-service
    • nova-compute:ceph-access
    • cinder-ceph:ceph-access

Ovn Central

    • ovn-central:certificates
    • vault:certificates
    • ovn-central:ovsdb-cms
    • neutron-api-plugin-ovn:ovsdb-cms

Ovn Chassis

    • ovn-chassis
    • nova-compute
    • ovn-chassis:certificates
    • vault:certificates
    • ovn-chassis:ovsdb
    • ovn-central:ovsdb

Neutron api

    • hacluster-neutron:ha
    • neutron-api:ha
    • neutron-api:identity-service
    • keystone:identity-service
    • neutron-api:shared-db
    • neutron-mysql-router:shared-db
    • neutron-api:amqp
    • rabbitmq-server:amqp
    • neutron-mysql-router:db-router
    • mysql-innodb-cluster:db-router
    • neutron-api-plugin-ovn:neutron-plugin
    • neutron-api:neutron-plugin-api-subordinate

Keystone

    • hacluster-keystone:ha
    • keystone:ha
    • keystone:shared-db
    • keystone-mysql-router:shared-db
    • keystone-mysql-router:db-router
    • mysql-innodb-cluster:db-router

Glance

    • hacluster-glance:ha
    • glance:ha
    • glance:identity-service
    • keystone:identity-service
    • glance:shared-db
    • glance-mysql-router:shared-db
    • glance:amqp
    • rabbitmq-server:amqp
    • glance-mysql-router:db-router
    • mysql-innodb-cluster:db-router

Nova Cloud Controller

    • hacluster-nova:ha
    • nova-cloud-controller:ha
    • nova-cloud-controller:identity-service
    • keystone:identity-service
    • nova-cloud-controller:image-service
    • glance:image-service
    • nova-cloud-controller:cloud-compute
    • nova-compute:cloud-compute
    • nova-cloud-controller:amqp
    • rabbitmq-server:amqp
    • nova-cloud-controller:shared-db
    • nova-mysql-router:shared-db
    • nova-cloud-controller:neutron-api
    • neutron-api:neutron-api
    • nova-mysql-router:db-router
    • mysql-innodb-cluster:db-router
    • nova-cloud-controller:memcache
    • memcached

Cinder

    • hacluster-cinder:ha
    • cinder:ha
    • cinder:image-service
    • glance:image-service
    • cinder:amqp
    • rabbitmq-server:amqp
    • cinder:identity-service
    • keystone:identity-service
    • cinder:cinder-volume-service
    • nova-cloud-controller:cinder-volume-service
    • cinder-ceph:storage-backend
    • cinder:storage-backend
    • cinder:shared-db
    • cinder-mysql-router:shared-db
    • cinder-mysql-router:db-router
    • mysql-innodb-cluster:db-router

Dashboard

    • hacluster-horizon:ha
    • openstack-dashboard:ha
    • openstack-dashboard:identity-service
    • keystone:identity-service
    • openstack-dashboard:shared-db
    • dashboard-mysql-router:shared-db
    • dashboard-mysql-router:db-router
    • mysql-innodb-cluster:db-router

NTP

    • ntp:juju-info
    • nova-compute:juju-info

# Ceilometer

- - ceilometer:ha

- hacluster-ceilometer:ha

- - ceilometer-agent

- nova-compute

- - ceilometer:amqp

- rabbitmq-server

- - ceilometer:identity-credentials

- keystone:identity-credentials

- - ceilometer

- keystone:identity-notifications

- - ceilometer:ceilometer-service

- ceilometer-agent:ceilometer-service

# Heat

- - heat:shared-db

- mysql:shared-db

- - heat:identity-service

- keystone:identity-service

- - heat:ha

- hacluster-heat:ha

- - heat:amqp

- rabbitmq-server:amqp

# Gnocchi

- - gnocchi:shared-db

- mysql:shared-db

- - gnocchi:identity-service

- keystone:identity-service

- - gnocchi:ha

- hacluster-gnocchi:ha

- - gnocchi:storage-ceph

- ceph-mon

- - gnocchi

- ceilometer

- - gnocchi:coordinator-memcached

- memcached

# Barbican

- - barbican:shared-db

- mysql:shared-db

- - barbican:ha

- hacluster-barbican:ha

- - barbican:identity-service

- keystone:identity-service

- - barbican:amqp

- rabbitmq-server:amqp

Placement

    • hacluster-placement:ha
    • placement:ha
    • placement:shared-db
    • placement-mysql-router:shared-db
    • placement:identity-service
    • keystone:identity-service
    • placement:placement
    • nova-cloud-controller:placement
    • placement-mysql-router:db-router
    • mysql-innodb-cluster:db-router

Vault

    • hacluster-vault:ha
    • vault:ha
    • vault:shared-db
    • vault-mysql-router:shared-db
    • vault-mysql-router:db-router
    • mysql-innodb-cluster:db-router
    • vault:certificates
    • neutron-api-plugin-ovn:certificates
    • vault:certificates
    • cinder:certificates
    • vault:certificates
    • glance:certificates
    • vault:certificates
    • keystone:certificates
    • vault:certificates
    • nova-cloud-controller:certificates
    • vault:certificates
    • openstack-dashboard:certificates
    • etcd:db
    • vault:etcd
    • etcd:certificates
    • easyrsa:client

services: hacluster-horizon: annotations: “gui-x”: “629.4802856445312” “gui-y”: “-187.46556091308594” charm: cs:focal/hacluster-72 options: maas_source: ppa:maas/2.9 corosync_key: | cluster_count: 3 maas_url: *maas_url maas_credentials: *maas_credentials

hacluster-cinder: annotations: “gui-x”: “1113.6179809570312” “gui-y”: “-182.85367584228516” charm: cs:focal/hacluster-72 options: maas_source: ppa:maas/2.9 corosync_key: | cluster_count: 3 maas_url: *maas_url maas_credentials: *maas_credentials

hacluster-keystone: annotations: “gui-x”: “364.57623291015625” “gui-y”: “-183.88646697998047” charm: cs:focal/hacluster-72 options: maas_source: ppa:maas/2.9 corosync_key: | cluster_count: 3 maas_url: *maas_url maas_credentials: *maas_credentials

hacluster-nova: annotations: “gui-x”: “-500.8421058654785” “gui-y”: “-188.8864517211914” charm: cs:focal/hacluster-72 options: maas_source: ppa:maas/2.9 corosync_key: | cluster_count: 3 maas_url: *maas_url maas_credentials: *maas_credentials

hacluster-glance: annotations: “gui-x”: “125.77049398422241” “gui-y”: “-182.24116134643555” charm: cs:focal/hacluster-72 options: maas_source: ppa:maas/2.9 corosync_key: | cluster_count: 3 maas_url: *maas_url maas_credentials: *maas_credentials

hacluster-neutron: annotations: “gui-x”: “-244.68324279785156” “gui-y”: “-180.88827514648438” charm: cs:focal/hacluster-72 options: maas_source: ppa:maas/2.9 corosync_key: | cluster_count: 3 maas_url: *maas_url maas_credentials: *maas_credentials

hacluster-vault: annotations: “gui-x”: “-244.68324279785156” “gui-y”: “-180.88827514648438” charm: cs:focal/hacluster-72 options: maas_source: ppa:maas/2.9 corosync_key: | cluster_count: 3 maas_url: *maas_url maas_credentials: *maas_credentials

hacluster-placement: annotations: “gui-x”: “-244.68324279785156” “gui-y”: “-180.88827514648438” charm: cs:focal/hacluster-72 options: maas_source: ppa:maas/2.9 corosync_key: | cluster_count: 3 maas_url: *maas_url maas_credentials: *maas_credentials

ceph-mon: annotations: “gui-x”: “-31.47058868408203” “gui-y”: “1241.4407653808594” charm: cs:focal/ceph-mon num_units: 6 options: expected-osd-count: *expected-osd-count monitor-count: *expected-mon-count #fsid: 51f54ee0-829b-11e7-89d3-cf1d259b9b2c monitor-secret: to: - ‘1’ - ‘2’ - ‘3’ - ‘4’ - ‘5’ - ‘6’

ceph-osd: annotations: “gui-x”: “-350.8948059082031” “gui-y”: “1241.34326171875” charm: cs:focal/ceph-osd num_units: 6 options: osd-devices: *osd-devices #osd-reformat: ‘yes’ to: - ‘1’ - ‘2’ - ‘3’ - ‘4’ - ‘5’ - ‘6’

ceph-radosgw: annotations: “gui-x”: “330.8906555175781” “gui-y”: “1242.5653076171875” charm: cs:focal/ceph-radosgw num_units: 3 # options: # use-embedded-webserver: true to: - lxd:1 - lxd:2 - lxd:3

cinder: annotations: “gui-x”: “1079.5557861328125” “gui-y”: “354.4442138671875” charm: cs:focal/cinder num_units: 3 options: openstack-origin: *openstack-origin dns-ha: true os-internal-network: x.x.x.0/24 os-public-hostname: “cinder..com” os-internal-hostname: “cinder..internal” os-admin-hostname: “cinder..internal” block-device: None glance-api-version: 2 worker-multiplier: *worker-multiplier to: - lxd:7 - lxd:8 - lxd:9

cinder-ceph: annotations: “gui-x”: “1110.9674072265625” “gui-y”: “707.72314453125” charm: cs:focal/cinder-ceph num_units: 0

cinder-mysql-router: annotations: gui-x: ‘900’ gui-y: ‘1400’ charm: cs:focal/mysql-router num_units: 0

glance-mysql-router: annotations: gui-x: ‘-290’ gui-y: ‘1400’ charm: cs:focal/mysql-router num_units: 0

glance: annotations: “gui-x”: “91.16439819335938” “gui-y”: “93.23036193847656” charm: cs:focal/glance options: dns-ha: true openstack-origin: *openstack-origin os-internal-network: x.x.x.0/24 os-public-hostname: “glance..com” os-internal-hostname: “glance..internal” os-admin-hostname: “glance..internal” num_units: 3 to: - lxd:7 - lxd:8 - lxd:9

keystone-mysql-router: annotations: gui-x: ‘230’ gui-y: ‘1400’ charm: cs:focal/mysql-router num_units: 0

keystone: annotations: “gui-x”: “336.1510925292969” “gui-y”: “87.35755920410156” charm: cs:focal/keystone num_units: 3 options: openstack-origin: *openstack-origin admin-user: admin-password: dns-ha: true os-internal-network: x.x.x.0/24 os-public-hostname: “keystone..com” os-internal-hostname: “keystone..internal” os-admin-hostname: “keystone..internal” to: - lxd:7 - lxd:8 - lxd:9

mysql-innodb-cluster: annotations: “gui-x”: “169.0853271484375” “gui-y”: “669.0663452148438” charm: cs:focal/mysql-innodb-cluster num_units: 3 options: innodb-buffer-pool-size: 512M max-connections: 50000 os-db-router-hostname: “dbserver..internal” to: - lxd:7 - lxd:8 - lxd:9

neutron-mysql-router: annotations: gui-x: ‘505’ gui-y: ‘1385’ charm: cs:focal/mysql-router num_units: 0

neutron-api-plugin-ovn: annotations: gui-x: ‘690’ gui-y: ‘1385’ charm: cs:focal/neutron-api-plugin-ovn

neutron-api: annotations: “gui-x”: “-276.74143981933594” “gui-y”: “97.98399353027344” charm: cs:focal/neutron-api num_units: 3 options: flat-network-providers: physnet1 neutron-security-groups: true openstack-origin: *openstack-origin worker-multiplier: *worker-multiplier dns-ha: true os-internal-network: x.x.x.0/24 os-public-hostname: “neutron-api..com” os-internal-hostname: “neutron-api..internal” os-admin-hostname: “neutron-api..internal” to: - lxd:7 - lxd:8 - lxd:9

ovn-central: annotations: gui-x: ‘70’ gui-y: ‘1550’ charm: cs:focal/ovn-central num_units: 3 to: - lxd:7 - lxd:8 - lxd:9

ovn-chassis: annotations: gui-x: ‘120’ gui-y: ‘1030’ charm: cs:focal/ovn-chassis num_units: 0 options: ovn-bridge-mappings: physnet1:br-ex bridge-interface-mappings: *data-port

nova-mysql-router: annotations: gui-x: ‘-30’ gui-y: ‘1385’ charm: cs:focal/mysql-router num_units: 0

nova-cloud-controller: annotations: “gui-x”: “-523.3897094726562” “gui-y”: “97.57771301269531” charm: cs:focal/nova-cloud-controller num_units: 3 options: network-manager: Neutron openstack-origin: *openstack-origin worker-multiplier: *worker-multiplier console-access-protocol: novnc console-proxy-ip: “cloud..com” #config-flags: “ssl_only=true,novncproxy_host=0.0.0.0,novncproxy_port=6080,novncproxy_base_url=https://cloud..com:6080/vnc_auto.html” config-flags: “novncproxy_host=0.0.0.0,novncproxy_port=6080,novncproxy_base_url=https://cloud..com:6080/vnc_auto.html” dns-ha: true os-internal-network: x.x.x.0/24 os-public-hostname: “nova..com” os-internal-hostname: “nova..internal” os-admin-hostname: “nova..internal” cpu-allocation-ratio: 2.5 ram-allocation-ratio: 1.5 to: - lxd:7 - lxd:8 - lxd:9

placement-mysql-router: annotations: gui-x: ‘1320’ gui-y: ‘1385’ charm: cs:focal/mysql-router num_units: 0

placement: annotations: “gui-x”: “-523.3897094726562” “gui-y”: “97.57771301269531” charm: cs:focal/placement num_units: 3 options: os-internal-network: x.x.x.0/24 os-public-hostname: “placement..com” os-internal-hostname: “placement..internal” os-admin-hostname: “placement..internal” openstack-origin: *openstack-origin to: - lxd:7 - lxd:8 - lxd:9

nova-compute: annotations: “gui-x”: “-1058.333251953125” “gui-y”: “328.2070007324219” charm: cs:focal/nova-compute num_units: 4 options: openstack-origin: *openstack-origin enable-live-migration: true enable-resize: true migration-auth-type: ssh config-flags: default_ephemeral_format=ext4 to: - ‘10’ - ‘11’ - ‘12’ - ‘13’

ntp: annotations: “gui-x”: “-1034.0950012207031” “gui-y”: “635.7695922851562” charm: cs:focal/ntp num_units: 0

dashboard-mysql-router: annotations: gui-x: ‘510’ gui-y: ‘1030’ charm: cs:focal/mysql-router num_units: 0

openstack-dashboard: annotations: “gui-x”: “599.20654296875” “gui-y”: “81.98934936523438” charm: cs:focal/openstack-dashboard num_units: 3 options: openstack-origin: *openstack-origin neutron-network-vpn: true neutron-network-lb: true ubuntu-theme: “no” cinder-backup: true neutron-network-l3ha: true webroot: “/” dns-ha: true os-public-hostname: “sws..com” endpoint-type: internalURL #os-internal-hostname: “horizon..internal” #os-admin-hostname: “horizon..internal” to: - lxd:7 - lxd:8 - lxd:9

rabbitmq-server: annotations: “gui-x”: “-184” “gui-y”: “634” charm: cs:focal/rabbitmq-server num_units: 3 options: min-cluster-size: 3 to: - lxd:7 - lxd:8 - lxd:9

vault-mysql-router: annotations: gui-x: ‘1535’ gui-y: ‘1560’ charm: cs:focal/mysql-router num_units: 0

vault: annotations: gui-x: ‘1610’ gui-y: ‘1430’ charm: cs:vault num_units: 3 options: dns-ha-access-record: “vault..internal” hostname: “horizon..internal” to: - lxd:7 - lxd:8 - lxd:9

Memcached is used for nova-cloud-controller

memcached: annotations: “gui-x”: “1647.76416015625” “gui-y”: “82.0185546875” charm: cs:focal/memcached num_units: 3 options: repcached: False ## Repcached replication only can take place between two units. If cluster is > 2 then this should be false repcached_origin: ‘ppa:iacobs/memcached-repcached’ annotations: gui-x: “2528.00146484375” gui-y: “794.591796875” to: - lxd:7 - lxd:8 - lxd:9

Etcd is used for vault ha

etcd: annotations: “gui-x”: “1647.76416015625” “gui-y”: “82.0185546875” charm: cs:focal/etcd num_units: 3 annotations: gui-x: “2528.00146484375” gui-y: “794.591796875” to: - lxd:7 - lxd:8 - lxd:9

Etcd is used for vault ha

easyrsa: annotations: “gui-x”: “1647.76416015625” “gui-y”: “82.0185546875” charm: cs:~containers/easyrsa num_units: 1 annotations: gui-x: “2528.00146484375” gui-y: “794.591796875” to: - lxd:7

4

Unfortunately, I think the formatting has messed up the paste of the bundle (perhaps try it as a file). e.g. I couldn’t find machine 62 in the bundle anywhere. Maybe a bit more of the log to get context of what is going on?

5

There is no machine 62 cause it does not exist. The way the bundle will work is that it will find the appropriate machine with the tag and then assign the number. The numbers inside the bundle have nothing to do with the actual assigned number.

6

Yes, I aware of that, although allocating to machine tends to make things clearer. My point (badly made) is that there isn’t enough detail in what you’ve posted so far to work out what is going on. Perhaps if you could please attach/upload (not copy/paste into the message body) the bundle and the debug log from the deploy, and the juju status (text is easiest to read). (e.g. from juju status).

Is the bundle you posted above the one you used to deploy with? (e.g. juju deploy <some-bundle>)

  • Where did the bundle come from?
  • What version of juju are you using?
  • What/where is the controller bootstrapped to? (e.g. the version of MaaS = 2.9?)
  • Is there a bit more context from the log? (i.e. could you add the whole thing?)

Unfortunately, the formatting of the message above makes it very difficult to see what the bundle is actually saying (although I can get the gist of it).

I suspect that it something to do with the configuration of MaaS, but it’s hard to tell at the moment. The error is coming from within the MaaS provisioner (in validate_constraint_label_name(...)) so it’s what’s being handed to MaaS that is causing the problem.

Thanks.