I want to build ha base on openstack-base-70 with juju commands as fellowed:
# rebuild keystone-hacluster
juju remove-unit keystone/0 --force --no-wait
juju remove-applicationg keystone --force --no-wait
juju deploy -n 3 --to lxd:0,lxd:1,lxd:2 --config vip=10.0.7.13 --series focal ./openstack-base-1/keystone --debug
juju deploy --config cluster_count=3 hacluster keystone-hacluster
juju add-relation keystone-hacluster:ha keystone:ha
# rebuild rabbitmq cluster
juju remove-unit rabbitmq-server/0 --force --no-wait
juju remove-applicationg rabbitmq-server --force --no-wait
juju deploy -n 3 --to lxd:0,lxd:1,lxd:2 --series focal ./openstack-base-1/ rabbitmq-server --debug
#rebuild vault ha-cluster
juju remove-unit vault/0 --force --no-wait
juju remove-application vault --force --no-wait
juju deploy -n 3 --to lxd:0,lxd:1,lxd:2 --config vip=10.0.7.21 --series focal vault --debug
juju deploy --config cluster_count=3 – series focal hacluster vault-hacluster
juju deploy -n 3 --to lxd:0,lxd:1,lxd:2 --series focal etcd
juju deploy --to lxd:0 --series focal cs:~containers/easyrsa
juju add-relation vault:ha vault-hacluster:ha
juju add-relation vault:shared-db mysql-innodb-cluster:shared-db
juju add-relation etcd:db vault:etcd
juju add-relation etcd:certificates easyrsa:client
then show the juju status:
juju status --relations
Model Controller Cloud/Region Version SLA Timestamp
openstack maas-controller mymaas/default 2.8.7 unsupported 16:14:36+08:00
App Version Status Scale Charm Store Rev OS Notes
ceph-mon 15.2.7 active 3 ceph-mon local 0 ubuntu
ceph-osd 15.2.7 active 3 ceph-osd local 15 ubuntu
ceph-radosgw 15.2.7 error 1 ceph-radosgw local 26 ubuntu exposed
cinder 16.2.0 active 1 cinder local 136 ubuntu
cinder-ceph 16.2.0 active 1 cinder-ceph local 2 ubuntu
cinder-mysql-router 8.0.22 active 1 mysql-router local 0 ubuntu
dashboard-mysql-router 8.0.22 active 1 mysql-router local 0 ubuntu
easyrsa 3.0.1 active 1 easyrsa jujucharms 342 ubuntu
etcd 3.4.5 active 3 etcd jujucharms 546 ubuntu
glance 20.0.1 active 1 glance local 150 ubuntu
glance-mysql-router 8.0.22 active 1 mysql-router local 0 ubuntu
keystone 17.0.0 active 3 keystone jujucharms 317 ubuntu
keystone-hacluster active 3 hacluster jujucharms 72 ubuntu
keystone-mysql-router 8.0.22 active 3 mysql-router local 0 ubuntu
mysql-innodb-cluster 8.0.22 active 3 mysql-innodb-cluster local 0 ubuntu
neutron-api 16.2.0 blocked 0/1 neutron-api local 0 ubuntu
neutron-api-plugin-ovn 16.2.0 blocked 0/1 neutron-api-plugin-ovn local 0 ubuntu
neutron-mysql-router 8.0.22 active 1 mysql-router local 0 ubuntu
nova-cloud-controller 21.1.0 active 1 nova-cloud-controller local 501 ubuntu
nova-compute 21.1.0 active 3 nova-compute local 133 ubuntu
nova-mysql-router 8.0.22 active 1 mysql-router local 0 ubuntu
ntp 3.5 active 3 ntp local 0 ubuntu
openstack-dashboard 18.3.2 active 1 openstack-dashboard local 32 ubuntu
ovn-central 20.03.1 waiting 3 ovn-central local 0 ubuntu
ovn-chassis 20.03.1 blocked 0/3 ovn-chassis local 0 ubuntu
placement 3.0.0 error 1 placement local 0 ubuntu
placement-mysql-router 8.0.22 active 1 mysql-router local 0 ubuntu
rabbitmq-server 3.8.2 active 3 rabbitmq-server local 151 ubuntu
vault 1.5.4 blocked 3 vault local 1 ubuntu
vault-hacluster active 3 hacluster jujucharms 72 ubuntu
vault-mysql-router 8.0.22 active 3 mysql-router local 0 ubuntu
Unit Workload Agent Machine Public address Ports Message
ceph-mon/0 active idle 0/lxd/0 10.0.0.250 Unit is ready and clustered
ceph-mon/1 active idle 1/lxd/0 10.0.7.145 Unit is ready and clustered
ceph-mon/2* active idle 2/lxd/0 10.0.7.137 Unit is ready and clustered
ceph-osd/0* active idle 0 10.0.0.159 Unit is ready (1 OSD)
ceph-osd/1 active idle 1 10.0.0.156 Unit is ready (1 OSD)
ceph-osd/2 active idle 2 10.0.0.157 Unit is ready (1 OSD)
ceph-radosgw/0* error idle 0/lxd/1 10.0.7.131 80/tcp hook failed: "certificates-relation-broken"
cinder/0* active idle 1/lxd/1 10.0.7.143 8776/tcp Unit is ready
cinder-ceph/0* active idle 10.0.7.143 Unit is ready
cinder-mysql-router/0* active idle 10.0.7.143 Unit is ready
easyrsa/0* active idle 0/lxd/12 10.0.7.163 Certificate Authority connected.
etcd/0 active idle 0/lxd/11 10.0.7.162 2379/tcp Healthy with 3 known peers
etcd/1* active idle 1/lxd/11 10.0.7.161 2379/tcp Healthy with 3 known peers
etcd/2 active idle 2/lxd/10 10.0.7.160 2379/tcp Healthy with 3 known peers
glance/0* active idle 2/lxd/1 10.0.7.140 9292/tcp Unit is ready
glance-mysql-router/0* active idle 10.0.7.140 Unit is ready
keystone/1 active idle 0/lxd/7 10.0.7.149 5000/tcp Unit is ready
keystone-hacluster/2 active idle 10.0.7.149 Unit is ready and clustered
keystone-mysql-router/3 active idle 10.0.7.149 Unit is ready
keystone/2* active idle 1/lxd/6 10.0.7.148 5000/tcp Unit is ready
keystone-hacluster/0* active idle 10.0.7.148 Unit is ready and clustered
keystone-mysql-router/1* active idle 10.0.7.148 Unit is ready
keystone/3 active idle 2/lxd/6 10.0.7.150 5000/tcp Unit is ready
keystone-hacluster/1 active idle 10.0.7.150 Unit is ready and clustered
keystone-mysql-router/2 active idle 10.0.7.150 Unit is ready
mysql-innodb-cluster/0* active idle 0/lxd/3 10.0.7.133 Unit is ready: Mode: R/W
mysql-innodb-cluster/1 active idle 1/lxd/2 10.0.7.142 Unit is ready: Mode: R/O
mysql-innodb-cluster/2 active idle 2/lxd/2 10.0.7.136 Unit is ready: Mode: R/O
neutron-api/0* blocked failed 1/lxd/3 10.0.7.147 9696/tcp Missing relations: messaging
neutron-api-plugin-ovn/0* blocked failed 10.0.7.147 'certificates' missing, 'ovsdb-cms' incomplete
neutron-mysql-router/0* active idle 10.0.7.147 Unit is ready
nova-cloud-controller/0* active idle 0/lxd/4 10.0.7.132 8774/tcp,8775/tcp Unit is ready
nova-mysql-router/0* active idle 10.0.7.132 Unit is ready
nova-compute/0* active idle 0 10.0.0.159 Unit is ready
ntp/0* active idle 10.0.0.159 123/udp chrony: Ready
ovn-chassis/0* blocked failed 10.0.0.159 'certificates' missing, 'ovsdb' incomplete
nova-compute/1 active idle 1 10.0.0.156 Unit is ready
ntp/2 active idle 10.0.0.156 123/udp chrony: Ready
ovn-chassis/2 blocked failed 10.0.0.156 'certificates' missing, 'ovsdb' incomplete
nova-compute/2 active idle 2 10.0.0.157 Unit is ready
ntp/1 active idle 10.0.0.157 123/udp chrony: Ready
ovn-chassis/1 blocked failed 10.0.0.157 'certificates' missing, 'ovsdb' incomplete
openstack-dashboard/0* active idle 1/lxd/4 10.0.7.144 80/tcp,443/tcp Unit is ready
dashboard-mysql-router/0* active idle 10.0.7.144 Unit is ready
ovn-central/0* waiting idle 0/lxd/5 10.0.7.134 6641/tcp,6642/tcp 'ovsdb-peer' incomplete, 'certificates' awaiting server certificate data
ovn-central/1 waiting idle 1/lxd/5 10.0.7.146 6641/tcp,6642/tcp 'ovsdb-peer' incomplete, 'certificates' awaiting server certificate data
ovn-central/2 waiting idle 2/lxd/3 10.0.7.138 6641/tcp,6642/tcp 'ovsdb-peer' incomplete, 'certificates' awaiting server certificate data
placement/0* error idle 2/lxd/4 10.0.7.141 8778/tcp hook failed: "certificates-relation-broken"
placement-mysql-router/0* active idle 10.0.7.141 Unit is ready
rabbitmq-server/7* active idle 0/lxd/9 10.0.7.153 5672/tcp Unit is ready and clustered
rabbitmq-server/8 active idle 1/lxd/8 10.0.7.154 5672/tcp Unit is ready and clustered
rabbitmq-server/9 active idle 2/lxd/8 10.0.7.155 5672/tcp Unit is ready and clustered
vault/1 blocked idle 0/lxd/10 10.0.7.157 8200/tcp Vault needs to be initialized
vault-hacluster/1 active idle 10.0.7.157 Unit is ready and clustered
vault-mysql-router/2 active idle 10.0.7.157 Unit is ready
vault/2 blocked idle 1/lxd/10 10.0.7.159 8200/tcp Vault needs to be initialized
vault-hacluster/2 active idle 10.0.7.159 Unit is ready and clustered
vault-mysql-router/3 active idle 10.0.7.159 Unit is ready
vault/3* blocked idle 2/lxd/9 10.0.7.158 8200/tcp Vault needs to be initialized
vault-hacluster/0* active idle 10.0.7.158 Unit is ready and clustered
vault-mysql-router/1* active idle 10.0.7.158 Unit is ready
Machine State DNS Inst id Series AZ Message
0 started 10.0.0.159 node4 focal default Deployed
0/lxd/0 started 10.0.0.250 juju-baaea2-0-lxd-0 focal default Container started
0/lxd/1 started 10.0.7.131 juju-baaea2-0-lxd-1 focal default Container started
0/lxd/3 started 10.0.7.133 juju-baaea2-0-lxd-3 focal default Container started
0/lxd/4 started 10.0.7.132 juju-baaea2-0-lxd-4 focal default Container started
0/lxd/5 started 10.0.7.134 juju-baaea2-0-lxd-5 focal default Container started
0/lxd/7 started 10.0.7.149 juju-baaea2-0-lxd-7 focal default Container started
0/lxd/9 started 10.0.7.153 juju-baaea2-0-lxd-9 focal default Container started
0/lxd/10 started 10.0.7.157 juju-baaea2-0-lxd-10 focal default Container started
0/lxd/11 started 10.0.7.162 juju-baaea2-0-lxd-11 focal default Container started
0/lxd/12 started 10.0.7.163 juju-baaea2-0-lxd-12 focal default Container started
1 started 10.0.0.156 node2 focal default Deployed
1/lxd/0 started 10.0.7.145 juju-baaea2-1-lxd-0 focal default Container started
1/lxd/1 started 10.0.7.143 juju-baaea2-1-lxd-1 focal default Container started
1/lxd/2 started 10.0.7.142 juju-baaea2-1-lxd-2 focal default Container started
1/lxd/3 started 10.0.7.147 juju-baaea2-1-lxd-3 focal default Container started
1/lxd/4 started 10.0.7.144 juju-baaea2-1-lxd-4 focal default Container started
1/lxd/5 started 10.0.7.146 juju-baaea2-1-lxd-5 focal default Container started
1/lxd/6 started 10.0.7.148 juju-baaea2-1-lxd-6 focal default Container started
1/lxd/8 started 10.0.7.154 juju-baaea2-1-lxd-8 focal default Container started
1/lxd/10 started 10.0.7.159 juju-baaea2-1-lxd-10 focal default Container started
1/lxd/11 started 10.0.7.161 juju-baaea2-1-lxd-11 focal default Container started
2 started 10.0.0.157 node1 focal default Deployed
2/lxd/0 started 10.0.7.137 juju-baaea2-2-lxd-0 focal default Container started
2/lxd/1 started 10.0.7.140 juju-baaea2-2-lxd-1 focal default Container started
2/lxd/2 started 10.0.7.136 juju-baaea2-2-lxd-2 focal default Container started
2/lxd/3 started 10.0.7.138 juju-baaea2-2-lxd-3 focal default Container started
2/lxd/4 started 10.0.7.141 juju-baaea2-2-lxd-4 focal default Container started
2/lxd/6 started 10.0.7.150 juju-baaea2-2-lxd-6 focal default Container started
2/lxd/8 started 10.0.7.155 juju-baaea2-2-lxd-8 focal default Container started
2/lxd/9 started 10.0.7.158 juju-baaea2-2-lxd-9 focal default Container started
2/lxd/10 started 10.0.7.160 juju-baaea2-2-lxd-10 focal default Container started
Relation provider Requirer Interface Type Message
ceph-mon:client cinder-ceph:ceph ceph-client regular
ceph-mon:client glance:ceph ceph-client regular
ceph-mon:client nova-compute:ceph ceph-client regular
ceph-mon:mon ceph-mon:mon ceph peer
ceph-mon:osd ceph-osd:mon ceph-osd regular
ceph-mon:radosgw ceph-radosgw:mon ceph-radosgw regular
ceph-radosgw:cluster ceph-radosgw:cluster swift-ha peer
cinder-ceph:ceph-access nova-compute:ceph-access cinder-ceph-key regular
cinder-ceph:storage-backend cinder:storage-backend cinder-backend subordinate
cinder-mysql-router:shared-db cinder:shared-db mysql-shared subordinate
cinder:cinder-volume-service nova-cloud-controller:cinder-volume-service cinder regular
cinder:cluster cinder:cluster cinder-ha peer
dashboard-mysql-router:shared-db openstack-dashboard:shared-db mysql-shared subordinate
easyrsa:client etcd:certificates tls-certificates regular
etcd:cluster etcd:cluster etcd peer
etcd:db vault:etcd etcd regular
glance-mysql-router:shared-db glance:shared-db mysql-shared subordinate
glance:cluster glance:cluster glance-ha peer
glance:image-service cinder:image-service glance regular
glance:image-service nova-cloud-controller:image-service glance regular
glance:image-service nova-compute:image-service glance regular
keystone-hacluster:ha ceph-radosgw:ha hacluster subordinate joining
keystone-hacluster:ha keystone:ha hacluster subordinate
keystone-hacluster:hanode keystone-hacluster:hanode hacluster peer
keystone-mysql-router:shared-db keystone:shared-db mysql-shared subordinate
keystone:cluster keystone:cluster keystone-ha peer
keystone:identity-service ceph-radosgw:identity-service keystone regular
keystone:identity-service cinder:identity-service keystone regular
keystone:identity-service glance:identity-service keystone regular
keystone:identity-service neutron-api:identity-service keystone regular
keystone:identity-service nova-cloud-controller:identity-service keystone regular
keystone:identity-service openstack-dashboard:identity-service keystone regular
keystone:identity-service placement:identity-service keystone regular
mysql-innodb-cluster:cluster mysql-innodb-cluster:cluster mysql-innodb-cluster peer
mysql-innodb-cluster:coordinator mysql-innodb-cluster:coordinator coordinator peer
mysql-innodb-cluster:db-router cinder-mysql-router:db-router mysql-router regular
mysql-innodb-cluster:db-router dashboard-mysql-router:db-router mysql-router regular
mysql-innodb-cluster:db-router glance-mysql-router:db-router mysql-router regular
mysql-innodb-cluster:db-router keystone-mysql-router:db-router mysql-router regular
mysql-innodb-cluster:db-router neutron-mysql-router:db-router mysql-router regular
mysql-innodb-cluster:db-router nova-mysql-router:db-router mysql-router regular
mysql-innodb-cluster:db-router placement-mysql-router:db-router mysql-router regular
mysql-innodb-cluster:db-router vault-mysql-router:db-router mysql-router regular
mysql-innodb-cluster:shared-db vault:shared-db mysql-shared regular
neutron-api-plugin-ovn:neutron-plugin neutron-api:neutron-plugin-api-subordinate neutron-plugin-api-subordinate subordinate
neutron-api:cluster neutron-api:cluster neutron-api-ha peer
neutron-api:neutron-api nova-cloud-controller:neutron-api neutron-api regular
neutron-mysql-router:shared-db neutron-api:shared-db mysql-shared subordinate
nova-cloud-controller:cluster nova-cloud-controller:cluster nova-ha peer
nova-compute:cloud-compute nova-cloud-controller:cloud-compute nova-compute regular
nova-compute:compute-peer nova-compute:compute-peer nova peer
nova-compute:juju-info ntp:juju-info juju-info subordinate
nova-mysql-router:shared-db nova-cloud-controller:shared-db mysql-shared subordinate
ntp:ntp-peers ntp:ntp-peers ntp peer
openstack-dashboard:cluster openstack-dashboard:cluster openstack-dashboard-ha peer
ovn-central:ovsdb ovn-chassis:ovsdb ovsdb regular
ovn-central:ovsdb-cms neutron-api-plugin-ovn:ovsdb-cms ovsdb-cms regular
ovn-central:ovsdb-peer ovn-central:ovsdb-peer ovsdb-cluster peer
ovn-chassis:nova-compute nova-compute:neutron-plugin neutron-plugin subordinate
placement-mysql-router:shared-db placement:shared-db mysql-shared subordinate
placement:cluster placement:cluster openstack-ha peer
placement:placement nova-cloud-controller:placement placement regular
rabbitmq-server:amqp cinder:amqp rabbitmq regular
rabbitmq-server:amqp glance:amqp rabbitmq regular
rabbitmq-server:amqp neutron-api:amqp rabbitmq regular
rabbitmq-server:amqp nova-cloud-controller:amqp rabbitmq regular
rabbitmq-server:amqp nova-compute:amqp rabbitmq regular
rabbitmq-server:cluster rabbitmq-server:cluster rabbitmq-ha peer
vault-hacluster:ha vault:ha hacluster subordinate
vault-hacluster:hanode vault-hacluster:hanode hacluster peer
vault-mysql-router:shared-db vault:shared-db mysql-shared subordinate
vault:certificates ceph-radosgw:certificates tls-certificates regular
vault:certificates cinder:certificates tls-certificates regular
vault:certificates glance:certificates tls-certificates regular
vault:certificates keystone:certificates tls-certificates regular
vault:certificates neutron-api-plugin-ovn:certificates tls-certificates regular
vault:certificates neutron-api:certificates tls-certificates regular
vault:certificates nova-cloud-controller:certificates tls-certificates regular
vault:certificates openstack-dashboard:certificates tls-certificates regular
vault:certificates ovn-central:certificates tls-certificates regular
vault:certificates ovn-chassis:certificates tls-certificates regular
vault:certificates placement:certificates tls-certificates regular
vault:cluster vault:cluster vault-ha peer
why “ceph-radosgw/0* error idle 0/lxd/1 10.0.7.131 80/tcp hook failed: “certificates-relation-broken””
and
" placement/0* error idle 2/lxd/4 10.0.7.141 8778/tcp hook failed: “certificates-relation-broken”"
two error happend ?