Percona-cluster woes

1

I have managed to get this percona-cluster in a strange state whereby it does not respond to the normal bootstrap-pxc/notify-bootstrapped restart process. With the five members freshly booted the juju status is like this;

Unit                  Workload  Agent  Machine    Public address   Ports     Message
mysql/0*              blocked   idle   0/lxd/4    192.168.200.236  3306/tcp  MySQL is down. Sequence Number: -1. Safe To Bootstrap: 0
  mysql-hacluster/1*  blocked   idle              192.168.200.236            Resource: res_mysql_57654ca_vip not running
  mysql-syslog/0*     active    idle              192.168.200.236            Unit is ready
mysql/2               blocked   idle   38/lxd/1   192.168.200.229  3306/tcp  MySQL is down. Sequence Number: 4300. Safe To Bootstrap: 0
  mysql-hacluster/0   active    idle              192.168.200.229            Unit is ready and clustered
  mysql-syslog/1      active    idle              192.168.200.229            Unit is ready
mysql/3               blocked   idle   38/lxd/25  192.168.200.219  3306/tcp  MySQL is down. Sequence Number: 4300. Safe To Bootstrap: 0
  mysql-hacluster/2   active    idle              192.168.200.219            Unit is ready and clustered
  mysql-syslog/2      active    idle              192.168.200.219            Unit is ready
mysql/4               blocked   idle   0/lxd/23   192.168.200.56   3306/tcp  MySQL is down. Sequence Number: -1. Safe To Bootstrap: 0
  mysql-hacluster/3   blocked   idle              192.168.200.56             Resource: res_mysql_57654ca_vip not running
  mysql-syslog/3      active    idle              192.168.200.56             Unit is ready
mysql/5               waiting   idle   9/lxd/10   192.168.200.226  3306/tcp  Unit waiting for cluster bootstrap
  mysql-hacluster/4   active    idle              192.168.200.226            Unit is ready and clustered
  mysql-syslog/4      unknown   idle              192.168.200.226

I can’t find a way to make all the units cluster. Any suggestions please?

Many thanks in advance!

1 Like
2

I’ll update on what happened here. MySQL is in fact running, with a cluster of three working units. Machine 0 is broken. It suffered an incomplete series upgrade and despite juju reporting it running focal, it was mostly still bionic. The bionic containers running mysql on machine 0 are somehow broken as a result of this, but because they exist juju is not acknowledging the cluster.
I removed the two broken units and all is green again.