How to remove dead relation which don't recognize by juju status --relations

juju
#1

Dear all , I am new to juju, and I’m facing the issue of an old - outdated production OpenStack system. Our system is working on juju 2.8. On the past, the vault got an issue with connecting to mysql-innodb-cluster. I did remove-relation, RELATE relation, but it looks like the process has stuck on some points.

This is the juju status juju status

This is error on mysql-innodb-cluster/1 Error relation-get

Error on relation-get relation-get

This is the juju status --relations juju status --relations

You can see we don’t have any relations that relate to shared-db on mysql-innodb-cluster. But the hook update status shows errors with shared-db:83, which can’t be accessed with permission denied.

At this time, vault can’t issue-certificate, so in this case, what is the best way to do to make the system comeback work, redeploy the vault (the single unit in the system) ? or try to delete the dead relation and re-add the relation to mysql-innodb-cluster.

Please give me some advice. Thanks,

#2

Looks like you environment is running juju 2.8.0. This is a pretty old version now, I would strongly recommend you look at upgrading both your controllers and models to the latest 2.9.x release (2.9.46 currently) as a minimum.

The most common cause of this has been the following bug:

This was fixed in juju 2.8.8, if you at least upgrade your environment to the latest 2.8.x it may help here.

However note that is also an issue where the juju unit incorrectly uses a Virtual IP (VIP) for OpenStack services as a units IP address in 2.8.x that can cause service outages in multiple services after this upgrade. This only happens in some environments as it depends on the sort-order of the Virtual IP addresses relative to the machine’s normal IP address. In some environments it picks the Virtual IP and in some it doesn’t. I am unsure if this issue existed in 2.8.0 or not, however, that may require you to upgrade to 2.9.x if you run into issues like that after first upgrading to the latest 2.8.x

juju upgrade-juju -m controller

Watch ‘juju debug-log -m controller’ and ‘juju status -m controller’ as well as ‘juju status’ for your openstack models and wait for the upgrade to settle. This can take 10s of minutes or more.

Once the controller upgrade has settled, and no more hooks are executing in your non-controller models (e.g. the openstack model), then run juju upgrade-juju against the other models as well.

There can be some other causes so if you get to 2.9.46 and it still doesn’t work you may need to consider some other actions.

Note: If you are an Ubuntu Pro customer with active support for this environment, I’d encourage you to open a support ticket.

1 Like
#3

Thanks, Lathiat, for your advice! Unfortunately, we were not a Ubuntu Pro customer. So we will back up the current and try to upgrade Juju to 2.9, hoping. the upgrade will fix the errors.

#4

Just update the status of my bundle. After updating juju controller to 2.9, the issues of dead relation have gone out. I can replace the application and re-deploy vault, generate a new root ca and issue certificates to applications. But only keystone can not receive the certificate. I don’t know why. But the solution on this issue have worked perfectly.

Thanks again Lathiat,