Vault Charm unit error hook failed: "start"

juju
, , ,
1

Hi,
i have an openstack deployment with vault deployed in HA mode with 3 units. I upgraded juju from 2.5 to 2.8.5. Since the upgrade one vault unit is in error with “hook failed: “start””.
The juju log inside the machine is showing the following error:
2020-10-22 16:00:01 ERROR juju-log Hook error:
Traceback (most recent call last):
File “/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.6/site-packages/charms/reactive/init.py”, line 74, in main
bus.dispatch(restricted=restricted_mode)
File “/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.6/site-packages/charms/reactive/bus.py”, line 390, in dispatch
_invoke(other_handlers)
File “/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.6/site-packages/charms/reactive/bus.py”, line 359, in _invoke
handler.invoke()
File “/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.6/site-packages/charms/reactive/bus.py”, line 181, in invoke
self._action(*args)
File “/var/lib/juju/agents/unit-vault-1/charm/reactive/vault_handlers.py”, line 220, in configure_vault_mysql
if local_unit() not in mysql.allowed_units():
TypeError: argument of type ‘NoneType’ is not iterable

2020-10-22 16:00:01 WARNING start Traceback (most recent call last):
2020-10-22 16:00:01 WARNING start File “/var/lib/juju/agents/unit-vault-1/charm/hooks/start”, line 22, in
2020-10-22 16:00:01 WARNING start main()
2020-10-22 16:00:01 WARNING start File “/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.6/site-packages/charms/reactive/init.py”, line 74, in main
2020-10-22 16:00:01 WARNING start bus.dispatch(restricted=restricted_mode)
2020-10-22 16:00:01 WARNING start File “/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.6/site-packages/charms/reactive/bus.py”, line 390, in dispatch
2020-10-22 16:00:01 WARNING start _invoke(other_handlers)
2020-10-22 16:00:01 WARNING start File “/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.6/site-packages/charms/reactive/bus.py”, line 359, in _invoke
2020-10-22 16:00:01 WARNING start handler.invoke()
2020-10-22 16:00:01 WARNING start File “/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.6/site-packages/charms/reactive/bus.py”, line 181, in invoke
2020-10-22 16:00:01 WARNING start self._action(*args)
2020-10-22 16:00:01 WARNING start File “/var/lib/juju/agents/unit-vault-1/charm/reactive/vault_handlers.py”, line 220, in configure_vault_mysql
2020-10-22 16:00:01 WARNING start if local_unit() not in mysql.allowed_units():
2020-10-22 16:00:01 WARNING start TypeError: argument of type ‘NoneType’ is not iterable
2020-10-22 16:00:01 ERROR juju.worker.uniter.operation runhook.go:136 hook “start” (via explicit, bespoke hook script) failed: exit status 1

Vault service is running:
root@juju-9d7331-0-lxd-34:/var/log/juju# systemctl status vault
● vault.service - HashiCorp Vault
Loaded: loaded (/etc/systemd/system/vault.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2020-09-04 14:54:20 UTC; 1 months 17 days ago
Main PID: 281 (vault)
Tasks: 40 (limit: 4915)
CGroup: /system.slice/vault.service
└─281 /snap/vault/1822/bin/vault server -config /var/snap/vault/common/vault.hcl
Any idea on how i can solve this ?
Thx in advance.
RZ

2

I suspect this may be related to a juju bug [0] that lost unit state.

Somehow there is no relation data for the mysql relation and specifically the allowed_hosts entries:
if local_unit() not in mysql.allowed_units():
TypeError: argument of type ‘NoneType’ is not iterable

To get you out of this we may need to do some code surgery. Add another check similar to:

Remove: if local_unit() not in mysql.allowed_units():
Add: if local_unit() not in mysql.allowed_units() or []:

def configure_vault_mysql(mysql):
if local_unit() not in mysql.allowed_units() or []:

I’ll file a bug on this against vault.

[0] Bug #1890465 “charm install hook fired on upgrade to juju 2.8.2” : Bugs : juju

3

Thx for your reply !! sounds exactly my problem.
Let me know when you file the bug report.
Thx

4

I tried your proposed fix but it did not solve it.