Hi,
Environment:
- Vault Version: 1.8/stable
- Deployment: Juju Charm
- Openstack version: Yoga
Description:
The machine running the Vault leader was rebooted, causing the leader role to move to another unit. However, the new leader is reporting a hook failed: "leader-elected" error.
Upon checking the logs, I found the following warning:
$ cat /var/log/juju/unit-vault-1.log | tail
WARNING unit.vault/1.leader-elected logger.go:60 hvac.exceptions.InvalidRequest: backend must be configured with a CA certificate/key, on get http://127.0.0.1:8220/v1/charm-pki-local/cert/ca_chain
ERROR juju.worker.uniter.operation runhook.go:180 hook "leader-elected" (via explicit, bespoke hook script) failed: exit status 1
INFO juju.worker.uniter resolver.go:180 awaiting error resolution for "leader-elected" hook
Vault status :
vault 1.8.8 error 3 vault 1.8/stable 372 no hook failed: "leader-elected"
vault-mysql-router 8.0.45 active 3 mysql-router 8.0/stable 257 no Unit is ready
vault/0 active idle 3/lxd/11 10.10.10.123 8200/tcp Unit is ready (active: true, mlock: disabled)
hacluster-vault/1 active idle 10.10.10.123 Unit is ready and clustered
vault-mysql-router/1* active idle 10.10.10.123 Unit is ready
vault/1* error idle 4/lxd/11 10.10.10.133 8200/tcp hook failed: "leader-elected"
hacluster-vault/2* active idle 10.10.10.133 Unit is ready and clustered
vault-mysql-router/2 active idle 10.10.10.133 Unit is ready
vault/2 active idle 5/lxd/11 10.10.10.94 8200/tcp Unit is ready (active: true, mlock: disabled)
hacluster-vault/0 active idle 10.10.10.94 Unit is ready and clustered
vault-mysql-router/0 active idle 10.10.10.94 Unit is ready
i’ve also check on all unit and here is the output, only 1 unit that seems normal.
$ juju exec -a vault "curl -s http://127.0.0.1:8220/v1/charm-pki-local/cert/ca_chain"
vault/0:
{"errors":["missing client token"]}
vault/1:
{"errors":["backend must be configured with a CA certificate/key"]}
vault/2:
{"errors":["backend must be configured with a CA certificate/key"]}
FYI, CA is already generated. Only unit-0 is normal.
$ juju run vault/0 get-root-ca
Running operation 209 with 1 task
- task 210 on unit-vault-0
Waiting for task 210...
output: |-
-----BEGIN CERTIFICATE-----
MIIDazCCAlOgAwIBAgIUb+u3lDPtTKW5cub769tQU+2hkVswDQYJKoZIhvcNAQEL
lxc
active
lxc
$ juju run vault/1 get-root-ca
Running operation 211 with 1 task
- task 212 on unit-vault-1
Waiting for task 212...
Action id 212 failed: hvac.exceptions.InvalidRequest: backend must be configured with a CA certificate/key, on get http://127.0.0.1:8220/v1/charm-pki-local/cert/ca_chain
output: |-
-----BEGIN CERTIFICATE-----
MIIDazCCAlOgAwIBAgIUb+u3lDPtTKW5cub769tQU+2hkVswDQYJKoZIhvcNAQEL
lxc
active
Please help.
Thanks.