Since you refer to the controller in the singular above, I am assuming that you are not running in HA.
There might be trouble starting Raft here, which would explain both the leader-get
and provisioning hangs.
If you are running Juju 2.9+, you can introspect Raft’s view of leases by SSHing directly to the controller machine and running juju_leases
.
If you have had storage issues, it may be that the Raft log/snapshots have been corrupted. These are stored in /var/lib/juju/raft. If this is the case, you can resolve his by:
- Stopping the controller.
- Deleting the contents of /var/lib/juju/raft.
- Starting the controller.
Note that this will effectively expire all current leases, leading to new claims and potentially some churn.