Cannot run command juju add-machine It stays in Pending State

Since you refer to the controller in the singular above, I am assuming that you are not running in HA.

There might be trouble starting Raft here, which would explain both the leader-get and provisioning hangs.

If you are running Juju 2.9+, you can introspect Raft’s view of leases by SSHing directly to the controller machine and running juju_leases.

If you have had storage issues, it may be that the Raft log/snapshots have been corrupted. These are stored in /var/lib/juju/raft. If this is the case, you can resolve his by:

  • Stopping the controller.
  • Deleting the contents of /var/lib/juju/raft.
  • Starting the controller.

Note that this will effectively expire all current leases, leading to new claims and potentially some churn.