Introduction
In recent testing it has come to our attention that Juju is not able to Bootstrap and work with Rancher clusters directly out of the box.
Problem
When bootstrapping to Rancher clusters it’s expected that the user will come across the following error message during the bootstrap process.
unable to determine legacy status for namespace add-cloud: the server has asked for the client to provide credentials
Why
The Kubernetes config credentials provided to the user by Rancher clusters force Kubernetes clients to make connections to Kubernetes through the Rancher api server that acts as a proxy onto the given cluster. The reason for this is so that the Rancher API server can validate it’s own OAuth tokens.
Due to the way Juju bootstraps we create an admin service account that is both used for the bootstrap process and for Juju to operate with. The Rancher api that is proxying connections does not understand nor can validate credentials that are not it’s own and created in cluster.
This issue is documented here on the Rancher Github issues page.
Workaround
This issue can currently be worked around by getting Rancher to generate Kubeconfig files that point directly to the Kubernetes cluster being interacted with. As per the documentation here
Solution
Juju is currently investigating several long term solutions for this problem. We believe this can be solved by changing the credentials used as part of the Juju bootstrap process.
We would appreciate long term feedback from users that run into this problem and how it affects you.
This is being further tracked in this Launch Pad Bug