Command 'juju add-k8s'

The information in this doc is based on Juju version 3.5.5, and may not accurately reflect other versions of Juju.

See also: remove-k8s

Summary

Adds a k8s endpoint and credential to Juju.

Usage

juju add-k8s [options] <k8s name>

Options

Flag Default Usage
-B, --no-browser-login false Do not use web browser for authentication
-c, --controller Controller to operate in
--client false Client operation
--cloud k8s cluster cloud
--cluster-name Specify the k8s cluster to import
--context-name Specify the k8s context to import
--credential the credential to use when accessing the cluster
--region k8s cluster region or cloud/region
--skip-storage false used when adding a cluster that doesn’t have storage
--storage k8s storage class for workload storage

Examples

When your kubeconfig file is in the default location:

juju add-k8s myk8scloud
juju add-k8s myk8scloud --client
juju add-k8s myk8scloud --controller mycontroller
juju add-k8s --context-name mycontext myk8scloud
juju add-k8s myk8scloud --region cloudNameOrCloudType/someregion
juju add-k8s myk8scloud --cloud cloudNameOrCloudType
juju add-k8s myk8scloud --cloud cloudNameOrCloudType --region=someregion
juju add-k8s myk8scloud --cloud cloudNameOrCloudType --storage mystorageclass

To add a Kubernetes cloud using data from your kubeconfig file, when this file is not in the default location:

KUBECONFIG=path-to-kubeconfig-file juju add-k8s myk8scloud --cluster-name=my_cluster_name

To add a Kubernetes cloud using data from kubectl, when your kubeconfig file is not in the default location:

kubectl config view --raw | juju add-k8s myk8scloud --cluster-name=my_cluster_name

Details

Creates a user-defined cloud based on a k8s cluster.

The new k8s cloud can then be used to bootstrap into, or it can be added to an existing controller.

Use --controller option to add k8s cloud to a controller. Use --client option to add k8s cloud to this client.

Specify a non default kubeconfig file location using $KUBECONFIG environment variable or pipe in file content from stdin.

The config file can contain definitions for different k8s clusters, use --cluster-name to pick which one to use. It’s also possible to select a context by name using --context-name.

When running add-k8s the underlying cloud/region hosting the cluster needs to be detected to enable storage to be correctly configured. If the cloud/region cannot be detected automatically, use either –cloud <cloudType|cloudName> to specify the host cloud or –region <cloudType|cloudName>/<someregion> to specify the host cloud type and region.

Region is strictly necessary only when adding a k8s cluster to a JAAS controller. When using a standalone Juju controller, usually just --cloud is required.

Once Juju is aware of the underlying cloud type, it looks for a suitably configured storage class to provide operator and workload storage. If none is found, use of the --storage option is required so that Juju will create a storage class with the specified name.

If the cluster does not have a storage provisioning capability, use the –skip-storage option to add the cluster without any workload storage configured.


Could you please elaborate on what it means to “add k8s cloud to a controller”? I thought it’s impossible to bootstrap a controller without a cloud, and that a controller can have only one cloud associated with it.

Adding a cloud (whether a machine cloud, via add-cloud, or a Kubernetes cloud, via add-K8s) to a controller (by using the --controller flag) results in what is called a ‘multi-cloud’ controller. It’s a possibility that can be used to minimize the cloud resources used to maintain a controller plus, in the case of adding a Kubernetes cloud to another cloud, to take advantage, e.g., of the fact that a controller on a machine cloud can be made highly available whereas a controller on a Kubernetes cloud currently cannot. We mention this in our How to guide on how to add a cloud.

It seems it is not working when we have a config file for a broken microk8s cluster in the “.kube/config”.

It is still trying to communicate with the API server of the broken microk8s cluster:

$ KUBECONFIG=new-microk8s.yaml juju add-k8s microk8s-new --debug
08:29:07 INFO  juju.cmd supercommand.go:56 running juju [3.5.4 31b4b0914740b84ad8166993635ac797a44276de gc go1.23.1]
08:29:07 DEBUG juju.cmd supercommand.go:57   args: []string{"/snap/juju/28520/bin/juju", "add-k8s", "microk8s-new", "--debug"}
08:29:07 DEBUG juju.environs.tools build.go:123 looking for: /snap/juju/28520/bin/juju
08:29:07 DEBUG juju.environs.tools versionfile.go:54 looking for sha256 86e52c09e852fb74e18905d90385b53e23af6533a868e301bd717cbbb8bd01da
08:29:07 DEBUG juju.kubernetes.provider detectcloud.go:33 failed to query local microk8s: "/var/snap/juju/28520/microk8s/credentials/client.config" does not exist: juju "3.5.4" can only work with strictly confined microk8s
ERROR making juju admin credentials in cluster: ensuring cluster role "juju-credential-1de4d752" in namespace "kube-system": Get "https://10.149.16.44:16443/apis/rbac.authorization.k8s.io/v1/clusterroles/juju-credential-1de4d752": dial tcp 10.149.16.44:16443: connect: no route to host
08:29:10 DEBUG cmd supercommand.go:549 error stack: 
Get "https://10.149.16.44:16443/apis/rbac.authorization.k8s.io/v1/clusterroles/juju-credential-1de4d752": dial tcp 10.149.16.44:16443: connect: no route to host
github.com/juju/juju/caas/kubernetes/clientconfig.getOrCreateClusterRole:167: 
github.com/juju/juju/caas/kubernetes/clientconfig.ensureJujuAdminServiceAccount:88: ensuring cluster role "juju-credential-1de4d752" in namespace "kube-system"
github.com/juju/juju/cmd/juju/caas.(*AddCAASCommand).Run:509: making juju admin credentials in cluster

In the new-microk8s.yaml file I have the server address as “https://10.149.16.72:16443”:

$ cat new-microk8s.yaml | grep server
    server: https://10.149.16.72:16443

It might be related to this one: Juju 3.x requires Microk8s snap install in strictly confinement (Doc bug) · Issue #830 · canonical/bundle-kubeflow · GitHub

But I could not add the new cloud after implementing the workaround mentioned here

I am getting the following error:

$ KUBECONFIG=new-microk8s.yaml juju add-k8s new-test --debug
10:06:12 INFO  juju.cmd supercommand.go:56 running juju [3.5.4 31b4b0914740b84ad8166993635ac797a44276de gc go1.23.1]
10:06:12 DEBUG juju.cmd supercommand.go:57   args: []string{"/snap/juju/28520/bin/juju", "add-k8s", "new-test", "--debug"}
10:06:12 DEBUG juju.environs.tools build.go:123 looking for: /snap/juju/28520/bin/juju
10:06:13 DEBUG juju.environs.tools versionfile.go:54 looking for sha256 86e52c09e852fb74e18905d90385b53e23af6533a868e301bd717cbbb8bd01da
10:06:13 DEBUG juju.kubernetes.provider detectcloud.go:33 failed to query local microk8s: microk8s is not installed: kubernetes cluster "microk8s-cluster" not found not found
ERROR making juju admin credentials in cluster: ensuring cluster role "juju-credential-bd8efe10" in namespace "kube-system": Unauthorized
10:06:13 DEBUG cmd supercommand.go:549 error stack: 
Unauthorized
github.com/juju/juju/caas/kubernetes/clientconfig.getOrCreateClusterRole:167: 
github.com/juju/juju/caas/kubernetes/clientconfig.ensureJujuAdminServiceAccount:88: ensuring cluster role "juju-credential-bd8efe10" in namespace "kube-system"
github.com/juju/juju/cmd/juju/caas.(*AddCAASCommand).Run:509: making juju admin credentials in cluster