How to install Charmed Kubeflow on Kubernetes

[knkski (Kenneth Koski)]

This guide shows how to install Charmed Kubeflow on any conformant Kubernetes, including AKS, EKS, GKE, Openshift and any kubeadm-deployed cluster.

Contents:

1. Get a Juju controller in a Kubernetes cluster
2. Create a model called kubeflow
3. Deploy the kubeflow bundle
4. Access the Kubeflow dashboard

Get a Juju controller in a Kubernetes cluster

To deploy Charmed Kubeflow, you need a Juju controller deployed in a Kubernetes instance. There are many ways to satisfy this, such as using your own local Kubernetes cluster or deploying into a managed Kubernetes in a cloud.

• If you do not have access to a Kubernetes cluster:
  See Get started with Charmed Kubeflow for an example of how to deploy a Juju onto a local Kubernetes cluster using MicroK8s.

• If you do have a Kubernetes cluster but have not bootstrapped a Juju controller to it yet:
  See Juju OLM | Get started wtih the Juju OLM.

---

Requirements of your Kubernetes instance

Charmed Kubeflow requires your Kubernetes instance to:

• be version 1.22, 1.23, or 1.24
• have a (default) storage class configured
• have dns configured
• for accessing the dashboard, have a LoadBalancer and Ingress

---

Create a model called kubeflow

To create a Juju model for Charmed Kubeflow, run:

Caution:

Charmed Kubeflow must be deployed to a model named kubeflow.

juju add-model kubeflow

This creates a model named kubeflow through our Juju controller and sets the model as active. A model in Juju is a blank canvas where your operators will be deployed, and it holds a 1:1 relationship with a Kubernetes namespace.

See more: Juju OLM | juju add-model

To view the newly created model, run:

juju models

See more: Juju OLM | juju models

Deploy the kubeflow bundle

To install the most recent stable version of Charmed Kubeflow, deploy the kubeflow bundle:

juju deploy kubeflow --trust

See more: Charmhub | kubeflow

Congratulations, Kubeflow is now installing!

Depending on your internet speed, this may take 10-15 minutes. You can observe your Kubeflow deployment getting spun-up with the command:

watch -c juju status --color

and see the corresponding resources in your Kubernetes cluster with the command:

watch kubectl get pods -n kubeflow

Access the Kubeflow dashboard

To access the Kubeflow dashboard, you need to prepare the IP and the credentials and then use them to log in to the dashboard.

• Find the IP address and set the access credentials
• Use the IP address to access the dashboard and log in

Find the IP address and set the access credentials

Find the IP address. To find the IP address of the Kubeflow dashboard for your deployment, run the code below (where kubeflow is the name you gave to your Juju model, and hence the namespace of your Kubeflow deployment).

If you’re using MicroK8s, with the typical metallb add-on:
You don’t need to do this. The URL is always http://10.64.140.43.nip.io. Note that, if you have set up DNS, you should use the resolvable address used by istio-ingressgateway.

kubectl get services -n kubeflow

The IP you need is the one attached to the istio-ingressgateway component.

See more: Kubernetes | kubectl get

Use the IP to configure the authentication components. Now, use the IP address you just obtained to set the URL in the dex-auth and oidc-gatekeeper components of the newly installed kubeflow, as below:

juju config dex-auth public-url=http://<IP address>
juju config oidc-gatekeeper public-url=http://<IP address>

Also configure the authentication components with a username and password. Configure the dex-auth component with a username and password:

juju config dex-auth static-username=<new username>
juju config dex-auth static-password=<new password>

See more: Juju OLM | juju config, Charmhub > dex-auth > Configure

Use the IP address to access the dashboard and log in

The instructions for how to access the dashboard depend on whether your deployment is local or remote.

• Local deployments
• Remote deployments

Local deployments

For local Kubeflow deployments, such as in a workstation, you can simply access the link found in the previous step, appending nip.io, for example: http://10.64.140.43.nip.io.

Remote deployments

However, for remote deployments, or running on a virtual machine, creating a SOCKS proxy is required to access the dashboard. This can be done as follows:

1. Log out from the current session with the exit command.

2. Re-establish connection to the machine using ssh with SOCKS proxy enabled through the -D 9999 parameter, as below:

   ssh -D 9999 ubuntu@<machine_public_ip>
   

3. On your computer, go to Settings > Network > Network Proxy, and enable SOCKS proxy pointing to: 127.0.0.1:9999

4. On a new browser window, copy the link from the previous step with .nip.io appended at the end, for example: http://10.64.140.43.nip.io

Expand to see the resulting settings for the Firefox browser

[sabdfl (Mark Shuttleworth)]

Surely just one page will suffice, since Kubeflow would install the same on any K8s cluster, with an extra paragraph for the fact that it’s built in to MicroK8s?

[zmraul (Raúl Zamora)]

I think steps 6 and 7 are reversed. When installing on Charmed Kubernetes, the istio-ingressgateway operator doesn’t create an external IP address until the role it’s patched.

[evilnick (Nick Veitch)]

Thanks for pointing this out, for some k8s that could be an issue so I will swap it around

[natalia-nowakowska]

Point 4 needs a small update: it says ‘You can create a model and give it a name, e.g. kubeflow’ while it has to be named exactly kubeflow, otherwise the deployment leads to one of the apps getting blocked with message ‘kubeflow-dashboard must be deployed to model named kubeflow: https://git.io/J6d35’.

[rohanrehman (R.)]

Has this been tested on 22.04? Jesus! what an ordeal. #NotaQuickStart

[ca-scribner (Andrew Scribner)]

Hey @rohanrehman,

I do use Charmed Kubeflow with 22.04 and don’t know of anything that blocks it. Do you have any details you could share to look into?

Thanks!

[rohanrehman (R.)]

thanks for getting back. Adding juju bootstrap microk8s --agent-version="2.9.22" was the only solution that worked mind you… 2 clean installs (22.04 and 20.04), 32gig of ram (500GB SSD) on-prem bare metal.

full kubeflow was problematic

Any documentation on what features are missing in the kubeflow-lite?

From a kubeflow-lite deploy to kubeflow, will this destroy all containers and “start from scratch”.

Thanks