Installing Kubeflow on MicroK8s/AWS for testing and development

Kubeflow can be installed on any compatible version of Kubernetes, but with some setups there are a few extra steps to ensure everything runs smoothly. This page documents how to install Kubeflow on top of MicroK8s running in an AWS instance - a useful setup for testing, developing Kubeflow or short-lived experiments.

Requirements

  • The latest version of the Juju OLM. On Ubuntu or other snap enabled operating systems, just run
sudo snap install juju --classic
  • You should have an active AWS account, with the credentials already stored in Juju. See the Juju documentation if you need to add credentials now.

1. Bootstrap AWS instance

Juju requires a controller instance to manage the deployment. If you already have a controller for AWS, you can skip to step 2. To bootstrap a controller on AWS, run the following command on your local machine:

juju bootstrap \
          --bootstrap-constraints "cores=2 mem=4G" \
          --config mongo-memory-profile=low \
          aws/us-east-1 \
          mk8s-controller

You can substitute in a different region instead of aws/us-east-1, though bear in mind that not all AWS instances are available in some regions. You can also use a different name for the controller (mk8s-controller is used here).

2. Deploy a host instance

The host instance will run MicroK8s, and will require enough resources to deploy the Kubeflow components. If you want to run GPU workloads, you can also choose a GPU enabled instance. For example:

  • Standard deployment
juju deploy ubuntu demo --constraints="root-disk=100G instance-type=t2.2xlarge" --series focal && juju wait -wv
  • GPU deployment
 juju deploy ubuntu demo --constraints="root-disk=100G instance-type=g4dn.xlarge" --series focal && juju wait -wv

3. Enable access to the instance

Run these commands to be able to connect to the running instance from your local machine (or elsewhere).

juju run --unit demo/0 "open-port 80"
juju expose demo

4. Connect to AWS instance

You can now use Juju to connect to the AWS instance using SSH:

juju ssh demo/0

or if you prefer, directly from another terminal on the local machine:

ssh ubuntu@(juju status --format yaml | yq e '.applications.demo.units["demo/0"]'.public-address -)

Note: The rest of these steps should be performed on the AWS instance, through the SSH connection which has just been made.

5. Install requirements on AWS instance

To run and deploy Kubeflow, the remote instance will need some software installed. As well as MicroK8s itself and Juju, some additional applications will be needed for use as a development system.

for snap in microk8s juju juju-wait juju-kubectl juju-bundle; do
    sudo snap install $snap --classic;
done
sudo usermod -a -G microk8s ubuntu

For a GPU enabled system, you should also install and initialise the required drivers:

sudo apt install -y nvidia-headless-460
sudo modprobe nvidia

To complete the setup, you will need to log in again (to pick up group changes). You can just run exit from the SSH terminal, and re-run the Juju or SSH command from step 4 to re-connect.

6. Deploy the Kubeflow bundle

For the scripts to work, you need to supply the public address of your AWS instance in place of x.x.x.x below:

screen
git clone https://github.com/canonical/bundle-kubeflow.git
cd bundle-kubeflow/
python3 scripts/cli.py microk8s setup --test-mode
python3 scripts/cli.py deploy-to uk8s --public-address x.x.x.x.nip.io --bundle full --password=foobar

7. Create an Ingress resource

Again, substitute in the public IP address of the instance in place of x.x.x.x in the code below:

echo "apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: istio-ingressgateway
spec:
  rules:
  - host: x.x.x.x.nip.io
    http:
      paths:
      - path: /
        backend:
          serviceName: istio-ingressgateway
          servicePort: 80
" | microk8s kubectl apply -n kubeflow -f -

8. Run tests

To run the tests on the AWS instance

sudo apt install -y python3-pytest
sudo ln -s $(which pytest-3) /usr/local/bin/pytest
pip3 install --user kfp sh
./tests/run.sh -m full

If you want to run the tests (such as the selenium tests) from your laptop:

pytest -svv tests/ -m full -k selenium --headful --url http://(juju status --format yaml | yq e '.applications.demo.units["demo/0"]'.public-address -).nip.io/ --password=foobar

Useful commands

# Watch Juju status
watch -tcn2 timeout 2s juju status --color

# Watch MicroK8s pods
watch -tn2 timeout 2s microk8s kubectl get pods -nkubeflow --sort-by=.metadata.name
1 Like