Charmed Kubeflow 1.6 beta is out: try it now!

announcements and community
#1

We are happy to announce that Charmed Kubeflow 1.6 is now available in Beta. Kubeflow has evolved into an end-to-end MLOps platform for optimised complex model training. We’re looking for data scientists, ML engineers and developers to take the Beta release for a drive and share their feedback! Our blog is available if you want to read more

On 18 August 2022, at 5 PM GMT, Canonical will offer a livestream about Charmed Kubeflow 1.6 beta. Together with Dominik Fleischmann and Daniela Plascencia, our resident Charmed Kubeflow developers, we will answer your questions and talk about:

  • The new release
  • The importance of being part of an open source community
  • The challenges involved in developing an open-source product
  • The differences between the upstream release and Canonical’s Charmed Kubeflow

Google | Outlook | Office 365 | Yahoo | Other calendars

Once the stable version of Charmed Kubeflow 1.6 is released, we will update you and guide you through the deployment process. Share your feedback here while it happens and our team will be here to support you.

1 Like
#4
#5
#6

These are the known issues that Charmed Kubeflow 1.6 Beta has:

  • On micok8s sometimes the containers go into an error state with the following message too many open files if that is the case execute the following commands in your system
sudo sysctl fs.inotify.max_user_instances=1280
sudo sysctl fs.inotify.max_user_watches=655360
  • Tensorboards-controller remains in a waiting state, as a workaround execute the following commands:
juju remove-relation tensorboard-controller istio-pilot
juju add-relation tensorboard-controller istio-pilot

This is an ongoing list that is going to be updated with your help. Contribute now and share your feedback with us!

#7

There is another (rare) known issue that prevents the kubeflow-gateway creation. If for any reason, after setting up everything to access the dashboard, it is unreachable, please:

  1. Check if the kubeflow-gateway is there

kubectl get gateway -nkubeflow kubeflow-gateway

  1. If you get no output from the above command, please run the following and wait for istio-pilot to be active and idle before trying to access the dashboard again:
juju run --unit istio-pilot/0 -- "export JUJU_DISPATCH_PATH=hooks/config-changed; ./dispatch"
1 Like
#8

@munteanuandreea

I’ve got an issue. When my notebook status is stop, then I can’t restart it anymore. (All my notebook got Event.SuccessfulCreate then Event.SuccessfulDelete)

All I can do is delete the notebook, create the new one then mount the volume again.

I am looking forward to hear any thoughts.

#9

Hi @amo-mycena, sorry to hear that!

Is this with 1.6 beta or 1.6 stable? I’m not sure if it makes a difference, but just checking.

I think I saw something yesterday in Kubeflow’s community meeting that sounded similar. Is this bug what you are seeing? In their case, they had notebooks shut down from inactivity that could not be restarted.

#10

Hi @ca-scribner, I am using 1.6 stable.

Yes this bug is the same as mine.

Is there anything I can do to fix the bug or just wait for the Canonical’s team to update the charms?

#11

Hello @amo-mycena we are currently waiting for upstream to release 1.6.1 which should fix this. We will update the charms once it is out in the coming week.

They just released the rc0 image for it, if you want to try replacing the image in the charm and see if that fixes it please do, we haven’t been able to try it yet but that might unblock you in the short run.