hi, I tested this new version on my platform. Everything works except like version 1.6, Istio problem is still there . Needs commands to fix it.
Thank you Moula! The team is working now on the upgrade path for Istio. How did you find the deployment experience of 1.7 Beta?
Hi Andreea, The deployment works well except istio which always looks for the loadbalancer ip but never finds it
Hi @Moula,
That looks like whatever is providing your load balancers in k8s might be malfunctioning. I see you’re using microk8s, can you check kubectl get pods -A
and look at the metallb pods? I have a feeling something is wrong there.
Did you deploy this in the past day or two? microk8s historically pulled metallb images from dockerhub, but metallb’s dockerhub has been shut down just this week (see this for some context if interested). If your metallb pods are having image pull problems and their images are pointing to dockerhub, I think this is your problem. To work around this, microk8s 1.24/edge
points to the up to date metallb repo, and that’ll hit other microk8s risks soon. Or, if you want to recover your current deployment, you should also be able to edit the metallb deployments and update the images like here
Hi @ca-scribner I’m not the only one having this problem. See: https://github.com/canonical/bundle-kubeflow/issues/559. The problem comes from the image of Metallb as you say.
I did the same deployment of my microk8s-ha cluster as with kubeflow 1.6 which worked after a few modifications. The deployment I redid it twice today and there is always the same problem.
Hey @Moula,
Yeah I too was frustrated with this yesterday I feel your pain.
microk8s 1.24/edge works for me. Does that help? Manually changing those metallb images should also get it working, I just haven’t done that yet myself to solve things
welcome! Sorry for the frustration. Let us know how it goes
By the time you try it, things might have moved through the release ci too. atm I see
snap info microk8s
(truncated)
1.24/stable: v1.24.10 2023-02-02 (4561) 224MB classic
1.24/candidate: v1.24.10 2023-01-27 (4561) 224MB classic
1.24/beta: v1.24.10 2023-01-27 (4561) 224MB classic
1.24/edge: v1.24.11 2023-03-16 (4891) 225MB classic
where that v1.24.11 (rev 4891) works for me. Check that out before you install in case it has promoted through to the others already
Hi @Moula,
From the messages in the status of the units, I think you need to juju trust
these charms - that should get things going. Did you deploy this from one of the premade bundles, or did you add these in separately? If a premade bundle, please let me know which one as we might have missed adding trust somewhere in our bundles.
@ca-scribner I just redid the deployment by bundle. The problem is still there. We must add trust to the charm :
Thanks @moula, just so I don’t get it wrong can you tell me which bundle you’re using? Is the command juju deploy kubeflow --channel 1.7/beta
?
Hi @Moula , thanks for reporting this. You can resolve this issue by running juju trust knative-eventing --scope=cluster
, and the same for knative-serving
. We have merged the fix for that already, you should not run into this issue in latest kubeflow (1.7/beta revision 333). Please let us know if you find any other issue.
Just confirmed that latest (revision 333) deploys all components: juju deploy kubeflow --channel 1.7/beta --trust
I have successfully installed charmed-kubeflow on GCP but I don’t have access to It, how do I have access to it? (newby) I did run the following code already but no success:
juju config dex-auth public-url=http://<IP address>
juju config oidc-gatekeeper public-url=http://<IP address>
with my istio-ingressgateway ClusterIP
Hey!
I’ve never tried myself on GCP, so I’m guessing a bit…
I’m guessing you’ve done the Configure the components and login to charmed kubeflow sections of the tutorial, but no luck? Theres an “expand to troubleshoot” under that section too that might help.
Are you trying to access the dashboard from your local machine but it is hosted on the GCP machine? You might need something like a SOCKS proxy to help out. This section talks about that.
Hope these help. Please let me know!
With my deployment I found a bug in the UI interface of kubeflow. The “LOGOUT” button never works!!! I tried with 3 different deployments, the bug is still there. Capture d’écran du 2023-03-27 10-50-19|690x515