X509: certificate signed by unknown authority

charm
,
1

I’m trying to setup kubeflow using microk8s.

However, when I deploy the kubeflow-lite bundle with juju, argo-controler is in “blocked” status, with a “Waiting for object-storage relation data” message and everything else (except for admission-webhook) is waiting to install the agent.

image
image1307×854 339 KB

The deployment is behind a corporate proxy, so I have configured the proxy address during bootstrap my controller using model-default to be sure that it would propagate everywhere. I have setup the http-proxy, https-proxy and no-proxy options. Checking “juju model-config” shows that the proxy options has the good values.

Looking at “microk8s kubectl get events”, I see a “FailedCreate” Warning for most of the apps in Waiting status, all with the same message:

“create Pod training-operator-0 in StatefulSet training-operator failed error: Internal error occurred: failed calling webhook “admission-webhook.kubeflow.org”: Post “https://admission-webhook.kubeflow.svc:4443/apply-poddefault?timeout=10s”: x509: certificate signed by unknown authority”

Can someone help me find from where is that certificate error is coming from?

2

Hiya, I’ve just put together this how-to guide to deploying Charmed Kubeflow behind a proxy:

Please check it and let me know if you have further problems. Note that if you choose to change the Juju controller name you will need to adjust the no_proxy and NO_PROXY environment variables accordingly.

Rob

3

Delete old webhooks of kubeflow might work:

# check current webhooks
kubectl get validatingWebhookConfiguration
kubectl get mutatingWebhookConfiguration

# delete old webhooks
kubectl delete validatingWebhookConfiguration <old_webhook>
kubectl delete mutatingWebhookConfiguration <old_webhook>
4

Hi, I’m seeing this issue for the admission-webhook with Kubeflow 1.6. But I’m not using a proxy. If I describe the mutatingWebhookConfiguration I see "Ca Bundle: Cg== ". If I remove-application admission-webhook and re-deploy it, it works until I reboot the node. Then, Ca Bundle gets equal to Cg== again.