Microk8s / Kubeflow 1.6 / NFS failed. Alternative storage solution?

I have a single node microk8s cluster using the nfs-csi storage class. I installed Kubeflow 1.6 following the install guide.

I first had this error in kfp-db, which was related to the NFS server’s permissions:

chown: changing ownership of '/var/lib/mysql/': Operation not permitted

I was able to solve that by editing the kfp-db statefulset by change the securityContext to:

securityContext:
  runAsUser: 999

Then I encountered a second error, this time in kfp-api:

I1021 18:40:19.862833       7 client_manager.go:160] Initializing client manager
I1021 18:40:19.862899       7 config.go:57] Config DBConfig.ExtraParams not specified, skipping
F1021 18:40:25.859359       7 error.go:337] commands out of sync. Did you run multiple statements at once?

Tried this solution but didn’t work.


I’m guessing this is caused by using NFS as my storage class, since when I used hostpath storage, there were no such issues.

Are there ways to solve this issue? Or is there another way to set this up without using hostpath storage?

Ended up using OpenEBS’s Jiva for storage.