Cdk & existing layer 2 services

i’m trying to get the pods deployed with the canonical distribution of kubernetes to communicate with the existing layer 2 vlan fabric, like ceph. question is how can i attach volumes from maas provisioned bare metal ceph to maas provisioned baremetal kubernetes pods. lemme know if i’m off track with this but the idea is to replace the flannel CNI with something like a bridge interface for the pods to network with the services on one of the VLANs in the datacenter. I’ve tried deploying the CDK without flannel and the kubeconfig never gets made. I’ve also tried deploying cdk with flannel and appending multus cni. Anyone ever had any luck replacing the default flannel CNI with something including layer 2? below is some bash i’ve stepped thru

    juju deploy ./core.yaml

    #wait a bit 

    juju config kubernetes-worker allow-privileged=true

    juju config kubernetes-master allow-privileged=true

    juju config kubernetes-master authorization-mode="RBAC,Node"

    #wait a bit
    juju scp kubernetes-master/0:config ~/.kube/config
    #git clone https://github.com/intel/multus-cni.git 
    cd multus-cni

    cat ./images/multus-daemonset.yml | kubectl apply -f -
    #cat ./images/{multus-daemonset.yml,flannel-daemonset.yml} | kubectl apply -f -

    cd..
    kubectl apply -f cnicrddhcp.yaml
    Kubectl apply -f testpod.yaml

maybe i just use the calico charm?

Let’s see if @tvansteenburgh and the CDK crew have any tips/tricks for you.