Metallb won't assign IP-addresses to services after reboot

charm
, ,
#1

Hey, I have setup COS-lite observability stack on my computer with microk8s and juju by following some threads here in the discourse.

The thing is, after I enable the addon metallb with the following IP-range everything works just fine. But after a reboot of the system traefik won’t get an IP-address assigned from metallb.

What am I missing here? and is metallb the way to go in a production environment? We are planning to use this with grafana-agent later in a local environment.

After deploying cos-lite

image
image843×285 107 KB

After rebooting the machine

image
image843×285 111 KB

#2

I believe I’ve seen this issue and its related to the fact that the service definition for the deployed charms gets reverted back to type: ClusterIP rather than type: LoadBalancer as patched by the traefik charm.

The code that does this patching and hence exposes a service via a LoadBalancer only runs on install/upgrade-charm in older lib versions - newer versions of the patcher lib do this on upgrade charm as well.

1 Like
#3

@james-page is this an issue that we can resolve by configuring the charms/models in Juju or is this a bug?

#4

I think what he just said is that it is/(used to be) a bug in the KubernetesServicePatch lib that should be fixed. What revision of traefik are you using? and what is the version of KubernetesServicePatch?

tip: jhack charm vinfo traefik

for me it shows: image

2 Likes
#5

image

A bit older version it seems. Should I build the package my self instead to get the latest versions? Been using the bundle (juju deploy cos-lite --trust)

EDIT: ah you have the edge version. will try that instead. Thanks!

#6

@marcus - to you mean the edge version of traefik ?

#7

Yeah, we use the stable version.

#8

Yeah, totally - but I think we can simply switch then to the edge without redeploying perhaps… @ppasotti ?

#9

I think you can upgrade the charm to edge without breaking stuff, but either way redeploying on k8s is cheap (I presume you’re talking about a development environment)

1 Like
#10

Yeah.

Is your @ppasotti recommendation to do a “export-bundle” and then change the channel to edge for traefik ?

#11

@erik-lonroth no need: you can deploy the edge bundle j deploy cos-lite --channel=edge --trust

#12

@marcus I chased down the issue a bit and it turns out this pr https://github.com/canonical/observability-libs/pull/46/files is not released (even in the lib). so the bugfix isn’t live. Working to see if we can squeeze out a hotfix sometime soon

2 Likes
#13

Oh thanks! got it to work by just change traefiks channel to edge with juju refresh traefik --channel=edge everything works now after a reboot! :smiley:

2 Likes
#14

I can confirm that I also got this to work with

juju refresh traefik --channel=edge

The address is still there now which is great. I think this for sure would be a great thing to update the stable version since this will haunt us for sure going forward.

3 Likes
Cross Model Integration COS light lxd-plus-microk8 makes juju error