Observability (elk/logstash) charm proxy request

charm
,
#1

Hi,

i have an external ELK (elastic-kibana) stack for log collection and i would like to use it with juju ecosystem.

if i look at Deploy Filebeat using Charmhub - The Open Operator Collection it always needs 2 relations: one to the target application (even a simple machine) the other to the logstash server.

i wonder if we might skip the second one and manually add infos such ad fqdn/ip and keys or auth infos, or provide a proxy charm to do this for us.

Thank you

#2

Hi Patrizio,

The charm you are referring to is currently in the maintenance phase, meaning that it only receives critical updates. However, if this is something you’d be willing to contribute yourself, I’m happy for us to try to accommodate with reviews and such to make it happen.

Another, possibly much easier, alternative would be to create a new small charm, which serves only the purpose of providing this information through the relation. This way, no changes would be needed in the existing charm.

Best, Simon

#3

Dear Simon,

can you tell me what technology/client is under active development?

#4

Hi Patrizio,

Have a look at the Canonical Observability Stack, and in particular, COS Lite. In particular, loki, promtail, as well as grafana-agent, would be the main tools we’re betting on for log ingestion.

Also feel free to join the observability channel on mattermost if you want to stay in the loop about what we’re doing around observability going forward! :sunny:

Best, Simon

#5

Hi,

we know COS and we are waiting for GA release. But what about a “basic” Ubuntu VM? what would be a suggested or supported path for log shipping? filebeat was a easy component to deploy and subordinate and ship logs. Is for instance Telegraf with elasticsearch plugin supported?

#6

we are waiting for GA release

Good news! COS Lite is GA as of the beginning of November! :tada:

But what about a “basic” Ubuntu VM?

The grafana agent machine charm we’re currently building will be the solution for this going forward. You will attach it to an existing machine as a subordinate and have it ship the log files for you.