Log Aggregator Documentation

charm
, ,
1

Log Aggregator

The Log Aggregator is a Juju charm for collecting and forwarding logs in a Kubernetes environment.

Built on OpenTelemetry Collector, it receives logs from workloads via the Loki Push API and forwards them to one or more backends simultaneously:

  • Syslog servers (e.g. rsyslog, Wazuh) — via configurable syslog endpoints, with optional mutual TLS.

  • Loki — the primary COS log store.

  • Grafana Cloud — via the cloud-config relation.

It also integrates with the Canonical Observability Stack (COS) for self-monitoring: it exposes its own metrics, Grafana dashboard, and Prometheus alert rules.

Deployment

Deploy the log-aggregator charm on a Kubernetes model:


juju deploy log-aggregator

The charm requires Juju >= 3.6 and a Kubernetes cloud with the k8s-api feature available.

Configuration

Log forwarding

Forward logs to a Loki instance by integrating via send-loki-logs (see Integrations).

Forward logs to one or more syslog servers by setting the syslog_endpoints config option:


juju config log-aggregator syslog_endpoints='

- endpoint: rsyslog.example.com:514

protocol: rfc5424

network: tcp

tls_enabled: false

'

Each syslog endpoint supports the following fields:

  • endpoint (required) — Syslog server address in host:port format.

  • protocol (optional, default: rfc5424) — Message format: rfc5424 (modern) or rfc3164 (legacy BSD).

  • network (optional, default: tcp) — Transport: tcp (reliable) or udp (lossy but lower overhead).

  • tls_enabled (optional, default: false) — Enable TLS encryption.

  • tls_secret (optional) — Juju secret URI holding client certificates for mutual TLS (see Syslog with mutual TLS).

  • tls_skip_verify (optional, default: false) — Skip server certificate validation.

Syslog with mutual TLS

If your syslog server requires mutual TLS (mTLS), store your client certificate, key, and CA certificate in a Juju secret:


juju add-secret syslog-certs \

client-cert="$(cat client.crt)" \

client-key="$(cat client.key)" \

ca-cert="$(cat ca.crt)"

Note the secret-id from the output, then grant the secret to the charm and reference it in your endpoint config:


juju grant-secret syslog-certs log-aggregator

juju config log-aggregator syslog_endpoints='

- endpoint: wazuh-rsyslog.example.com:6514

protocol: rfc5424

network: tcp

tls_enabled: true

tls_secret: secret:<secret-id>

tls_skip_verify: false

'

The charm retrieves the certificate material from the Juju secret on every hook. If the secret becomes temporarily unavailable, the charm will continue to use the last successfully retrieved certificates (graceful degradation) and report the situation in the unit status — it will not overwrite a working configuration with a broken one.

Custom OTel processors

Apply custom OpenTelemetry Collector processor configuration to all pipelines. Provide a YAML snippet (without the top-level processors: key):


# processors.yaml

batch:

memory_limiter:

limit_mib: 4000



juju config log-aggregator processors=@processors.yaml

See more: charm configuration parameters, Juju config command reference.

Integrations

Receiving logs

receive-loki-logs (provides, loki_push_api) — Workloads and Promtail instances send logs to the collector via the Loki Push API.


juju integrate <workload-charm>:logging log-aggregator:receive-loki-logs

Forwarding logs

send-loki-logs (requires, loki_push_api) — Forward collected logs to a Loki backend.

cloud-config (requires, grafana_cloud_config) — Forward logs and metrics to Grafana Cloud or another COS stack.


juju integrate log-aggregator:send-loki-logs loki:logging

juju integrate log-aggregator:cloud-config grafana-cloud-integrator

COS self-monitoring

grafana-dashboards-provider (provides, grafana_dashboard) — Expose the built-in collector dashboard to Grafana.

grafana-dashboards-consumer (requires, grafana_dashboard) — Aggregate dashboards from related charms and forward them to Grafana.


juju integrate log-aggregator:grafana-dashboards-provider grafana:grafana-dashboard

juju integrate <workload-charm>:grafana-dashboard log-aggregator:grafana-dashboards-consumer

TLS

receive-server-cert (requires, tls-certificates) — Obtain a signed TLS certificate for the collector’s incoming endpoints.

receive-ca-cert (requires, certificate_transfer) — Trust the CA of servers the collector connects to (Loki, Tempo, syslog, etc.).

Service mesh

service-mesh (requires, service_mesh) — Subscribe into a service mesh (e.g. Istio) to enforce authorisation policies.

provide-cmr-mesh (provides, cross_model_mesh) — Allow cross-model applications to reach the collector via the service mesh.

require-cmr-mesh (requires, cross_model_mesh) — Allow the collector to reach a cross-model application via the service mesh.


juju integrate log-aggregator:service-mesh istio-beacon:service-mesh

The charm also supports metrics scraping (metrics-endpoint), trace forwarding (receive-traces, send-traces), and profiling (receive-profiles, send-profiles). See the full Integrations reference for details.

Actions

reconcile

Force the charm to regenerate its full configuration from scratch, including the OpenTelemetry Collector config file and all relation data:


juju run log-aggregator/leader reconcile

This is useful after manually resolving a stuck state or after updating a Juju secret (e.g. syslog TLS certificates) when the charm has not yet reacted to the secret-changed event.

Troubleshooting

Check the unit status

The charm sets descriptive status messages to help diagnose issues:


juju status log-aggregator

Common status messages:

  • Active — Collector is running with a valid configuration.

  • Active (using cached certs for syslog endpoints: N) — Secret retrieval failed temporarily; the last successfully fetched certificates are in use. The charm will retry automatically.

  • Active (syslog endpoints degraded — no certs: N) — Secret retrieval failed and no cached certs exist; those endpoints are using TLS without client certificates.

  • Waiting — CSR sent; otelcol down while waiting for a cert — A TLS relation exists but the signed certificate has not yet arrived.

  • Blocked — A configuration error needs to be resolved (the message will say what).

Inspect the collector logs


juju debug-log --include log-aggregator

Or exec directly into the container:


juju ssh --container otelcol log-aggregator/0 /bin/sh

Force a configuration reload


juju run log-aggregator/leader reconcile

Further information

Project and community

The Log Aggregator charm is an open-source project that welcomes community contributions, suggestions, fixes and constructive feedback.