The upstream OpenTelemetry Collector (otelcol) supports a wide range of architectures. The charmed version simplifies deployment and integration into various architectures, allowing administrators to configure multiple otelcol instances with specific microservice-like roles instead of relying on a single otelcol with numerous processors. This chaining approach clearly defines the responsibilities of each otelcol instance, making it easy to replace one without losing all processing at once.
One imaginable scenario is splitting a log stream into hot and cold data based on log levels. The first Otel-collector collecting logs directly from Flog has a batch processor to improve the efficiency of both log streams via compression. Low-severity levels like DEBUG and INFO often have a greater frequency in log streams and indicate normal workload operation. This can be filtered out in a log stream which is sent to long-term (cold) storage to minimize cost while maintaining compliance. Alternatively, the hot storage could include INFO logs, since storage is less costly, while filtering out DEBUG logs.
To understand how to filter telemetry with otelcol, refer to the selectively drop telemetry documentation or see the examples for log-level filtering.
Hot storage processor config
filter/include:
logs:
include:
severity_number:
min: "INFO"
match_undefined: true
This filters out all logs below “INFO” level (the whole DEBUG and TRACE ranges). Logs with no defined severity will also be matched.
Cold storage processor config
filter/include:
logs:
include:
severity_number:
min: "WARN"
match_undefined: true