Logs take up quite a bit of storage, and querying them can be cpu-intensive. Node sizing depends on:
- Logging rate
- Desired buffer for spikes in logging rate
- Desired retention period
With such information at hand, you can refer to the performance envelope for sizing guidelines.
How to measure a workload’s logging rate
The loki distriburor already exports the metrics we need to determine logging rate. Deploy your solution, and relate workload under test to loki, for example:
Be sure to relate to loki only the workload(s) you’re measuring the rate for.
Usually a 48hr view of the following expressions should give a good idea about logging rate:
- GB/day:
sum(rate(loki_distributor_bytes_received_total[5m]) / 1e9 * 60*60*24) - Lines/min:
sum(rate(loki_distributor_lines_received_total[5m]) * 60)
