Hello, I am deploying cos-lite, and I added an overlay in order to assign a 2G storage to prometheus.
small-storage.yaml
---
applications:
prometheus:
storage:
database: 2G
The problem is that while juju storage shows the right size (2G), more than 2G can be stored in this storage. After some times this causes a “no space left” on my machine.
Here is an example of my prometheus storage:
root@prometheus-0:/var/lib/juju/storage/database/0# ls -lah
total 2.3G
drwxrwxrwx 4 root root 4.0K Apr 24 10:46 .
drwxr-xr-x 3 root root 4.0K Apr 24 09:56 ..
drwxr-xr-x 2 root root 4.0K Apr 24 10:00 chunks_head
-rw-r--r-- 1 root root 0 Apr 24 09:56 lock
-rw-r--r-- 1 root root 20K Apr 24 10:45 queries.active
-rw-r--r-- 1 root root 210M Apr 24 10:45 ros1-noetic-2022-05-23-17-23-41.bag
-rw-r--r-- 1 root root 210M Apr 24 10:46 ros1-noetic-2022-05-23-17-23-41.bag1
-rw-r--r-- 1 root root 210M Apr 24 10:46 ros1-noetic-2022-05-23-17-23-41.bag10
-rw-r--r-- 1 root root 210M Apr 24 10:46 ros1-noetic-2022-05-23-17-23-41.bag2
-rw-r--r-- 1 root root 210M Apr 24 10:46 ros1-noetic-2022-05-23-17-23-41.bag3
-rw-r--r-- 1 root root 210M Apr 24 10:46 ros1-noetic-2022-05-23-17-23-41.bag4
-rw-r--r-- 1 root root 210M Apr 24 10:46 ros1-noetic-2022-05-23-17-23-41.bag5
-rw-r--r-- 1 root root 210M Apr 24 10:46 ros1-noetic-2022-05-23-17-23-41.bag6
-rw-r--r-- 1 root root 210M Apr 24 10:46 ros1-noetic-2022-05-23-17-23-41.bag7
-rw-r--r-- 1 root root 210M Apr 24 10:46 ros1-noetic-2022-05-23-17-23-41.bag8
-rw-r--r-- 1 root root 210M Apr 24 10:46 ros1-noetic-2022-05-23-17-23-41.bag9
drwxr-xr-x 2 root root 4.0K Apr 24 09:56 wal
It’s 2.3G while I assigned 2G. This is not specific to Prometheus, any charm with a storage can actually store more than assigned.
Is there a way to enforce that an application cannot store more than the associated storage? This would avoid an application to fill the whole disk in case it accumulates too much data.
my model: cos* microk8s/localhost kubernetes available 9 admin 6 minutes ago
juju version: 3.1.8-genericlinux-amd64]