1. Determine what you want to profile
Say you have a juju model with a postgresql instance
juju add-model test-profiling
juju deploy postgresql \
# request a virtual machine, else Juju may give us a container incapable of eBPF
--constraints="virt-type=virtual-machine" \
# otel-ebpf-profiler only has ubuntu@24.04-base releases, we need this version so the bases match
--base ubuntu@24.04 --channel 16/edge \
pgql
Now after some time the juju status will show that the pgql application is being deployed on a machine with a certain ID, in my case 1:
Model Controller Cloud/Region Version SLA Timestamp
test-profiling lxd localhost/localhost 3.6.2 unsupported 14:30:06+02:00
App Version Status Scale Charm Channel Rev Exposed Message
pgql 16.9 maintenance 1 postgresql 16/edge 885 no installing PostgreSQL
Unit Workload Agent Machine Public address Ports Message
pgql/0* maintenance executing 1 fd42:a207:65c0:dd4f:216:3eff:fe4b:8a62 (install) installing PostgreSQL
Machine State Address Inst id Base AZ Message
1 started fd42:a207:65c0:dd4f:216:3eff:fe4b:8a62 juju-a7a9ba-1 ubuntu@24.04 Running
Integration provider Requirer Interface Type Message
pgql:database-peers pgql:database-peers postgresql_peers peer joining
pgql:refresh-v-three pgql:refresh-v-three refresh peer joining
pgql:restart pgql:restart rolling_op peer joining
Make note of the machine you want to profile (1 in this case), because next we’ll need to deploy the otel-ebpf-profiler charm to the same machine.
2. deploy the otel-ebpf-profiler
juju deploy otel-ebpf-profiler --channel latest/edge --base ubuntu@24.04 \
# deploy to the same machine!
--to 1 \
profiler
Co-locating the charms on machine 1 ensures that the otel-ebpf-profiler will be collecting profiles from the same (v)CPU that’s also running postgresql.
Deploying the otel-ebpf-profiler to a different machine will mean you’re profiling a machine that’s not running anything interesting (other than the otel-ebpf-profiler itself, naturally).
Deploying otel-ebpf-profiler to a machine which already has an otel-ebpf-profiler running on it is not supported. otel-ebpf-profiler, on install, claims an exclusive lock on the (Juju) machine it’s deployed on, to prevent duplicate profiling data to be generated (which causes unnecessary overhead and is most likely a user error).
If you are trying to multiplex the output stream of profiling data, you should send the profiles to an opentelemetry-collector instance and configure its profiling pipeline to send copies of all profile data to multiple remote receivers.
At this point the juju model should look something like:
Unit Workload Agent Machine Public address Ports Message
pgql/0* active idle 1 fd42:a207:65c0:dd4f:216:3eff:fe4b:8a62 5432/tcp Primary
profiler/0* active idle 1 fd42:a207:65c0:dd4f:216:3eff:fe4b:8a62 profiling machine 1
3. Send profiles to pyroscope through opentelemetry-collector
In most (production) scenarios we recommend to send all profiling data to an opentelemetry collector instance co-located with the profiler (to minimise traffic cost and optimize the value gained from any sampling being applied by the collector). The opentelemetry collector will then forward profiles to a backend enabled to store and process them, such as pyroscope.
If you have access to a Juju model somewhere with Pyroscope (see tutorial), you can cross-model relate the otel-ebpf-profiler with pyroscope over otelcol by doing:
juju deploy opentelemetry-collector otelcol --channel 2/edge
juju relate otelcol:cos-agent profiler # to get otelcol deployed and assigned to a machine
juju relate profiler:profiling otelcol # to start forwarding profiles
juju consume k8s-controller:admin/pyro.pyroscope
juju relate otelcol:send-profiles pyro:profiling
As soon as Juju settles and the applications report active/idle, you should be able to open the Pyroscope UI (at <pyroscope application IP>:8080) and inspect the pyroscope datasource to find profiles from the postgresql process.
4. Integrate directly with pyroscope and COS
Instead of integrating through an otel collector, it is also possible (for development/testing purposes) to integrate the otel-ebpf-profiler directly to a pyroscope instance.
# in the pyroscope model
juju offer pyroscope:profiling
#back in the test-profiling model
juju consume k8s-controller:admin/pyro.pyroscope
juju relate profiler:profiling pyro:profiling
this will require you to bridge the networks manually in such a way that the profiler application can resolve the pyroscope URL. If pyroscope is not related to an ingress, it will advertise its cluster-internal FQDN which the profiler application will probably not be able to resolve in most vanilla DNS configurations.