Ceilometer/Gnocchi metrics and duplicate "services" project

I have a functional test/PoC openstack deployment based on ubuntu@22.04/stable, 2023.2/stable and ceph reef. All services running and I can deploy working instances with functional network etc.

I am attempting to configure Ceilometer and Gnocchi, and while the configuration appears correct, I get errors in Ceilometer about nonUnique project names in Keystone. The issue relates to the fact their are 2 projects called “services”, one in the Default domain and the other in “service_domain”.

  • ceilometer-agent-notification.log
2024-09-11 10:49:02.754 798958 ERROR ceilometer.publisher.gnocchi [-] Failed to retrieve filtered project [services].: keystoneclient.exceptions.NoUniqueMatch: ClientException
2024-09-11 10:49:02.754 798958 ERROR ceilometer.publisher.gnocchi Traceback (most recent call last):
2024-09-11 10:49:02.754 798958 ERROR ceilometer.publisher.gnocchi   File "/usr/lib/python3/dist-packages/ceilometer/publisher/gnocchi.py", line 288, in gnocchi_project_id
2024-09-11 10:49:02.754 798958 ERROR ceilometer.publisher.gnocchi     project = self._ks_client.projects.find(
2024-09-11 10:49:02.754 798958 ERROR ceilometer.publisher.gnocchi   File "/usr/lib/python3/dist-packages/keystoneclient/v3/projects.py", line 225, in find
2024-09-11 10:49:02.754 798958 ERROR ceilometer.publisher.gnocchi     p = super(ProjectManager, self).find(**kwargs)
2024-09-11 10:49:02.754 798958 ERROR ceilometer.publisher.gnocchi   File "/usr/lib/python3/dist-packages/keystoneclient/base.py", line 86, in func
2024-09-11 10:49:02.754 798958 ERROR ceilometer.publisher.gnocchi     return f(*args, **new_kwargs)
2024-09-11 10:49:02.754 798958 ERROR ceilometer.publisher.gnocchi   File "/usr/lib/python3/dist-packages/keystoneclient/base.py", line 506, in find
2024-09-11 10:49:02.754 798958 ERROR ceilometer.publisher.gnocchi     raise ksc_exceptions.NoUniqueMatch
2024-09-11 10:49:02.754 798958 ERROR ceilometer.publisher.gnocchi keystoneclient.exceptions.NoUniqueMatch: ClientException
2024-09-11 10:49:02.754 798958 ERROR ceilometer.publisher.gnocchi
2024-09-11 10:49:02.754 798958 ERROR ceilometer.pipeline.sample [-] Pipeline network_sink: Continue after error from publisher <ceilometer.publisher.gnocchi.GnocchiPublisher object at 0x7fde60103340>: keystoneclient.exceptions.NoUniqueMatch: ClientException
+----------------------------------+----------------------------+----------------------------------+-----------------------------------------------+---------+
| ID                               | Name                       | Domain ID                        | Description                                   | Enabled |
+----------------------------------+----------------------------+----------------------------------+-----------------------------------------------+---------+
| ec5964382c5b422bb76bfaa46d69a983 | services                   | 5be35b1bbb5a41a5a60cef025ebf9f93 | Created by Juju                               | True    |
| f17bbdea04344547bd31dece98dc787a | services                   | default                          | Created by Juju                               | True    |
+----------------------------------+----------------------------+----------------------------------+-----------------------------------------------+---------+
+----------------------------------+----------------+---------+--------------------+
| ID                               | Name           | Enabled | Description        |
+----------------------------------+----------------+---------+--------------------+
| 5be35b1bbb5a41a5a60cef025ebf9f93 | service_domain | True    | Created by Juju    |
| default                          | Default        | True    | The default domain |
+----------------------------------+----------------+---------+--------------------+

This duplicate I believe was/is to support the v2 and v3 Keystone API’s with v3 introducing “domains”.

  • ceilometer.conf
[service_credentials]
...
project_name = services
project_domain_name = service_domain
user_domain_name = service_domain
...
[keystone_authtoken]
...
project_domain_name = service_domain
user_domain_name = service_domain
project_name = services
...
  • pipeline.yaml
sinks:
  - name: cpu_sink
    publishers:
      - gnocchi://?filter_domain=service_domain&filter_project=services&archive_policy=fleio_policy
  - name: network_sink
    publishers:
      - gnocchi://?filter_domain=service_domain&filter_project=services&archive_policy=fleio_policy
  - name: object_storage_sink
    publishers:
      - gnocchi://?filter_domain=service_domain&filter_project=services&archive_policy=fleio_policy
  - name: ceph_sink
    publishers:
      - gnocchi://?filter_domain=service_domain&filter_project=services&archive_policy=fleio_policy
...

^ Have attempted with and without filter_domain, but that does seem to be a valid filter in /usr/lib/python3/dist-packages/ceilometer/publisher/gnocchi.py

I did attempt to prove the issue was the duplicate, by deleting one of the services projects, prior to a full redeployment, and the metrics were then successful and above error gone. This however breaks everything, and I can no longer deploy/delete instances etc. It seems Keystone grants to both “services” projects when deploying and fails to deploy services when either version is deleted.

2024-09-10 17:01:10 INFO unit.keystone/2.juju-log server.go:325 Granting user 'neutron' role 'Admin' on tenant 'services' in domain 'default'
2024-09-10 17:01:12 INFO unit.keystone/2.juju-log server.go:325 Granting user 'neutron' role 'Admin' on tenant 'services' in domain 'service_domain'
2024-09-10 17:01:22 INFO unit.keystone/2.juju-log server.go:325 Granting user 'neutron' role 'Admin' on tenant 'services' in domain 'default'
2024-09-10 17:01:24 INFO unit.keystone/2.juju-log server.go:325 Granting user 'neutron' role 'Admin' on tenant 'services' in domain 'service_domain'
2024-09-10 17:01:33 INFO unit.keystone/2.juju-log server.go:325 Granting user 'neutron' role 'Admin' on tenant 'services' in domain 'default'
2024-09-10 17:01:35 INFO unit.keystone/2.juju-log server.go:325 Granting user 'neutron' role 'Admin' on tenant 'services' in domain 'service_domain'
2024-09-10 17:01:44 INFO unit.keystone/2.juju-log server.go:325 Granting user 'nova' role 'Admin' on tenant 'services' in domain 'default'
2024-09-10 17:01:46 INFO unit.keystone/2.juju-log server.go:325 Granting user 'nova' role 'Admin' on tenant 'services' in domain 'service_domain'
2024-09-10 17:01:55 INFO unit.keystone/2.juju-log server.go:325 Granting user 'nova' role 'Admin' on tenant 'services' in domain 'default'
2024-09-10 17:01:58 INFO unit.keystone/2.juju-log server.go:325 Granting user 'nova' role 'Admin' on tenant 'services' in domain 'service_domain'
2024-09-10 17:02:06 INFO unit.keystone/2.juju-log server.go:325 Granting user 'nova' role 'Admin' on tenant 'services' in domain 'default'
2024-09-10 17:02:08 INFO unit.keystone/2.juju-log server.go:325 Granting user 'nova' role 'Admin' on tenant 'services' in domain 'service_domain'

Any pointers in how I can fix this issue without deleting one of the services projects, which broke other services. Both seem to be required, yet the fact there are 2 projects called “services” is breaking ceilometer.

1 Like

Replying to my own post, just in case someone else runs into the same issue. The problem is with the lines above. &filter_domain= does nothing, it does not actually filter the domain and so you end up with 2 results for “services”. Switched to &filter_project_id= giving it the specific ID for the services project in the service_domain domain.

e.g.

gnocchi://?filter_project_id=1234567890abde&archive_policy=fleio_policy

Metrics are now working!