Integrate your charm with PostgreSQL

sorry, all this discussion is out of scope of this tutorial

please ask you question on: Mattermost

1 Like

For the history:

I have a tricky situation now, since I can’t seem to re-deploy the old charm (revision 246) charm, since its no longer available in the store.

It is available in the store, but you have faced a bug Bug #2020899 “cannot resolve charm or bundle “mongodb”: resolvin...” : Bugs : MongoDB Charm (the legacy charm should be installable again from latest/stable channel soon). The possible workaround there is to define “series” on deployment.

The manual How-to Migrate is on the way, the link point to the proper placeholder.

1 Like

I was reading through this and noticed that the link “The location of a charm library inside a charm” is to an archived page and inaccessible on the production docs.

The link should now be pointing to a subsection inside of the general “Library” explanation, which is here: Library

Another thing I noticed was that the postgresql-k8s charm itself required --trust to be deployed. I found this out by looking at the charm documentation and saw that the machine charm can be run without --trust but the k8s charm requires some changes to service accounts (maybe? or at least something with privileges).

So the line:

juju deploy postgresql-k8s --channel=14/stable

Should become

juju deploy postgresql-k8s --channel=14/stable --trust

@tmihoc can you please have a look

I recently tested these instructions. From what I remember, the code as given works – so, without trust. I’ll check again though.

Fixed, thanks!

I think what I found was that it would deploy, but end up at a “blocked” state. It wasn’t until I dived into the logs that I saw it trying to do things with service accounts and then I found the postgresql-k8s charm page itself (Charmhub | Deploy Charmed PostgreSQL K8s using Charmhub - The Open Operator Collection) had a --trust in its tutorial.

1 Like

@tmihoc if psql itself points to be trusted, then we need to add trust here

1 Like

Done, thanks!

key is never used in the loop of the above code. If there is no reason to evaluate the key attribute we may want to make the code easier to understand, e.g. like this:

def fetch_postgres_relation_data(self) -> dict:
    relations = self.database.fetch_relation_data()
    logger.debug("Got following database data: %s", relations)
    for data in relations.values():
        if not data:
            continue
        logger.info("New PSQL database endpoint is %s", data["endpoints"])
        ...

true! thanks

@tmihoc can you please update the post and git repo ?

2 Likes

Done, thanks!

1 Like

The method fetch_postgres_relation_data in the charms.py is broken from lection 4 on. https://github.com/canonical/juju-sdk-tutorial-k8s/blob/04_integrate_with_psql/src/charm.py#L107C1-L131C28

    def fetch_postgres_relation_data(self) -> dict:
        """Fetch postgres relation data.

        This function retrieves relation data from a postgres database using
        the `fetch_relation_data` method of the `database` object. The retrieved data is
        then logged for debugging purposes, and any non-empty data is processed to extract
        endpoint information, username, and password. This processed data is then returned as
        a dictionary. If no data is retrieved, the unit is set to waiting status and
        the program exits with a zero status code."""
        relations = self.database.fetch_relation_data()
        logger.debug("Got following database data: %s", relations)
        for data in relations.values():
            if not data:
                continue
            logger.info("New PSQL database endpoint is %s", data["endpoints"])
            host, port = val["endpoints"].split(":")
            db_data = {
                "db_host": host,
                "db_port": port,
                "db_username": val["username"],
                "db_password": val["password"],
            }
            return db_data
        self.unit.status = WaitingStatus("Waiting for database relation")
        raise SystemExit(0)

should be

    def fetch_postgres_relation_data(self) -> dict:
        """Fetch postgres relation data.

        This function retrieves relation data from a postgres database using
        the `fetch_relation_data` method of the `database` object. The retrieved data is
        then logged for debugging purposes, and any non-empty data is processed to extract
        endpoint information, username, and password. This processed data is then returned as
        a dictionary. If no data is retrieved, the unit is set to waiting status and
        the program exits with a zero status code."""
        relations = self.database.fetch_relation_data()
        logger.debug("Got following database data: %s", relations)
        for data in relations.values():
            if not data:
                continue
            logger.info("New PSQL database endpoint is %s", data["endpoints"])
            host, port = data["endpoints"].split(":")
            db_data = {
                "db_host": host,
                "db_port": port,
                "db_username": data["username"],
                "db_password": data["password"],
            }
            return db_data
        self.unit.status = WaitingStatus("Waiting for database relation")
        raise SystemExit(0)

(Variable name val is not defined and should be data)

I already changed that in the doc post. Should I do a PR for the Github repo or directly pushing it to the branch?

1 Like

@tmihoc please suggest how to go with it on GitHub

@bschimke95 The process we’ve followed so far is direct push to the branch + updates to the subsequent branches by successive merge (switch to the next branch, merge the previous branch into it, repeat).

Done, thanks!

M1/M2 mac users, arm64 users might get the following error:

sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) SCRAM authentication requires libpq version 10 or above

This is because of a bug mentioned here: https://github.com/psycopg/psycopg2/issues/1360.

I believe the docker image used in the tutorial is from this repo: https://github.com/canonical/api_demo_server, and it’s using a relatively older version of the psycopg2, see here: https://github.com/canonical/api_demo_server/blob/master/pyproject.toml#L17.

The bug is fixed from 2.9.5.

A workaround would be to clone the above repo, change the version of psycopg2 to the latest (I used 2.9.9) and build the image.

I’ll submit a PR to fix this issue in the original repo as well.

2 Likes

The docstring in the function below is over-indented and produces an error when I use it.

def fetch_postgres_relation_data(self) -> dict:
        """Fetch postgres relation data.

        This function retrieves relation data from a postgres database using
        the `fetch_relation_data` method of the `database` object. The retrieved data is
        then logged for debugging purposes, and any non-empty data is processed to extract
        endpoint information, username, and password. This processed data is then returned as
        a dictionary. If no data is retrieved, the unit is set to waiting status and
        the program exits with a zero status code."""
    relations = self.database.fetch_relation_data()
    logger.debug("Got following database data: %s", relations)
    for data in relations.values():
        if not data:
            continue
        logger.info("New PSQL database endpoint is %s", data["endpoints"])
        host, port = data["endpoints"].split(":")
        db_data = {
            "db_host": host,
            "db_port": port,
            "db_username": data["username"],
            "db_password": data["password"],
        }
        return db_data
    raise DatabaseNotReady()

I also notice in that same function in the tutorial we use a previously-created Exception class called DatabaseNotReady(), but in the provided code, that class is not used and the line self.unit.status = WaitingStatus("Waiting for Pebble in the workload container) is used instead. I have seen WaitingStatus() used in other instructional material so it may best to keep things consistent unless there was another reason we have the Exception class in the tutorial.

Thanks - fixed!

The status is still set to Waiting (code higher up catches the exception and sets the status). The older version of the code did a system exit in the handler, which isn’t something that we recommend.

The code’s a little behind the docs - there was a bit of a mixup with handling changes to the branches, but we’re getting that fixed this week.

1 Like