From Zero to Hero: Write your first Kubernetes charm > Integrate your charm with PostgreSQL
See previous: Expose the version of the application behind your charm
This document is part of a series, and we recommend you follow it in sequence. However, you can also jump straight in by checking out the code from the previous branches:
git clone https://github.com/beliaev-maksim/k8s-charm-tutorial.git
cd k8s-charm-tutorial
git checkout 03_set_workload_version
git checkout -b 04_integrate_with_psql
A charm often requires or supports relations to other charms. For example, to make our application fully functional we need to connect it to the PostgreSQL database. In this chapter of the tutorial we will update our charm so that it can be integrated with the existing PostgreSQL charm.
Contents:
- Add the necessary database interface charm libraries
- Define the charm relation interface
- Import the database interface libraries and define database event handlers
- Validate your charm
- Review the final code
Add the necessary database interface charm libraries
To set up a database relation in a charm, we need to fetch the data_interfaces charm library from Charmhub. Charm libraries are fetched from Charmhub using Charmcraft. Below we use Charmcraft to log in to Charmhub and then fetch the library.
Log in to Charmhub
To log in to Charmhub, run:
charmcraft login --export ~/secrets.auth
This should output:
Opening an authorization web page in your browser.
If it does not open, please open this URL:
https://api.jujucharms.com/identity/login?did=48d45d919ca2b897a81470dc5e98b1a3e1e0b521b2fbcd2e8dfd414fd0e3fa96
As we are inside a Multipass instance, this will not open the authorization web page directly. Copy-paste the provided web link into your web browser instead. Use your Ubuntu SSO to log in.
When you’re done, you should see in your terminal the following:
Login successful. Credentials exported to '~/secrets.auth'.
Now set an environment variable with the new token:
export CHARMCRAFT_AUTH=$(cat ~/secrets.auth)
Read more: How to authenticate Charmcraft in remote environments
Fetch the required database interface charm libraries
Now we’re ready to fetch the data_interfaces charm library.
Navigate to your charm directory and run:
ubuntu@charm-dev:~/fastapi-demo$ charmcraft fetch-lib charms.data_platform_libs.v0.data_interfaces
Your charm directory should now contain the structure below:
lib
└── charms
└── data_platform_libs
└── v0
└── data_interfaces.py
Well done, you now have all the charm libraries required to set up the relation.
Define the charm relation interface
First, find out the name of the interface that PostgreSQL offers for other charms to connect to it. According to the documentation of the PostgreSQL charm, the interface is called postgresql_client.
Next, open the metadata.yaml file of your charm and, before the containers section, define a relation endpoint using a requires block, as below. This endpoint says that our charm is requesting a relation called database over an interface called postgresql_client with a maximum number of supported connections of 1. (Note: Here, database is a custom relation name, though in general we recommend sticking to default recommended names for each charm.)
requires:
database:
interface: postgresql_client
limit: 1
That will tell juju that our charm can be integrated with charms that provide the same postgresql_client interface, for example, the official PostgreSQL charm.
Read more: File ‘metadata.yaml’
Import the database interface libraries and define database event handlers
We now need to implement the logic that wires our application to a database. When a relation between our application and the data platform is formed, the provider side (i.e., the data platform) will create a database for us and it will provide us with all the information we need to connect to it over the relation – e.g., username, password, host, port, etc. On our side, we nevertheless still need to set the relevant environment variables to point to the database and restart the service.
To do so, we need to update our charm “src/charm.py” to do all of the following:
- Import the
DataRequiresclass from the interface library; this class represents the relation data exchanged in the client-server communication. - Define the event handlers that will be called during the relation lifecycle.
- Bind the event handlers to the observed relation events.
Import the database interface libraries
First, at the top of the file, import the database interfaces library:
# Import the 'data_interfaces' library.
# The import statement omits the top-level 'lib' directory
# because 'charmcraft pack' copies its contents to the project root.
from charms.data_platform_libs.v0.data_interfaces import DatabaseCreatedEvent
from charms.data_platform_libs.v0.data_interfaces import DatabaseRequires
Add relation event observers
Next, in the __init__ method, define a new instance of the ‘DatabaseRequires’ class. This is required to set the right permissions scope for the PostgreSQL charm. It will create a new user with a password and a database with the required name (below, names_db), and limit the user permissions to only this particular database (that is, below, names_db).
# The 'relation_name' comes from the 'metadata.yaml file'.
# The 'database_name' is the name of the database that our application requires.
# See the application documentation in the GitHub repository.
self.database = DatabaseRequires(self, relation_name="database", database_name="names_db")
Now, add event observers for all the database events:
# See https://charmhub.io/data-platform-libs/libraries/data_interfaces
self.framework.observe(self.database.on.database_created, self._on_database_created)
self.framework.observe(self.database.on.endpoints_changed, self._on_database_created)
Finally, set up an event observer for the standard Juju OLM relation-broken event, as below. This will handle the situation where the charm user decides to remove the relation to the database.
self.framework.observe(self.on.database_relation_broken, self._on_database_relation_removed)
Read more: Event
<relation name>-relation-broken
Fetch the database authentication data
Now we need to extract the database authentication data and endpoints information. We can do that by adding a fetch_postgres_relation_data method to our class. Inside this method, we first retrieve relation data from the PostgreSQL using the fetch_relation_data method of the database object. We then log the retrieved data for debugging purposes. Next we process any non-empty data to extract endpoint information, the username, and the password and return this process data as a dictionary. Finally, we ensure that, if no data is retrieved, the unit status is set to WaitingStatus and the program exits with a zero status code.
def fetch_postgres_relation_data(self) -> dict:
data = self.database.fetch_relation_data()
logger.debug("Got following database data: %s", data)
for key, val in data.items():
if not val:
continue
logger.info("New PSQL database endpoint is %s", val["endpoints"])
host, port = val["endpoints"].split(":")
db_data = {
"db_host": host,
"db_port": port,
"db_username": val["username"],
"db_password": val["password"],
}
return db_data
self.unit.status = WaitingStatus("Waiting for database relation")
raise SystemExit(0)
Share the authentication information with your application
Our application consumes database authentication information in the form of environment variables. Let’s update the Pebble service definition with an environment key and let’s set this key to a dynamic value – the class property self.app_environment. Your _pebble_layer property should look as below:
@property
def _pebble_layer(self):
"""Return a dictionary representing a Pebble layer."""
command = " ".join(
[
"uvicorn",
"api_demo_server.app:app",
"--host=0.0.0.0",
f"--port={self.config['server-port']}",
]
)
pebble_layer = {
"summary": "FastAPI demo service",
"description": "pebble config layer for FastAPI demo server",
"services": {
self.pebble_service_name: {
"override": "replace",
"summary": "fastapi demo",
"command": command,
"startup": "enabled",
"environment": self.app_environment,
}
},
}
return Layer(pebble_layer)
Now, let’s define this property such that, every time it is called, it dynamically fetches database authentication data and also prepares the output in a form that our application can consume, as below:
@property
def app_environment(self):
"""This property method creates a dictionary containing environment variables
for the application. It retrieves the database authentication data by calling
the `fetch_postgres_relation_data` method and uses it to populate the dictionary.
If any of the values are not present, it will be set to None.
The method returns this dictionary as output.
"""
db_data = self.fetch_postgres_relation_data()
env = {
"DEMO_SERVER_DB_HOST": db_data.get("db_host", None),
"DEMO_SERVER_DB_PORT": db_data.get("db_port", None),
"DEMO_SERVER_DB_USER": db_data.get("db_username", None),
"DEMO_SERVER_DB_PASSWORD": db_data.get("db_password", None),
}
return env
Finally, let’s define the methods that are called on database events:
def _on_database_created(self, event: DatabaseCreatedEvent) -> None:
"""Event is fired when postgres database is created."""
self._update_layer_and_restart(None)
def _on_database_relation_removed(self, event) -> None:
"""Event is fired when relation with postgres is broken."""
self.unit.status = WaitingStatus("Waiting for database relation")
raise SystemExit(0)
The diagram below illustrates the workflow for the case where the database integration exists and for the case where it does not:
Validate your charm
Time to check the results!
First, repack and refresh your charm:
charmcraft pack
juju refresh \
--path="./demo-api-charm_ubuntu-22.04-amd64.charm" \
demo-api-charm --force-units --resource \
demo-server-image=ghcr.io/beliaev-maksim/api_demo_server:0.0.9
Next, deploy the posgresql-k8s charm:
juju deploy postgresql-k8s --channel=14/stable
Now, integrate our charm with the newly deployed postgresql-k8s charm:
juju integrate postgresql-k8s demo-api-charm
Read more: Integration,
juju integrate
Finally, run:
juju status --relations
You should see both applications in the active status, and also that the postgresql-k8s charm
has a relation to the demo-api-charm over the postgresql_client interface, as below:
Model Controller Cloud/Region Version SLA Timestamp
charm-model tutorial-controller microk8s/localhost 3.0.0 unsupported 13:50:39+01:00
App Version Status Scale Charm Channel Rev Address Exposed Message
demo-api-charm 0.0.9 active 1 demo-api-charm 1 10.152.183.233 no
postgresql-k8s active 1 postgresql-k8s 14/stable 29 10.152.183.195 no Primary
Unit Workload Agent Address Ports Message
demo-api-charm/0* active idle 10.1.157.90
postgresql-k8s/0* active idle 10.1.157.92 Primary
Relation provider Requirer Interface Type Message
postgresql-k8s:database demo-api-charm:database postgresql_client regular
postgresql-k8s:database-peers postgresql-k8s:database-peers postgresql_peers peer
postgresql-k8s:restart postgresql-k8s:restart rolling_op peer
The relation appears to be up and running, but we should also test that it’s working as intended. First, let’s try to write something to the database by posting some name to the database via API using curl as below – where 10.1.157.90 is a pod IP and 8000 is our app port. You can repeat the command for multiple names.
curl -X 'POST' \
'http://10.1.157.90:8000/addname/' \
-H 'accept: application/json' \
-H 'Content-Type: application/x-www-form-urlencoded' \
-d 'name=maksim'
Second, let’s try to read something from the database by running:
curl 10.1.157.90:8000/names
This should produce something similar to the output below (of course, with the names that you decided to use):
{"names":{"1":"maksim","2":"simon"}}
Congratulations, your integration with PostgreSQL is functional!
Review the final code
For the full code see: 04_integrate_with_psql
For a comparative view of the code before and after this doc see: Comparison
See next: Preserve your charm’s data