I’ve been working on the process to migrate the Nexcloud charm from the old (to be deprecated) Postgresql charm to the new.
This this is a fairly sensitive operation since we use Nextcloud with Dwellir, so I thought to share my (positive) experience getting to a point where we can migrate to the new ops Postgresql.
Note! This process will not fit everyone. Especially if you have very large databases which may not be easily migrated using a simple pg_dump. You would then need to figure out how to best migrate your data as part of a migration process. Our Nextcloud has a small database, which makes this process work well.
Overview
First, read this document by @taurus: Charmed PostgreSQL How-to - Migrate from Legacy (overview) which by the time of writing was very much a DRAFT. But contained enough information to get going.
The Nextcloud charm use the new “ops” lib interface “postgresql_client” (database) and keeps the metadata.yaml reference to the old interface name (db). The migration method described here involves Nextcloud having two relations to Postgresql before the first/old one is removed.
charm.py
from charms.data_platform_libs.v0.data_interfaces import (DatabaseCreatedEvent, DatabaseRequires)
metadata.yaml
requires:
db:
interface: pgsql
database:
interface: postgresql_client
limit: 1
The migration process, takes you from the old version of the “old Nextcloud” (released as v1.0.0) + old Postresql (12) -> into the “new Nextcloud (rev23+)” + “new Postgresql (14)” . It will work on new revisions too.
Practice
Checkout, build, deploy the old version of the Nextcloud charm together with old Postgresql.
Practice the migration gives you confidence and saves heartache doing it on your live instance.
You need to get the old code for Nextcloud (deprecated as of 2023-june) to practice the real migration.
git clone https://github.com/nextcloud-charmers/nextcloud-charms/
git checkout v1.0.0
cd operator-nextcloud
charmcraft pack
juju add-model old-nextcloud
juju deploy postgresql –channel stable/290 –series focal
juju deploy nextcloud-old.charm
juju relate nextcloud:db postgresql:db
Backup Nextcloud config.php
Make sure to make a copy the /var/www/nextcloud/config/config.php as you can always revert if something goes sideways along the way.
Dump database old Postgresql
Extract a database dump of the nextcloud database. It will be restored in to a new instance of Postgresql later. Remember to place Nextcloud into maintenance mode to avoid any database changes while we perform the work.
juju run-action nextcloud/leader maintenance enable=true
juju ssh postgresql/0
sudo -u postgres pg_dump -Fc -d nextcloud > nextcloud-database-backup-file.db
(The option “-Fc” is needed to be able to restore with pg_restore)
Deploy a new Postresql database & test import
Test that you can import the database into a fresh install of Postgresql.
juju add-model test-importing-database
juju deploy postgresql
juju run-action postgresql/leader get-password –wait
juju scp nextcloud-database-backup-file.db postgresql/0:
juju ssh postgresql/0
createdb --host=10.51.45.221 --username=operator --password testdb
pg_restore --username=operator --password -d testdb nextcloud-database-backup-file.db
This should now work without errors and that means you have tested your backup/restore.
Upgrade the Nextcloud charm
Upgrade the Nextcloud charm, available in charmhub on the edge channel.
juju upgrade-charm nextcloud --switch "ch:nextcloud" –channel=edge
Nextcloud becomes operational with the new relation (database) interface now available.
Deploy a new postgresql instance in the same model as your Nextcloud
Deploy the new postgresql charm under a different name (since it will be conflicting with your old postgresql).
juju deploy postgresql postgresql-ops --series jammy
Relate new Postgresql with Nextcloud
Now relate Nextcloud with the new postgresq-ops application. This is safe and will not cause any issues, because the Nextcloud charm will only request the creation of a new database for us which we will extract the needed credentials from and modify the primary config.php manually.
juju relate nextcloud:database postgresql:database
Get the login information form the new database relation.
Extract the connection information I needed to connect to the new instance.
juju show-unit nextcloud/0 | grep -E "database|endpoints:|password"
Restore Nextcloud database into the new instance.
We can import the exported database into the new one which has been created for us by Juju.
pg_restore --no-owner --clean --host=10.51.45.221 --username="relation-6" --password -d nextcloud nextcloud-database-backup-file.db
Use the password from the output of the above juju show-unit command.
Update config.php
Use the information about the new db instance (host,port,username and password.) to edit the configuration file on the nextcloud/leader unit: /var/www/nextcloud/config/config.php
<?php
$CONFIG = array (
'passwordsalt' => 'rSGbWXO179oaGDFMP/9HxGv2aoraq9',
'secret' => '88IswSZ2nkY9xc6I8QEvwBhdt/hmtq56MuYnlyey5kxtsXWM',
'trusted_domains' =>
array (
0 => 'localhost',
2 => '10.51.45.74',
),
'datadirectory' => '/var/www/nextcloud/data/',
'dbtype' => 'pgsql',
'version' => '23.0.0.10',
'overwrite.cli.url' => '',
'dbname' => 'nextcloud',
'dbhost' => '10.51.45.221',
'dbport' => '5432',
'dbtableprefix' => 'oc_',
'dbuser' => 'relation-6',
'dbpassword' => 'dgJ3d9slL6iHirJi',
'installed' => true,
'instanceid' => 'ocq6dltb3pbg',
'htaccess.RewriteBase' => '\'/\'',
'overwriteprotocol' => 'http',
'default_phone_region' => 'SE',
);
Check Nextcloud works.
After the change, check that your instance works as normal (first disable maintenance mode).
juju run-action nextcloud/leader maintenance enable=false
http://nextcloud.instance.com
If all is OK - great! If not, just recover the old config.php you backed up initially.
Break relation to old Nextcloud
Remove the old relation.
juju remove-relation nextcloud:db postgresql:db
Nextcloud will now detect your manual changes and report it in the status output. This is all fine as this is a means to let you know that a config has happened outside of Juju. We can remedy this by a simple config change that will re-sync the config from disk.
Invoke a random config-change event by altering some non critical config variable:
juju config nextcloud php_upload_max_filesize="128M"
Remove the old Postgresql application
Remove the old relation concludes the migration process.
juju remove-application postgresql
Extra material: Interacting with Postgresql
The below is useful if you need to validate your database etc.
-
Old Postgresql charm:
sudo -u postgres psql -
New Postgresql:
sudo apt install postgresql-client juju run-action postgresql/leader get-password –wait psql --host=10.4.210.100 --username=operator --password postgres