How to migrate Charmed MLflow v1 to v2

charm
,
1

Welcome to the guide on how to migrate Charmed MLflow version 1 to version 2. This guide assumes you are running the old Charmed MLflow stack version 1, which runs with MariaDB. With the MLflow version 2.1.1, we support only MySQL integration. This guide sums up how to move data from MariaDB to MySQL and how to migrate data from version 1 to version 2. Data from the object store doesn’t need to be migrated.

Prerequisites: Before proceeding with this guide, make sure you have completed the following prerequisites on your deployment environment:

  1. Deployed MLflow version 1 with MariaDB, MLflow server version 1.x, and MinIO.
  2. CLI access to the machine where the Juju controller is deployed (all of the commands will be executed from there).

Contents:

MariaDB Backup

Install the mysqldump command with: 
sudo apt update
sudo apt install mysql-client

Backup the MariaDB database with the following command:

mysqldump --host=<mariadb-charm-ip-address> --user=root --password=root --column-statistics=0 --databases database > mlflow-db.sql

Deploy MySQL Charm

Deploy the MySQL charm, which is needed for your desired MLflow bundle. For MLflow version v.2.1, please deploy the 8.0/beta with the command: 
juju deploy mysql-k8s --channel 8.0/beta --series jammy --trust

Please wait until the charm goes to active in juju status.

Now the database is ready to get the password for MySQL. Run the following command:

juju run-action mysql-k8s/0 get-password --wait

Adjust the Database Backup

Rename the database from database (used in MariaDB) to mlflow (used in MySQL): 
sed 's/`database`/`mlflow`/g' mlflow-db.sql > mlflow-db-updated.sql

Rename one duplicate constraint as MySQL does not allow that:

sed -i '0,/`CONSTRAINT_1`/s//`CONSTRAINT-1`/' mlflow-db-updated.sql

You can do all the above modifications in the text editor of your choice if you prefer.

Move Database to MySQL

Install the MySQL CLI tool: 
sudo apt update
sudo apt-get install mysql-shell

Connect to the MySQL charm:

mysql --user=root --host=<mysql-unit-ip> -p
# you will be prompted for password

Create the MySQL database called mlflow :

CREATE TABLE mlflow;

Leave the client with ctrl + D

Move the updated database dump file to the MySQL

mysql -u root -p <mysql_password> mlflow <mlflow-db-updated.sql

Migrate MySQL Database

Install the MLflow Python client version 2.1.1: 
pip install mlflow==2.1.1

Run the migration script against the MySQL mlflow database:

mlflow db upgrade mysql+pymysql://root:<mysql-password>@<mysql-ip>/mlflow

Update MLflow Server

Remove relations from the old MLflow server: 
juju remove-relation mlflow-db:mysql mlflow-server:db
juju remove-relation minio mlflow-server

Update the MLflow server:

juju refresh mlflow-server --channel 2.1/edge

Create relations with MinIO and MySQL:

juju relate mysql-k8s mlflow-server
juju relate minio mlflow-server