How to upgrade your deployment from 2.9 to 3.x

See also: Deployment, Juju 3.0 release notes, Upgrading things, How to upgrade a controller’s minor or major versions

This document gives the steps for how to upgrade your Juju deployment from 2.9 to 3.x.

The steps are the same across machine and Kubernetes deployment, with two exceptions:

Contents:

  1. Ensure your controller is at the latest patch release of 2.9
  2. Ensure your models are at the latest patch release of 2.9
  3. Ensure that your workload machines are Ubuntu focal or later
  4. Upgrade the client to 3.x
  5. Bootstrap a 3.x controller
  6. Clone your old controller configuration into the 3.x controller
  7. Help users connect to the 3.x controller
  8. Migrate your models to the 3.x controller
  9. Upgrade your models to match the agent version of the 3.x controller

Ensure your controller is at the latest patch release of 2.9

The only way to upgrade a 2.9 controller to 3.x is to use the 3.x client to bootstrap a new 3.x controller, and then migrate your models from the old controller to the new one. Now, if your 2.9 controller’s patch version is too old, your 3.x client might not be able to talk to it. To avoid that, upgrade your controller to the latest patch version of 2.9, as below:

juju upgrade-controller <controller>

See more: How to upgrade a controller’s patch version

Ensure your models are at the latest patch release of 2.9

Since the only way to upgrade a 2.9 controller to 3.x is to bootstrap a new 3.x controller, the only way to connect your 2.9 models to the new controller is to migrate them to it. Now, if your model’s version is too old, the migration might be rejected. To avoid that – as well as to ensure your model has picked up all the latest bug fixes – upgrade your models to the latest patch release of 2.9, as below:

juju upgrade-model <model>

See more: How to upgrade a model

Ensure that your workload machines are Ubuntu focal or later

See also: Ubuntu Wiki | Releases

If your deployment is on Kubernetes: Skip ahead to the next step.

A 3.x controller no longer supports workload machines running on Windows or Ubuntu series older than focal. Before you begin your upgrade, make sure that all your existing or new machines comply.

If your charm doesn’t support Ubuntu focal or later, you will have to either
(a) provide a charm revision that does yourself or
(b) remove the charm from your deployment.

Make existing machines Ubuntu focal or later

If you have any Windows machines, you must remove them. If you have machines running on Ubuntu bionic (18.04) or older, you must upgrade them:

# Ask juju to take the machine out of circulation, 
# in preparation for the upgrade:
juju upgrade-machine <machine number> prepare <target upgrade series>
# If you get an error that your charms don't support the target Ubuntu series, 
# you will have to either 
# (a) provide a charm revision that does yourself or 
# (b) remove the charm from your deployment.

# SSH into the machine:
juju ssh <machine number>

# Initiate system upgrade:
do-release-upgrade
# Stand by as you'll have to confirm ('y' or ENTER) various steps.

# Ask juju to put the machine back in circulation as the upgrade is complete:
juju upgrade-machine <machine number> complete 

See more: How to upgrade a workload machine

Make new machines Ubuntu focal or later

For each application in your deployment, refresh its charm to the latest revision and check if it supports Ubuntu focal or later, then set the application base to Ubuntu focal or later.

# Upgrade the charm to the latest revision, to get the latest supported bases:
juju refresh <charm>

# Check the supported bases:
juju info <charm>
# The supported bases are listed under `supports`.
# If your charm doesn't support Ubuntu focal or later, you will have to either
# (a) provide a charm revision that does yourself or 
# (b) remove the charm from your deployment.

# Set the base of applications deployed from this charm to one of the supported bases:
juju set-application-base <charm> <base>
# Formerly, 'juju set-series'.

See more: How to upgrade an application

Upgrade the client to 3.x

To bootstrap a 3.x controller, you will need a 3.x client. To upgrade your client to 3.x, refresh the snap. For example, for juju v.3.1, do:

snap refresh juju --channel 3.1/stable

See more: How to upgrade juju

While you need a 3.x client to bootstrap a 3.x controller, you can technically continue to manage a 3.x controller with a 2.9 client, at least until Juju release 3.1. On Linux, you can do that via parallel snap installs:
(a) enable the experimental parallel-install feature: sudo snap set system experimental.parallel-instances=true;
(b) reboot; and
(c) parallel-install juju 3.0: snap install --channel 3.0/stable juju_30,
where your juju_30 is your juju 3.0 client’s name.

Bootstrap a 3.x controller

Now, use the 3.x client to bootstrap a 3.x controller.

juju bootstrap <cloud> <3.x controller name>

If you’ve chosen to parallel-install a 3.0 client (instead of upgrading), run juju_30 bootstrap <cloud> <3.x controller name>. Once you’re done bootstrapping, you can continue to manage this controller with the 2.9 client (just juju instead of juju_30).

See more: How to upgrade a controller’s minor or major version

Clone your old controller configuration into the 3.x controller

If your deployment is on Kubernetes: You will have to replicate this logic manually.

Your old 2.9 controller was already configured to your liking. To retrieve all this knowledge of clouds, cloud configurations, users, user credentials, user permissions, etc., back up the old 2.9 controller and then use the stand-alone juju-restore tool to clone its configuration from the backup into the new 3.0 controller.

Create a backup of the 2.9 controller

To create a backup of the 2.9 controller model, run:

juju create-backup -m <2.9 controller>:controller
# Sample output:
# >>> ...
# >>>  Downloaded to juju-backup-20221109-090646.tar.gz

See more: How to create a controller backup

Copy the configuration of the 2.9 controller into the new 3.x controller

To clone the configuration of the 2.9 controller into the 3.x controller, you must download the stand-alone juju-restore tool, switch to the 3.x controller, copy this along with the 2.9 controller backup file to the primary controller machine, and then run ./juju-restore with the --copy-controller command:

# Download the latest release binary (Linux, AMD64):
wget https://github.com/juju/juju-restore/releases/latest/download/juju-restore
chmod +x juju-restore

# Switch to the controller model:
juju switch controller

# Copy juju-restore to the primary controller machine:  
juju scp juju-restore 0:

# Copy the backup file to the primary controller machine:
juju scp <path-to-backup> 0:

# SSH into the primary controller machine:
juju ssh 0

# Start the restore with the '--copy-controller' flag:
./juju-restore --copy-controller <path-to-backup>
# Congratulations, your 2.9.37+ controller config has been cloned into your 3.0 controller.

See more: How to restore a controller from a backup

Help users connect to the 3.x controller

Your 2.9 controller likely had other users connected to it. To help these users connect to the new 3.x controller, reset their password from the new controller and send them the new registration string. Then they’ll be able register the new 3.0 controller on their client.

# Make sure you're on the 3.x controller:
juju switch <3.x controller name>

# Reset the users' password:
juju change-user-password <user> --reset
# >>> Password for "bob" has been reset.
# >>> Ask the user to run:
# >>>     juju register 
# >>> MEcTA2JvYjAWExQxMC4xMzYuMTM2LjIxNToxNzA3MAQgJCOhZjyTflOmFjl-mTx__qkvr3bAN4HAm7nxWssNDwETBnRlc3QyOQAA
# When they use this registration string, they will be prompted to create a login for the new controller.

See more: How to manage users

Migrate your models to the 3.x controller

To migrate your 2.9.x models to the 3.x controller, run:

juju migrate <model> <3.x controller>

See more: How to migrate a workload model to another controller

Upgrade your models to match the agent version of the 3.x controller

To upgrade your migrated models to match the agent version of the 3.x controller, run:

# Find out the exact current version of the controller by inspecting the output of 
# 'juju status' or 'juju show-controller <3.x controller>'.

# Upgrade the model to that version:
juju upgrade-model  <model>  --agent-version=<the version of the 3.x controller>

See more: How to upgrade a model

Congratulations, you have upgraded your deployment to 3.x!

Contributors: @wallyworld

–agent-version=3.0.0 or the like is required here, otherwise the upgrade won’t find it - it’ll be looking for newer versions of 2.9 by default.

1 Like

Updated, thanks! (I had this in an earlier version of the doc and then lost it somehow.)

Its mentioned a tool here “the stand-alone juju-restore tool” but there is no link to it which makes the documentation somewhat incomplete.

I’ll try find it.

Thanks for catching this, @erik-lonroth, I’ve added a link now. It’s this one here: https://github.com/juju/juju-restore

1 Like

This page is titled “from 2.9 to 3.x” but it seems you can’t actually easily upgrade from 2.9 to 3.1. This is maybe alluded to with the note of

While you need a 3.x client to bootstrap a 3.x controller, you can technically continue to manage a 3.x controller with a 2.9 client, at least until Juju release 3.1 ."

If you switch to 3.1, bootstrap a controller, once you switch back to the 2.9.42 controller it won’t connect so you have to downgrade back to the 2.9 client at least.

If the intended path is truly 2.9->3.0->3.1 then the page should be updated to reflect that explicitly, especially now that 3.1 is out.

1 Like

Hi @lathiat, thanks for your feedback. Insofar as I know, you should be able to upgrade from 2.9.x to 3.1 in one step. But I will double-check.

Hi,

after spinning a new 3.1 controller and importing all data from a 2.9.42 controller i cannot import models into the new controller.

infact when i issue a migrate command the migration starts but than hangs saying (json output):

“model-status”: { “current”: “available”, “message”: “migrating: aborted, removing model from target controller: model data transfer failed, failed to import model into target controller: remote-applications: version 3 not valid (not val id)”, “since”: “14 Apr 2023 17:24:57+02:00” },

what am i missing?

forgot to add: i’m using juju 2.9 client as when trying to use the 3.1 version it stops immediately

ERROR juju client with major version 3 used with a controller having major version 2 not supported

update your juju client to match the version running on the controller

There are some migration issues with importing to juju 3.1.0 which are fixed in the current 3.1 candidate. We’ll have to see if this is one of them. I don’t have a release date, but soon.

1 Like

The other thing would be nice to have in this guide are steps to get the current model in a clean state. Particularly, I have seen cases where the user needs to run mongo purge before re-trying a failed migration, and increasing agent-ratelimit-max that always make model migrations to fail.

The See more: How to upgrade a model link points to a archived link, which I was able to seen only after login. I believe the new link is How to manage models

1 Like

Fixed, thanks!

1 Like

Step 4 says to upgrade juju client to 3.x but then juju create-backup fails

juju create-backup -m ...
ERROR juju client with version 3.1 used with a controller having major version 2 not supported
re-install your juju client to match the version running on the controller

Another thing is backing up container controller

$ juju create-backup -m ...
ERROR Juju command "create-backup" not supported on container controllers

There is a note saying If your deployment is on Kubernetes: You will have to replicate this logic manually. but no details, like what to backup.

Hello @tmihoc,

Could you help us to solve how to backup for if the deployment is on K8s?

Many thanks.

Hi @marosg & @amo-mycena,

We currently don’t support backups on Kubernetes. There is a bit of enablement work we need to do to have this supported. At the moment we don’t advertise Kubernetes controllers as a production support Juju because of this and our ability to not provide HA.

The steps above for manually cloning a controller is different and not a backup. The steps involved logging into the Mongo database container and exporting several collections and then importing these into the new Database.

I will have a talk with @wallyworld and we will get these written in a formal manner for someone to follow.

1 Like

Hi,

It is great now that we have the Juju roadmap and release schedule, but I think some extra clarity would be great regarding upgrade paths now that we have some more minor versions, and 3.5 (LTS) is soon to be released, so I’m basically getting back to the question from @lathiat one year ago.

It is clear that minor versions (3.1->3.2, 3.2->3.3 and so on) can’t be carried out without model migration, but when migrating from 2.9 to 3.latest with as few migrations as possible, can we expect LTS->LTS to work, i.e. 2.9->3.5 directly or will certain stops in between be required?

Thanks in advance!

Hi @hallback , this doc might help: https://juju.is/docs/juju/juju-version-compatibility-matrix Do let me know if if it still has issues (e.g., missing info or unclear) and I’ll work to improve it.

1 Like