Configuration backup and restore
Charmed Apache Kafka K8s’ configuration is distributed using Charmed Apache ZooKeeper K8s.
A Charmed Apache ZooKeeper K8s backup can be stored on any S3-compatible storage.
S3 access and configurations are managed with the s3-integrator charm.
This guide will teach you how to deploy and configure the s3-integrator charm for AWS S3, send the configurations to the Charmed Apache ZooKeeper K8s application, and finally manage your Charmed Apache ZooKeeper K8s backups.
Configure s3-integrator
First, deploy and run the charm:
juju deploy s3-integrator
juju run s3-integrator/leader sync-s3-credentials access-key=<access-key-here> secret-key=<secret-key-here>
Then, use juju config to add your configuration parameters. For example:
juju config s3-integrator \
endpoint="https://s3.us-west-2.amazonaws.com" \
bucket="zk-backups-bucket-1" \
path="/zk-backups" \
region="us-west-2"
The only mandatory configuration parameter in the command above is bucket.
Integrate with Charmed Apache ZooKeeper K8s
To pass these configurations to Charmed Apache ZooKeeper K8s, integrate the two applications:
juju integrate s3-integrator zookeeper-k8s
You can create, list, and restore backups now:
juju run zookeeper-k8s/leader list-backups
juju run zookeeper-k8s/leader create-backup
juju run zookeeper-k8s/leader list-backups
juju run zookeeper-k8s/leader restore backup-id=<backup-id-here>
Create a backup
Once you have a Charmed Apache ZooKeeper K8s deployment with configurations set for S3 storage, check that it is active and idle with juju status.
Once Charmed Apache ZooKeeper K8s is active and idle, you can create your first backup with the create-backup command.
juju run zookeeper-k8s/leader create-backup
Charmed Apache ZooKeeper K8s backups created with the command above will always be full backups: a copy of all the Charmed Apache Kafka K8s configuration will be stored in S3.
The command will output the ID of the newly created backup:
Backup created
┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Id ┃ Log-sequence-number ┃ Path ┃
┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 2024-09-12T14:32:46Z │ 8589934621 │ zookeeper_backups/2024-09-12T1 │
│ │ │ 4:32:46Z/snapshot │
└──────────────────────┴─────────────────────┴────────────────────────────────┘
List backups
You can list your available backups by running the list-backups command:
juju run zookeeper-k8s/leader list-backups
This should show your available backups, like in the sample output below:
Backups
┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Id ┃ Log-sequence-number ┃ Path ┃
┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 2024-09-12T14:32:46Z │ 8589934621 │ zookeeper_backups/2024-09-12T1 │
│ │ │ 4:32:46Z/snapshot │
│ 2024-09-12T14:32:00Z │ 8589934621 │ zookeeper_backups/2024-09-12T1 │
│ │ │ 4:32:00Z/snapshot │
│ 2024-09-12T14:26:12Z │ 8589934621 │ zookeeper_backups/2024-09-12T1 │
│ │ │ 4:26:12Z/snapshot │
└──────────────────────┴─────────────────────┴────────────────────────────────┘
Below is a list of parameters shown for each backup:
Id: identifier of the backup.Log-Sequence-number: a database-specific number to identify its state. Learn more about the Zxid on Apache ZooKeeper documentation.Path: path of the snapshot file in the S3 repository.
Restore a backup
This operation puts you at risk of losing unsaved configuration data. We recommend creating a backup first.
To restore a backup from that list, run the restore command and pass the backup-id (in the form of YYYY-MM-DDTHH:MM:SSZ) that is listed in the list-backups action output:
juju run zookeeper-k8s/leader restore backup-id=<backup-id-here>
The restore will then proceed. Follow its progress using juju status.