The Charmed ZooKeeper Operator delivers automated operations management from Day 0 to Day 2 on the Apache ZooKeeper highly reliable distributed coordination platform.
In this tutorial we will walk through how to:
- Set up an environment using Multipass with MicroK8s and Juju.
- Deploy ZooKeeper using juju.
- Change the super-user password.
- Scale to more replicas
- Test service.
Minimum requirements
- 4 to 8 GB of RAM
- 2 to 4 cores
- 1 storage device, 64GB
Step-by-step guide
Setup an environment
Multipass is a quick and easy way to launch virtual machines running Ubuntu. It uses “cloud-init” standard to install and configure all the necessary parts automatically.
Let’s install Multipass from Snap and launch a new VM using “charm-dev” cloud-init config:
sudo snap install multipass
multipass launch --cpus 4 --memory 8G --disk 64G --name charm-dev-zookeeper charm-dev
Use the multipass list command to wait until the VM is running:
watch -n 15s multipass list
Note: This could take a long time to get everything setup. If it fails retry the launch command after cleaning up the VM with multipass delete --purge charm-dev-zookeeper
Once the VM is ready the multipass list should show an output similar to this:
$ multipass list
Name State IPv4 Image
charm-dev-zookeeper Running 10.230.220.9 Ubuntu 22.04 LTS
10.45.105.1
Press CTRL+C to exit the watch display and start a shell in the newly created VM using this:
multipass shell charm-dev-zookeeper
The image comes with two juju controllers already setup, one for LXD and one for MicroK8s, for this tutorial, use the MicroK8s:
$ juju list-controllers
Use --refresh option with this command to see the latest information.
Controller Model User Access Cloud/Region Models Nodes HA Version
lxd* welcome-lxd admin superuser localhost/localhost 2 1 none 3.5.4
microk8s zookeeper admin superuser microk8s/localhost 3 1 - 3.5.4
$ juju switch microk8s
Now, create a new model for this tutorial:
juju add-model zookeeper
Deploy ZooKeeper
To deploy ZooKeeper with 3 replicas:
juju deploy zookeeper-k8s -n 3
To monitor the deployment, run juju status:
juju status --watch 5s
Once all the units are active/idle. Get the super user password:
$ juju run zookeeper-k8s/leader get-super-password
Running operation 1 with 1 task
- task 2 on unit-zookeeper-k8s-0
Waiting for task 2...
super-password: CQeh2HUaBavIHg8yhkj6VLatypPGIza5
Change the super-user password
The Charmed ZooKeeper K8s Operator has two internal users:
super: admin user for the cluster. Used by the Kafka operator.sync: specific to the internal quorum handling. The actionset-passwordcan be used to rotate the password of the internal users, if passed with option, it will rotate the password of thesuperuser.
# to set a specific password for the super user:
juju run zookeeper-k8s/leader set-password username=super password=2p14jdpsa024834sadpf234
# to set a random password for the super user:
juju run zookeeper-k8s/leader set-password username=super
Scale the application to 5 replicas
In order to scale up or down the application, run this command with 5 is the target replicas:
juju scale-application zookeeper-k8s 5
Verify that the replicas have been added:
juju status --watch=15s
Test service
First determine the IP address of the leader unit:
$ juju status
Model Controller Cloud/Region Version SLA Timestamp
zookeeper microk8s microk8s/localhost 3.5.4 unsupported 15:26:10-04:00
App Version Status Scale Charm Channel Rev Address Exposed Message
zookeeper-k8s active 3 zookeeper-k8s 3/stable 51 10.152.183.18 no
Unit Workload Agent Address Ports Message
zookeeper-k8s/0* active idle 10.1.198.58
zookeeper-k8s/1 active idle 10.1.198.59
zookeeper-k8s/2 active idle 10.1.198.60
In this example, the leader unit IP is 10.1.198.58, then test of the port is reachable from the VM:
nc -zv 10.1.198.58:2181
If it is succesful, you can run this little python test:
git clone https://github.com/mathmarchand/test-zookeeper.git
python3 -m venv zk
source zk/bin/activate
python3 -m pip install -r test-zookeeper/requirements.txt
$ python3 test-zookeeper/test_zookeeper.py --host 10.1.198.52:2181
INFO:__main__:Will connection to 10.1.198.48:2181
INFO:__main__:Connecting to 10.1.198.48:2181
INFO:kazoo.client:Connecting to 10.1.198.48(10.1.198.48):2181, use_ssl: False
INFO:kazoo.client:Zookeeper connection established, state: CONNECTED
INFO:__main__:Ensuring the path exist, creating if needed
INFO:__main__:Writing a test value to /test/zookeeper/node-test
INFO:__main__:znode was created
INFO:__main__:Version: 0, data: a test value
INFO:__main__:Test successful!
INFO:__main__:Cleaning up
INFO:kazoo.client:Closing connection to 10.1.198.48:2181
INFO:kazoo.client:Zookeeper session closed, state: CLOSED
Next Steps
Charmed ZooKeeper is mostly used by Charmed Kafka, you can read more here.