Kafka Test App
The test application charm used in various test scenarios for Charmed Kafka K8s/VM installations.
While this tutorial intends to guide and teach you as you deploy Kafka Test App, it will be most beneficial if you already have a familiarity with:
- Basic terminal commands.
- Kafka concepts such as topics, producers and consumers.
Kafka Test App is the workload-less charm. It can be deployed and used in both K8s and VM installations.
Just deploy and relate it:
# Deploy
juju deploy kafka-test-app producer --config role=producer --channel edge
juju deploy kafka-test-app consumer --config role=consumer --channel edge
juju deploy kafka
juju deploy zookeeper
# Integrate
juju integrate kafka producer
juju integrate kafka consumer
juju integrate kafka zookeeper
# The endless test will start automatically, follow:
juju ssh consumer/leader tail -f /tmp/*_consumer.log
==> /tmp/1714675860_consumer.log <==
2024-05-02 18:58:09,873 INFO [__main__] (MainThread) (<module>) ConsumerRecord(topic='test-topic', partition=4, offset=145, timestamp=1714676289867, timestamp_type=0, key=None, value=b'{"timestamp": 1714676289.866915, "_id": "b09368e996e0463a81859f09172ce402", "origin": "juju-b12b18-3 (10.184.219.11)", "content": "Message #821"}', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=145, serialized_header_size=-1)
2024-05-02 18:58:10,375 INFO [__main__] (MainThread) (<module>) ConsumerRecord(topic='test-topic', partition=4, offset=146, timestamp=1714676290371, timestamp_type=0, key=None, value=b'{"timestamp": 1714676290.371619, "_id": "fde91ab429bd40f1b07f765498ebdff5", "origin": "juju-b12b18-3 (10.184.219.11)", "content": "Message #822"}', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=145, serialized_header_size=-1)
2024-05-02 18:58:10,882 INFO [__main__] (MainThread) (<module>) ConsumerRecord(topic='test-topic', partition=2, offset=177, timestamp=1714676290875, timestamp_type=0, key=None, value=b'{"timestamp": 1714676290.8754, "_id": "1208f40acb5441588e036a654aff43af", "origin": "juju-b12b18-3 (10.184.219.11)", "content": "Message #823"}', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=143, serialized_header_size=-1)
2024-05-02 18:58:11,382 INFO [__main__] (MainThread) (<module>) ConsumerRecord(topic='test-topic', partition=2, offset=178, timestamp=1714676291380, timestamp_type=0, key=None, value=b'{"timestamp": 1714676291.380164, "_id": "7e3528a19d15473ca0e3d6d9ab9d09aa", "origin": "juju-b12b18-3 (10.184.219.11)", "content": "Message #824"}', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=145, serialized_header_size=-1)
2024-05-02 18:58:11,888 INFO [__main__] (MainThread) (<module>) ConsumerRecord(topic='test-topic', partition=2, offset=179, timestamp=1714676291882, timestamp_type=0, key=None, value=b'{"timestamp": 1714676291.882081, "_id": "1ff770274d224cb38c7a758e243a38cc", "origin": "juju-b12b18-3 (10.184.219.11)", "content": "Message #825"}', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=145, serialized_header_size=-1)
Example of juju status output:
Model Controller Cloud/Region Version SLA Timestamp
test53 lxd localhost/localhost 3.4.2 unsupported 20:51:50+02:00
App Version Status Scale Charm Channel Rev Exposed Message
consumer active 1 kafka-test-app latest/edge 10 no Topic test-topic enabled with process consumer
kafka active 1 kafka 3/stable 156 no
producer active 1 kafka-test-app latest/edge 10 no Topic test-topic enabled with process producer
zookeeper active 1 zookeeper 3/stable 126 no
Unit Workload Agent Machine Public address Ports Message
consumer/0* active idle 2 10.184.219.5 Topic test-topic enabled with process consumer
kafka/0* active idle 0 10.184.219.254
producer/0* active idle 3 10.184.219.11 Topic test-topic enabled with process producer
zookeeper/0* active idle 1 10.184.219.31
Machine State Address Inst id Base AZ Message
0 started 10.184.219.254 juju-b12b18-0 ubuntu@22.04 Running
1 started 10.184.219.31 juju-b12b18-1 ubuntu@22.04 Running
2 started 10.184.219.5 juju-b12b18-2 ubuntu@22.04 Running
3 started 10.184.219.11 juju-b12b18-3 ubuntu@22.04 Running
Integration provider Requirer Interface Type Message
consumer:app-peers consumer:app-peers app-peers peer
kafka:cluster kafka:cluster cluster peer
kafka:kafka-client consumer:kafka-cluster kafka_client regular
kafka:kafka-client producer:kafka-cluster kafka_client regular
kafka:restart kafka:restart rolling_op peer
kafka:upgrade kafka:upgrade upgrade peer
producer:app-peers producer:app-peers app-peers peer
zookeeper:cluster zookeeper:cluster cluster peer
zookeeper:restart zookeeper:restart rolling_op peer
zookeeper:upgrade zookeeper:upgrade upgrade peer
zookeeper:zookeeper kafka:zookeeper zookeeper regular