Charmed Kafka Documentation - How to create mTLS Client Credentials

Create mTLS Client Credentials


Goal: Create mTLS credentials for a client application to be able to connect to the Kafka cluster.


# ---------- Environment

# Kafka ports

# Kafka servers

# Java keystore and trustore

# Client information

Retrieve Root CA

If you are using the tls-certificates-operator charm, retrieve the Root CA executing the following commands:

# ---------- Root CA
# Get the root CA to be used by the client
juju show-unit tls-certificates-operator/0 --format json | jq -r '.[]."relation-info"[]."application-data"."self_signed_ca_certificate" // empty' > ss_ca.pem

# Get the root CA private key and password to be used by the client
juju show-unit tls-certificates-operator/0 --format json | jq -r '.[]."relation-info"[]."application-data"."self_signed_ca_private_key" // empty' > ss_ca.key
SS_KEY_PASSWORD=$(juju show-unit tls-certificates-operator/0 --format json | jq -r '.[]."relation-info"[]."application-data"."self_signed_ca_private_key_password" // empty')

Otherwise retrieve Root CA from your certs provider.

Retrieve Server CA

# ---------- Server CA
# getting the CA used by the server
juju show-unit kafka/0 --format json | jq -r '.[]."relation-info"[]."local-unit" // empty' > kafka_ca.pem

Create Keystore (Client Cert)

Create a client cert signed by the server

# ---------- Keystore
# create new private key --> client_key.pem
openssl genrsa -out client_key.pem 4096

# create new csr --> client_csr.pem
openssl req -new -key client_key.pem -out client_csr.pem -subj "/C=US/ST=Denial/L=Springfield/O=Dis/CN=$KAFKA_CLIENT_MTLS_CN"

# sign new csr using new client CA --> client_cert.pem
openssl x509 -req -CA ss_ca.pem -CAkey ss_ca.key -in client_csr.pem -out client_cert.pem -days 365 -CAcreateserial -passin pass:$SS_KEY_PASSWORD

# create new chain --> client_chain.pem
cat ss_ca.pem client_cert.pem client_key.pem > client_chain.pem

# create p12 keystore from chain --> client.keystore.p12
openssl pkcs12 -export -in client_chain.pem \
-out client.keystore.p12 -password pass:$KAFKA_CLIENT_KEYSTORE_PASSWORD \
-name client-chain -noiter -nomaciter

Create Trustsore (Server Cert)

Inject Root CA and Server CA into the truststore file:

# ---------- Truststore
charmed-kafka.keytool -keystore client.truststore.jks -storepass $KAFKA_CLIENT_TRUSTSTORE_PASSWORD -noprompt \
  -importcert -alias kafka-ca -file kafka_ca.pem
charmed-kafka.keytool -keystore client.truststore.jks -storepass $KAFKA_CLIENT_TRUSTSTORE_PASSWORD -noprompt \
  -importcert -alias CARoot -file ss_ca.pem

Check certificates validity

# ---------- Check certs validity
echo "Client certs in Keystore:"
charmed-kafka.keytool -list -keystore client.keystore.p12 -storepass $KAFKA_CLIENT_KEYSTORE_PASSWORD -rfc | grep "Alias name"
charmed-kafka.keytool -list -keystore client.keystore.p12 -storepass $KAFKA_CLIENT_KEYSTORE_PASSWORD -v | grep until

echo "Server certs in Truststore:"
charmed-kafka.keytool -list -keystore client.truststore.jks -storepass $KAFKA_CLIENT_TRUSTSTORE_PASSWORD -rfc | grep "Alias name"
charmed-kafka.keytool -list -keystore client.truststore.jks -storepass $KAFKA_CLIENT_TRUSTSTORE_PASSWORD -v | grep until


This is a mutual TLS communication which means:

  1. The client needs to trust the server certificates.

  2. Instead of username and passwords, the client needs its own certificate signed by a certificate trusted by the server for the authentication.

# Map the CN on the cert to be considered the principal (username)
juju config kafka ssl_principal_mapping_rules='RULE:^.*[Cc][Nn]=([a-zA-Z0-9\.-]*).*$/$1/L,DEFAULT'
# ---------- Create mTLS User credentials
juju ssh kafka/leader "
sudo charmed-kafka.configs \
--bootstrap-server $KAFKA_SERVERS_SASL \
--command-config $SNAP_KAFKA_PATH/ \
--alter --entity-type=users \
--entity-name=$KAFKA_CLIENT_MTLS_CN \

Create a file called that should look like:



Manage Authorization through ACLs

Grant read and write privileges to user over group, topic and transaction resources:

juju ssh kafka/leader "
echo 'LOG: Creating ACLs for SASL user'

sudo charmed-kafka.acls --bootstrap-server $KAFKA_SERVERS_SASL --command-config $SNAP_KAFKA_PATH/ \
--add --allow-principal User:$KAFKA_CLIENT_MTLS_CN \
--operation READ --operation DESCRIBE --group='*'
sudo charmed-kafka.acls --bootstrap-server $KAFKA_SERVERS_SASL --command-config $SNAP_KAFKA_PATH/ \
--add --allow-principal User:$KAFKA_CLIENT_MTLS_CN \
--operation READ --operation DESCRIBE --operation CREATE --operation WRITE --operation DELETE --operation ALTER --operation ALTERCONFIGS --topic='*'

sudo charmed-kafka.acls --bootstrap-server $KAFKA_SERVERS_SASL --command-config $SNAP_KAFKA_PATH/ \
--add --allow-principal User:$KAFKA_CLIENT_MTLS_CN \
--operation DESCRIBE --operation WRITE --transactional-id '*'

Test Access

# ---------- Test Access
# Copy the files to a path readable by the `charmed-kafka` snap commands
sudo cp client.truststore.jks $SNAP_KAFKA_PATH/
sudo cp client.keystore.p12 $SNAP_KAFKA_PATH/

# Apply file permissions to be readable by the snap
sudo chown snap_daemon:root $SNAP_KAFKA_PATH/
sudo chown snap_daemon:root $SNAP_KAFKA_PATH/client.keystore.p12
sudo chown snap_daemon:root $SNAP_KAFKA_PATH/client.truststore.jks

# Use newly created credentials to create a topic and list existing topics
sudo charmed-kafka.topics --bootstrap-server $KAFKA_SERVERS_MTLS --command-config $SNAP_KAFKA_PATH/ \
--create --topic EXAMPLE-TOPIC

sudo charmed-kafka.topics --list --bootstrap-server $KAFKA_SERVERS_MTLS --command-config $SNAP_KAFKA_PATH/

Hi @gustavosr98!

  1. Looks like this is a charm doc. If so, the page’s title should be prefixed with Kafka docs: “Kafka docs - Create mTLS Client Credentials”.
  2. Iiuc, it seems like the setup depicted here is equivalent to trusting self-signed certs, and is not really mTLS, because for mTLS we need both sides to separately trust a separate CA.

Hi @sed-i ! Sorry for jumping in here a tad late. Anyhow, I believe this is indeed mTLS, in the sense that both server and clients presents certificates to qualify themself. The server is also using the certificate presented by the client (beside trusting) to also authenticate/authorize (based on ACLs) the access.

Whether the client certificates is self-signed and which one is the CA does not really matter.