Getting Started

Hello World Juju Kubernetes

Here’s a basic set of instructions for setting up Juju to work with a given Kubernetes cloud.

First, you’ll need to be running Juju:

sudo snap install juju --classic

Limitations and Prerequisites

  1. The Juju GUI can display a Kubernetes model but anything else - creating units, deploying charms, status etc - is currently unsupported.

Setting Up The Juju Controller

You need a running Juju controller and also a Kubernetes cluster to work with. For the Kubernetes cluster, there’s several choices:

  1. use MicroK8s.
  2. deploy your own Kubernetes cluster
  3. use a Kubernetes cluster on a public cloud (eg GKE)

Once a Juju controller is up and running and you have your Kubernetes cluster, you need to import the cluster and user credential into Juju. The Juju add-k8s command extracts the information it needs from the Kubernetes configuration file (the same one that kubectl uses). Once imported, the cluster appears as a cloud. See below for instructions relevant to each of the above scenarios.

Note: add-k8s will import whatever credential values exist in ~/.kube/config. There’s plans to add support for juju add-credential so that you can add arbitrary k8s clusters as named Juju clouds.

Local setup with MicroK8s

If you want to try a simple deployment on your own server or laptop, the microk8s option is perhaps the easiest. See this post for how to get things set up. The TL;DR: is that you’ll need to enable dns and storage on MicroK8s.

juju bootstrap lxd
microk8s.config | juju add-k8s myk8scloud

You can use your own cloud name in place of myk8scloud.

Running your own Kubernetes cluster

Deploying your own Kubernetes cluster may be done using conjure-up or deploying the kubernetes core bundle or deploying a production ready Canonical Distribution of Kubernetes. Kubernetes may be run locally on a Juju LXD cloud, or on a public cloud like AWS.

Assume we want to run Kubernetes on a Juju instance running on AWS.

juju bootstrap aws
juju deploy kubernetes-core
juju deploy cs:~containers/aws-integrator
juju trust aws-integrator
juju relate aws-integrator kubernetes-master
juju relate aws-integrator kubernetes-worker

The juju trust command sets up the necessary configuration to allow Juju to request dynamic persistent volumes to use for storage (covered in another topic).

You’ll now need to wait for things to stabilise before taking the next step. You can run watch -c juju status --color and wait for everything to go green.

Finally, you need to copy the kubectl config file for the cluster to your local machine and register the cluster as a cloud known to Juju using add-k8s.

juju scp kubernetes-master/0:config ~/.kube/config
juju add-k8s myk8scloud

You can use your own cloud name in place of myk8scloud.

Creating a Kubernetes model in Juju.

Now that the Juju controller is set up and the Kubernetes cluster has been registered as a cloud known to Juju, you can create a new Juju model on that cloud.

juju add-model myk8smodel myk8scloud

The cloud name is whatever was used previously with add-k8s. The model name is whatever works for you. Juju will create a Kubernetes namespace in the cluster to host all of the pods and other resources for that model. The namespace is used to separate resources from different models.

Optionally set up storage

Now that the model is created, you’ll may also need to configure a storage pool to provide storage for the charm operator pods. If Juju is running on a cluster that has suitable storage already configured, then you don’t need to do anything and Juju will use that storage class (Juju on microk8s requires no additional setup apart from ensuring microk8s storage is enabled via microk8s.enable storage). But you may want to set up a bespoke storage class per application, or for the model itself. See this post for more detail.

Deploying a Kubernetes charm

There are some very early proof of concept charms and bundles in the staging charm store.

These charms are not for production and are not complete. They are proof of concept only.

Let’s deploy a gitlab and mariadb charm and relate them. Ensure that the Kubernetes model is in focus.
We’ll need storage for the mariadb charm - if there’s a default storage class already set up for the cluster, there’s no need to do anything and you’ll get some storage allocated out of the box. If you want to configure some bespoke storage, see this post.

If we’re deploying to MicroK8s, that comes with a default hostpath storage class, so there’s no need to create a storage pool for mariadb. But we may want to override the default 1GiB storage allocation and only ask for 10MiB.

Now deploy and relate the charms:

juju switch myk8smodel
juju deploy cs:~juju/mariadb-k8s --storage database=10M
juju deploy cs:~juju/gitlab-k8s
juju relate gitlab-k8s mariadb-k8s

You can use juju status to watch the progress of the deployment. Note that even after Juju status indicates that things have finished, the gitlab image is churning away setting up the database tables it needs. We don’t currently have a way of exposing this to Juju status,

Juju status will surface the current pod status for each unit and the last error message (if any). You can also use kubectl to describe relevant pods or other artefacts if more detail is required.

Exposing gitlab

To be able to connect to gitlab externally with a web browser, it needs to be exposed. The means to do this depends on the underlying cloud on which Kubernetes is running and how the deployment was set up.

If the Kubernetes bundle was deployed on AWS using the aws-integrator charm, then an AWS Elastic Load Balancer is automatically configured to route external traffic to the gitlab service. Use juju status to see the FQDN of the service and point the browser at that address.

Similarly, when using MicroK8s, you can simply access the workload using the IP address of the Kubernetes service resource (shown as the application address in Juju status).

For deployments on other substrates, you’ll need to juju expose gitlab and also supply a hostname for the service. The easiest way to get a hostname to test with is to use the facility provided by For our simple test deployment, using the kubernetes-core bundle, there’s only 1 worker node and that’s where gitlab will be running. The IP address of the worker node is what’s needed. Run juju status on the Juju model hosting the actual Kubernetes deployment and note the IP addresses of the worker node.

Use this Juju command to configure the gitlab application:

juju config gitlab

Obviously replace with the correct IP address.

Now gitlab can be exposed:

juju expose gitlab

Note: it may take a minute for the exposed workload to become available. Until then you get an nginx error page trying to view the gitlab web page.

Using Storage

Persistent storage for charms is supported. See this topic for more details.


You can specify a placement directive using the standard Juju --to syntax.

Right now, we support a node selector placement based on matching labels. The labels can be either a built-in label or any user defined label added to the node.
juju deploy mariadb-k8s --to


You can specify resource limits for memory and cpu. The cpu units are milli cpu 2. The standard Juju constraint syntax is used.

Note: right now, the constraint values are mapped to resource limits. There’s no support for resource requests. This conforms to the behaviour of LXD constraints.

juju deploy mariadb-k8s --constraints "mem=4G cpu-power=500"

The cpu-power value specified is an int, and the implicit unit is “milli CPUs”.
K8s requires a value plus unit, so 500 is translated to “500m” to pass to Kubernetes.

1 Like

This command resulted in a failed hook error for me:

juju deploy cs:~johnsca/kube-core-aws && juju trust aws-integrator

I deployed Kubernetes using the canonical-kubernetes bundle on the localhost cloud and I’m stuck setting up storage. Apparently I need to create a storage pool by supplying a value for a Kubernetes storage class but I don’t see anything for LXD.

The storage backends supported by Kubernetes are listed here

For LXD, there’s no native Kubernetes dynamic persistent volume support as far as I know.
You’ll need to use static persistent volumes instead as described in the post on storage.

The status output provided doesn’t include enough info to diagnose the issue, but typically you will get an error with the aws-integrator if the account being used doesn’t have sufficient permissions to support the IAM profiles needed.

I’ve followed these instruction (on a vsphere setup) and I’m currently stuck after deploying a k8s charm. It seems that there are some authentication issues between the charm agent and the controller.

2018-10-16 13:24:35 DEBUG juju.worker.apicaller connect.go:125 connecting with old password
2018-10-16 13:24:35 DEBUG juju.api apiclient.go:877 successfully dialed “wss://”
2018-10-16 13:24:35 INFO juju.api apiclient.go:599 connection established to “wss://”
2018-10-16 13:24:35 DEBUG juju.worker.apicaller connect.go:152 failed to connect
2018-10-16 13:24:35 DEBUG juju.worker.dependency engine.go:538 “api-caller” manifold worker stopped: cannot open api: invalid entity name or password (unauthorized access)

Did I miss something here? Juju version is 2.5-beta1. Charm used is mysql-k8s.

It may be that the cached docker jujud image in your k8s cluster is stale (we use PullIfNotPresent). Because we’re still in development, APIs can change. This won’t be a problem once the final release goes out as the tagged docker image will be stable.

You can try deleting the caas-jujud-operator docker image from your k8s cluster. I’ve just tested with a clean microk8s setup with the 2.5 edge Juju snap and things were fine.

I’ve had this happen as well. In each case, I’d removed the charm from Juju, but did not delete the operator pod’s persistent volume. It seems that they are not automatically removed (as of juju 2.5-beta1) if you juju remove-application the charm.

As with cloud deployments, we specifically do not remove storage when deleting an application unless the user asks for it with --destroy-sorage.

But, we should remove the operator volume. That’s a bug that needs fixing.

I’ve just landed a fix so that the operator storage is deleted when the operator is deleted.

I tried this today with cdk on gce with the gcp-integrator charm.

I have the cdk up and running and setup the operator storage using the gce example on the storage post.

juju list-storage-pools
operator-storage  kubernetes  parameters.type=pd-standard storage-class=juju-operator-storage

When I deploy mysql/gitlab charms I get the following:

pod has unbound immediate PersistentVolumeClaims

On both of the applications and I’m unsure how to proceed.

pod has unbound immediate PersistentVolumeClaims means that the underlying cloud could not provision the requested storage. This may be due to an issue with the integrator charm, or it could be an account limitation, or something else. It’s not a juju-k8s issue per se but something with the substrate on which Kubernetes is running. You’ll need to kubectl describe the affected PVC to dig into the details of why the claim could not be satisfied.

Also, don’t use mysql - use mariadb instead. There’s an issue with the upstream mysql docker image that causes the db not to come up correctly.

How to write a statefuleset container with kuernetes charm. Can you help me to giving some documents or examples.

Juju automatically creates a k8s stateful set if your charm requires storage; otherwise a deployment controller is used to managed to pods. The stateful set allows the storage pv’s to be correctly re-attached to new pods which are spun up to replace existing pods. The creation of a stateful set or deployment controller is done automatically by Juju; it’s not something the charm author needs to worry about.

So, based on the above, the mariadb charm is an example of a charm which uses a stateful set; the gitlab charm will use a deployment controller.

There’s also this post on writing k8s charms in general.

If i want deploy pod on some kubernetes-worker with specific tag. How can i do it.

1 Like

This post explains how you can do what you want:

1 Like

I used a whole kubernetes built by kubespray. Can I create storage by “storage-class=microk8s-hostpath”. specially:

  1. how to create a hostpath type storage for high disk-IO requirements.
  2. how to create a storage on a distribute storage system(maybe ceph).

“hostpath” is a class of storage setup specifically by microk8s.
To create storage for any given k8s cluster, you need to use what’s supported by the underlying cloud or cluster itself as the storage class provisioner. Juju just uses what it’s told to - knowledge of the underlying k8s cluster is necessary to know how to set things up.