What's new in Juju 2.8

[timClicks (Tim McNamara)]

The Juju 2.8 series brings several features to the Juju community.

Usability enhancements

Improved resource usage

Juju agents (jujud processes) now use less memory, particularly for long-lived models.

Increased speed

Several internal changes have led to speed improvements while executing hooks. This means that all Juju operations should run faster on Juju 2.8 than previous versions.

Further details

As an implementation detail, the Juju controller stores its state within a database. Research carried out through the cycle identified areas where unnecessary database calls were being made.

Fewer, more informative log messages

Juju 2.8 increases the default logging level to INFO. This allows debugging information to be accessible, but not presented to users by default.

Juju logs

There are various logging resources available to the Juju operator. This page will explain these and show how to use them. It will cover: Model logs Remote logging Audit logging Model logs Model logs can be considered as Juju’s “regular logs” and are intended to be inspected with the debug-log command. This method provides logs on a per-model basis and is therefore more convenient than reading individual logs on multiple (Juju) machines directly on the file system. The latter can nonetheless …

Custom machine images are now easier to deploy

You are now able to specify virtual machine images that Juju should consider when provisioning new compute capacity. The juju metadata add-image command informs Juju that an image should be considered when the series is selected.

juju metadata add-image --series <series> <image-id>

Here are some provider-specific guides for identifying which <image-id> to select:

• AWS EC2 refers to images as “AMIs” (Amazon Machine Image). AMIs are region-specific.
  • Finding a Linux AMI
  • Finding a Windows AMI
• Google Compute Engine Images  |  Compute Engine Documentation  |  Google Cloud
• Azure Machine Images
  • Find Windows VM images in the Azure Marketplace with Azure PowerShell
  • Find Linux VM images in the Azure Marketplace with the Azure CLI

Juju 2.8 improves support for adding MS Windows machines via Windows Remote Management (WinRM).

Features added to the core

Kubernetes

Juju continues to strengthen its ability to support Kuberentes-based workloads.

Charms no longer need a persistent volume for charm state

The Juju controller can now store charm state in juju-db (an instance of MongoDB deployed alongside the Juju controller), rather than on disk where the charm is located.

This removes the requirement for Juju to provision a persistent volume for every charm operator that is deployed.

To take advantage of this feature, charms declare their min-juju-version in metadata.yaml to this release:

# metadata.yaml
min-juju-version: 2.8.0

For such charms, Juju will no longer create a k8s PV when deploying the operator pod to run the charm.

Operator charms

Developer preview: This feature is available for experimentation via a controller feature flag, but is subject to further refinement.

Juju now supports the ability for charms’ workloads to run within the charm’s workload pod itself. As background, Juju co-deploys an “operator pod” to execute hooks alongside workloads.

An “operator”, as a k8s concept, is defined as:

… clients of the Kubernetes API that act as controllers for a Custom Resource. An operator uses the k8s APIs to automate and control the provisioning of resources in the cluster.

Experimental new feature - k8s charms as operators

The 2.8 edge snap contains a new experimental feature to allow k8s charms to be deployed as Kubernetes operators. As such, these charms the will not define a separate OCI image resource for the workload, and will not call k8s-spec-set hook command to have the controller create and manage the workload. Instead, the charm itself will do that using the Python k8s API added to the container which hosts the charm (pip installed to /usr/local/lib/python3.6/dist-packages/kubernetes). Note: this featu…

Storage on Deployment and DaemonSet workloads

Upcoming in 2.8 - Support for storage on Deployments and DaemonSets

New in 2.8 beta1… Juju k8s charms are able to specify a deployment type: stateless (maps to DeploymentController) stateful (maps to StatefulSet) daemon (maps to DaemonSet) Until now, if a charm wanted to declare storage, Juju required that a StatefulSet be used. It’s now possible for a charm to use Juju storage and have its workload provisioned as a DeploymentController or DaemonSet. Such deployments need to have the shared) volume mounted with access mode as ReadWriteMany or ReadOnlyMany. …

Extended volume support and other features

• ability to define hostpath, emptydir volumes
• volumes backed by secrets and config maps
• define lifecycle or CRDs
• value reference container config

K8s Spec v3 changes

Available in the 2.8 edge snap… New extended volume support It’s now possible to configure volumes backed by: config map secret host path empty dir To do this, you’ll need to mark your YAML as version 3. This new version 3 also: renames the config block to envConfig (to better reflect its purpose). renames the files block to volumeConfig. allows file mode to be specified With secret and config map, these must be defined in the elsewhere YAML handed to Juju - you can’t reference existing r…

Tracking of resources created outside of Juju

Charms deployed by a Juju application may now create custom resources themselves. These will now be annotated in the same way as any resources created by Juju - with the model and controller UUIDs and application name.

This change allows for simpler management over time. They to be removed when the parent application is removed.

Resource lifetime management

It’s now possible for a charm to allow resources to exceed the charm’s own lifetime.

Sometimes, certain k8s workloads expect that some global resources (eg kubeflow and User profiles) are retained even when the application (or model) which created them is removed.

Charms now define a juju-resource-lifecycle label. The possible values are:

• model, which ties the resource’s lifetime to the lifetime of the model. The resource is deleted when the model is destroyed.
• persistent (do not delete even when the model is destroyed)

The default is to delete resources when the application which created them is removed.

Support for running ad hoc commands inside the operator pod

Both juju run (on the client) and juju-run (inside a operator or workload pod) now both accept the --operator argument to specify that the requested script should be run on the operator pod rather than the workload pod.

Webhook global names

Juju will prefix any global resources with the model name to ensure applications deployed multiple times into different namespaces do not conflict. However, some workloads which Juju has no control over (yet) expect web hooks (in particular) to have fixed names. Charms can now define an annotation on mutating nad validating webhooks to disable this name qualification:

annotations:
  juju.io/disable-name-prefix: "true"

Other Clouds

vSphere

Until Juju 2.7, organisations running vSphere would require Juju administrators to be provided with write permissions at the datacenter level. Organisations would prefer to be able to give access to users to a bootstrap controller in a specific location without opening up the whole datacenter.

The current behaviour of the vSphere provider is to create virtual machines within in a top-level folder in the specified datacenter named “Juju Controller ”. Template VMs are under another folder called “juju-vmdks”.

OpenStack

Juju now appropriately models OpenStack clouds with with multiple NIC devices or “spaces”.

Openstack Multi-Space Support

The first patches have landed in Juju edge that supports multi-space/multi-NIC support for Openstack. If you want to begin testing this feature, there is no better way than with MicroStack. Installing MicroStack via Snap is trivial and the process is outlined in the documentation. For brevity, simply run the following. At the time of writing, only --edge with --devmode currently works. This is ongoing work to provide confinement to microstack. sudo snap install microstack --edge --devmode …

New Actions UX

Actions have gained a new user experience (UX), including the ability to cancel a running action. Making use of these new commands requires setting an environment variable, JUJU_FEATURES, as they conflict with the hold behaviour.

Juju Actions: opt-in to new behaviour from Juju 2.8

The UX around Juju Actions has undergone a fairly radical re-design to address some long standing issues and deliver some nice usability improvements. An early version of this was delivered in Juju 2.7. Coming in Juju 2.8 is a more polished iteration. Key points: terminology changes running an action defaults to synchronous (block until done) running actions can be cancelled new ability to display progress log messages cleaner plain text output option stdout and stderr included in output nume…

Internal Changes

New capability for charms: controller-backed key/value store

Charms now have the ability to request that the Juju controller store data between hook invocations. This should avoid the need for charms to modify the charm directory. This enables charms to be deployed as “stateless” workloads on clouds that may not have storage support, such as some Kubernetes clusters.

Three hook tools have been introduced:

Hook tool	Role
state-set	Modify values in the store.
state-get	Load a value from the
state-delete	Remove a key/value pair from the store.

Juju enforces the following limits during state-set:

• Keys cannot exceed 256 bytes
• Values cannot exceed 64kb
• Total size of persisted charm state cannot exceed the limit specified by the max-charm-state-size controller configuration option. Its default is 2MB.

This feature is yet to be integrated into charming frameworks, but is available for experimentation today.

New hook family: *-relation-created

A symmetric hook family to *-relation-broken. Allows charms to receive data from the relation, regardless of the presence of other units.

It should also allow charms to not needlessly enter the blocked state when a relation has been created, but before it the first unit is available.

A comment in the original bug report explains some of the rationale:

This is fired as soon as a relation is established, and before any units are accounted as joined as far as the local unit agent knows about. This hook fires once for each relation ID, and is the missing complement of -relation-broken.

Hook execution sequence

Changes have been made to the sequence of hooks that are fired to include the new hooks. This should not affect Juju deployments.

Authentication internals

Juju makes use of macaroons for its cabability-based authorisation. After lots of behind-the-scenes effort, Juju 2.8 moves to the macaroon-bakery.v2 implementation.

Removal of “legacy” leases

Juju now exclusively uses the Raft consensus algorithm to implement high-availability controllers.

Juju client libraries

Python

The pylibjuju project has released been updated to support Juju 2.8.0

pip install -U juju

Juju Dashboard

New in Juju 2.8 is the Juju Dashboard. A shift in direction from the preceding Juju GUI which had a focus on the deployment of models. The new Dashboard is focused on the operations and observability of your Juju models across one or more controllers.

Model List Overview

Tailored for enterprises and large environments; your primary view in the Juju Dashboard is a list of your models. It provies at a glance, a live, autoupdating overview of the statuses of your systems without having to repeatedly query the status on the command line.

Model Details

To provide deeper observability into your model you also have access to the Model Details view. It’s here that you’ll find a detailed overview of all the entities in your model, applications, units, machines, and relations.

The topology that was the center of the old GUI hasn’t completely disappeared. We still show you the same representation of your model on this details page complete with application icons and relation connections to give you that same top down view of your models.

What’s Coming Up?

• The topology model is currently a static view, we’ll be re-introducing the ability to position your application icons to allow for a better representation of your model.
• Cross Model Relations have become an increasingly popular feature of Juju so we’ll be highlighting those and providing insight into the relations both in the relations list and in the topology view.
• Surfacing your permissions and disabled-commands.
• From an operations perspective Juju Actions are integral to the day to day operations of most models. We’re going to be investigating how we can surface the actions and provide feedback on their execution.
• When you’re tracking down an issue with a deployment nothing is more important than logs. We’re actively investigating the best approach to surface the various logs that you have access to in your deployment.
• And much much more…

We hope you enjoy using the new Juju Dashboard!

Changes for charms

New hook: remove

Juju 2.8 introduces the remove hook. This is symmetric to install and provides an opportunity for charms to clean up after the unit has been removed.

This functionality is especially important when multiple charms are deployed to the same machines via placement directives.

New command: juju debug-code

The juju debug-code sub-command allows charm authors to enter a debugger as a hook is executing. Initial support has been added to the Operator Framework.

Live debugging at Python level

The Operator framework offers the possibility of debugging running code, very similar to what any Python developer is used to when doing pdb.set_trace() in their code. That concept is carried to charm processes running inside live systems, where breakpoints will be enabled while handling Juju events and/or actions. This feature is used through the Juju command debug-code; you can learn more about it here, but in a nutshell it will leave you in a Python debugger console, interrupting the runnin…

Feedback Welcome

Dashboard

We have a vision for what this new dashboard will become and your feedback is integral to that vision. Please feel free to file issues on our GitHub repository or start a discussion. We’re actively seeking input on existing and upcoming features.

[jameinel (John A Meinel)]

‘juju debug-code’. I’m working on writing up a doc entry on it, but we should call it out here as well.

[jameinel (John A Meinel)]

We also want a section to discuss changing the default logging level to INFO everywhere. In Juju 2.7 the default logging is "<root>=INFO;unit=DEBUG". In 2.8 the default is changing to just "<root>=INFO". We won’t change the logging-config during ‘juju upgrade-controller’, but new bootstrapped controllers will use the new default logging level.

If people want to restore the original behavior, they can run:

  juju model-config logging-config="<root>=INFO;unit=DEBUG"

Note they can also run:

  juju model-default logging-config="<root>=INFO;unit=DEBUG"

If they want to change the behavior for newly created models.

[timClicks (Tim McNamara)]

@timClicks

annotations:
  juju.io/disable-name-prefix: "true"

Hi @wallyworld, does this require a string literal containing “true” or can it also accept a YAML boolean value, e.g. true, yes, …?

[timClicks (Tim McNamara)]

@jameinel how do charms interact with the persistent state now offered by the uniter?

[jameinel (John A Meinel)]

They don’t currently. It is one of the open issues that we want to address in the operator framework. (We want to move from saving the persisted state and events in a local sqlite database into saving that state in the controller via ‘save-state’.)

The expectation is that to do it well, involving all the pieces at play is likely to take about a month’s work.

[timClicks (Tim McNamara)]

So, it’s save-state on the Juju side? What is the load function?

[jameinel (John A Meinel)]

My apologies, save-state was my internal shorthand for it. The actual methods are:

    state-delete             delete server-side-state key value pair
    state-get                print server-side-state value
    state-set                set server-side-state values

You can use

  juju help-tool state-delete

etc to get the help text. (Though it seems a bit bare bones at the moment.)