Have you ever deployed the Canonical Observabilty Stack (COS) and wondered if there was a one-liner that can do this for you? Imagine, you are an o11y-core developer and need to spin up ephemeral COS(s) in different configurations for e.g. demos, bugs, etc. Then this post is for you!
Repo setup
The main strategy of a good COS one-liner is that you can deploy it locally-built and from source. For this reason, we will clone the canonical/observability-stack repo.
Since we are working within a popular repo, add these lines to your ~/.gitignore_global to avoid committing dev files:
.terraform
**/.terraform/*
*.tfstate
*.tfstate.*
*.tfplan
.terraformrc
terraform.rc
.terraform.lock.hcl
**/testing/root.tf
Create a testing directory within observability-stack/terraform with this structure:
./observability-stack/terraform/testing/
├── get_nori_ip.sh
├── cos
│ └── root.tf
├── external-ca
│ └── root.tf
├── loki
│ └── root.tf
└── nori
└── root.tf
Now, we can update the root.tf file contents. Make sure that each file defines a Juju provider:
terraform {
required_version = ">= 1.5"
required_providers {
juju = {
source = "juju/juju"
version = "~> 1.0"
}
}
}
Note: I use the upstream TF module for COS Lite, so I will not include it in this post.
Note: each of the following root modules define and create their own model.
nori/root.tf
An S3 backend is required to run COS, so we can use the seaweed-fs charm:
resource "juju_model" "nori" {
name = "nori"
}
resource "juju_application" "nori" {
name = "sw"
model_uuid = juju_model.nori.uuid
charm {
name = "seaweedfs-k8s"
channel = "latest/edge"
}
}
external-ca/root.tf
Assuming you want full TLS config control over COS:
resource "juju_model" "external-ca" {
name = "external-ca"
}
module "ssc" {
source = "git::https://github.com/canonical/self-signed-certificates-operator//terraform"
model_uuid = juju_model.external-ca.uuid
}
cos/root.tf
From the root module, you can choose between the git source (source = git::) or the local source (source = ../../cos), but not both. This is useful if you want to edit some files in observability-stack/terraform and test it without pushing to a git remote. I also set all units to 1 to reduce system strain.
resource "juju_model" "cos" {
name = "cos"
}
data "external" "juju_nori_address" {
program = ["bash", "${path.module}/../get_nori_ip.sh"]
}
module "cos" {
source = "git::https://github.com/canonical/observability-stack//terraform/cos"
# source = "../../cos"
model_uuid = juju_model.cos.uuid
channel = "dev/edge"
internal_tls = true
external_certificates_offer_url = "admin/external-ca.certificates"
external_ca_cert_offer_url = "admin/external-ca.send-ca-cert"
s3_endpoint = "http://${data.external.juju_nori_address.result.address}:8333" # Seaweed
s3_secret_key = "placeholder"
s3_access_key = "placeholder"
loki_coordinator = { units = 1 }
loki_worker = { backend_units = 1, read_units = 1, write_units = 1 }
mimir_coordinator = { units = 1 }
mimir_worker = { backend_units = 1, read_units = 1, write_units = 1 }
tempo_coordinator = { units = 1 }
tempo_worker = { compactor_units = 1, distributor_units = 1, ingester_units = 1, metrics_generator_units = 1, querier_units = 1, query_frontend_units = 1 }
ssc = { channel = "1/stable" }
traefik = { channel = "latest/edge" }
}
get_nori_ip.sh
This requires that both jq and yq are installed.
#!/bin/bash
set -e
ADDRESS=$(juju status -m ck8s:sw --format yaml | yq -r ".applications.sw.units.\"sw/0\".address")
jq -n --arg address "$ADDRESS" '{"address": $address}'
Shell functions
Now that the repo is set up, we can add the one-liner. First, update <ABS_PATH> so that we can deploy from any directory. Then, add these two shell functions to your .bashrc or .zshrc:
tf-purge() {
rm -rf <ABS_PATH>/observability-stack/terraform/testing/"$1"/.terraform*
rm -rf <ABS_PATH>/observability-stack/terraform/testing/"$1"/terraform.*
}
tf-testing() {
tf-purge "$1" && {
local tf_dir="<ABS_PATH>/observability-stack/terraform/testing/$1"
terraform -chdir="$tf_dir" init
terraform -chdir="$tf_dir" apply
}
}
Deploy COS
To deploy COS:
- switch to a
k8sJuju controller you are theadminuser for - no pre-existing models named:
cos,nori,external-cain the controller
Then run:
# enter "yes" for each prompt
tf-testing nori
tf-testing external-ca
tf-testing cos
# check the deployment status
juju status -m k8s:cos --watch 1s
The fine print
The tf-testing command removes the TF state before deploying so you have confidence that nothing weird is affecting your deployment. If you want to work with the TF state after the deployment, you can:
terraform -chdir=./observability-stack/terraform/testing/cos plan
Deploy a standalone coordinated-worker
The coordinated workers in COS (Loki, Mimir, and Tempo) are components which have a lot of submodules, making them difficult to deploy standalone. To simplify this, I created this root module for Loki:
resource "juju_model" "coordinated-worker" {
name = "loki-standalone"
}
locals {
tls_termination = var.external_certificates_offer_url != null ? true : false
}
variable "internal_tls" {
type = bool
default = true
}
variable "external_certificates_offer_url" {
type = string
default = "admin/external-ca.certificates"
}
variable "external_ca_cert_offer_url" {
type = string
default = "admin/external-ca.send-ca-cert"
}
data "external" "juju_nori_address" {
program = ["bash", "${path.module}/../get_nori_ip.sh"]
}
module "loki" {
source = "git::https://github.com/canonical/observability-stack//terraform/loki"
model_uuid = juju_model.coordinated-worker.uuid
channel = "dev/edge"
s3_endpoint = "http://${data.external.juju_nori_address.result.address}:8333" # Seaweed
s3_secret_key = "placeholder"
s3_access_key = "placeholder"
coordinator_units = 1
backend_units = 1
read_units = 1
write_units = 1
}
module "ssc" {
count = var.internal_tls ? 1 : 0
source = "git::https://github.com/canonical/self-signed-certificates-operator//terraform"
app_name = "ca"
model_uuid = juju_model.coordinated-worker.uuid
units = 1
}
module "traefik" {
source = "git::https://github.com/canonical/traefik-k8s-operator//terraform"
channel = "latest/stable"
model_uuid = juju_model.coordinated-worker.uuid
units = 1
}
# -------------- # Provided by Traefik --------------
resource "juju_integration" "ingress" {
for_each = {
loki = {
app_name = module.loki.app_names.loki_coordinator
endpoint = module.loki.endpoints.ingress
}
}
model_uuid = juju_model.coordinated-worker.uuid
application {
name = module.traefik.app_name
endpoint = module.traefik.endpoints.ingress
}
application {
name = each.value.app_name
endpoint = each.value.endpoint
}
}
# -------------- # Provided by Self-Signed-Certificates --------------
resource "juju_integration" "internal_certificates" {
for_each = var.internal_tls ? {
loki = {
app_name = module.loki.app_names.loki_coordinator
endpoint = module.loki.endpoints.certificates
}
} : {}
model_uuid = juju_model.coordinated-worker.uuid
application {
name = module.ssc[0].app_name
endpoint = module.ssc[0].provides.certificates
}
application {
name = each.value.app_name
endpoint = each.value.endpoint
}
}
resource "juju_integration" "traefik_receive_ca_certificate" {
count = var.internal_tls ? 1 : 0
model_uuid = juju_model.coordinated-worker.uuid
application {
name = module.ssc[0].app_name
endpoint = module.ssc[0].provides.send-ca-cert
}
application {
name = module.traefik.app_name
endpoint = module.traefik.endpoints.receive_ca_cert
}
}
# -------------- # Provided by an external CA --------------
resource "juju_integration" "external_traefik_certificates" {
count = local.tls_termination ? 1 : 0
model_uuid = juju_model.coordinated-worker.uuid
application {
offer_url = var.external_certificates_offer_url
}
application {
name = module.traefik.app_name
endpoint = module.traefik.endpoints.certificates
}
}
This provides external TLS and ingress via Traefik; two common deployment scenarios we test for. This root module can also be deployed with:
# enter "yes" each prompt
# tf-testing nori
# tf-testing external-ca
tf-testing loki
# check the deployment status
juju status -m k8s:loki-standalone --watch 1s
If you want to deploy a different coordinated-worker, you can replace all loki instances and update the juju_integration resources to match the new charm.
Summary
You have achieved a TF module deploy one-liner, a COS deploy three-liner, and a standalone coordinated-worker deploy three-liner.
The best part; you can configure it how you want. This serves as a framework for reproducible and configurable dev deployments.