How to quickly clean unused LXD instances from charmcraft pack

Sometimes, when you pack a lot of charms, you may end up with lots of wasted space in /var/snap/lxd. For a single charm, you can solve this issue by running charmcraft clean; but what if you have 10, 20, 100 charms?

With the original idea from @sed-i, I wrote a small bash script that cleans up all the LXD instances that are used to pack charms. Two cool features:

  • --last-used <date> allows you to specify a cutoff time (i.e., “delete the instances that have not been used since <date>”);
  • --dry-run lets you check what would be deleted, without touching any instance.

I use argc to parse input arguments in my bash scripts, so I’ll include both my original script (you’ll need to download argc, but it’s easier to read and modify to your liking) and the “compiled” version (no dependency, but more boilerplate).

Original script (argc dependency)

#!/usr/bin/env bash
# charmcraft-purge
# @describe Delete all the LXC instances that charmcraft creates to pack charms
# @author Luca Bello
# @meta version 0.3
# @meta require-tools lxc,jq

# @option --last-used Delete instances that were last use before this date (format: anything parsed by 'date') (default: one month ago)
# @flag --dry-run Perform a dry run (don't delete instances)

eval "$(argc --argc-eval "$0" "$@")"

# Default --last-used to one week ago (if not specified)
if [[ -z $argc_last_used ]]; then
  argc_last_used="$(date -d '-7 days' +'%F')";
else
  argc_last_used="$(date -d "$argc_last_used" +'%F')"
fi

lxc_instances=$(lxc list --project=charmcraft --format=json | jq --arg cutoff "$argc_last_used" '[.[] | select(.last_used_at | . < $cutoff)]')
lxc_instances_count=$(echo "$lxc_instances" | jq -r '[ .[] | select(.name | startswith("charmcraft-")) | .name ] | length')
echo "$lxc_instances_count lxc containers will be removed (not used since $argc_last_used)"
for instance in $(echo $lxc_instances | jq -r '.[] | select(.name | startswith("charmcraft-")) | .name'); do
  echo "Deleting $instance"
  if [ "$argc_dry_run" = 0 ]; then
    lxc --project=charmcraft delete "$instance"
  fi
done

Compiled script (no dependency, lots of boilerplate)

#!/usr/bin/env bash
# charmcraft-purge
# @describe Delete all the LXC instances that charmcraft creates to pack charms
# @author Luca Bello
# @meta version 0.3
# @meta require-tools lxc,jq

# @option --last-used Delete instances that were last use before this date (format: anything parsed by 'date') (default: one month ago)
# @flag --dry-run Perform a dry run (don't delete instances)

# ARGC-BUILD {
# This block was generated by argc (https://github.com/sigoden/argc).
# Modifying it manually is not recommended

_argc_run() {
    if [[ "${1:-}" == "___internal___" ]]; then
        _argc_die "error: unsupported ___internal___ command"
    fi
    if [[ "${OS:-}" == "Windows_NT" ]] && [[ -n "${MSYSTEM:-}" ]]; then
        set -o igncr
    fi
    argc__args=("$(basename "$0" .sh)" "$@")
    argc__positionals=()
    _argc_index=1
    _argc_len="${#argc__args[@]}"
    _argc_tools=()
    _argc_parse
    _argc_require_tools "${_argc_tools[@]}"
    if [ -n "${argc__fn:-}" ]; then
        $argc__fn "${argc__positionals[@]}"
    fi
}

_argc_usage() {
    cat <<-'EOF'
charmcraft-purge 0.3
Luca Bello
Delete all the LXC instances that charmcraft creates to pack charms

USAGE: charmcraft-purge [OPTIONS]

OPTIONS:
      --last-used <LAST-USED>  Delete instances that were last use before this date (format: anything parsed by 'date') (default: one month ago)
      --dry-run                Perform a dry run (don't delete instances)
  -h, --help                   Print help
  -V, --version                Print version
EOF
    exit
}

_argc_version() {
    echo charmcraft-purge 0.3
    exit
}

_argc_parse() {
    local _argc_key _argc_action
    local _argc_subcmds=""
    while [[ $_argc_index -lt $_argc_len ]]; do
        _argc_item="${argc__args[_argc_index]}"
        _argc_key="${_argc_item%%=*}"
        case "$_argc_key" in
        --help | -help | -h)
            _argc_usage
            ;;
        --version | -version | -V)
            _argc_version
            ;;
        --)
            _argc_dash="${#argc__positionals[@]}"
            argc__positionals+=("${argc__args[@]:$((_argc_index + 1))}")
            _argc_index=$_argc_len
            break
            ;;
        --last-used)
            _argc_take_args "--last-used <LAST-USED>" 1 1 "-" ""
            _argc_index=$((_argc_index + _argc_take_args_len + 1))
            if [[ -z "${argc_last_used:-}" ]]; then
                argc_last_used="${_argc_take_args_values[0]:-}"
            else
                _argc_die "error: the argument \`--last-used\` cannot be used multiple times"
            fi
            ;;
        --dry-run)
            if [[ "$_argc_item" == *=* ]]; then
                _argc_die "error: flag \`--dry-run\` don't accept any value"
            fi
            _argc_index=$((_argc_index + 1))
            if [[ -n "${argc_dry_run:-}" ]]; then
                _argc_die "error: the argument \`--dry-run\` cannot be used multiple times"
            else
                argc_dry_run=1
            fi
            ;;
        *)
            if _argc_maybe_flag_option "-" "$_argc_item"; then
                _argc_die "error: unexpected argument \`$_argc_key\` found"
            fi
            argc__positionals+=("$_argc_item")
            _argc_index=$((_argc_index + 1))
            ;;
        esac
    done
    _argc_tools=(lxc jq)
    if [[ -n "${_argc_action:-}" ]]; then
        $_argc_action
    else
        if [[ "${argc__positionals[0]:-}" == "help" ]] && [[ "${#argc__positionals[@]}" -eq 1 ]]; then
            _argc_usage
        fi
    fi
}

_argc_take_args() {
    _argc_take_args_values=()
    _argc_take_args_len=0
    local param="$1" min="$2" max="$3" signs="$4" delimiter="$5"
    if [[ "$min" -eq 0 ]] && [[ "$max" -eq 0 ]]; then
        return
    fi
    local _argc_take_index=$((_argc_index + 1)) _argc_take_value
    if [[ "$_argc_item" == *=* ]]; then
        _argc_take_args_values=("${_argc_item##*=}")
    else
        while [[ $_argc_take_index -lt $_argc_len ]]; do
            _argc_take_value="${argc__args[_argc_take_index]}"
            if _argc_maybe_flag_option "$signs" "$_argc_take_value"; then
                if [[ "${#_argc_take_value}" -gt 1 ]]; then
                    break
                fi
            fi
            _argc_take_args_values+=("$_argc_take_value")
            _argc_take_args_len=$((_argc_take_args_len + 1))
            if [[ "$_argc_take_args_len" -ge "$max" ]]; then
                break
            fi
            _argc_take_index=$((_argc_take_index + 1))
        done
    fi
    if [[ "${#_argc_take_args_values[@]}" -lt "$min" ]]; then
        _argc_die "error: incorrect number of values for \`$param\`"
    fi
    if [[ -n "$delimiter" ]] && [[ "${#_argc_take_args_values[@]}" -gt 0 ]]; then
        local item values arr=()
        for item in "${_argc_take_args_values[@]}"; do
            IFS="$delimiter" read -r -a values <<<"$item"
            arr+=("${values[@]}")
        done
        _argc_take_args_values=("${arr[@]}")
    fi
}

_argc_maybe_flag_option() {
    local signs="$1" arg="$2"
    if [[ -z "$signs" ]]; then
        return 1
    fi
    local cond=false
    if [[ "$signs" == *"+"* ]]; then
        if [[ "$arg" =~ ^\+[^+].* ]]; then
            cond=true
        fi
    elif [[ "$arg" == -* ]]; then
        if (( ${#arg} < 3 )) || [[ ! "$arg" =~ ^---.* ]]; then
            cond=true
        fi
    fi
    if [[ "$cond" == "false" ]]; then
        return 1
    fi
    local value="${arg%%=*}"
    if [[ "$value" =~ [[:space:]] ]]; then
        return 1
    fi
    return 0
}

_argc_require_tools() {
    local tool missing_tools=()
    for tool in "$@"; do
        if ! command -v "$tool" >/dev/null 2>&1; then
            missing_tools+=("$tool")
        fi
    done
    if [[ "${#missing_tools[@]}" -gt 0 ]]; then
        echo "error: missing tools: ${missing_tools[*]}" >&2
        exit 1
    fi
}

_argc_die() {
    if [[ $# -eq 0 ]]; then
        cat
    else
        echo "$*" >&2
    fi
    exit 1
}

_argc_run "$@"

# ARGC-BUILD }

# Default --last-used to one week ago (if not specified)
if [[ -z $argc_last_used ]]; then
  argc_last_used="$(date -d '-7 days' +'%F')";
else
  argc_last_used="$(date -d "$argc_last_used" +'%F')"
fi

lxc_instances=$(lxc list --project=charmcraft --format=json | jq --arg cutoff "$argc_last_used" '[.[] | select(.last_used_at | . < $cutoff)]')
lxc_instances_count=$(echo "$lxc_instances" | jq -r '[ .[] | select(.name | startswith("charmcraft-")) | .name ] | length')
echo "$lxc_instances_count lxc containers will be removed (not used since $argc_last_used)"
for instance in $(echo $lxc_instances | jq -r '.[] | select(.name | startswith("charmcraft-")) | .name'); do
  echo "Deleting $instance"
  if [ "$argc_dry_run" = 0 ]; then
    lxc --project=charmcraft delete "$instance"
  fi
done

I hope that was useful, enjoy! :sparkles:

1 Like

will we get a charmcraft clean, or would you like to bundle it in jhack while we get there?

AFAIK this is not planned to be a charmcraft command at the moment, so having it in jhack sounds good! A version of rockcraft-purge would also be great (the same exact script, but instances start with rockcraft- instead of charmcraft-), and I guess snaps might benefit from that too :slight_smile:

you know where to find me :slight_smile:

@lucabello @ppasotti please make yourselves heard :slight_smile:

1 Like

it seems it’s WIP feat(provider): list provider instances by lengau · Pull Request #645 · canonical/craft-providers · GitHub

Awesome work (it helped me figure out I had 150Gb of leftover --rockcraft instances)

1 Like