Charmcraft Programmatic Outputs

Abstract

Definition of some Charmcraft commands’ outputs that are easily consumable by other programs/scripts.

Rationale

Charmcraft is used more and more in automatic systems (e.g. CI/CD). To simplify the automation scripts, Charmcraft should be able to be used programmatically. This means that it should provide a simple way to parse its outputs.

Currently one command provides a way to format its output for easy programmatic consumption: analyze. It has a --format option that allows to specify in which format the output should be encoded (currently only JSON). If the option is not used, the output is formatted for easier human readability.

This proposal formalizes the extension of that mechanism to most Charmcraft commands.

Exceptions

Not all commands need to be consumed programmatically, though. Here is a list of those exceptions and the reasoning behind the exclusion.

  • help: its output is always intended for human consumption; furthermore, it’s automatically handled by Craft CLI, so we will not interfere from Charmcraft.

  • login and logout: the login process always needs human interaction; logout is included here for symmetry.

  • init: there is no information in the final output other than reminders for the developer on how to complete the provided template; if this command would be ever used programmatically, it would be enough to check the process’ return code.

  • build: it’s deprecated.

Commands Without Outputs

The following are commands that even as they may be used programmatically, there is no information in the final output (just a message to the user that all ended correctly) so it’s enough to check the process’ return code.

  • clean
  • close
  • register
  • register-bundle
  • release

Outputs Formats

The following would be the formats produced by the different commands.

Possible future changes in these formats should be specified including the Charmcraft version where they happened. Backwards incompatible changes should be minimized, and only be introduced in major Charmcraft versions.

  • pack: pkgtype may be “charms” or “bundles”; nameN are the filenames of the produced binaries.
    {
      pkgtype: [name1, ...],
    }
  • version:
    {
      "version": the_version_string,
    }
  • whoami: name, username and id are optional, they will be present only if logged is True. Also permissions, channel and packages are optional keys, as those restrictions may not exist in the current credentials (the later may include charms/bundles or not). When charms or bundles are included, for each case it’s specified which type of identifier is used (name or id) and its value.
    {
      "logged": bool,
      "name": account_name,   <-- optional
      "username": account_username,   <-- optional
      "id": account_id,   <-- optional
      "permissions": [perm1, ...],   <-- optional
      "packages":   <-- optional
        {
          "charms": [{identifier_type: identifier_value}, ...],   <-- optional
          "bundles": {identifier_type: identifier_value}, ...],   <-- optional
        },
      "channels": [channel1, ...]   <-- optional
    }
  • names:
    [
      {
        "name": name1,
        "type": type1,
        "visibility": visibility1,
        "status": status1,
      },
      ...
    ]
  • upload: errors xor revision may be present, according to success or not (which can also be verified using process’ return code)
    {
      "revision": uploaded_revision,   <-- optional
      "errors": [{"code": code1, "message": message1}, ...]   <-- optional
    }
  • revisions: if the status is not approved an errors key may be present with the relevant information.
    [
      {
        "revision": revision1,
        "version": version1,
        "created_at": created_at1,
        "status": status1,
        "errors": [{"message": message1, "code": code1}, ...],  # <-- optional
      },
      ...
    ]
  • status:
    {
      "track": track_name,
      "mappings": [
          {
            "base":  # <--- this can be None (no dict with detailed info in that case)
              {
                "name": base_name,
                "channel": channel,
                "architecture": architecture,
              },
            "releases":
              [
                {
                  "status": status,  # `open`, `tracking` or `closed`
                  "channel": channel,
                  "version": version,
                  "revision": revision,
                  "resources": [{"name": name1, "revision", ...],
                  "expires_at": timestamp_or_None,
                },
               ...
             ],
           },
           ...
         ],
     }

  • create-lib:
    {
      "library_id": the_lib_id,
    }
  • publish-lib:
   [
     {
       "charm_name": charm_name1,
       "library_name": lib_name1,
       "library_id": lib_id1,
       "api": api_version1,
       "published":   <-- optional
         {   
           "patch": patch_version1,
           "content_hash": hash1,
         },
       "error_message": explanation,  <-- optional, and see note 1 below
     },
     ...
   ]
  • fetch-lib:
    [
      {
        "charm_name": charm_name1,
        "library_name": lib_name1,
        "library_id": lib_id1,
        "api": api_version1,
        "fetched":   <-- optional
          {
            "patch": patch_version1,
            "content_hash": hash1,
          },
        "error_message": explanation,  <-- optional, and see note 1 below
      },
      ...
    ]
  • list-lib:
    [
      {
        "charm_name": charm_name1,
        "library_name": lib_name1,
        "library_id": lib_id1,
        "api": api_version1,
        "patch": patch_version1,
        "content_hash": hash1,
      },
      ...
    ]
  • list-resources:
    [
      {
        "charm_revision": charm_rev1,
        "name": resource1,
        "type": type1,
        "optional": optional1,
      },
      ...
    ]
  • upload-resource: errors xor revision may be present, according to success or not (which can also be verified using process’ return code)
    {
      "revision": uploaded_revision,   <-- optional
      "errors": [{"code": code1, "message": message1}, ...]   <-- optional
    }
  • list-resource-revisions
    [
      {
          "revision": revision1,
          "created at": created_at1,
          "size": size1,
      },
      ...
    ]

Note 1: the error results for library related operations can be improved later if there is really need (having specific codes and extra metadata for the different errors); the current key is purposefully error_message so we could add a complex error in the future without breaking compatibility.

4 Likes

Excited for this - @dominik.f / @0x12b this should make the Github Actions you’ve been working on a bit more solid I’d have thought! :slight_smile:

2 Likes

Love it! This is a much needed addition! :star_struck:

1 Like

The implementation of this spec is currently done, so far in main, will be released in edge in a few.

As of yesterday, this has now reached stable :tada:

  ❯ snap info charmcraft
  name:      charmcraft
  summary:   The charming tool
  publisher: Canonical✓
  store-url: https://snapcraft.io/charmcraft
  license:   Apache-2.0
  description: |
    Charmcraft enables charm creators to build, publish, and manage charmed operators for Kubernetes,
    metal and virtual machines.
  commands:
    - charmcraft
  snap-id:      gcqfpVCOUvmDuYT0Dh5PjdeGypSEzNdV
  tracking:     latest/candidate
  refresh-date: 6 days ago, at 10:13 CEST
  channels:
+   latest/stable:    2.0.0             2022-08-01 (1033) 59MB classic
    latest/candidate: 2.0.0             2022-07-18 (1033) 59MB classic
    latest/beta:      2.0.0             2022-07-12 (1033) 59MB classic

Thank you, @facundo and team!

1 Like