Charmhub Transition FAQ

From 9th February 2022, charm publishing and consumption are served exclusively by Charmhub.io. This FAQ (Frequently Asked Questions) document explains what will happen after this transition away from Charm Store and if there is any action you need to take as a charm author. Here are some common charm operations, with equivalent Charmhub<->Charm Store commands that you may find useful. Please ask in the forum if you feel there is any other question that you would like us to answer.

How were things working until now?

For over a year, Charmhub and Charm Store have coexisted, sharing the Charm Store charm collection under both services.

Charm Store existed, unchanged until February 9th, and had its own web frontend at https://jaas.ai, as well as the API consumed by all Juju clients (2.8 and older, and 2.9 for charm identifiers prefixed with cs:).

Charmhub has its web frontend at https://charmhub.io, and provides an API consumed by Juju 2.9 for non-prefixed and ch: charm identifiers.

The web team has redirected most charm pages from https://jaas.ai to Charmhub.io, leveraging the fact that most of the same content is available on both.

The collection of charms offered by Charmhub includes all the charms present in Charm Store, as well as any charms registered directly on Charmhub.io or via charmcraft.

The import process is not two-way: charms created natively on Charmhub are not reflected in Charm Store.

What happened on 9th February 2022?

  1. We placed redirects on api.jujucharms.com to redirect the relevant parts of the Charm Store API to Charmhub. This caused all Juju clients to transparently start talking to Charmhub.
  2. Charm Store became read-only; any write operations are redirected to the Charmhub compatibility API, which does not support write operations. The old backends and data are still functional but are inaccessible (except for administrative purposes).
  3. All charms were switched to read-write mode, meaning that publishers are now able to manage them on Charmhub.io using the web frontend and charmcraft tool.

After this, Charm Store will continue to exist for an undetermined amount of time. It will act only as a data backup as most parts of it will not be accessible.

Note that other services which live under api.jujucharms.com, such as Omnibus, Candid and the Charm Store image registry, will continue to exist there, as there is no direct replacement planned for them.

What is the compatibility API?

To allow pre-2.9 Juju clients to continue working after transitioning charms to Charmhub, an API compatible with a subset of the old Charm Store API was implemented in Charmhub. Only the URL changes; everything else stays compatible so these clients can deploy from Charmhub using the old protocol.

The compatibility API is aimed primarily at pre-2.9 clients, which have no knowledge of the native Charmhub API. It is also consumed by Juju 2.9 and newer for cs:-prefixed charms (to ensure bundles using the old syntax continue working).

Note the compatibility API only covers consumption of charms and does not implement all of the old Charm Store API. Therefore, any other uses of the old API (including publishing and web display of charm data) should be (and are already, mostly) migrated to use the new, native Charmhub API.

As a charm author, how do I publish my charm on Charm Store?

As Charm Store is now deprecated (all the APIs are redirected to Charmhub), it’s no longer possible to publish charms directly to Charm Store. Instead, use charmcraft to publish your charm to Charmhub.

Note that charms published to Charmhub are still visible and deployable by Juju using cs: notation.

As a charm author, how do I publish my charm on Charmhub?

As of February 9th, please use charmcraft to upload and publish builds for all charms (both charmstore-imported and charmhub-native).

How can I manage my old charm on Charmhub?

From February 9th, all charms are managed on Charmhub using the charmcraft tool.

If you had an old charm in Charm Store, you should be the owner on Charmhub as well, and you can upload and release updates using charmcraft.

How can my charms use Charmhub features like tracks?

These features are now available to all charms. To request creation of tracks, ownership transfers, collaborator additions, and toggling visibility for charms, please post your request in the Charmhub forum.

What will happen with old Juju clients?

Any Juju client that talked to Charm Store (Juju 2.8 and older, and Juju 2.9 for cs:-prefixed charms) will be transparently redirected to Charmhub’s compatibility API. Clients older than Juju 2.0 use an older v4 API which is not supported, so those may stop working.

How can I have a namespace for my charms on Charmhub?

There is no namespacing in Charmhub. Read more here

Why isn’t my Charm store charm on Charmhub?

All Charm Store charms should be on Charmhub, unless they are private.

This is usually related to “findability”. Charmhub has a “show charm in search results” flag, which, for imported charms, is only set for promulgated charms.

This means that if your charm was not promulgated, it will not show up in search results.

Additionally, because there are no namespaces on Charmhub (see above question), all charm names are “converted” to a flat format. For non-promulgated charms, this has the form ~user/charm -> user-charm. For example, a charm that was available on Charm Store under https://jaas.ai/u/containers/metallb-controller is now https://charmhub/containers-metallb-controller.

Any bundles referencing the charm, or installing it directly via Juju, should still work as intended.

Charms that are natively registered on Charmhub are also created initially “unlisted” and don’t show up in search results.

The above policies were put in place to showcase promulgated charms and avoid potentially low-quality, experimental namespaced charms polluting search results.

To change visibility status, please ask the Store team (or in the Charmhub forum) to make them listed, once you’re ready to have them show up in search results. Note that any change in visibility status should be authorized by the Charmhub product managers.

Why are revision numbers different in Charmhub and Charm Store?

The way each service models charm revisions is different, so upon importing from Charm Store, Charmhub cannot always assign the same revision number to each revision. For example, if a single Charm Store revision supports multiple Ubuntu series (e.g. cs:xenial/foo-5, cs:trusty/foo-5), it will be imported as several Charmhub revisions, each with an incrementing number.

However, note that revision numbers presented via the compatibility API (that is, installed with Juju using the cs prefix) will be consistent with Charm Store and should return the expected payload.

Navigation

2 Likes

Hey @hollyhall,

I am a bit confused by the FAQ.

  1. Charm Store will become read-only, as any write operations will be redirected to the Charmhub compatibility API, which does not support write operations.

Here we clearly state that all Charm Store APIs will be switched to read-only mode. There will be no further write operations possible after the transition.

Further down we say:

Continue using the charm tool to publish charms on Charm Store. Any charm updates published this way will also be imported and visible on charmhub.io.

This implies the above is not correct and we still allow write operations through the Charm Store API after the transition has happened to get imported charms updated.

What will be the correct way to maintain charms which got imported after the transition has happened? Can I still update them with the old charm tool or do I must use charmcraft for that?

Thanks!

Simon

CC @roadmr

  1. Please use charm tool to publish charms on charm store …
  2. … until we do the cutover, at which point Charm Store will become read-only, as any write operations will be redirected to the Charmhub compatibility API, which does not support write operations.

Use charmcraft. Per the above, once we cutover, charm will no longer work as it needs to perform write operations using the old API which will no longer work.

Hopefully that makes it clearer.

  • Daniel

Thanks a lot @roadmr!

If we have to use charmcraft for publishing, that means we also have to change the charm name inside the metadata.yaml to reflect the new naming, e.g. name: foo (cs:~bar/foo) to name: bar-foo to allow charmcraft to detect which charm to publish to?

Simon

I believe this will be necessary, yes.

  • Daniel

Thanks @roadmr!

I’ve opened a separate thread for that part at Updating imported charms through Charmhub

Simon

Hi guys, I just found Kubeflow has been broken since yesterday which was the charmstore cutover date. It seems the registry.jujucharms.com is broken.

$ juju status --relations --color --storage -m k1:t3
Model  Controller  Cloud/Region        Version  SLA          Timestamp
t3     k1          microk8s/localhost  2.9.25   unsupported  15:52:43+11:00

App                        Version                    Status   Scale  Charm                    Store     Channel      Rev  OS          Address         Message
admission-webhook          res:oci-image@fc124ea      waiting      1  admission-webhook        charmhub  stable        12  kubernetes                  waiting for container
argo-controller            res:oci-image@0eec3c1      waiting      1  argo-controller          charmhub  stable        56  kubernetes
dex-auth                   res:oci-image@a74f783      waiting      1  dex-auth                 charmhub  2.28/stable   78  kubernetes                  waiting for container
envoy                      res:oci-image@b4adee5      waiting      1  envoy                    charmhub  stable         6  kubernetes  10.152.183.74
istio-ingressgateway                                  waiting      1  istio-gateway            charmhub  1.5/stable    40  kubernetes                  Waiting for istio-pilot relation data
istio-pilot                res:oci-image@87fc646      waiting      1  istio-pilot              charmhub  1.5/stable    61  kubernetes                  List of ingress versions not found for apps: kubeflow-dashboard
jupyter-controller         res:oci-image@62a1ccf      waiting      1  jupyter-controller       charmhub  stable        61  kubernetes                  waiting for container
jupyter-ui                 res:oci-image@5536a2d      waiting      1  jupyter-ui               charmhub  stable        21  kubernetes  10.152.183.2
kfp-api                    res:oci-image@81e784a      waiting      1  kfp-api                  charmhub  stable        33  kubernetes  10.152.183.30
kfp-db                     mariadb/server:10.3        active       1  charmed-osm-mariadb-k8s  charmhub  stable        35  kubernetes  10.152.183.251
kfp-persistence            res:oci-image@1012943      waiting      1  kfp-persistence          charmhub  stable        29  kubernetes
kfp-profile-controller     res:oci-image@14ec522      waiting      1  kfp-profile-controller   charmhub  stable        16  kubernetes  10.152.183.4
kfp-schedwf                res:oci-image@34e7e9e      waiting      1  kfp-schedwf              charmhub  stable        32  kubernetes                  waiting for container
kfp-ui                     res:oci-image@b67a29c      waiting      1  kfp-ui                   charmhub  stable        32  kubernetes  10.152.183.114
kfp-viewer                 res:oci-image@c208ebd      waiting      1  kfp-viewer               charmhub  stable        31  kubernetes                  waiting for container
kfp-viz                    res:oci-image@13c46cf      waiting      1  kfp-viz                  charmhub  stable        28  kubernetes  10.152.183.43
kubeflow-dashboard                                    blocked      1  kubeflow-dashboard       charmhub  stable        64  kubernetes                  kubeflow-dashboard must be deployed to model named `kubeflow`: https://git.io/J6d35
kubeflow-profiles          res:profile-image@f4450cf  waiting      1  kubeflow-profiles        charmhub  stable        57  kubernetes                  List of kubeflow-profiles versions not found for apps: kubeflow-dashboard
kubeflow-roles                                        active       1  kubeflow-roles           charmhub  stable         1  ubuntu      10.152.183.31
kubeflow-volumes           res:oci-image@fedee0e      waiting      1  kubeflow-volumes         charmhub  stable        11  kubernetes  10.152.183.44
metacontroller-operator                               active       1  metacontroller-operator  charmhub  stable         2  ubuntu      10.152.183.126
minio                      res:oci-image@1755999      waiting      1  minio                    charmhub  stable        57  kubernetes  10.152.183.88
mlmd                       res:oci-image@e2cb9ce      waiting      1  mlmd                     charmhub  stable        10  kubernetes  10.152.183.80
oidc-gatekeeper            res:oci-image@4e7f8dd      waiting      1  oidc-gatekeeper          charmhub  stable        57  kubernetes  10.152.183.169
seldon-controller-manager  res:oci-image@047f2fc      waiting      1  seldon-core              charmhub  stable        52  kubernetes  10.152.183.3
training-operator                                     active     0/1  training-operator        charmhub  stable         6  kubernetes  10.152.183.127

Unit                          Workload  Agent       Address      Ports              Message
admission-webhook/0*          waiting   idle                                        waiting for container
argo-controller/0*            error     idle        10.1.97.195                     unknown container reason "ImagePullBackOff": Back-off pulling image "registry.jujucharms.com/charm/inj6efkysev7imlwdaew4g01y37n9igvzovw9/oci-image@sha256:0eec3c14f3d908ce7bf6fcad49f4883a756786da9e7c8c70f24b270ffba16946"
dex-auth/0*                   waiting   idle                                        waiting for container
envoy/0*                      error     idle        10.1.97.218  9901/TCP,9090/TCP  unknown container reason "ImagePullBackOff": Back-off pulling image "registry.jujucharms.com/charm/c3jh1tja0aj8qdj21dodkderf10kmkbeboth3/oci-image@sha256:b4adee54f7754edefe0ad0d40ceefb4d72af2203e315aaca6f39b079c810fcf5"
istio-ingressgateway/0*       waiting   idle                                        Waiting for istio-pilot relation data
istio-pilot/0*                waiting   idle                                        List of ingress versions not found for apps: kubeflow-dashboard
jupyter-controller/0*         waiting   idle                                        waiting for container
jupyter-ui/0*                 error     idle        10.1.97.211  5000/TCP           unknown container reason "ImagePullBackOff": Back-off pulling image "registry.jujucharms.com/charm/9lw7s63ewtlyew486jjn1ez4fe2mh813xni8f/oci-image@sha256:5536a2da977f5d6d10d3c21296e1cc83aa4e7bddc9cd42616cb700a4f31e89ed"
kfp-api/0*                    error     idle        10.1.97.215  8888/TCP,8887/TCP  unknown container reason "ImagePullBackOff": Back-off pulling image "registry.jujucharms.com/charm/6dmow0tt4o9ss0keyowwkifpww9nhegg6exlj/oci-image@sha256:81e784a137925307ad8ffc95e3b6134f1cb681f0aeb151aff4873fd759491129"
kfp-db/0*                     active    idle        10.1.97.227  3306/TCP           ready
kfp-persistence/0*            error     idle        10.1.97.194                     unknown container reason "ImagePullBackOff": Back-off pulling image "registry.jujucharms.com/charm/530iqpar7ey6vmhy5v19z5dnqywn1439aqet2/oci-image@sha256:1012943122e9ffc3eca4b9c54b2259d85a71a617ea14dff6a1ebf6fc6b91580b"
kfp-profile-controller/0*     error     idle        10.1.97.199  80/TCP             unknown container reason "ImagePullBackOff": Back-off pulling image "registry.jujucharms.com/charm/gm1axzm8pxqlan75l3a7znu2mv5bf0pm1wfar/oci-image@sha256:14ec52252771f8fa904afbdac497c80fc3234d518b1e0bced0c810d5748a7347"
kfp-schedwf/0*                waiting   idle                                        waiting for container
kfp-ui/0*                     error     idle        10.1.97.209  3000/TCP           unknown container reason "ImagePullBackOff": Back-off pulling image "registry.jujucharms.com/charm/3fs6abrv95ts1n20tdw7irqnkfawe3pmtnzrc/oci-image@sha256:b67a29cd66cc88c7c16883005757ab6886d7172d3476f85270cb77fe068c9c43"
kfp-viewer/0*                 waiting   idle                                        waiting for container
kfp-viz/0*                    error     idle        10.1.97.204  8888/TCP           unknown container reason "ImagePullBackOff": Back-off pulling image "registry.jujucharms.com/charm/c2o31yht1y825t6n49mwko4wyel0rracnrjn5/oci-image@sha256:13c46cf878062fd6ad672cbec4854eba7e869cd0123a8975bea49b9d75d4e698"
kubeflow-dashboard/0*         blocked   idle                                        kubeflow-dashboard must be deployed to model named `kubeflow`: https://git.io/J6d35
kubeflow-profiles/0*          waiting   idle                                        List of kubeflow-profiles versions not found for apps: kubeflow-dashboard
kubeflow-roles/0*             active    idle        10.1.97.249
kubeflow-volumes/0*           error     idle        10.1.97.255  5000/TCP           unknown container reason "ImagePullBackOff": Back-off pulling image "registry.jujucharms.com/charm/6tvhkd5az86qo4m4zxeoqip1iejoxusiphggc/oci-image@sha256:fedee0ed496093fc72bf4a5fd11549d13224960c526235c0b23bb19186d053ae"
metacontroller-operator/0*    active    idle        10.1.97.232
minio/0*                      error     idle        10.1.97.206  9000/TCP           OCI image pull error: rpc error: code = Unknown desc = failed to pull and unpack image "registry.jujucharms.com/charm/81j63o4a2ldarn1umc22iyjz1q9l9g0sx5b8j/oci-image@sha256:1755999849a392bdf00b778705f4cf5c1c971a1ef55a17e9075e56f8d58bdc2f": failed to resolve reference "registry.jujucharms.com/charm/81j63o4a2ldarn1umc22iyjz1q9l9g0sx5b8j/oci-image@sha256:1755999849a392bdf00b778705f4cf5c1c971a1ef55a17e9075e56f8d58bdc2f": failed to authorize: failed to fetch oauth token: unexpected status: 400 Bad request
mlmd/0*                       error     idle        10.1.97.203  8080/TCP           unknown container reason "ImagePullBackOff": Back-off pulling image "registry.jujucharms.com/charm/4cyk13qm8dv3h6331811wlclu917dl2ifj5ax/oci-image@sha256:e2cb9ce0e007fd9306ebc93fbae54ebd0db2cfa04cfde53c1ce03adeb05b0f8a"
oidc-gatekeeper/0*            error     idle        10.1.97.198  8080/TCP           unknown container reason "ImagePullBackOff": Back-off pulling image "registry.jujucharms.com/charm/8msvfykzw1otf8kg225c5eqctq3fcppryljdq/oci-image@sha256:4e7f8ddd122c2f02dee29c14fd05ffd620686549f71795740b69f112a8cde6a6"
seldon-controller-manager/0*  error     idle        10.1.97.234  8080/TCP,4443/TCP  unknown container reason "ImagePullBackOff": Back-off pulling image "registry.jujucharms.com/charm/c5e3s519ko1quc9tqnysy92oeeqmgvwvhxvgf/oci-image@sha256:047f2fc1a8adfd72871af0f0c5cf92c9297536180293d413fba93d5a411aa96d"
training-operator/0*          waiting   allocating  10.1.97.251                     Back-off pulling image "registry.jujucharms.com/charm/7brniqppl0lq3h7zopjcmmsx9k1bxi3pvilrj/training-operator-image@sha256:85272e1974b8bd3714d5a76ee6d1315546a97e29e6e8ec3414db237c038b4dca"
  Warning  Failed            2m47s (x4 over 4m27s)  kubelet            Failed to pull image "registry.jujucharms.com/charm/81j63o4a2ldarn1umc22iyjz1q9l9g0sx5b8j/oci-image@sha256:1755999849a392bdf00b778705f4cf5c1c971a1ef55a17e9075e56f8d58bdc2f": rpc error: code = Unknown desc = failed to pull and unpack image "registry.jujucharms.com/charm/81j63o4a2ldarn1umc22iyjz1q9l9g0sx5b8j/oci-image@sha256:1755999849a392bdf00b778705f4cf5c1c971a1ef55a17e9075e56f8d58bdc2f": failed to resolve reference "registry.jujucharms.com/charm/81j63o4a2ldarn1umc22iyjz1q9l9g0sx5b8j/oci-image@sha256:1755999849a392bdf00b778705f4cf5c1c971a1ef55a17e9075e56f8d58bdc2f": failed to authorize: failed to fetch oauth token: unexpected status: 400 Bad request

  Warning  Failed     2m50s (x4 over 4m17s)  kubelet            Failed to pull image "registry.jujucharms.com/charm/c5e3s519ko1quc9tqnysy92oeeqmgvwvhxvgf/oci-image@sha256:047f2fc1a8adfd72871af0f0c5cf92c9297536180293d413fba93d5a411aa96d": rpc error: code = Unknown desc = failed to pull and unpack image "registry.jujucharms.com/charm/c5e3s519ko1quc9tqnysy92oeeqmgvwvhxvgf/oci-image@sha256:047f2fc1a8adfd72871af0f0c5cf92c9297536180293d413fba93d5a411aa96d": failed to resolve reference "registry.jujucharms.com/charm/c5e3s519ko1quc9tqnysy92oeeqmgvwvhxvgf/oci-image@sha256:047f2fc1a8adfd72871af0f0c5cf92c9297536180293d413fba93d5a411aa96d": failed to authorize: failed to fetch oauth token: unexpected status: 400 Bad request

Did anyone find a similar issue?

Hi Kelvin - I believe this was an issue with the cut over, but I’m told that they have fixed this (it was an API that was not meant to be changed yet for the registry to do authentication). This should be working today, if there are any issues we should bring them up quickly.

Yeah, I saw the registry is now working. Thanks!

The migration seems to have broken links to bundles that were published to the charm store.

Ex: All the kubernetes bundles linked here Installing Charmed Kubernetes | Ubuntu like https://api.jujucharms.com/charmstore/v5/charmed-kubernetes-862/archive/bundle.yaml are now returning 404 errors.

Somewhat related, will Charmhub have an equivalent feature to Charm Store where you are able to view the files included in a revision of a charm/bundle?

Found a workaround to my above issue by downloading via the charm client:

charm pull ch:charmed-kubernetes-862