Charms for centos - lets begin

juju
21

@erik-lonroth, as you’ve probably seen, me and my team are working on developing the Lucky charming framework and Centos support is something that we are very interested in.

To address your stated problems:

  1. Lucky doesn’t use charmhelpers and is a standalone binary that doesn’t have any dependencies on Python or anything else, so that shouldn’t be a problem
  2. Lucky doesn’t have layers so that isn’t a problem
  3. lucky charm create will create a charm that should be able to be deployed anywhere

That said, there are other potential problems to consider:

Installing Docker

Because Lucky charms don’t have layers, the primary way to install software dependencies will be to create a Docker image that contains all of the aplication’s dependencies. This means that we must be able to install Docker on all target systems ( Ubuntu, CentOS, etc. ).

I have heard of people having problems installing Docker on RHEL/CentOS, so that may be somewhat difficult, but it is possible to do. We will just have to build in to Lucky different methods of installing Docker for the different distros. ( BTW, we might do this with the Docker snap, but it has problems on CentOS too. )

The good thing about putting your application dependencies in Docker is that the application’s software packages are now portable across CentOS and Ubuntu. If I write a charm and my charm’s Docker image is based on Ubuntu and I use apt to install my application dependencies, then that charm will still run on CentOS, and even other versions of Ubuntu, without changes because it is in a Docker image.

Bash vs. Python

Lucky will initially target bash scripts as the primary way of developing charms. We believe that this should be fully sufficient for most use-cases, but that is up to interpretation.

We are not sure exactly what Python support in Lucky will look like on the first release, but at the very least you will have to interact with Lucky by calling the Lucky CLI from your Python script instead of having the luxury of a Python library with Python functions to call. This will most-likely change in the future, but it won’t be the first thing out of the gate.

Relations

Related to using Bash instead of Python, you will have to interact with the Juju relations without any helper Python libraries that come with the Juju interface layers. The Lucky CLI will have commands to make interacting with the relations as easy as possible, but it will not initially have support for interface-specific helpers. You will have to read and write the key-value pairs on the relation yourself. My initial observation leads me to believe that interacting with the relations like this will not be a burden, but that is up to interpretation.

We are going to work to make this as easy as possible, but we are going to have to do some experimentation to find out what the best workflow will look like.

Under Development

The final concern with Lucky is that it isn’t finished yet. We are expecting to be able to get a first release out within the next two months, but hopefully much sooner. Even after a first release, the Lucy CLI will probably be changing and you may have to keep up with that for the first while, but if you try it out and have use for it we will do what we can to make it work for your use-case.

Assuming that the CentOS support problems come only from the charming framework and not Juju itself ( which is what it looked like from your stated problems, if I understand correctly ), I think that we will reasonably be able to support CentOS with Lucky.

Does this sound like something that would help you? I’m open to questions or suggestions. Is there anything that you think would be a blocker for you, such as needing Python or not wanting to use Docker?

22

I think your effort highlight the need. Amzing work! However, the approach will bring more issues.

  • docker will add in a significant overhead to the running device. This will not be good for deployment on small devices where docker might be a no no. Also, it will alienate other container architectures such as singularity.
  • bash only charms will not be cool. A huge ecosystem of great software will be alienated. I don’t think bash can be default/preferred. It should be supported.

I think it’s critical to think “Linux” instead of “distros” for a framework such as juju since otherwise it will be just another Ubuntu tool. It’s great, but won’t be able to reach a broader community of developers from domains where Ubuntu not already has a strong precence.

I for one hope to be able to use juju for deploying software on embedded devices. This might be an impossible task today, but it also serves as a good benchmark for the architectural challenge of getting juju in shape for broader adoption.

1 Like
23

That’s a valid point.

I do think that, if you want to write charms that can run on any distro, you will either have to use containers or something like snaps, flatpaks, etc. . That said, writing charms that run on any distro and having a charming framework that supports any distro are two different things. Maybe you don’t need your charm to run on any distro, but you do need your charming framework to allow you to write a charm for the distro you are interested in.

It is very possible that we might extend support to different container engines such as singularity. We could probably even abstract the CLI over the different engines by providing a single way in which to add volumes, environment variables, etc. . If there is a use-case for a different container engine then it would be worth looking into at least.

In fact, if Singularity is lighter-weight than Docker and still supports the Docker image format, then it could be a great option to use with Lucky. I’ll look into it and discuss it with my team.

If you don’t mind, I would like to understand a bit more about what would be alienated. What kind of software are you using that would be pushed out?

I come from a background where I’ve written many Docker images and I use bash as the primary method of automation for all of them. My assumption on how charming would be like with Lucky is something like this:

  1. I have an application I need to charm, so I write a Docker container for it or I use a container that the vendor has already made.
  2. In the charm I add the logic needed to set environment variables, change config files, start/stop the application, etc., all this in response to the Juju hooks and relations I am interested in.

The logic for the charm, I imagine, should be fundamentally simple for most applications. The charm is doing the operations that an operator would otherwise do manually, often with a CLI. My thought is that you would not need Python to accomplish these tasks.

This is just my understanding in the realm of my own experience so I would like to understand more of your use-case.

1 Like
24

We’ve definitely found through experience that relying on individual charm authors to get the details of the key-value mapping of relations right in every charm leads, somewhat paradoxically, to both fragmentation as well as difficulty in evolving the interface. Part of this is not having a good pattern to use for versioning an interface API, but even just having the KV pattern abstracted into a shared code-base that abstracts it into a defined API makes it so that only a single author has to think and code for compatibility changes in the interface protocol, as opposed to every single charm author having to keep all of that in mind for all interfaces that they want to use.

25

If you haven’t seen it already, it would be good to get your input on https://github.com/canonical/operator/ which is another replacement framework being worked on which has some of the same goals, though it is targeting (pure) Python as the language of choice. Certainly having your team’s perspective on the challenges you’ve faced and how you overcame them would be helpful on that, and vice versa for your team seeing how that framework addresses issues you may run into.

26

In effort to help that, it would be simple to have shell scripts that serve essentially the same exact purpose of the Python interface layers. You would just drop the shell script into a charm utils dir or use a built-in “package manager” tool to do it for you and it would put the tool in your charm script’s PATH so that you could call it to interact over the relation.

The issue with the system of using layers or utility libraries to communicate over relations is that they are specific to a framework while properly documenting the key-value requirements of the relation like we have started doing with the #docs:interfaces category allows any developer to properly design their charm for the interface regardless of the framework they are using.

When I tried to write a bash reactive charm that simply consumed a PostgreSQL relation, the pgsql layer documentation was extremely minimal and I found that the only indication of how to actually interact with the relation in bash was in the README of the PostgreSQL charm. I think that the first step to defining a relation should be to define it in terms of the key-value behavior, even if you proceed afterwards to create a layer or other library for it.

Unfortunately the method of providing that library to the community for use has to be framework specific unless we come up with a universal charm interface definition of some sort that can be supported by any charming framework. ( That might be impossible, but probably worth brainstorming. )

Versioning relations is an interesting topic, too. Honestly it seems like the only safe way to do this is to put the version name in the interface name. For example, instead of pgsql you would have pgsql.1 or pgsql.2. That would be the only way to ensure that you couldn’t ever connect a charm that only supports an old version of the protocol to a charm that provides the new version of the protocol.

Every time a backward incompatible change was made to the interfaces behavior you would have to increment the version number. I think that making a practice of this might actually be required if we want to advance the defined interfaces while ensuring an ecosystem where charms can still safely relate to each-other.

Yes, I’d heard of it, but I didn’t know there was a public repo for it or what the name was. :slight_smile:

Absolutely. I’ll definitely try that out. We’ve almost got Lucky’s first release out so it will be very interesting to compare the frameworks and see what they might be able to learn from each-other.

27

I’ve ben involved with the charming community for a relatively short amount of time, so would love to hear more background related to this.

My perception is that a “shared code-base” typically becomes dominated by a single developer/team. And they’re more than happy to make changes without seeking input from other parties. From what I’ve seen, layer developers don’t see interfaces as shared or common, but a private implementation detail.

IMO introducing breaking changes to an interface would be better handled by creating another interface. But that opinion isn’t very well formed.