Can this be done by setting default cloud-init data in MAAS? In this way all instance would have this setup?
If not there, it can be done with custom cloud-init data with Juju itself.
My point is that leveraging cloud-init means you don’t have to customize the image, but basically create a “pre step” during deployments.
I will try this above since it seems to be able to provide a shorter path to getting at first “yaml” on to the nodes in models using centos.
But I still think that basic reactive charms would be possible to get going without this preparation. Oh, given python3 perhaps as a single requirement =/ At least for charms.reactive kind of things…
At the moment, I’m down to trying to understand how layer-options could be made to include “pyyaml” as a wheelhouse installation. But so far I haven’t been able to figure out how.
https://github.com/juju-solutions/layer-options/blob/master/lib/charms/layer/options.py
After that, @hallback:s fixes and some in layer-basic should perhaps be enough to get a better start for centos-like: distros.
I tried your recommendation, with no avail. Below is the situation.
I’ll keep going =)
I’m making another attempt like this now, trying to get a modified “basic.py” and @hallback fixes to reactive.charms
Following the advice here: https://serverfault.com/questions/988874/how-can-i-enable-epel-release-before-installing-packages-with-cloud-init
I really need to get some LXD instances up to be able to test this faster… Machines takes forever to boot.
K, can you file a bug on the centos not working with the additional packages? That seems worth making right with our current setup.
I was definitely thinking you’d have to leverage the preruncmd to get that setup ahead of time, but could make it consistent and automated using that.
Definitely agree that testing that setup up for charming on lxd is great.
Hey @rick_h, I will update on this soon as I actually got something that worked efter a few tries with your suggestions.
I have basically used the model config adding in a repo, added in a few packages, modified layer-basic and the patch from @hallback.
With all that, I deployed the minimal centos reactive charm.
I would consider that a very complex activity today but could be greatly reduced with the code fixes.
But I think it highlights the need to work on getting charms out of being a ‘Ubuntu only’ tool.
No doubt, not defending doing nothing here at all. I think this is just really good that we can use this to help work out and validate what needs doing, what doesn’t, etc.
For speeding up development I wonder if the cloud-init config can be made a model-default value so that all models on the controller have it and kind of allow for quicker iteration, but again, that’s a hack not a solution.
Thanx @rick_h I’m not holding you guilty =) I’m ready to pitch in with my findings and contribute with thoughts on getting juju stronger. Its a great tool with potential to be even better. A good thought is to strive to get juju linux agnostic if possible.
Did anyone say ‘snap’?
I have produced and tested a patch for layer-basic https://github.com/juju-solutions/layer-basic/pull/148
This works along-side with this patch:https://github.com/erik78se/charm-helpers/tree/hallback/centosfixes which @hallback might want to create a PR for.
I’m not sure I should make the PR against master, but I’ll fix that if needed.
Here is a charm that deploys on my local maas with NO modifications to the Centos7 image:
The charm has the patches I mentioned above and the model contains only this.
$ juju model-config cloudinit-userdata
yum_repos:
epel-release:
baseurl: http://download.fedoraproject.org/pub/epel/7/$basearch
enabled: true
failovermethod: priority
gpgcheck: true
gpgkey: http://download.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-7
name: Extra Packages for Enterprise Linux 7 - Release
packages:
- 'python3'
- 'python36-PyYAML'
However, the direct drawback, is that having BOTH centos and ubuntu series within the same model will become impossible as package-names will differ and cloud-init will bork.
OK, @rick_h @hallback with the following “hack” I have managed to get running a model with mixed centos/ubuntu in a lxd localhost cloud behind a corporate proxy.
The experience achieving this was horrible, but at least its possible without messing up the stock images with centos.
It deploys a centos7 reactive charm I made for the purpose: cs~erik-lonroth/tiny-reactive-centos along with the cs~erik-lonroth/tiny-bash, which is a non reactive ubuntu charm.
The trick I had to use, was to make yum use the proxy config with the ‘–setopt=’ argument. This is needed to happen ‘before’ the preruncmd is executed. Installing packages from the “EPEL” repo is not possible otherwise since the proxy is not set at this moment. Patching cloud-init to allow the yum proxy to be configured as a configuration item in the ‘cloudinit-userdata’ would have made this alot nicer.
Also, a configuration for “yum” in the model config might make sense down the road.
This is how I end up for now prepping my model and deploying charms:
modelconfig.yaml
cloudinit-userdata: |
yum_repos:
epel-release:
baseurl: http://download.fedoraproject.org/pub/epel/7/$basearch
enabled: true
failovermethod: priority
gpgcheck: true
gpgkey: http://download.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-7
name: Extra Packages for Enterprise Linux 7 - Release
preruncmd:
- yum --setopt=proxy=http://some.proxy.com:8080 -y install epel-release redhat-lsb-core sudo openssh-server python36-virtualenv python36-PyYAML python3 snapd
- echo "proxy=http://some.proxy.com:8080" >> /etc/yum.conf
postruncmd:
- usermod -p '*' ubuntu
- 'systemctl start sshd'
- 'systemctl enable snapd'
http-proxy: http://some.proxy.com:8080
https-proxy: http://some.proxy.com:8080
snap-http-proxy: http://some.proxy.com:8080
snap-https-proxy: http://some.proxy.com:8080
After takin this path, I’ve now come to try to deploy a centos-charm. A very basic one really. But that is not working out so good.
I started off with a few attempts like:
“charm create -t python-basic subsub”.
“charm create -t python subsub”
The outcome renders always into a fail/error on centos.
unit-subsub-0: 15:27:05 DEBUG unit.subsub/0.install Traceback (most recent call last):
unit-subsub-0: 15:27:05 DEBUG unit.subsub/0.install File “/var/lib/juju/agents/unit-subsub-0/charm/hooks/install”, line 8, in
unit-subsub-0: 15:27:05 DEBUG unit.subsub/0.install from charmhelpers.core import (
unit-subsub-0: 15:27:05 DEBUG unit.subsub/0.install ImportError: No module named charmhelpers.core
unit-subsub-0: 15:27:05 ERROR juju.worker.uniter.operation hook “install” failed: exit status 1
unit-subsub-0: 15:28:27 DEBUG unit.subsub/0.install Traceback (most recent call last):
unit-subsub-0: 15:28:27 DEBUG unit.subsub/0.install File “/var/lib/juju/agents/unit-subsub-0/charm/hooks/install”, line 8, in
unit-subsub-0: 15:28:27 DEBUG unit.subsub/0.install from charmhelpers.core import (
unit-subsub-0: 15:28:27 DEBUG unit.subsub/0.install ImportError: No module named charmhelpers.core
unit-subsub-0: 15:28:27 ERROR juju.worker.uniter.operation hook “install” failed: exit status 1
Even if this error might be for something not directly related to centos, the three most critical pieces that all fails today are:
There are a number of PR:s out for charmhelpers and layer-basic, which would perhaps restore some support for non-ubuntu charms, but there is not much activity in those repos or response on the PR:s made?
https://github.com/juju-solutions/layer-basic/pull/148
https://github.com/juju/charm-helpers/pull/400
It’s very difficult to get a proper path working with all obstacles needing to be covered for to be able to juju with centos.
The current state of juju is that it does not support anything but Ubuntu. I think that is something that MUST change.
I think any future framework for charming needs to carefully consider a much more general linux support for juju/charms rather than Ubuntu only.
For now, is the only remedy is to use hooks only and bash to create my charms?
@erik-lonroth, as you’ve probably seen, me and my team are working on developing the Lucky charming framework and Centos support is something that we are very interested in.
To address your stated problems:
lucky charm create will create a charm that should be able to be deployed anywhereThat said, there are other potential problems to consider:
Because Lucky charms don’t have layers, the primary way to install software dependencies will be to create a Docker image that contains all of the aplication’s dependencies. This means that we must be able to install Docker on all target systems ( Ubuntu, CentOS, etc. ).
I have heard of people having problems installing Docker on RHEL/CentOS, so that may be somewhat difficult, but it is possible to do. We will just have to build in to Lucky different methods of installing Docker for the different distros. ( BTW, we might do this with the Docker snap, but it has problems on CentOS too. )
The good thing about putting your application dependencies in Docker is that the application’s software packages are now portable across CentOS and Ubuntu. If I write a charm and my charm’s Docker image is based on Ubuntu and I use apt to install my application dependencies, then that charm will still run on CentOS, and even other versions of Ubuntu, without changes because it is in a Docker image.
Lucky will initially target bash scripts as the primary way of developing charms. We believe that this should be fully sufficient for most use-cases, but that is up to interpretation.
We are not sure exactly what Python support in Lucky will look like on the first release, but at the very least you will have to interact with Lucky by calling the Lucky CLI from your Python script instead of having the luxury of a Python library with Python functions to call. This will most-likely change in the future, but it won’t be the first thing out of the gate.
Related to using Bash instead of Python, you will have to interact with the Juju relations without any helper Python libraries that come with the Juju interface layers. The Lucky CLI will have commands to make interacting with the relations as easy as possible, but it will not initially have support for interface-specific helpers. You will have to read and write the key-value pairs on the relation yourself. My initial observation leads me to believe that interacting with the relations like this will not be a burden, but that is up to interpretation.
We are going to work to make this as easy as possible, but we are going to have to do some experimentation to find out what the best workflow will look like.
The final concern with Lucky is that it isn’t finished yet. We are expecting to be able to get a first release out within the next two months, but hopefully much sooner. Even after a first release, the Lucy CLI will probably be changing and you may have to keep up with that for the first while, but if you try it out and have use for it we will do what we can to make it work for your use-case.
Assuming that the CentOS support problems come only from the charming framework and not Juju itself ( which is what it looked like from your stated problems, if I understand correctly ), I think that we will reasonably be able to support CentOS with Lucky.
Does this sound like something that would help you? I’m open to questions or suggestions. Is there anything that you think would be a blocker for you, such as needing Python or not wanting to use Docker?
I think your effort highlight the need. Amzing work! However, the approach will bring more issues.
I think it’s critical to think “Linux” instead of “distros” for a framework such as juju since otherwise it will be just another Ubuntu tool. It’s great, but won’t be able to reach a broader community of developers from domains where Ubuntu not already has a strong precence.
I for one hope to be able to use juju for deploying software on embedded devices. This might be an impossible task today, but it also serves as a good benchmark for the architectural challenge of getting juju in shape for broader adoption.
That’s a valid point.
I do think that, if you want to write charms that can run on any distro, you will either have to use containers or something like snaps, flatpaks, etc. . That said, writing charms that run on any distro and having a charming framework that supports any distro are two different things. Maybe you don’t need your charm to run on any distro, but you do need your charming framework to allow you to write a charm for the distro you are interested in.
It is very possible that we might extend support to different container engines such as singularity. We could probably even abstract the CLI over the different engines by providing a single way in which to add volumes, environment variables, etc. . If there is a use-case for a different container engine then it would be worth looking into at least.
In fact, if Singularity is lighter-weight than Docker and still supports the Docker image format, then it could be a great option to use with Lucky. I’ll look into it and discuss it with my team.
If you don’t mind, I would like to understand a bit more about what would be alienated. What kind of software are you using that would be pushed out?
I come from a background where I’ve written many Docker images and I use bash as the primary method of automation for all of them. My assumption on how charming would be like with Lucky is something like this:
The logic for the charm, I imagine, should be fundamentally simple for most applications. The charm is doing the operations that an operator would otherwise do manually, often with a CLI. My thought is that you would not need Python to accomplish these tasks.
This is just my understanding in the realm of my own experience so I would like to understand more of your use-case.
We’ve definitely found through experience that relying on individual charm authors to get the details of the key-value mapping of relations right in every charm leads, somewhat paradoxically, to both fragmentation as well as difficulty in evolving the interface. Part of this is not having a good pattern to use for versioning an interface API, but even just having the KV pattern abstracted into a shared code-base that abstracts it into a defined API makes it so that only a single author has to think and code for compatibility changes in the interface protocol, as opposed to every single charm author having to keep all of that in mind for all interfaces that they want to use.
If you haven’t seen it already, it would be good to get your input on https://github.com/canonical/operator/ which is another replacement framework being worked on which has some of the same goals, though it is targeting (pure) Python as the language of choice. Certainly having your team’s perspective on the challenges you’ve faced and how you overcame them would be helpful on that, and vice versa for your team seeing how that framework addresses issues you may run into.
In effort to help that, it would be simple to have shell scripts that serve essentially the same exact purpose of the Python interface layers. You would just drop the shell script into a charm utils dir or use a built-in “package manager” tool to do it for you and it would put the tool in your charm script’s PATH so that you could call it to interact over the relation.
The issue with the system of using layers or utility libraries to communicate over relations is that they are specific to a framework while properly documenting the key-value requirements of the relation like we have started doing with the #docs:interfaces category allows any developer to properly design their charm for the interface regardless of the framework they are using.
When I tried to write a bash reactive charm that simply consumed a PostgreSQL relation, the pgsql layer documentation was extremely minimal and I found that the only indication of how to actually interact with the relation in bash was in the README of the PostgreSQL charm. I think that the first step to defining a relation should be to define it in terms of the key-value behavior, even if you proceed afterwards to create a layer or other library for it.
Unfortunately the method of providing that library to the community for use has to be framework specific unless we come up with a universal charm interface definition of some sort that can be supported by any charming framework. ( That might be impossible, but probably worth brainstorming. )
Versioning relations is an interesting topic, too. Honestly it seems like the only safe way to do this is to put the version name in the interface name. For example, instead of pgsql you would have pgsql.1 or pgsql.2. That would be the only way to ensure that you couldn’t ever connect a charm that only supports an old version of the protocol to a charm that provides the new version of the protocol.
Every time a backward incompatible change was made to the interfaces behavior you would have to increment the version number. I think that making a practice of this might actually be required if we want to advance the defined interfaces while ensuring an ecosystem where charms can still safely relate to each-other.
Yes, I’d heard of it, but I didn’t know there was a public repo for it or what the name was. 
Absolutely. I’ll definitely try that out. We’ve almost got Lucky’s first release out so it will be very interesting to compare the frameworks and see what they might be able to learn from each-other.
I’ve ben involved with the charming community for a relatively short amount of time, so would love to hear more background related to this.
My perception is that a “shared code-base” typically becomes dominated by a single developer/team. And they’re more than happy to make changes without seeking input from other parties. From what I’ve seen, layer developers don’t see interfaces as shared or common, but a private implementation detail.
IMO introducing breaking changes to an interface would be better handled by creating another interface. But that opinion isn’t very well formed.