Hi Friends,
Can you please give me a idea if I can use Juju for regular websites like these?
Howdy, I think that this is a case where Juju isn’t normally the best fit. I’d suggest that Juju is best for the person looking to run the software stack to host many websites as a service. You might have charms and associated applications for things like wordpress, mediawiki, or other content platforms along with the back end database, caching, and https endpoint services needed. It’s a bit much for a single site, but would make a lot of sense for a website hosting service.
It might be overkill to develop custom charms for a single website.
I’ve done it for our client’s websites that all have the exact same structure and it’s worked very well, just took a lot of time to get it all together and working perfectly.
You might look around for existing charms in the charms store and you could adapt it to your needs. I know that there is one for Wordpress.
Just my 2 cents here, @terrellstanton you could encapsulate these websites as Juju k8s charms! If you run k8s in the cloud, you could write a fairly basic charm that pulls down your docker image for the website onto k8s and sets up an elb and a cert in front of each of your respective website container. You can use service annotations in the pod spec set by your k8s charm to facilitate creating the loadbalancer and associating the cert. See an unofficial short list of aws-elb related service annotations here. By doing this, you could write a single charm that would be reused to deploy all of your websites via Juju.
Pretty sure I am closing in on something similar: https://discourse.jujucharms.com/t/k8s-charms-annotations-charm-config-use-case/1489
1 Like
@terrellstanton thanks for your question. It’s a very interesting one.
Today, @rick_h is right: Juju does not justify itself for this scenario. As time goes on though, I would like to be able to see web sites as something that we support well.
Two reasons why:
-
Stacks grow over time
There are very few web applications that speak to a single database forever. As things grow, you need a full text search engine (Elastic Search/Lucece + Solr/…), logging and monitoring (Nagios, …) and work queues (Celery). You might also want to add functionality via microservices. And that’s just the start - perhaps you need to also manage LDAP, a mail server, 2FA service and other things. Juju definitely makes sense in cases like these. -
To learn something new, we build from what we know
Juju is a complicated system. To start with, it includes over 100 subcommands. (Don’t worry about this - 6 commands do most of the work). Starting by deploying something that you already understand, such as a website, makes it easier to learn Juju.
Why does Juju not justify itself for simple cases? Because users will usually need to write at least one charm to describe how to manage their workload. Charm writing is difficult (well, in easy cases it’s easy - but it’s still difficult to learn). The patten described by @jamesbeedy is a good one that works well with developer workflows. I need to write that pattern up - because it’s a great one.
@maaudet I am very interested to learn more. Well done on your perseverance! If I may ask, how many customer sites do you manage and what size is your team?
I’m alone in my team that works on Juju Charms. I’ve built about 15 custom charms in order to make everything work. Our business is mainly Magento websites but I’ve adapted the code to support Wordpress and others.
Right now we have 2 clients in production using Juju, but we have spun easily over 2000 sandboxes. I’ve automated everything so much that we only have to deploy a bundle and we get a ready-to-use environment from scratch in under 20 minutes (and for any versions of Magento, as long as the commands work/don’t change).
The reason why I used custom charms all the way is because there was slight missings in some parts of the available public charms at the time.
Right now I have made charms for:
- MariaDB
- Automated Galera Cluster with MariaDB
- Redis
- Redis with Redis Sentinel (only works through Haproxy)
- Haproxy with support for HTTP & HTTPS, MySQL and Redis Sentinel (auto master failover), with configurable failover modes (2 as of now, 1 to connect to the nearest backend and use the other as backups, the other is simply round-robin or least-connections depending on which integration, Redis doesn’t offer a choice because it’s a single-master setup)
- Magento 1 (with NGINX as a base layer)
- Magento 2 (with NGINX as a base layer)
- Wordpress (with NGINX as a base layer)
- GlusterFS Cluster, it’s managed using custom run commands
- Subordinate: PHP 5.6, 7.0, 7.1, 7.2, 7.3 (I support the use of multiple PHP versions for each web charms, so we can have a Magento on both PHP 7.0 and 7.2 [for testings] and we can decide which paths runs on which PHP versions)
- Subordinate: “Mount point” for GlusterFS (For web charms), the Web charms all create their structures automatically based on the directory communicated from this charm, this charm setups the mounts that are required and send the mounted directory to it’s parent, then the parent is able to create it’s structure (if required)
All of that is generated through ~25 layers. I have 3 layer structures:
- Base Layers, they all include 1 “master” base layer and provide the basis for MariaDB, Redis, NGINX, PHP-Based Web Platforms, Node.js-Based Web Platforms
- Integration Layers, they provide tools to integrate other charms or to add new features, ex: I have a Sendmail integration layer that automatically installs sendmail and configure it for Amazon SES if the configuration enables it. If it’s disabled, then we uninstall Sendmail (if it was installed). I have one for crons management as well that makes use of the /etc/cron.d directory and takes a file provided using attach-resource and installs it there and the parent charm can disable and enable crons on automation. etc.
- Charm Layers, they put everything together to make the charms.
I use a “only change what you have to change” model by verifying the current state at all times before doing an action and correcting the situation. So I usually end up just deleting whole configuration directories and rewriting them using templates and files.
I also automated deployments using attach-resource as the entry point for an artifact. If the charm detects a change in the “code” resource, then we run a synchronized deployment. Those deployments are synchronized by using cluster states (interface communications) and can only switch to the next state if every cluster members are at their state or higher. This way they can all go into maintenance mode, apply database modifications and remove maintenance mode at the same time (database modifications are only run on the leader unit, the other units only wait on the leader, I also have error detection where if the leader breaks and changes to a new node during deployment, then the new leader will execute the code, but not the previous leader).
Also, every configuration files are managed using the application configurations or information from interfaces and make use of Jinja2 templates. There is also some configurations sent as YAML files (using attach-resource), mainly for array-type configurations.
I’ve written all that in a bit less than a year (part time, I technically work in cybersecurity 
), what took most of my time was reducing complexity. But now I can create a fully-featured charm in less than half a day if I use the “framework” I’ve built (over the charms.reactive one). The only exceptions being clustered charms where it might take me about a week depending on the complexity.
I’m planning on adding new features next year and start doing more production environments because I see Juju as an opportunity to remove developers root accesses to the servers while providing run commands so that they can still run the commands they need to do (I won’t sleep well at night until the day no devs will have root access to production environments 
). And I also created a synchronisation script between Okta and Juju that takes the public keys from Okta and installs them on the right models, it also creates users with a register command and send it by email to our users so that they can get started. It also automates access to the right models for each users depending on their groups. (the inverse is also true, users and keys are removed if no longer required)
I’m also planning on open-sourcing some parts of it and sending patches to add more features to currently existing interfaces next year and try to integrate with the ecosystem a bit more. It will depend a lot on my planning, but I’m getting a DEVOPS engineer in my team, we’ll see how that goes 
.
4 Likes
Wow, you’ve done well! This is an excellent post, and I’m really glad that you’ve taken the time to write it.
Fantastic to see that charming is helping e-commerce.
This is a really interesting quote. So in a sense, your charms are allowing your team to iterate very fast and experiment quickly. What do they think of the support that you can provide? Your colleagues must think that you have superpowers.
This makes sense. Developing (and learning) by yourself can be much simpler than working with an open source project. And for charms, every charm is an open source project. Those improvements seem very impressive though. If you’re able to open source some of your work, many would benefit. Let me know if you would like any assistance in this area.
What an innovative way to use subordinates. Did you see this done anywhere else or did you think of it yourself?
Fantastic to see. You should read through the upcoming changes (and perhaps help test!):
Great to see. Speaking from a security perspective, are you happy with Juju? Can you see anything that needs hardening from its defaults?
Most of the times yes, other times I get “it should be faster” 
The Sales team are very happy with it though, because I’m able to setup a demonstration sandbox per prospects now rather than have 2 or 3 that they share.
I would absolutely love to open-source at least key aspects of my work, I will have a lot more availabilities at the start of next year because right now I’m working on implementing new security standards internally, so I have tons of sensitivity materials to send out and policies to rewrite.
Once I’m ready to put more time, or once I actually get my DEVOPS engineer I will poke you and we’ll see what can be done
I though of it myself, most other charms that I saw integrated PHP directly in the Charm, but I was required to work in an environment that uses many different versions of PHP. I though about the subordinates because it would give me a lot more flexibility in terms of maintenance and would allow me to completely separate the automation code and provide a better service-level isolation. The fact that we can install multiple versions at the same time is a consequence of this isolation and was actually a requirement for more legacy systems, so it ended up being extremely useful.
For the GlusterFS part, it’s in order to manage the charm-local part of the mount and allows me to write more connectors in the future. Even if it could be integrated within the charms, I prefer to support services separately.
Absolutely, I love every concepts around Juju and it’s so simple to implement once you get the basic concepts. I’ll be testing every new features .
The thing I would personally love to see would be SSO integration directly on the controller. I imitate that through my synchronisation script, but I think that it’s something most companies would look into.
I haven’t looked at what each rights level give in details yet though, but I’m sure that more granularity in rights would be a good thing to see, even maybe at the action-level? Like being able to define a minimum rights to execute an action. This could be configurable at a model-level.
Also, maybe being able to automatically configure (through Juju) stricter Security Groups (at least for AWS), by default the SSH port and the Juju port are open to the public, but I like to limit their access only to a select few IPs to reduce risks through exposure.
We’re investigating adding support for RBAC and Candid (https://github.com/CanonicalLtd/candid) in the future. You can actually use Candid but it needs some work to get it held just right.
Interesting, I’ll look into that!
Thank You so much for the effort you have put in here!