A "Terraform" Cloud Backend

tl;dr: What if we provided a Terraform “cloud” for Juju that allowed you to provide custom, templatized Terraform config files that Juju would use to spin up Juju infrastructure.

I’m going to go through the thought process that led to this idea and how it might be implemented.


Me and my partner were just discussing how one of the limitations of Juju is the fact that it can’t deploy highly specialized infrastructure because it focuses on simple abstractions over infrastructure so that it can be deployed similarly to multiple clouds without modifications.

Just to clarity what I mean, when you specify the servers you need as part of a Juju bundle, for example, you say that you need at least this much CPU, at least this much memory, and you need it to be a certain Linux distribution. By specifying only these relatively universal parameters, Juju can then spin up specific infrastructure that matches those requirements on almost any cloud.

This works really well to allow you to target multiple clouds, but want it doesn’t allow you to do is spin up specialized infrastructure that is specific to a certain cloud provider.

For example, in a certain enterprise infrastructure use-case, there was the requirement to spin up servers with specialized network configuration including a VPN tunnel. This needed to provision cloud-specific resources in a targeted configuration. Juju would not be suitable for this use-case because of its high-level abstractions. Instead Terraform was used because, instead of abstracting over clouds for portability, Terraform goes the route of allowing you to define the specific infrastructure you want provisioned in each cloud.

Both of the paradigms make sense and you can’t have specific control and high-level abstractions at the same time, so both tools have their place.


Still, me and my partner were thinking about how we might be able to marry the two ideas. What if I wanted to be able to use Juju to deploy my applications on infrastructure that I had customized for a specific cloud/use-case.

The first thought is that you can use Juju’s manual cloud to add infrastructure that was provisioned with Terraform to the Juju controller. You could probably even get terraform to automatically add the provisioned infrastructure to the Juju controller for you. The obvious disadvantage of the manual cloud is that Juju cannot automatically provision servers for you when you do something such as a juju add-unit mysql or when you deploy Juju bundles.

Then I had an idea. What if you used a templating language such as the Go templating language, Mustaches, Liquid, etc. to write a Terraform file ( which I’m not going to call a Terraform template for the sake of confusing terminology ) where the inputs to the template were the requirements that Juju has for the machines, such as memory, CPU, and distro. So the workflow would be like this:

mermaid-diagram-20200125153504

The user would create a template that uses the infrastructure requirements from Juju models and bundles and renders to a valid Terraform file that will provision the infrastructure.

When somebody does something like juju add-unit, Juju would re-render the template with the updated list of servers required and then re-run terraform apply to update the infrastructure.

This model allows the user to to create specific infrastructure tailored to the deployment that they need while still allowing them to get the first class user-experience of having Juju trigger the provisioning of the machines.


I think that this could be relatively feasible and open Juju up to a lot of situations where it might not otherwise be used.

This is a really interesting idea. This would extend the manual cloud with “provisioners”. I wonder if it could also help with the problem of the difficulty people have when incrementally adopting Juju. At the moment, new Juju users need to use it 100%. This would allow them to re-use a lot of their pre-existing Terraform work.

1 Like

I think this could really help people who want to use different cloud constructs like subnets. It sounds like it would help this situation for example:

I’m not sure I fully understand the situation, but if you had a Terraform template backend, you could use it to create an subnet an in AWS VPC for every Juju space or potentially use Terraform’s import feature to import the existing infrastructure.


Another use-case I thought of was if you had applications you were running and you wanted to use a scalable load-balancer that will automatically update your DNS records as it scales, AWS ELB and Route56 DNS would be a natural fit, but I don’t know that it is possible to have Juju spin up machines connected to ELB.

Or what if you wanted to spin up Amazon Lightsail servers instead of EC2 instanaces?


Still, something I realized is that maybe you wouldn’t want to make it specific to Terraform. While Terraform sticks out as the de-facto way to do this kind of provisioning right now, what if some other tool came up later, or what if you wanted to use Chef? Maybe what we need is a way to create Juju cloud plugins that could use any provisioner that they wanted.

It seems like it could be a good idea to design these plugins like Docker volume plugins. The plugin listens on a Unix domain socket ( or a Windows named pipe? ) and serves an HTTP API that Juju uses to instruct the plugin on how to deploy infrastructure.

This would blow open the doors for all kinds of integrations and I think it would remove what may be Juju’s single largest limitation.

It would be a step in the direction of allowing integrations with other services like Spotinst which would allow you, for fault tolerant workloads, to get server hosting at massive scale at incredible discounts.

I definitely see the appeal in doing this. It would enable users to re-use tools that they were using before Juju and ease incremental adoption of Juju. Additionally, there may be specific operational constraints that Juju hasn’t implemented yet.

However, the Juju community has been down this road before to some degree and it caused lots of confusion. Several years ago now, Juju promoted the idea of being able to write charms in “any language” or tool. This created several charms in the short term, but observers became very confused. It appeared that users needed to learn Juju + Chef, rather than just Juju, to get their work done.

Getting the balance right is hard. At the moment, telling people to lay out their environment with their own tools then use the manual provider is probably okay. Not ideal.

Lastly, if there are weaknesses in our unit provisioning system, let’s improve Juju core. Subnets are modelled as spaces, for example.

I think that we still want to present Juju as a “ready out-of-the-box” setup where you don’t have to have any specific infrastructure, just give Jass/Juju cloud credentials and get going by selecting an app from the store. This shouldn’t change, but when customers end up with the need for specialized configuration we are going to want to be able to account for that.

We don’t need to push Terraform/Chef or any other technology as the primary way to deploy Juju, and we would still put a marketing emphasis on the easiest way to use Juju, but I think that providing the ability to extend Juju with customized tools would give power-users the ability to get the most out of Juju as a tool, while still providing that beautiful high-level user-experience for the operators.

Absolutely, we just still need to keep Juju’s high-level components abstracted from the cloud-specific infrastructure so that those same constructs can be represented on any cloud.

I am trying to do something similar with cloud formation. @rick_h made a good suggestion about a proxy/wrapper charm; a charm that exists for the purpose of bringing more of the cloud tech into the reachable and model-able world of juju. Example: A cloudformation charm could be responsible for assembling and applying the cloudformation, and returning any defined cloudformation outputs to the charm. In this way juju could facilitate the full run of AWS services through a cloudfront wrapper charm. I think this would be similar for terraform… seems pretty close to what you are getting at above.

1 Like

FYI:

https://discourse.maas.io/t/maas-terraform-provider/1255

That sounds similar to what I’m thinking, but I don’t think that would allow you to deploy Juju units on the cloud formation provisioned infrastructure, like you do for other clouds using juju add-unit.

I admit that maybe this isn’t as much of a need as I initially thought. I’m still getting into Juju and I’ll need to do some more diverse testing and experimentation to get a good idea of what it can/can’t do today and what you an accomplish with new charms, too.

Coincidence. I posted that. I tried that because I like Terraform and I like MAAS and don’t want to use AWS.

OT, but I am tying to start a movement away from these two terms to something that I think is more suitable: envoy charms. Proxy feels too active to me. The role of an envoy is to connect the two sides together, but do very little actual work.

1 Like