What’s the design rationale for the hardcoded --via portion of this relation? In general, the zen of Juju is to avoid hardcoding things like IP addresses in a model like the plague, because all of these should be automated and intent-based.
I’m sure there is a reason, but I’m also sure we should invest in a better way that does not require these sorts of things to be hardcoded in a bundle or model by administrators.
I think the gap in modelling here is that the consumer of this cross model relation wants to let traffic through only from a specific blessed network, but we don’t have a way to convey that information across models or controllers without using a cidr.
We should probably solve this with spaces, but what do cross model spaces look like? Do I do a query on a controller, or a collection of JAAS managed controllers, to determine what space “foo” means in model “bar”? Do we briefly open up the firewall long enough to perform a handshake where we exchange information about the cidr behind space “foo”?
This is going to matter increasingly much as we do more with Kubernetes ingresses and Virtual Private Clouds. We want to give Enterprise Operators a way to lock down their deployments very tightly, without giving them inflexible bundles that require a lot of editing to use and share.
Right, my concern is the need to edit a bundle to reuse it. I don’t think CIDRs are a good idea at all! We should rethink this, and definitely avoid this pattern in future. Anything that’s tied to a particular IP address is poured in concrete, which won’t flow as things evolve.
This has exactly the same problem. A bundle won’t work without editing to hand-craft something that I am pretty sure Juju could figure out by itself.
Our guiding principle has to be that we suck the complexity of this into juju so that users don’t need to worry about it. @pengale could you conduct an analysis of both of these examples and make a proposal as to how we fix and in future avoid these anti-patterns?
This is a case where we haven’t moved to modeling links between spaces yet. The VIA in this case is the NAT subnet where traffic will appear to the consumer. In some circumstances a cross model relation doesn’t need a shadow address and in other cases it does, but because we don’t know the routes between subnets we can’t chose for you.
In this specific example, it is unclear to me why they would need via, since I would expect that postgresql would not be proxied for it to be seen by mattermost.
This is also a rather strange one. xip.io is explicitly a site that runs a DNS server that just always returns the IP address that you specify. (it solely exists so that anything.10.0.0.1.xip.io will return 10.0.0.1 for a gethostbyname request.)
The point of ‘juju-external-hostname’ is to tell Juju “what external DNS entry have you recorded for where your service will be available”. I believe this exists because K8s Ingress resources require a DNS name for the ingress.
I think that in this case, the tutorial is leveraging xip.io as a convenient provider for a no-waiting, no-registration domain name. Presumably, in a production environment, you’d need to specify a real domain, and pass that through to juju-external-hostname.
To abstract that away, we might do something like indicate what space we meant to point the DNS at, and then do a reverse lookup of our IP on that space to derive the domain name. That seems a bit fragile, though.
There are points where a production ready deployment does need to specify information unique to a deployment. Right now, that information is mixed in with config that could be generalized. I’ve opened up a doc internally to keep track of our thoughts on it, and of the results of the investigation that @sabdfl requested above.
The immediate question is whether the Juju server can get this information from Kubernetes directly. Juju is standing up the service on Kubernetes in the first place. I’m pretty sure, since Juju is on both sides of the equation, that we can engineer Juju to do the work so that users don’t have to, and that should absolutely be the standard we hold ourselves to.
juju-external-hostname is based on the HTTP Host header. The Ingress controller is terminating HTTP requests and needs to know based on the Host who to route it to.
In the case of our internal Mattermost, this would be “https://chat.canonical.com” rather than something that has an IP address in it. The only reason there is an IP there is because it is a service that XIP.io provides for people that want to be testing out their deployments.
I don’t know that we can discover “what is your pubic facing HTTP DNS name” from Kubernetes, which is why it needs to be some form of config being passed into the system. We can certainly look to write nicer documentation in the charm, but that is one of those places that is site specific, and will need to be customized for each user.
We could look to model it at a different level, and have it be something like “what is the root domain that you want to use for services”, and then have Juju default to exposing applications as “.<deploy-domain.com>”, which would be a reasonable default.
What makes you think that we can’t ask Kubernetes what the hostname is that it is mapping to the service? At the end of the day, that information must be in Kubernetes config somehow, and I would bet the other way, that it can be set and retrieved programmatically. That means we should design for Juju to know it, and Juju to share it automatically where that is needed, and for Juju to update it automatically too if it changes, so that a human never needs to worry about that.
Ah, I think I see now - the user needs to tell the outside world a DNS name, and they need to point that DNS name to the K8s ingress controller, and when the traffic arrives, ingres needs to know that the traffic is supposed to go to this particular service.
Still, I think we could do better.
First, I think it’s tasteless that we use juju- as a prefix on things for no good reason. It results in output that looks bad. When you look at your instance names in a cloud and its a mess of juju, it doesn’t create a positive impression. Why did this need to be juju-external-name?
Second, as you started to explore, I think we can make the DNS process more deliberate and designed, rather than requiring human interaction. We have app-name and model-name as keys that can be used in a DNS string, for example. If DNS is a key part of networking (it is) then Juju should have a designed and deliberate approach to it, which is zero-touch y default but can easily be tailored. For example, Juju could default to xip.io names unless you provide a custom one (meaning, you don’t have to provide a name for it to work). Juju could provide DNS itself, so this becomes app.model.controller-uid.local (but with some plan for how that’s useful).
Just falling back on ‘the human will do it manually every time’ is the anti-pattern I am asking us to avoid.
I like the idea of using xip.io. To future proof things, we could drop it in as a default into the controller or model config, so operators can switch things around if the folks at Basecamp ever decide to turn xip.io off (or if we decide to operate a foo.juju.is wildcard DNS). The config might looks something like:
domain: xip.io
domain_is_wildcard: True
That would potentially solve a lot of issues, including giving us a query-able way to share uris in cross model relations and the like.
I think we should treat DNS and LBAAS as a part of how we model networks. Then we can drive them through the model just like we drive other things.
There is a slight gotcha with things that are genuinely elective (I can map any name to a thing, and how would Juju know I had done that?) but those should also become things you can tell the model, which will then be shared across relations.
So for example:
Juju by default could use xip.io with no config needed.
Juju could also provide DNS using app.model.x and app-unit.model.x as names.
Admin could tell Juju to put its DNS under a parent DNS, so I could then get app.model.svcs.canonical.com. This could be done at controller or model level.
Expose could get additional names that override these, so something like “expose this service on that network with this name”.
It will take a fair bit of work and digging to think through all the issues, but I bet we can get this on rails.
