Hm… I have read documentation and I wonder how that would work.
VSphere cloud is linked to a VCenter host and datacenter object, not specific ESXi host. So if I add additional cloud definition with same VCenter hosts this do not change situation if I use same Juju controller. If I add new controller then I still would be limited to one application per controller, as per CMR scenario 2. This would not be intended way to deploy an app as I would loose resiliency in case of host failure.
Ok - it sounds like I was wrong about this, sorry. The vsphere instance I have access to for testing only has one host so I wasn’t able to try it.
What are the hosts named in the vsphere UI? When finding the resource group to create a VM in we match the name of the host in the inventory, not the IP address or the DNS name (unless that happens to also be the inventory name). So I wouldn’t expect that the commands specifying zones=x.y.z.24 would do what you want.
In general, you should be able to create machines on both hosts with only one controller, no cross-model relations needed.
Hosts in vSphere named via their IP addresses. Now they are part of the cluster Cluster01, but previously there was no cluster combined and specifying zones=‘IP_ADDRESS’ did not work. Cant it be that “.” in IP address are interpreted other than literal symbol, when matching name?
Will try to reconfigure that, when I find login to DNS VM running in same VCenter.
Hi @babbageclunk was the fix applied to version 2.7-beta1? As today I have tried to redeploy environment and first I have failed due to wrong “VM Network” being selected instead of distributed switch one. So I have rebuilt everything, including the controller. Version is still same 2.7-beta1, but now “juju status” shows the errors as boostraping controller if root-disk-source is not specified.
As soon as I have specified root-disk-source for all VM’s and shuffling VM’s between two nodes, Juju have created them on expected hosts. So it looks like the issue was fixed in 2.7-beta1.
The strange thing is that juju now selects wrong network. I have found some bug dated to 2017, where it was said something hardcoded “VM network” and this issue supposed to be fixed.
Hi @elvinas - yes, the root-disk-source fix was merged into the 2.7 branch on Friday so if you were using a new build for testing then it would have been there.
Have you set the primary-network (and possibly external-network) values in the model configuration? If not Juju might pick the wrong one.
OK. Thanks for the confirmation. Although it would be nice if juju would report build number or something so it can be identified that something have changed.
Regarding primary network. Yes, I have specified. And I see “DSwitch-VM Network” in bootstrap config file:
However VM’s end up in “VM Network”. Last week it seem to have worked and application was installing. Now deployment continues as expected, when I manually migrate VM’s from “VM Network” to “DSwitch-VM Network”.
Behavior is the same. Controller is booted on correct network, however VM’s ends up in the “VM Network”, instead of “DSwitch-VM Network”.
In addition to that, deployment with 2.7-beta1+develop-4819cdc stalls with all hosts showing “agent initializing” and logs full of for each cluster node:
_unit-etcd-1: 14:24:09 DEBUG juju.worker.dependency "uniter" manifold worker stopped: subnet "x.y.z.0/24" not found_
_unit-etcd-1: 14:24:09 ERROR juju.worker.dependency "uniter" manifold worker returned unexpected error: subnet "x.y.z.0/24" not found_
Hi @elvinas, sorry I missed your response! I think the problem might be that specifying --config applies to the controller model but not to any new models you create after that, so those are falling back to the default network. Could you try bootstrapping but setting the network and datastore with --model-default rather than --config?
The logging about missing subnets from the 2.7 edge version might be due to ongoing work that a couple of other team members are doing in that area - I’ll make sure they know about the problem. In the meantime it’s probably better to use the 2.6/edge snap.
Thanks, adding --model-default and specifying intended network for VM’s did help! I have added just primary network to avoid specifying VM’s on single host. So for the moment case is closed and I can continue development.
I have 100% the same issue you had. I think that in Juju they should specify the datastore limitation - is it REALLY is a limitation.
I have vCenter, Enteprise Plus, all bells and whistles, only thing is: each ESX host has multiple “local” datastores - no shared datastores across ESX hosts. Certainly it is not a common production setup, but as the cloud is connecting directly to vCenter, it would be nice for Juju to make use of all hosts, and all available resources in the vSphere Datacenter pool.
@nicowalker I do not have a connection myself to another environment (as I am a contractor there) but according to team members, in that environment Juju is able to properly allocate VM’s on different ESXi hosts. Initial comparison shows difference in the hosts and datastore naming. We will be reinstalling ESXi environment and will try to rename hosts.
Naming schema in working environment is as follows:
ESXi hosts in vCenter are named as short hostname: node1, node2
Datastores are named as “hostname”-“datastore name”: node1-Local-N, node2-Local-N.
I have the same config as you, 4 ESXi with no shared datastores, only local, and the problem I have is the following:
I bootstrap the controller to DataStore04: perfect
Then I try to deploy a Kubernetes charm, and set the datastore model config to DataStore04, but when juju tries to deploy the machines, on some of them, try to deploy it to another host, or cluster, given the following error: could not find datastore “DataStore04”, datastore(s) accesible: “DataStore01”, “DataStore01-SSD”; juju retries the creation of the machine, but after 10 attempts it fails and the machine stays in down state.
I have modified the charm to force the root-disk-source in constraints for each machine, with no results, and I have juju 2.70 version.
When deploying a charm it’s not allowed the --to zone=XXXX modifier, it would be great to deploy a charm to the cluster or host inside vCenter I choose, but it’s not possible.
Nothing worked. Every time a get a
ERROR cannot deploy bundle: cannot create machine for holding etcd unit: cannot add a new machine: availability zone “CLUSTERNAME/HOSTFQDN” not found
Edit: I tried it with “VM/Host Groups” inside the cluster, but it still fails.
Hi @panda - are you trying to deploy to a specific host inside a cluster? I don’t think the Juju zones constraint provides a way to specify a specific host inside a cluster - anything after the first slash is interpreted as a resource pool, not a host.
If you’re trying to deploy to a host or cluster, the name used should be whatever is displayed in the vSphere web client for the host/cluster, not the FQDN (unless they happen to be the same).
I don’t think we have a way to specify that in Juju at the moment, sorry!
Could you give me some more details about your situation? Why do you need to deploy to a specific host in the cluster?
One of the reasons is that juju deploys not equally on all available hosts inside a cluster. E. g. I have a 5 node cluster and deploy 10 instances of kubernetes-worker I would assume that every host will get two workers. But I have hosts with 3 workers and hosts with 0 worker and it is quite random which host gets how many.
Another reason is to get more control of where an instance is deployed. From the example above, one of the 5 hosts has less RAM and only one cpu socket instead of 2. Therefore it would be useful if there are less machines on than on the other four. Specifying hosts during deployment would help a lot. Same for if inside the cluster are more hosts than juju should use.
