Some notes for working with the VMware vSphere provider code

juju
1

Working through lp:1800940 has been a bit of a grind. One of the hardest things was for me to create a clearer mental model of what’s happening. Here are some notes:

Sea of red herrings

Small, invisible errors make debugging very hard. Plus the debugging cycle with vSphere is slow. Hopefully you’ll be able to avoid some of errors like setting newly provisioned disks’ capacity to 0 bytes.

Mapping vSphere concepts to Juju’s

They’re not in perfect alignment, but this provides a good

vSphere term Juju term Remarks
datacenter cloud
resource pool model Allows resources within a datacenter to be shared with other resource pools. Contains VApps.
VApp application VApps contain 1 or more VMs. Changes to the VApp affect all of its VMs.
VM unit

Tips for working with the vSphere API

The vSphere SDK is consistent across client languages. That makes for consistent documentation. It also means that examples for C# or Perl tend to be illustrative for Go code also.

The SDK is comprehensive. It’s hard to know what the right way to do things is. A RelocationSpec is not a CloneSpec, nor a CreationSpec. But several spec types can take on a CustomizationSpec. And every type embeds DynamicData.

Go isn’t very fond of deep inheritance hierarchies. Normally you’ll receive a base class from the API within a result struct and you’ll need to cast it to a specific type for things to work happily.

Even when you are doing the right thing, it’s still sometimes hard to remember that there’s often a layer or two to get to what you want. ovfManager.CreateImportSpec() doesn’t return an ImportSpec. It returns an OvfCreateImportSpecResult that may contain a BaseImportSpec, which may provide an ImportSpec via its GetImportSpec method. But watch out, a BaseImportSpec is not a VirtualMachineImportSpec.

Web API technology

vSphere uses SOAP for its web API, and data sent over the wire is mapped to github.com/vmware/govmomi/vim25/types. It’s a large file. GitHub refuses to render it.

Similar to Juju, govmomi calls across the wire use params structs (called specs) and return result structs. Juju client code is higher level though. Our internal wrapper doesn’t wrap very much and you’ll probably still need to understand what’s happening.

Take time to learn the terminology and workflow

To understand the API, you need to have an understanding of leases and managed object references. A managed object reference—normally abbreviated as mo in VMware’s code—is like C++'s std::unique_ptr or mutable borrow in Rust. Leases enable you to create managed object references. To mutate an object, you need to call lease.StartUpdater().

Here’s an example, with error handling elided:

ovfManager := ovf.NewManager(vsphereClient)
spec, _err := ovfManager.CreateImportSpec(ctx, ...)
lease, _err := resourcePool.ImportVApp(ctx, spec.ImportSpec, vmFolder, nil)
info, _err := lease.Wait(ctx, spec.FileItem)
updater := lease.StartUpdater(ctx, info)
defer updater.Done()

Here’s the pattern:

  • Some API method will return a lease (resourcePool.ImportVApp)
  • lease.Wait() returns managed object reference to what we’re waiting for (spec.FileItem is something we haven’t yet uploaded, it’s a reference to the VMDK file that we’re)
  • lease.SetUpdater() enables mutation on the object that you’re referring to

Web API technology

Be wary of Go’s XML marshaling combined with type zero-ing by default. Several structs in govmomi/.../types.go leave off the “omitempty” flag. This means that if a struct is not being properly initialised in client code, it will be set to 0 underneath you.

Here’s an example type definition:

type HttpNfcLeaseInfo struct {
	DynamicData

	Lease                 ManagedObjectReference           `xml:"lease"`
	Entity                ManagedObjectReference           `xml:"entity"`
	DeviceUrl             []HttpNfcLeaseDeviceUrl          `xml:"deviceUrl,omitempty"`
	TotalDiskCapacityInKB int64                            `xml:"totalDiskCapacityInKB"`
	LeaseTimeout          int32                            `xml:"leaseTimeout"`
	HostMap               []HttpNfcLeaseDatastoreLeaseInfo `xml:"hostMap,omitempty"`
}

Other points

Signing up to VMware {code} and asking questions in the Slack channel was quite a good choice. It made it obvious that our current approach is going against the grain.

1 Like