[Tutorial] How to deploy Centos7 images on VMware's vSphere using juju

We managed to get this working and I will describe it here in case there are other juju users looking for ways to deploy Centos7 images on VMware’s vSphere environment.

The juju version I’m using is:

$ juju --version
2.8.6-focal-amd64

Setup the streaming server, download the OS image, and configure metadata

juju provides a tool ( juju metadata ) to automatically generate metadata more easily by specifying the inputs as arguments. However, to my knowledge this tool does not allow input of a custom image file, only custom “image ids”, which refer to pre-defined images hosted by various public cloud platforms.

Here I will describe exactly how you can deploy your own cloud image to your VMware environment. I will use Centos7 here (but other Linux images should work too):

  1. The first step is to setup a streaming server, which is actually just a web server hosting your image(s). I use Nginx and the http protocol. Apache is of course another option. On the http server, create the directories for hosting metadata and images:
    # Install the http server (we will use the default configuration here)
    sudo apt install nginx
    
    cd /var/www/html
    sudo mkdir -p releases/streams/v1/
    
    # yes, releases/releases is correct:
    sudo mkdir -p releases/releases/centos7/
    
    # change permissions
    sudo chown -R www-data:www-data /var/www/html
  2. Still on the streaming server, create the index file (this is a “main” metadata file pointing to the other metadata files):
    # use any preferred editor
    emacs releases/streams/v1/index.json
  3. Add the following content:
    {
        "index": {
            "com.ubuntu.cloud:released:download": {
                "datatype": "image-downloads", 
                "path": "streams/v1/com.ubuntu.cloud:released:download.json", 
                "updated": "Fri, 23 Oct 2020 03:06:29 +0000", 
                "products": [
                    "com.ubuntu.cloud:server:centos7:amd64"
                ],
                "format": "products:1.0"
            }
        },
        "updated": "Fri, 23 Oct 2020 11:08:44 +0000", 
        "format": "index:1.0"
    }

    Note 1: JSON does not support comments! Everything is interpreted as data.

    Note 2: Don’t end arrays, objects, etc with commas. For example, this is invalid and will give an error: { "foo" : 42, "bar": -1, }

  4. Download the Centos7 image (or any other image):
    cd releases/releases/centos7/
    
    # I use this image, but you should use the image you want to use
    # (qcow2 is a file format for disk images used by QEMU)
    wget https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-2003.qcow2
    
    # convert qcow2 to vmdk (the disk image format used by VMware)
    sudo apt install qemu-utils
    
    qemu-img convert -f qcow2 \
             -O vmdk \
             -o subformat=streamOptimized \
             CentOS-7-x86_64-GenericCloud-2003.qcow2 \
             CentOS-7-x86_64-GenericCloud-2003.vmdk
  5. Create a file called CentOS-7-x86_64-GenericCloud-2003.ovf (still in releases/releases/centos7/ ) and put the following content in it:

    Note: update the size below if you downloaded a different image than above, of course you may also want to change other settings. For example, you might not need a floppy drive in your VM.

    <?xml version="1.0" encoding="UTF-8"?>
    <Envelope xmlns="http://schemas.dmtf.org/ovf/envelope/1" xmlns:cim="http://schemas.dmtf.org/wbem/wscim/1/common" xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1" xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData" xmlns:vmw="
      <References>
        <File ovf:href="CentOS-7-x86_64-GenericCloud-2003.vmdk" ovf:id="file1" ovf:size="369723392"/>
      </References>
      <DiskSection>
        <Info>Virtual disk information</Info>
        <Disk ovf:capacity="10737418240" ovf:capacityAllocationUnits="byte" ovf:diskId="vmdisk1" ovf:fileRef="file1" ovf:format="http://www.vmware.com/interfaces/specifications/vmdk.html#streamOptimized" ovf:populatedSize="0"/>
      </DiskSection>
      <NetworkSection>
        <Info>The list of logical networks</Info>
        <Network ovf:name="VM Network">
          <Description>The VM Network network</Description>
        </Network>
      </NetworkSection>
      <VirtualSystem ovf:id="CentOS-7-x86_64-GenericCloud-2003">
        <Info>A virtual machine</Info>
        <Name>CentOS-7-x86_64-GenericCloud-2003</Name>
        <OperatingSystemSection ovf:id="94" vmw:osType="centos7Guest">
          <Info>The kind of installed guest operating system</Info>
          <Description>Centos7 (64-bit)</Description>
        </OperatingSystemSection>
    
        <ProductSection ovf:required="false">
          <Info>Cloud-Init customization</Info>
          <Product>Centos7</Product>
          <Property ovf:key="instance-id" ovf:type="string" ovf:userConfigurable="true" ovf:value="id-ovf">
              <Label>A Unique Instance ID for this instance</Label>
              <Description>Specifies the instance id.  This is required and used to determine if the machine should take "first boot" actions</Description>
          </Property>
          <Property ovf:key="hostname" ovf:type="string" ovf:userConfigurable="true" ovf:value="centosguest">
              <Description>Specifies the hostname for the appliance</Description>
          </Property>
          <Property ovf:key="seedfrom" ovf:type="string" ovf:userConfigurable="true">
              <Label>Url to seed instance data from</Label>
              <Description>This field is optional, but indicates that the instance should 'seed' user-data and meta-data from the given url.  If set to 'http://tinyurl.com/sm-' is given, meta-data will be pulled from http://tinyurl.com/sm-meta-data and user-data from http://t
          </Property>
          <Property ovf:key="public-keys" ovf:type="string" ovf:userConfigurable="true" ovf:value="">
              <Label>ssh public keys</Label>
              <Description>This field is optional, but indicates that the instance should populate the default user's 'authorized_keys' with this value</Description>
          </Property>
          <Property ovf:key="user-data" ovf:type="string" ovf:userConfigurable="true" ovf:value="">
              <Label>Encoded user-data</Label>
              <Description>In order to fit into a xml attribute, this value is base64 encoded . It will be decoded, and then processed normally as user-data.</Description>
              <!--  The following represents '#!/bin/sh\necho "hi world"'
              ovf:value="IyEvYmluL3NoCmVjaG8gImhpIHdvcmxkIgo="
            -->
          </Property>
          <Property ovf:key="password" ovf:type="string" ovf:userConfigurable="true" ovf:value="">
              <Label>Default User's password</Label>
              <Description>If set, the default user's password will be set to this value to allow password based login.  The password will be good for only a single login.  If set to the string 'RANDOM' then a random password will be generated, and written to the console.</De
          </Property>
        </ProductSection>
    
        <VirtualHardwareSection ovf:transport="iso">
          <Info>Virtual hardware requirements</Info>
          <System>
            <vssd:ElementName>Virtual Hardware Family</vssd:ElementName>
            <vssd:InstanceID>0</vssd:InstanceID>
            <vssd:VirtualSystemIdentifier>CentOS-7-x86_64-GenericCloud-2003</vssd:VirtualSystemIdentifier>
            <vssd:VirtualSystemType>vmx-10</vssd:VirtualSystemType>
          </System>
          <Item>
            <rasd:AllocationUnits>hertz * 10^6</rasd:AllocationUnits>
            <rasd:Description>Number of Virtual CPUs</rasd:Description>
            <rasd:ElementName>2 virtual CPU(s)</rasd:ElementName>
            <rasd:InstanceID>1</rasd:InstanceID>
            <rasd:ResourceType>3</rasd:ResourceType>
            <rasd:VirtualQuantity>2</rasd:VirtualQuantity>
          </Item>
          <Item>
            <rasd:AllocationUnits>byte * 2^20</rasd:AllocationUnits>
            <rasd:Description>Memory Size</rasd:Description>
            <rasd:ElementName>1024MB of memory</rasd:ElementName>
            <rasd:InstanceID>2</rasd:InstanceID>
            <rasd:ResourceType>4</rasd:ResourceType>
            <rasd:VirtualQuantity>1024</rasd:VirtualQuantity>
          </Item>
          <Item>
            <rasd:Address>0</rasd:Address>
            <rasd:Description>SCSI Controller</rasd:Description>
            <rasd:ElementName>SCSI Controller 0</rasd:ElementName>
            <rasd:InstanceID>3</rasd:InstanceID>
            <rasd:ResourceSubType>VirtualSCSI</rasd:ResourceSubType>
            <rasd:ResourceType>6</rasd:ResourceType>
          </Item>
          <Item>
            <rasd:Address>1</rasd:Address>
            <rasd:Description>IDE Controller</rasd:Description>
            <rasd:ElementName>VirtualIDEController 1</rasd:ElementName>
            <rasd:InstanceID>4</rasd:InstanceID>
            <rasd:ResourceType>5</rasd:ResourceType>
          </Item>
          <Item>
            <rasd:Address>0</rasd:Address>
            <rasd:Description>IDE Controller</rasd:Description>
            <rasd:ElementName>VirtualIDEController 0</rasd:ElementName>
            <rasd:InstanceID>5</rasd:InstanceID>
            <rasd:ResourceType>5</rasd:ResourceType>
          </Item>
          <Item ovf:required="false">
            <rasd:AutomaticAllocation>false</rasd:AutomaticAllocation>
            <rasd:ElementName>VirtualVideoCard</rasd:ElementName>
            <rasd:InstanceID>6</rasd:InstanceID>
            <rasd:ResourceType>24</rasd:ResourceType>
            <vmw:Config ovf:required="false" vmw:key="enable3DSupport" vmw:value="false"/>
            <vmw:Config ovf:required="false" vmw:key="enableMPTSupport" vmw:value="false"/>
            <vmw:Config ovf:required="false" vmw:key="use3dRenderer" vmw:value="automatic"/>
            <vmw:Config ovf:required="false" vmw:key="useAutoDetect" vmw:value="false"/>
            <vmw:Config ovf:required="false" vmw:key="videoRamSizeInKB" vmw:value="4096"/>
          </Item>
          <Item ovf:required="false">
            <rasd:AutomaticAllocation>false</rasd:AutomaticAllocation>
            <rasd:ElementName>VirtualVMCIDevice</rasd:ElementName>
            <rasd:InstanceID>7</rasd:InstanceID>
            <rasd:ResourceSubType>vmware.vmci</rasd:ResourceSubType>
            <rasd:ResourceType>1</rasd:ResourceType>
            <vmw:Config ovf:required="false" vmw:key="allowUnrestrictedCommunication" vmw:value="false"/>
          </Item>
          <Item ovf:required="false">
            <rasd:AddressOnParent>0</rasd:AddressOnParent>
            <rasd:AutomaticAllocation>false</rasd:AutomaticAllocation>
            <rasd:ElementName>CD-ROM 1</rasd:ElementName>
            <rasd:InstanceID>8</rasd:InstanceID>
            <rasd:Parent>4</rasd:Parent>
            <rasd:ResourceSubType>vmware.cdrom.remotepassthrough</rasd:ResourceSubType>
            <rasd:ResourceType>15</rasd:ResourceType>
            <vmw:Config ovf:required="false" vmw:key="backing.exclusive" vmw:value="false"/>
          </Item>
          <Item>
            <rasd:AddressOnParent>0</rasd:AddressOnParent>
            <rasd:ElementName>Hard Disk 1</rasd:ElementName>
            <rasd:HostResource>ovf:/disk/vmdisk1</rasd:HostResource>
            <rasd:InstanceID>9</rasd:InstanceID>
            <rasd:Parent>3</rasd:Parent>
            <rasd:ResourceType>17</rasd:ResourceType>
            <vmw:Config ovf:required="false" vmw:key="backing.writeThrough" vmw:value="false"/>
          </Item>
          <Item ovf:required="false">
            <rasd:AddressOnParent>0</rasd:AddressOnParent>
            <rasd:AutomaticAllocation>false</rasd:AutomaticAllocation>
            <rasd:Description>Floppy Drive</rasd:Description>
            <rasd:ElementName>Floppy 1</rasd:ElementName>
            <rasd:InstanceID>10</rasd:InstanceID>
            <rasd:ResourceSubType>vmware.floppy.remotedevice</rasd:ResourceSubType>
            <rasd:ResourceType>14</rasd:ResourceType>
          </Item>
          <Item>
            <rasd:AddressOnParent>7</rasd:AddressOnParent>
            <rasd:AutomaticAllocation>true</rasd:AutomaticAllocation>
            <rasd:Connection>VM Network</rasd:Connection>
            <rasd:Description>VmxNet3 ethernet adapter on &quot;VM Network&quot;</rasd:Description>
            <rasd:ElementName>Ethernet 1</rasd:ElementName>
            <rasd:InstanceID>11</rasd:InstanceID>
            <rasd:ResourceSubType>VmxNet3</rasd:ResourceSubType>
            <rasd:ResourceType>10</rasd:ResourceType>
            <vmw:Config ovf:required="false" vmw:key="wakeOnLanEnabled" vmw:value="true"/>
          </Item>
          <vmw:Config ovf:required="false" vmw:key="cpuHotAddEnabled" vmw:value="false"/>
          <vmw:Config ovf:required="false" vmw:key="cpuHotRemoveEnabled" vmw:value="false"/>
          <vmw:Config ovf:required="false" vmw:key="firmware" vmw:value="bios"/>
          <vmw:Config ovf:required="false" vmw:key="virtualICH7MPresent" vmw:value="false"/>
          <vmw:Config ovf:required="false" vmw:key="virtualSMCPresent" vmw:value="false"/>
          <vmw:Config ovf:required="false" vmw:key="memoryHotAddEnabled" vmw:value="false"/>
          <vmw:Config ovf:required="false" vmw:key="nestedHVEnabled" vmw:value="false"/>
          <vmw:Config ovf:required="false" vmw:key="powerOpInfo.powerOffType" vmw:value="preset"/>
          <vmw:Config ovf:required="false" vmw:key="powerOpInfo.resetType" vmw:value="preset"/>
          <vmw:Config ovf:required="false" vmw:key="powerOpInfo.standbyAction" vmw:value="checkpoint"/>
          <vmw:Config ovf:required="false" vmw:key="powerOpInfo.suspendType" vmw:value="preset"/>
          <vmw:Config ovf:required="false" vmw:key="tools.afterPowerOn" vmw:value="true"/>
          <vmw:Config ovf:required="false" vmw:key="tools.afterResume" vmw:value="true"/>
          <vmw:Config ovf:required="false" vmw:key="tools.beforeGuestShutdown" vmw:value="true"/>
          <vmw:Config ovf:required="false" vmw:key="tools.beforeGuestStandby" vmw:value="true"/>
          <vmw:Config ovf:required="false" vmw:key="tools.syncTimeWithHost" vmw:value="false"/>
          <vmw:Config ovf:required="false" vmw:key="tools.toolsUpgradePolicy" vmw:value="manual"/>
        </VirtualHardwareSection>
      </VirtualSystem>
    </Envelope>
  6. Generate the checksums and save the output:

    sha256sum CentOS-7-x86_64-GenericCloud-2003.vmdk
    sha256sum CentOS-7-x86_64-GenericCloud-2003.ovf
  7. Create a file CentOS-7-x86_64-GenericCloud-2003.mf (still in releases/releases/centos7/ ) with the following content:

    SHA256(CentOS-7-x86_64-GenericCloud-2003.vmdk)= 0c189d665e6ba7150c74626a63f6eb32ef3936f459790f04d720a243f6586d9b
    SHA256(CentOS-7-x86_64-GenericCloud-2003.ovf)= 1074a4aac1b07c811d874233361dfa24bd3b2d5d7db5526b4f58a07666c048a6

    Replace the string to the right of the equal sign with the checksum calculated in step 5. The extra whitespace to the right of the equal sign might or might not be needed.

  8. Create the OVA file, which is what VMware accepts. The OVA file is just a tar archive containing the virtual machine’s metadata and the disk image:

    tar -vcf CentOS-7-x86_64-GenericCloud-2003.ova \
             CentOS-7-x86_64-GenericCloud-2003.vmdk \
             CentOS-7-x86_64-GenericCloud-2003.ovf \
             CentOS-7-x86_64-GenericCloud-2003.mf
  9. Calculate two checksums of the tar archive and its size in bytes:
    # save the number of bytes
    wc -c CentOS-7-x86_64-GenericCloud-2003.ova
    
    # save the md5 checksum
    md5sum CentOS-7-x86_64-GenericCloud-2003.ova
    
    # save the sha256 checksum
    sha256sum CentOS-7-x86_64-GenericCloud-2003.ova
  10. Change directory:
    cd ../../streams/v1/
  11. Create a file called com.ubuntu.cloud:released:download.json :

    emacs com.ubuntu.cloud:released:download.json

    Add the following JSON to the file:

    {                                  
        "content_id": "com.ubuntu.cloud:released:download",
        "datatype": "image-downloads",
        "format": "products:1.0",
        "license": "http://www.canonical.com/intellectual-property-policy",
        "products": {
            "com.ubuntu.cloud:server:centos7:amd64": {
                "arch": "amd64",
                "version": "centos7",
                "release": "centos7",
                "os": "centos7",
                "supported": true,
                "versions": {
                    "20201029": {
                        "items": {
                            "ova": {
                                "ftype": "ova",
                                "md5": "f087ed0604dbf00887c75d22c6cbf2b2",
                                "path": "releases/centos7/CentOS-7-x86_64-GenericCloud-2003.ova",
                                "sha256": "ef704a952ecec2e5976309a1f7e12c236d9c68f95f9019111b7616fe266b1cb9",
                                "size": 369745920
                            }
                        }
                    }
                }
            },
        },
        "updated": "Fri, 23 Oct 2020 03:06:29 +0000"
    }

    Update the keys md5 , sha256 and size with the values that you saved in step 8.

  12. At this point we want to verify that the JSON code does not have syntax errors, for this we will use a tool called sstream-query:

    sudo apt install simplestreams
    
    # replace IP with the IP-number of your web server
    sstream-query --no-verify --json --max=1 http://IP/releases/streams/v1/index.json

    If your output looks like JSON then it worked. If your JSON has errors then there will be errors reported. Errors must be corrected or else juju might not understand your metadata.

Add vSphere as a cloud

# this is where you type the endpoint, etc
juju add-cloud vsphere

# provide user/pass
juju add-credential vsphere

Configure your juju model to use the metadata

# replace "IP" with the IP-number of the server that was setup above
juju model-config image-metadata-url="http://IP/releases"

# try to deploy a machine using the image
juju add-machine --series centos7

It takes a few minutes for the IP address of the new VM to show up in juju status because cloud-init runs updates prior to running the juju agent ( jujud ). To get faster “boot” times via juju we can disable OS updates:

juju model-config enable-os-refresh-update=false
juju model-config enable-os-upgrade=false

juju add-machine --series centos7

Happy computing!

3 Likes

@jameinel @jamesbeedy @thumper This is a solid starting point to having this more “defaulted” to work perhaps on other substrates?

There are still some “errors” with cloud-init and missing capabilities to add “yum mirrors” (hence the need to disable update/upgrades to avoid very slow download speeds from yum internet repos)… and a few more errors. But all in all, this is useful at least.

Is related to the errors we get…

@petevg

Thank you for the detailed writeup, @oscarf! And thank you for linking the bug and the discussion, @erik-lonroth. That should be useful for reference going forward.

2 Likes