CoreOS on VMWare using VMWare GuestInfo API

CoreOS bills itself as “Linux for Massive Server Deployments”, but it turns out, it’s excellent for smaller deployments as well. CoreOS was intended to run in big cloud providers (Azure, AWS, Digital Ocean, etc) backed by OpenStack. I’ve been running CoreOS for a while now, on premises in VMWare. It is kind of a pain: clone the template; interrupt the boot to enable autologin, set a password for “core”; reboot again; re-mount the disk rw; paste in a hand crafted cloud-config.yml. Fortunately for me (and you!), VMWare has an API to inject settings into guests, and CoreOS has added support for using those settings in coreos-cloudinit. Information about the supported properties is here.

My new process is:

  1. Clone the template to a new VM
  2. Inject the config through the guest vmx file
  3. Boot the VM

It’s all automated though, so mostly I get other productive work or coffee drinking done while it’s conjured into existence.

The short version is that you base64 encode and inject your cloud-config, along with any additional parameters. For me it looks like:

Property Value Explanation
hostname coreos0 The hostname, obviously 🙂
interface.0.name ens192 The name of the interface to assign id #0 for future settings. The kernel names the first nic ens192 in VMWare.
interface.0.role private This is my internal (and only) NIC.
interface.0.dhcp no Turn off DHCP. I like it on, most like it off.
interface.0.ip.0.address 192.168.229.20/24 CIDR IP address
interface.0.route.0.gateway 192.168.229.2 Default gateway
interface.0.route.0.destination 0.0.0.0/0 Default gateway
dns.server.0 192.168.229.2 DNS server
coreos.config.data [some ugly string of base64] Base64 encoded cloud-config.yml
coreos.config.data.encoding base64 Tells cloudinit how to decode coreos.config.data

Ok, so this is all great, but how to you get it into VMWare? You have two choices, you can manually edit the .VMX file (boring!) or you can use powershell. The script I use is in github, but the workflow is basically:

  1. Clone the VM
  2. Connect to the datastore and fetch the vmx file
  3. Add the guestinfo to the VMX file
  4. Upload the VMX file back to the datastore
  5. Boot the VM

 

The script will pick up a cloud-config.yml and base64 encode it for you and inject it. Check out the source in github to learn more. If you’re looking at the CoreOS documentation on the VMWare backdoor, you need to put “guestinfo.” infront of all the properties. For example, guestinfo.dns.server.0. The VMWare RPC API only passes properties that start with guestinfo.

This is how it looks when it’s written out to the VMX:

guestinfo.coreos.config.data.encoding = "base64"
guestinfo.interface.0.dhcp = "no"
guestinfo.coreos.config.data = "I2Nsyb2JAd............WJ1bnR1MA0K"
guestinfo.dns.server.0 = "192.168.229.2"
guestinfo.interface.0.ip.0.address = "192.168.229.20/24"
guestinfo.interface.0.route.0.gateway = "192.168.229.2"
guestinfo.interface.0.role = "private"
guestinfo.interface.0.route.0.destination = "0.0.0.0/0"
guestinfo.interface.0.name = "ens192"
guestinfo.hostname = "coreos0"

Wanna see it in action?

Advertisements

About robertlabrie
DevOps Engineer at The Network Inc in metro Atlanta. Too many interests to list here, check out my posts, or look me up on LinkedIn

2 Responses to CoreOS on VMWare using VMWare GuestInfo API

  1. Greg says:

    Great article Robert! I ran into an issue when doing this where it doesn’t seem to want to pick up the IP information from the cloud-config. My system is esx 5.5. Are there any special settings that need to be in place on the vmware side for that to work? The api created the machines just fine, and the vmx file has the correct information in it. However, when I boot up, the machines are still DHCP. Kind of stumped at the moment.

    Greg

    • robertlabrie says:

      Hi Greg,

      Sorry for the long delay. Make sure the interface name is actually ens192 … depending on the VNIC you chose or the ESX version, the kernel might give it some other name.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: