CoreOS on VMWare using VMWare GuestInfo API

CoreOS bills itself as “Linux for Massive Server Deployments”, but it turns out, it’s excellent for smaller deployments as well. CoreOS was intended to run in big cloud providers (Azure, AWS, Digital Ocean, etc) backed by OpenStack. I’ve been running CoreOS for a while now, on premises in VMWare. It is kind of a pain: clone the template; interrupt the boot to enable autologin, set a password for “core”; reboot again; re-mount the disk rw; paste in a hand crafted cloud-config.yml. Fortunately for me (and you!), VMWare has an API to inject settings into guests, and CoreOS has added support for using those settings in coreos-cloudinit. Information about the supported properties is here.

My new process is:

1. Clone the template to a new VM
2. Inject the config through the guest vmx file
3. Boot the VM

It’s all automated though, so mostly I get other productive work or coffee drinking done while it’s conjured into existence.

The short version is that you base64 encode and inject your cloud-config, along with any additional parameters. For me it looks like:

Property	Value	Explanation
hostname	coreos0	The hostname, obviously 🙂
interface.0.name	ens192	The name of the interface to assign id #0 for future settings. The kernel names the first nic ens192 in VMWare.
interface.0.role	private	This is my internal (and only) NIC.
interface.0.dhcp	no	Turn off DHCP. I like it on, most like it off.
interface.0.ip.0.address	192.168.229.20/24	CIDR IP address
interface.0.route.0.gateway	192.168.229.2	Default gateway
interface.0.route.0.destination	0.0.0.0/0	Default gateway
dns.server.0	192.168.229.2	DNS server
coreos.config.data	[some ugly string of base64]	Base64 encoded cloud-config.yml
coreos.config.data.encoding	base64	Tells cloudinit how to decode coreos.config.data

Ok, so this is all great, but how to you get it into VMWare? You have two choices, you can manually edit the .VMX file (boring!) or you can use powershell. The script I use is in github, but the workflow is basically:

1. Clone the VM
2. Connect to the datastore and fetch the vmx file
3. Add the guestinfo to the VMX file
4. Upload the VMX file back to the datastore
5. Boot the VM

 

The script will pick up a cloud-config.yml and base64 encode it for you and inject it. Check out the source in github to learn more. If you’re looking at the CoreOS documentation on the VMWare backdoor, you need to put “guestinfo.” infront of all the properties. For example, guestinfo.dns.server.0. The VMWare RPC API only passes properties that start with guestinfo.

This is how it looks when it’s written out to the VMX:

guestinfo.coreos.config.data.encoding = "base64"
guestinfo.interface.0.dhcp = "no"
guestinfo.coreos.config.data = "I2Nsyb2JAd............WJ1bnR1MA0K"
guestinfo.dns.server.0 = "192.168.229.2"
guestinfo.interface.0.ip.0.address = "192.168.229.20/24"
guestinfo.interface.0.route.0.gateway = "192.168.229.2"
guestinfo.interface.0.role = "private"
guestinfo.interface.0.route.0.destination = "0.0.0.0/0"
guestinfo.interface.0.name = "ens192"
guestinfo.hostname = "coreos0"

Wanna see it in action?