CoreOS on VMWare using VMWare GuestInfo API
September 27, 2015 2 Comments
CoreOS bills itself as “Linux for Massive Server Deployments”, but it turns out, it’s excellent for smaller deployments as well. CoreOS was intended to run in big cloud providers (Azure, AWS, Digital Ocean, etc) backed by OpenStack. I’ve been running CoreOS for a while now, on premises in VMWare. It is kind of a pain: clone the template; interrupt the boot to enable autologin, set a password for “core”; reboot again; re-mount the disk rw; paste in a hand crafted cloud-config.yml. Fortunately for me (and you!), VMWare has an API to inject settings into guests, and CoreOS has added support for using those settings in coreos-cloudinit. Information about the supported properties is here.
My new process is:
- Clone the template to a new VM
- Inject the config through the guest vmx file
- Boot the VM
It’s all automated though, so mostly I get other productive work or coffee drinking done while it’s conjured into existence.
The short version is that you base64 encode and inject your cloud-config, along with any additional parameters. For me it looks like:
Property | Value | Explanation |
---|---|---|
hostname | coreos0 | The hostname, obviously 🙂 |
interface.0.name | ens192 | The name of the interface to assign id #0 for future settings. The kernel names the first nic ens192 in VMWare. |
interface.0.role | private | This is my internal (and only) NIC. |
interface.0.dhcp | no | Turn off DHCP. I like it on, most like it off. |
interface.0.ip.0.address | 192.168.229.20/24 | CIDR IP address |
interface.0.route.0.gateway | 192.168.229.2 | Default gateway |
interface.0.route.0.destination | 0.0.0.0/0 | Default gateway |
dns.server.0 | 192.168.229.2 | DNS server |
coreos.config.data | [some ugly string of base64] | Base64 encoded cloud-config.yml |
coreos.config.data.encoding | base64 | Tells cloudinit how to decode coreos.config.data |
Ok, so this is all great, but how to you get it into VMWare? You have two choices, you can manually edit the .VMX file (boring!) or you can use powershell. The script I use is in github, but the workflow is basically:
- Clone the VM
- Connect to the datastore and fetch the vmx file
- Add the guestinfo to the VMX file
- Upload the VMX file back to the datastore
- Boot the VM
The script will pick up a cloud-config.yml and base64 encode it for you and inject it. Check out the source in github to learn more. If you’re looking at the CoreOS documentation on the VMWare backdoor, you need to put “guestinfo.” infront of all the properties. For example, guestinfo.dns.server.0. The VMWare RPC API only passes properties that start with guestinfo.
This is how it looks when it’s written out to the VMX:
guestinfo.coreos.config.data.encoding = "base64" guestinfo.interface.0.dhcp = "no" guestinfo.coreos.config.data = "I2Nsyb2JAd............WJ1bnR1MA0K" guestinfo.dns.server.0 = "192.168.229.2" guestinfo.interface.0.ip.0.address = "192.168.229.20/24" guestinfo.interface.0.route.0.gateway = "192.168.229.2" guestinfo.interface.0.role = "private" guestinfo.interface.0.route.0.destination = "0.0.0.0/0" guestinfo.interface.0.name = "ens192" guestinfo.hostname = "coreos0"
Wanna see it in action?
Great article Robert! I ran into an issue when doing this where it doesn’t seem to want to pick up the IP information from the cloud-config. My system is esx 5.5. Are there any special settings that need to be in place on the vmware side for that to work? The api created the machines just fine, and the vmx file has the correct information in it. However, when I boot up, the machines are still DHCP. Kind of stumped at the moment.
Greg
Hi Greg,
Sorry for the long delay. Make sure the interface name is actually ens192 … depending on the VNIC you chose or the ESX version, the kernel might give it some other name.