Getting started with CoreOS on Nutanix Community Edition

24 07 2015

Containers seem to be the hot trend right now. I needed to get some more experience in this area, and instead of working with a single container machine, I actually wanted to get a “quick” distributed setup going. It wasn’t all that quick to start with, but I now have a working setup that can actually be rolled out and scaled in a pretty quick fashion.

Now, I’m assuming you already know what a container is, and have heard about CoreOS. Here are some quick steps to get you started. I’ll start off with the prerequisites:

  • You will have your Nutanix CE cluster up and running
  • You have a VLAN set up with IP address management and a DHCP server on Nutanix CE
  • You downloaded the CoreOS ISO image

Your further steps are relatively simple. First off, we will create an etcd master, the most important thing we need is a fixed IP, so define which IP you want to give it. Obviously we could use the CoreOS cluster discovery mechanism and rely on an internet connection, but I decided to just use my own instance instead.

Start off by creating a cloud-config file for your etcd master:

  - ssh-rsa AAAAB3NzaC1...
    name: etcdserver
    initial-cluster: etcdserver=http://<etcd-vm-ip-here>:2380
    initial-advertise-peer-urls: http://<etcd-vm-ip-here>:2380
    advertise-client-urls: http://<etcd-vm-ip-here>:2379
    # listen on both the official ports and the legacy ports
    # legacy ports can be omitted if your application doesn't depend on them
    - name: etcd2.service
    command: start
    - name:
    runtime: true
    content: |

Note that I’ve copied in the public ssh key from my laptop to get easier access to the VM. Now, save this file as a text file called user_data, and create an iso image using the ways described here. Copy that over to your container on CE using sftp to a controller VM on port 2222. You can use Prism credentials to authenticate.

Next step, create a new VM in Acropolis, attach the CoreOS ISO image as your primary CD drive, and the ISO you just created as the second CD drive, and power on the VM

Now, to create the actual CoreOS cluster, you create a second user_data file, that only contains the following:

 - ssh-rsa AAAAB3NzaC1...
    proxy: on
    initial-cluster: etcdserver=http://<etcd-vm-ip-here>:2380
    listen-client-urls: http://localhost:2379
    etcd_servers: http://localhost:2379
    metadata: "role=etcd"
    - name: etcd2.service
    command: start
    - name: fleet.service
    command: start

For quick deployment, I’d create a VM that you use as a template to clone from. Give the VM the newly created file as the secondary drive.

Now, just create some clones, power them on and wait for them to get their IP. You should then be able to ssh into the machine using the “core” user, and check your cluster:

core@CoreOS-1 ~ $ fleetctl list-machines
MACHINE     IP             METADATA
1c24fc23... role=etcd
8e974c05...  role=etcd
a899b944...  role=etcd
f916eb93...  role=etcd

With that, you can start implementing and rolling out your units:

core@CoreOS-1 ~ $ fleetctl load hello.service
Unit hello.service loaded on 1c24fc23.../

core@CoreOS-1 ~ $ fleetctl list-units
UNIT          MACHINE                    ACTIVE   SUB
hello.service 1c24fc23.../ inactive dead



One response

26 07 2015

Excellent guide Bas!

For those of you copying and pasting the config snippets and wondering why nothing is working, make sure to correct the whitespace, as the example is a bit broken. is a solid tool if your IDE doesn’t check for YAML parsing errors.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: