Containers seem to be the hot trend right now. I needed to get some more experience in this area, and instead of working with a single container machine, I actually wanted to get a “quick” distributed setup going. It wasn’t all that quick to start with, but I now have a working setup that can actually be rolled out and scaled in a pretty quick fashion.
Now, I’m assuming you already know what a container is, and have heard about CoreOS. Here are some quick steps to get you started. I’ll start off with the prerequisites:
- You will have your Nutanix CE cluster up and running
- You have a VLAN set up with IP address management and a DHCP server on Nutanix CE
- You downloaded the CoreOS ISO image
Your further steps are relatively simple. First off, we will create an etcd master, the most important thing we need is a fixed IP, so define which IP you want to give it. Obviously we could use the CoreOS cluster discovery mechanism and rely on an internet connection, but I decided to just use my own instance instead.
Start off by creating a cloud-config file for your etcd master:
#cloud-config ssh_authorized_keys: - ssh-rsa AAAAB3NzaC1... coreos: etcd2: name: etcdserver initial-cluster: etcdserver=http://<etcd-vm-ip-here>:2380 initial-advertise-peer-urls: http://<etcd-vm-ip-here>:2380 advertise-client-urls: http://<etcd-vm-ip-here>:2379 # listen on both the official ports and the legacy ports # legacy ports can be omitted if your application doesn't depend on them listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001 listen-peer-urls: http://0.0.0.0:2380 units: - name: etcd2.service command: start - name: 00-eth0.network runtime: true content: | [Match] Name=eth0 [Network] DNS=<your-dns-ip-here> Address=<etcd-vm-ip-here>/16 Gateway=<your-gateway-ip-here>
Note that I’ve copied in the public ssh key from my laptop to get easier access to the VM. Now, save this file as a text file called user_data, and create an iso image using the ways described here. Copy that over to your container on CE using sftp to a controller VM on port 2222. You can use Prism credentials to authenticate.
Next step, create a new VM in Acropolis, attach the CoreOS ISO image as your primary CD drive, and the ISO you just created as the second CD drive, and power on the VM
Now, to create the actual CoreOS cluster, you create a second user_data file, that only contains the following:
#cloud-config ssh_authorized_keys: - ssh-rsa AAAAB3NzaC1... coreos: etcd2: proxy: on initial-cluster: etcdserver=http://<etcd-vm-ip-here>:2380 listen-client-urls: http://localhost:2379 fleet: etcd_servers: http://localhost:2379 metadata: "role=etcd" units: - name: etcd2.service command: start - name: fleet.service command: start
For quick deployment, I’d create a VM that you use as a template to clone from. Give the VM the newly created file as the secondary drive.
Now, just create some clones, power them on and wait for them to get their IP. You should then be able to ssh into the machine using the “core” user, and check your cluster:
core@CoreOS-1 ~ $ fleetctl list-machines MACHINE IP METADATA 1c24fc23... 192.168.96.248 role=etcd 8e974c05... 192.168.79.25 role=etcd a899b944... 192.168.114.8 role=etcd f916eb93... 192.168.3.179 role=etcd
With that, you can start implementing and rolling out your units:
core@CoreOS-1 ~ $ fleetctl load hello.service Unit hello.service loaded on 1c24fc23.../192.168.96.248 core@CoreOS-1 ~ $ fleetctl list-units UNIT MACHINE ACTIVE SUB hello.service 1c24fc23.../192.168.96.248 inactive dead