CentOS – BasRaayman's Technical Take

I’m getting more and more customer requests that are looking at alternatives for VMware, and are considering a different hypervisor. Since I’m more of a VMware guy, but I am always willing to learn new stuff, I figured I might as well share some info on how to set up the Nutanix cluster on KVM, and create an initial virtual machine.

I’m assuming you have at least some Linux knowledge, and that you were able to get the hosts and the controller VMs configured with an IP address. After that, the basic setup is pretty much the same. Visit the cluser_init page using the IPv6 link local address, which is in the format:

http://ntnx-[block_serial_number]-[node_position]-cvm.local:2100

Which looks something like this:

Nutanix - Cluster Init — Nutanix – Cluster init

Fill out the information in that window, and click the “Create” button. Once that is done, you will see some messages popping up underneath:
Configuring IP addresses on node 13SM15400003/A... Configuring IP addresses on node 13SM15400003/B... Configuring IP addresses on node 13SM15400003/C... Configuring the Hypervisor DNS settings on node 13SM15400003/A... Configuring the Hypervisor DNS settings on node 13SM15400003/B... Configuring the Hypervisor DNS settings on node 13SM15400003/C... Configuring the Hypervisor NTP settings on node 13SM15400003/A... Configuring the Hypervisor NTP settings on node 13SM15400003/B... Configuring the Hypervisor NTP settings on node 13SM15400003/C... Configuring Zeus on node 13SM15400003/A... Configuring Zeus on node 13SM15400003/B... Configuring Zeus on node 13SM15400003/C... Initializing cluster... not ready, trying again in 5 seconds... Initializing cluster... Cluster successfully initialized! Initializing the CVM DNS and NTP servers... Successfully updated the CVM NTP and DNS server list

What we are doing, is actually configuring the cluster with all the IP addresses, writing the cluster configuration to the underlying services, and starting the cluster for you. Give it a couple of minutes (usually 2 or 3 minutes will suffice), and you can now log on to the IP address of a controller VM, or the “Cluster External IP” that you put in, using the default username and password:

Nutanix - Cluster logon — Nutanix – Cluster logon

By the way, I disabled the background video by simply adding “?novideo=true” to the logon URL. This disables the video, and makes logon a bit faster, especially when working via a link that might not have the bandwidth that you would prefer.

I then created a storage pool by the name of “default”, and created a container with the same name. Once that is done, your cluster is ready for its first VMs.

Nutanix - Cluster ready — Nutanix – Cluster ready

Now, Nutanix relies on the management tools that a Hypervisor offers. In the case of vSphere, this would be vCenter. With KVM, or in our case KVM on CentOS, the selection is a bit more limited. Especially, since we make use of the Open Virtual Switch. That means, right now, we use libvirt as the management API, and wrote some extensions of our own. After all, your VMs will be located on storage that is being provided by Nutanix, so it would be good if we gave you some commands to make use of that storage, right? 😉

If we want to start the installation of a VM, we are first going to need an installation medium that we can use. So, I’m going to whitelist the default container I just created, and copy over a Ubuntu iso image:

Nutanix - Filesystem whitelist — Nutanix – Filesystem whitelist

Since you want to be able to see what was uploaded to the container, you can check from any of the controller VMs what is on there:
nutanix@NTNX-13SM15400003-A-CVM:10.0.0.30:~$ nfs_ls ubuntu-13.04-server-amd64.iso

Now, just pick the host that you want to use for your VM, and create the VM using the virt_install command. For example:
virt_install --name bas_ubuntu_test --disk 32 --cdrom /default/ubuntu-13.04-server-amd64.iso --nic VM-Network --vcpus 2 --ram 4096

Which would result in the following:
nutanix@NTNX-13SM15400003-A-CVM:10.0.0.30:~$ virt_install --name bas_ubuntu_test --disk 32 --cdrom /default/ubuntu-13.04-server-amd64.iso --nic VM-Network --vcpus 2 --ram 4096 2014-02-27 15:26:54 INFO batch_worker.py:211 Preparing nutanix disks: 0% 2014-02-27 15:26:57 INFO batch_worker.py:211 Preparing nutanix disks: 50% 2014-02-27 15:26:57 INFO batch_worker.py:211 Preparing nutanix disks: 100% 2014-02-27 15:26:57 INFO batch_worker.py:211 Creating libvirt storage pools: 0% 2014-02-27 15:26:59 INFO batch_worker.py:211 Creating libvirt storage pools: 50% 2014-02-27 15:26:59 INFO batch_worker.py:211 Creating libvirt storage pools: 100% 2014-02-27 15:26:59 INFO kvm_domain_template.py:184 Running virt-install

Now you have multiple options. You could connect using virt-manager:

Nutanix - virt-manager — Nutanix – virt-manager

Or, alternatively, you could open up the VNC port that the VM is running on (or disable iptables alltogether), and use your favorite VNC client to manage the newly created VM:

Most of the commands that Nutanix implemented come with a syntax that is very similar to the native libvirt syntax, but the commands will be named with an underscore instead of a dash. For example live migration of a VM can be performed using:
virt_migrate --vm bas_ubuntu_test --destination 10.0.0.20 --live

That’s it for a rough overview. If you have any questions, feel free to contact your local SE, or leave a note in the comments. 🙂

So folks, I helped a colleague install the VMware vCloud Director. In case you are not aware of what the vCloud Director is I can give you a very rough description.

Think about how you deploy virtual machines. Usually you will deploy one machine at a time, which is a good thing if you only need one server. But usually in larger environments, you will find that applications or application systems are not based on a single server. You will find larger environments that consist of multiple servers that will segregate functions, so for example, your landscape could consist of a DB server, an application server, and one or more proxies that provide access to your application servers.

If you are lucky, the folks installing everything will only request one virtual machine at a time. Usually that isn’t the case though. Now, this is where vCloud Director comes in. This will allow you to roll out a set of virtual machines at a time as a landscape. But it doesn’t stop there, since you can do a lot more because you can pool things like storage, networks and you a tight integration with vShield to secure your environment. But this should give you a very rough idea of what you can do with the vCloud Director. For a more comprehensive overview, take a look at Duncan’s post here.

Anyway, let’s dig in to the technical part.

There are plenty of blog posts that cover how to set up the CentOS installation, so I won’t cover that at great length. If you are looking for that info, take a peek here. If you want to install the Oracle DB on CentOS, take a look here to see how it’s done.

Here are some tips that might come in useful during the install:

Use the full path to the keytool. There is a slight difference between /usr/bin/keytool, /usr/lib/jvm/java-1.4.2-gcj-1.4.2.0/jre/bin/keytool and /opt/vmware/cloud-director/jre/bin/keytool. Be sure to use one of those, and if the commands to create and import your self-signed certificates are not working for some reason be sure to try a different one.

If you just simply create a database and browsed through the installation guide, you might have a hard time once you install the binary. Basically you run the “dbca” tool to create an empty database. If you by any chance forget to create the database files and run the installation binary (or the vCD configuration tool for that matter), you will receive an error while running the .sql database initialization scripts under /opt/vmware/cloud-director/db/oracle. The error message will tell you that there was an error creating the database.

Well, if only you had read the installation guide properly. Bascially what you do is start up the database:

sqlplus "/ as sysdba" startup

Make sure that the path you use in the “create tablespace” command actually exists. If they don’t you need to perform “mkdir $ORACLE_HOME/oradata” first. Then create the tablespaces and corresponding files:

Create Tablespace CLOUD_DATA datafile '$ORACLE_HOME/oradata/cloud_data01.dbf' size 1000M autoextend on; Create Tablespace CLOUD_INDX datafile '$ORACLE_HOME/oradata/cloud_indx01.dbf' size 500M autoextend on;

Now create a seperate user that we will give right for the database. The password for the user is the thing you type after “identified by”:

create user vcloud identified by vcloud default tablespace CLOUD_DATA;

Make sure that you give the user the correct rights to perform all the DB operations:

grant CONNECT, RESOURCE, CREATE TRIGGER, CREATE TYPE, CREATE VIEW, CREATE MATERIALIZED VIEW, CREATE PROCEDURE, CREATE SEQUENCE, EXECUTE ANY PROCEDURE to vcloud;

Now run the setup script, or run the configure script and you should be set to go.