Uncategorized, Virtualization, VMware, vSphere

Shorts: What is it about cpuid.corespersocket on vSphere?

Time for another short! The google searches leading to this blog show searches coming in based on the cpuid.corespersocket setting. In this short I’ll try to explain what this setting is for and how it behaves. So, let’s dig right in!

The cpuid.corespersocket setting

In a sense, you would assume that the name of the setting says it all. And in a sense, it does. In the physical world, you will have a number of sockets on your motherboard, this number of sockets is normally also the number of physical CPU’s that you have on said motherboard (at least in an ideal world), and each CPU will have one or more cores on it.

Wikipedia describes this in the following way:

One can describe it as an integrated circuit to which two or more individual processors (called cores in this sense) have been attached.

…..

The amount of performance gained by the use of a multi-core processor depends very much on the software algorithms and implementation. In particular, the possible gains are limited by the fraction of the software that can be parallelized to run on multiple cores simultaneously; this effect is described by Amdahl’s law. In the best case, so-called embarrassingly parallel problems may realize speedup factors near the number of cores, or beyond even that if the problem is split up enough to fit within each processor’s or core’s cache(s) due to the fact that the much slower main memory system is avoided.

Now, this sounds quite good, but some of you may ask what kind of influence this has on my virtualized systems. The most obvious answer would be “none at all”. This is because by default your virtualized system will see the cores as physical CPU’s and be done with it.

So, now you are probably wondering why VMware would even distinguish between cores and sockets. The answer is quite simple; It’s due to licensing. Not so much by VMware, but by the software or operating system that you would like to virtualize. You see, some of that software is licensed per core, and some will license by the number of sockets (some even combine the two).

So how do I use it?

As with all things computer related… It depends. When you are using ESX 3.5 you have no chance of using it. With ESX 4, you can actually use this feature, but it is not officially supported (someone please point me in the right direction if this is incorrect). And starting with ESX 4.1 the setting is now officially supported, and even documented in the VMware Knowledge Base as KB Article: 1010184.

Simply put, you can now create a virtual machine with for example 4 vCPU’s and set the cpuid.corespersocket to 2. This will make your operating system assume that you have two CPU’s, and that each CPU has two cores. If you create a machine with 8 vCPU’s and again select a cpuid.corespersocket of 2, your operating system will report 4 dual-core CPU’s.

You can actually set this value by either going this route:

  1. Power off the virtual machine.
  2. Right-click on the virtual machine and click Edit Settings.
  3. Click Hardware and select CPUs.
  4. Choose the number of virtual processors.
  5. Click the Options tab.
  6. Click General, in the Advanced options section.
  7. Click Configuration Parameters.
  8. Include cpuid.coresPerSocket in the Name column.
  9. Enter a value ( try 2, 4, or 8 ) in the Value column.

    Note: This must hold:

    #VCPUs for the VM / cpuid.coresPerSocket = An integer

    That is, the number of vCPUs must be divisible by cpuid.coresPerSocket. So if your virtual machine is created with 8 vCPUs, coresPerSocket can only be 1, 2, 4, or 8.

    The virtual machine now appears to the operating system as having multi-core CPUs with the number of cores per CPU given by the value that you provided in step 9.

  10. Click OK.
  11. Power on the virtual machine.

If the setting isn’t shown, for example for those who want to experiment with it under ESX 4.0, you can create the values in the following way:

  1. Power off the virtual machine.
  2. Right-click on the virtual machine and click Edit Settings.
  3. Click the Options tab.
  4. Click General, under the Advanced options section.
  5. Click Configuration Parameters.
  6. Click Add Row.
  7. Enter “cpuid.coresPerSocket” in the Name column.
  8. Enter a value ( try 2, 4, or 8 ) in the Value column.
  9. Click OK.
  10. Power on the virtual machine.

To check if your settings actually worked, you can use the sysinternals tool called Coreinfo on your Windows systems, and on Linux you can perform a simple “cat /proc/cpuinfo” to see if everything works.

GestaltIT, Virtualization, vSphere

The RAM per CPU wall

There is one thing that keeps coming up lately in our office. We install a lot of systems virtually, and have over 8000 virtual servers installed.

RAM.jpgNow, as anyone who has installed SAP will probably tell you, our software can do a lot of things, but one of the things it swallows up is RAM. We seem to be doing a bit better on the central instance side, but moved a lot of the “RAM hunger” elsewhere. This is especially true for our new CRM 7.0 which will show you that your application server will need a lot more of it, when compared to older versions.

Now, since these kind of application servers are ideal candidates for virtual machines. With the new vSphere release you can actually give a VM up too 255 GB or RAM, and for larger CRM implementations you might actually consider this limit for your application servers. No problem at all, but you will face an entirely new problem that has been creeping up and has left most people unaware.

Since there’s no real name for it, I just call it the “RAM per CPU wall”, or perhaps even the “RAM per core wall”.

So, what about this wall?

Well, it used to be that computational power was quite expensive, and the same could be said about storage. Times have changed and right now we see that processing power is not that expensive anymore. We have seen a slow moving shift toward the x86 instruction set and it’s x64 extension. With the development of the multi-core processor we are seeing applications being optimized to run in parallel on the chipsets, and with solutions like vSphere we are using more and more of those features. Examples in vSphere would be “Hyperthreaded Core Sharing” or a setting like “cpuid.coresPerSocket”.

It’s great that we can tweak how how our VM platform and our VMs handle multiple cores, and I am certain that we will be seeing a lot more options in the future. Why I am so sure of that? Because we will be seeing the octo-core processors in the not too distant future (I’m betting somehwere around the end of Q2 2010). Just think about it, 8 cores on one CPU.

Now, combine that with an educated guess that we will be seeing 8 socket mainboards and servers, and we are talking about something like the HP DL780 or DL785 with a total of 64 cores. Quite the number right? Now just imagine the amount of RAM you can put in such a server.

Right?

The current HP DL785 G5 has a total of 32 cores and a limit of 512 GB, assuming you are willing to pay for the exuberant memory price tag for such a configuration. I’m assuming support for 128 GB RAM modules will take a bit longer, so just assuming 64 cores and 512 GB of RAM will give you 8GB of RAM per CPU core.

It might just be my point of view, but that’s not that much per core.

Now I can hear you saying that for smaller systems 8 GB of ram per core is just fine, and I hear you very good. But that’s not the typical configuration used in larger environments. If you take the CRM example I gave before, you can load up a bunch of smaller application servers and set your hopes on overcommitting, but that will only get you so far.

And the trend is not changing anytime soon. We will be seeing more cores per cpu coming, and the prices and limits of large amounts of RAM are not keeping up, and I doubt they will do so in the future.

So, will we continue to see larger systems being set-up on physical hardware? Actually, I honestly think so. For service providers and large environments things need to change if they want to get a big benefit out of the provisioning of larger machines. If things don’t change, they are facing the “RAM per CPU wall” and will be stuck provisioning smaller instances.

All in all I can only recommend digging down and asking your customers for their requirements. Check the sizing recommendations and try to get some hands on information or talk to people who have implemented said solutions to see how the application handles memory. Make a decision on a per-use case for larger environments. Virtualization is great, but make sure that the recommendation and implementation suits the needs that your customer has, even if that means installing on a physical platform.