Tag: General

Nutanix, Nutanix AHV, SAP, Virtualization

Bas, what the hell have you been doing? – an​ SAP HANA on Nutanix story

Bas Raayman

To say my last blog post is “a while ago” would a grave understatement. Unfortunately, I’ve mainly been busy with something that was entirely new for Nutanix, and with the amount of work involved and the sensitive nature of what I was working on, there was relatively little room left to blog. Especially since I usually ended up blogging about stuff, I stumbled upon while doing my job or researching.

This all started with me changing from my presales focussed role to our internal “Solutions & Performance Engineering” team, which focusses on the business-critical applications running on the Nutanix platform. In essence, those applications that are the lifeblood of a company. Applications which, if they are unavailable, will cost the company significant amounts of money.

One of those applications is SAP, or more specifically the SAP HANA in-memory database. My colleagues (mainly Kasim Hansia, Rainer Wacker and Alexander Thoma) had already been doing a great job, and all of the SAP applications were certified to run on the Nutanix platform in November 2016. The big question was always “When can we run SAP HANA on Nutanix?”.

Working on the answer to this question is what I’ve been busy with the last year or so. While I won’t bother you with the specific details on the business side of things, I do want to take a bit of time to show what it’s like to go through the process of validating a new application.

First off, the SAP HANA in-memory database is an application that scales to levels that many people won’t ever see in action. You can run HANA in two ways. You either scale up the resources of a single server, for example running with up to 20TB of memory, or you can scale out by adding multiple servers and distributing the load across all servers.

Now, SAP has given the customer two options to select hardware to run SAP HANA. One is an “appliance model” where you choose a configuration as a whole, and everything will run in a pre-tested and validated fashion. You are ensured of a specific behavior of the whole system while running your application. The other option is something called “Tailored Data Center Integration” or TDI in short, where in essence you select your hardware from a hardware compatibility list and have the freedom to mix and match.

What we have done is work with SAP to introduce a new third category called “Hyperconverged Infrastructure” or HCI. The HCI category assumes that we are running SAP HANA in a virtualized fashion, and “collapse” several infrastructure components such as compute and storage to an integrated system.

The limitations on the maximum sizes for this category are smaller than for the other two categories, but the requirements that are in place for this certification do not offer much more leeway. For example, a storage test to ensure storage performance, where initially log overwrite operations needed to have latency <= 400 microseconds (this changed later on). Another example is a test suite of close to 700 tests that emulate real-world issues, and the performance delta is then compared to a bare-metal installation, with only a specific maximum performance delta between the two giving you a passing grade.

All this meant that I had my work cut out for me from the start. We started off working with a server model that wasn’t qualified before, before switching to the validation hardware, namely a Lenovo SR950. A big four-socket server with the fastest CPUs we could use, namely the Intel Xeon Platinum 8180M Processors, 3072 GB of RAM, 3.84TB SSDs and 1.6 TB NVMes.

Now, as much as Nutanix is a software company, we do strictly check that hardware meets specific prerequisites to ensure a smooth user experience and to make sure that certain performance metrics are a given. The issue is that all of the checks and functionality in place didn’t work for this new hardware. Simple things like making the status indicator LED for the NVMe light up, or mapping the physical drive locations back to the diagram view in Prism. It meant modifying Python files that handle how hardware is accessed, packaging everything back up into Python egg files, restarting services and then magically seeing drives that the system was able to access. Or passing through NICs so that we could test with “RDMA over Converged Ethernet” or RoCE, and changing BIOS settings to ensure maximum performance.

And while pushing the underlying hardware to its limits, it also meant we had to dive deep, and I mean very deep, into the software side of things. From things like experimenting with c-states on the CPU and NIC multiqueueing in the virtual machines, down to changing parameters in our product, ensuring that specific CPU features are passed through, the pinning of virtual CPUs to their physical location or making changes to how often a vmexit instruction is called.

I can’t go into all of the specific details since some of it is Nutanix’ intellectual property, but I’ll check what I can share in future posts, and if you have any specific questions, please ask them, and I’ll try to answer as best I can. What I can say is that we pushed the limits of our platform and quite a couple of the things we found are going to be implemented back into the product, and I expect a lot of those changes to show up in the release notes of an upcoming release.

Fact is, I learned a ton of new things, and this all culminated in our validation for pre-production use of SAP HANA on Nutanix AHV as announced in https://www.nutanix.com/2018/06/05/finally-can-talk-sap-hana-nutanix/, and we are working full steam ahead on the last steps to get production support. It was and continues to be one hell of a journey, and I just wanted to give you guys a bit of insight into what it is like working on the engineering side of the Nutanix platform, and what a project like this entails.

I want to finish with a special thank you to (in no particular order), Rainer, Alexander, Kasim, Malcolm, Jonathan, and the extended team. It’s been one heck of an experience! 🙂

General

Staying in touch while traveling to the USA

Bas Raayman

Devang Panchigar over at storagenerve.com posted an article about traveling abroad from the USA and how you can stay in touch. I found it to be good, but it lacks some points for people who are visiting the US and originate from a different country, so I decided to write my own post and share some experiences.

One thing I want to share is something that my dad has taught me. He has been in shipping and forwarding almost his entire life and was on the road a lot. He always said that when you are travelling it boils down to the three main things, and I think that is good enough to share even though it’s not that relevant for staying in touch.

Three things that are most important:

  1. Passport / ID
  2. Medication
  3. Wallet

In Dutch and in German I always refer to them as the “three P’s” which would stand for “Pas, Pillen, Portemonnaie”. Even if you forget your cellphone, lose your bags, forget your tickets or something else happens. These are the main things to always look after, all else is a piece of luxury.

Making & receiving calls in the US

Most of the people I know have a cellphone of sorts, but aren’t necessarily aware of mobile networks that are used throughout the globe. Basically there are two main standards out there, one is called “Global System for Mobile Communications” (GSM), the other is called “Code Division Multiple Access” (CDMA). GSM seems to have a more dense coverage outside of North America (a couple of areas Asia are an exception), whereas CDMA is the common standard in the US.

Unfortunately both systems are not really compatible, and a lot of the folks from the US are wondering why the people using a GSM are going through the trouble of inserting and swapping out SIM cards. In this part I will just assume that you are currently using a GSM cellphone and are looking for an alternative.

Basically you have two options. One is getting a CDMA cellphone in the US. Since it’s uncommon to have the equivalent of a SIM-card, called a R-UIM card, for CDMA networks in the USA, you would need to get a phone that is linked to one carrier. If you want to use this option, it is important to check which carrier offers the best cellphone signal in the area that you will be visiting. Remember that signing up for a contract will be quite hard without a permanent address in the US, but perhaps you are able to work something out with friends/colleagues/family that you may know stateside. The other option is to just get a prepaid phone and use that.

If you plan on using your GSM cellphone while in the US, make sure that you have a phone that is able to use the appropriate frequencies. The GSM-850 and GSM-1900 bands are commonly used in America and in Canada, while the GSM-900 and GSM-1800 bands are usually used outside of those countries. Simply check on the website of your phone manufacturer to see which bands your mobile phone supports. If you are lucky it will actually be all four bands (commonly referred to as a quad-band cellphone) and you should be set to go. Since GSM is not a very common standard in the US, you will have a good chance of finding yourself using either Cingular, AT&T Wireless or T-Mobile USA.

Also, as a last tip, you could consider registering for a Google voice number and have it forward your calls to the prepaid number while you are in the USA, and forward calls to your regular cellphone number while you are back home. Just know that to change the settings while outside of the United States means using a proxy. This due to the fact that Google voice is only available in the US, and currently requires an invite. It’s slightly tricky to set up, but it could be a useful alternative.

Roaming charges

One thing to be extremely aware of when using your cellphone in the US is that you will be faced with roaming charges. Roaming is basically something that will allow you to use your phone on a network that is different from your own provider, or use your own provider in for example a different company. As you will probably already have guessed, this doesn’t come for free. Actually, it usually comes for quite the price. I would highly recommend checking the exact details with your carrier, but it is quite common that you will pay to receive calls. Basically the person making the call will pay for the cost up to the border of the country the caller is in, and you will pay for the call from that country up to the US. Check if it might make sense to upgrade your plan to include a flatrate to call to the US.

Another “feature” about roaming charges was introduced by using smartphones. These usually tend to rely heavily on a data connection, and most people find themselves having a flatrate for data connections. But be warned, this is usually not a flatrate outside of your own country! Things can get quite expensive really fast when you check your e-mails on your phone from a different country. Data is usually charged in chunks of 10 Kb and bills over €1.000,- can accumulate really fast. Turn data roaming off to prevent this, get a different plan, or another alternative could be to get a prepaid SIM in the US and use that to use a data connection. SIM cards from the US can for example be found on Ebay and will set you back around €5,- which might be a good alternative.

Charge it!

Having a lot of gadgets with you like your laptop, digital camera, cellphone means that you usually need to charge them on a regular basis. Easiest thing to do is check if they are able to automatically switch to the correct voltage. You will normally find something like 100v-240v written on the adapter or the device itself. If that is the case, you are in luck. You would just need a simple plug or socket converter and you can plug the device directly in the the socket once you are stateside.

If the device or adapter is not able to switch to the correct voltage, you might want to consider a so called “step up voltage converter”. I’ve had one with me on all my trips and it has been of great use to me. Just make sure that the output wattage is suitable for the device you are connecting to it.

Additionally having an extension cable and a USB cable can be of use. Lot’s of things can be charged via USB these days and it’s just handy to have one of those cables in your luggage. If you do happen to forget something, you can usually find an electronics store like Fry’s or BestBuy and get what you need there.

Being online

This topic has been covered in part by the roaming piece above. However, when using things like EDGE/UMTS/EVDO you are highly dependent on the reception of your cellphone. Having a Wi-Fi connection is a good way to reduce any roaming charges you might have and will get you a more stable connection. Besides that it’s usually also the faster option if you want to for example send over some pictures.

Usually hotels and motels will either offer you the option of using their Wi-Fi connection (for a surcharge) or have something like an internet PC. A laptop can be of use, and thankfully a fair amount of the hotels you will encounter will be able to offer you free Wi-Fi. If you are in doubt assume the worst and be prepared to pay for the connection or to be confronted with a screen that will have you enter your credit card or room details to charge you for the connection. One useful site to check out is wififreespot.com that will give you a rough overview of free Wi-Fi hotspots in the US. Pack an ethernet cable if you want to be on the safe side.

So in short?

Realize you are visiting the US and not a war zone. Usually you will be able to find everything you need when it comes to being online and staying in touch. Some advance planning will help you make your stay a bit easier. Things like a foreign SIM card or the voltage converter are a good help but not a necessity. Inform yourself about the roaming charges to avoid bad surprises. Oh, and most importantly, have some fun while you are there, be it a private or a business trip.

Update – March 30th 2010

Well, I am now back from a vacation in the US, where it was possible for me to use my own advice. And what do you know, one of the things that seemed to work so well isn’t actually working at all, but this was not a real problem.

I got the AT&T SIM card, filled it up and I could be reached by phone without a problem. The Google voice trick worked like a charm for me. The part that didn’t work was using the data part of the prepaid card. I tried it with an iPhone 3G and a BlackBerry Bold 9000 and both were unable to go online, both with the EDGE and 3G network that both phones found. The problem? According to an AT&T representative, AT&T uses proprietary cellular frequencies for their prepaid data plans in the US, which is why non-US phones simply aren’t able to get online with their prepaid cards. The only good thing? I was able to use the AT&T WiFi hotspots without any additional charge. I will be trying out the same trick with a T-Mobile prepaid card when I visit the US again next week and updating once more with my results.

As pointed out by Matt Simmons, you will have it easy when you visit one of the fast food chains and diners while you are there. A lot of chains like Denny’s, Mc Donalds, Burger King, Starbucks and the likes will offer you complimentary WiFi. I found that adding some of those hotspots to the wireless profiles in my BlackBerry to be quite useful. When driving through the cities my phone would suddenly start to vibrate because it automatically connected to the hotspots and was downloading my e-mails automatically.

Update – July 28th 2010

One more update. The T-Mobile card worked in the same manner as the AT&T card did. Unfortunately this didn’t really work, which might force me to change my strategy the next time I visit. If I do so I will be sure to update with a new post to see what might be the easiest solution.

On a different note, I just noticed via one of the comments that Fabio Rapposelli linked back to this article with some tips of his own on this subject. It includes a good list of tips that will help you in getting some decent food, how to pack and other stuff. Be sure to check out his post on this topic.