Shorts: How to reboot a Cisco UCS 6100 series fabric interconnect

5 07 2011

I actually had this written down in my notes somewhere, and today a colleague of mine called me because he was having some issues. Since it’s not necessarily obvious, and others might be having the same problem, I thought I’d post a quick note here.

To reset the fabric interconnect on a Cisco UCS 6100 series, connect via your serial cable or use putty to ssh in to your fabric, and use the following commands:

connect local-mgmt [enter]
reboot [enter]

Obviously you leave out the space after the command, and the [enter] should be replaced by you actually pressing the enter key. Not sure if this is of help to folks out there, but I figured it’s more useful out here than in my private notes.

Update – July 10th:

Dan gave a good hint in the comments, that you might want check if the fabric interconnect you are trying to reboot is the primary or the subordinate, so here goes.

Connect to your fabric interconnect just like you did in the top example, and enter the local management:

connect local-mgmt
Now, check to see the state of the fabric interconnect you have attached to:
show cluster extended-state

This will give you a bunch of output, but there is one part that you are interested in more than anything else:
B: UP, SUBORDINATE
A: UP, PRIMARY

B: memb state UP, lead state SUBORDINATE, mgmt services state: UP
A: memb state UP, lead state PRIMARY, mgmt services state: UP
heartbeat state PRIMARY_OK

At the prompt, you can see (and hopefully you already knew to begin with) to what node or member you are connecting to. If you are at the node that is in a subordinate state, use the following command to change the state of the node you are rebooting to subordinate:
cluster lead a
or
cluster lead b
Where a or b is the node that you want to become subordinate. After that you can reboot the respective member





Gestalt IT Tech Field Day – On Cisco and UCS

14 04 2010

There are a couple of words that are high on my list as being the buzzwords for 2010. The previous year brought us things like “green computing”, but the new hip seems to be “federation”, “unification”. And let’s not forget the one that seems to last longer then just one year, it’s the problem solving term “cloud”.

Last Friday (April 9th), I and the rest of the Gestalt IT tech field day delegates were invited by Cisco to get a briefing on Cisco’s Unified Computing System or in short “UCS”. Basically this is Cisco’s view that builds on the notion that we are currently viewing a server as being tied to the application, instead of seeing the server as a resource that allows us to run that application.

Anyone in marketing will know that the next question being asked is “What is your suggestion to change all that?”, and Cisco’s marketing department didn’t disappoint us and tried to answer that question for us. The key, in their opinion, is using a system consisting of building blocks that allow me to to give customers a solution stack.

As the trend can be spotted to go towards commodity hardware, Cisco is following suit by using industry standard servers that are equipped with Intel Xeon processors. Other key elements are a virtualization of services, a focus on automated provisioning and unification of the fabric by means of FCoE.

What this basically means is that you order building blocks from Cisco in the form of blade servers, blade chassis, fabric interconnects and virtual adapters. But instead of connecting this stuff up and expanding my connectivity like I do in a standard scenario, I instead wire my hardware depending on the bandwidth requirements and that’s pretty much it. Once I am done with that, I can assign virtual interfaces as I need them on a per blade basis, which in term removes the hassle of plugging in physical adapters and cabling all that stuff up. In a sense it reminded me of the take that Xsigo offered with their I/O director, but with the difference that Cisco uses FCoE instead of Infiniband, and with Cisco you add the I/O virtualization to a more complete management stack.

The management stack

This is in my opinion the key difference. I can bolt together my own pieces of hardware and use the Xsigo I/O director in combination with VMware and have a similar set-up, but I will be missing out on one important element. A central management utility.

This UCS unified management offers me some advantages that I have not seen from other vendors. I can now tie the properties to the resources that I want, meaning that I can set up properties tied to a blade, but can also tie them to the VM or application running on that blade in form of service profiles. Things like MAC, WWN or QoS profiles are defined inside of these service profiles in an XML format and then applied to my resources as I see fit.

Sounds good, but…..?

There is always a but, that’s something that is almost impossible to avoid. Even though Cisco offers a solution that seems to offer some technical advantages, there are some potential drawbacks.

  • Vendor lock in:
    This is something that is quite easy to see. The benefit of getting everything from one vendor also means that my experience is only as good as the vendors support is in case of trouble. Same thing applies when ordering new hardware and there are unexpected problems somewhere in the ordering/delivery chain
  • The price tag:
    Cisco is not know to be cheap. Some would even say that Cisco is very expensive, and it will all boil down to one thing. Is the investment that I need to make for a UCS solution going to give me the return on invest? And is it going to do that anytime soon? Sure it can reduce my management overhead and complexity, sure it can lower my operational expense, but I want to see something in return for the money I gave Cisco and preferably today, not tomorrow.
  • Interoperability with my existing environment:
    This sort of stuff works great when you are lucky enough to create something new. A new landscape, a new data center or something along those lines. Truth is that usually we will end up adding something new to our existing environment. It’s great that I can manage all of my UCS stack with one management interface. But what about the other stuff? What if I already have other Cisco switches that are not connected to this new UCS landscape? Can I manage those using the built in UCS features? Or is this another thing that my admins have to learn?
  • The fact that UCS is unified does not mean that my company is:
    In smaller companies, you have a couple of sysadmins that do everything. They install hardware, configure the operating system, upload firewall policies to their routers and zone some new storage. So far so good, I’ll give them my new UCS gear and they usually know what goes where and will get going. Now I end up in the enterprise segment where I talk to one department to change my kernel parameters, a different to configure my switch port to auto-negotiate and the third one will check on the WWN of my fibre-channel HBA to see if this is matching to the one configured on the storage side. Now I need to get all of them together to work on creating the service profiles, although not all will be able to work outside of their knowledge silo. The other alternative would be to create a completely new team that just does UCS, but do I want that?

Besides the things that are fairly obvious and not necessarily Cisco’s fault, I think that Cisco was actually one of the first companies to go this way and one of the first to show an actual example of a federated and consolidated solution. Because that is what this is all about, it’s not about offering a piece of hardware, it’s about offering a solution. Initiatives like VCE and VCN only show us that Cisco is moving forward and is actually pushing towards offering complete solution stacks.

My opinion? I like it. I think Cisco have delivered something that is a usable showcase, and although unfortunately I have not been able to actually test it so far, I do really like the potential it offers and the way it was designed. If I ever get the chance to do some testing on a complete UCS stack, I’ll be sure to let you know more, but until then I at least hope that this post has made things a bit clearer and removed some of the questions you might have. And if that’s not the case, leave a comment and I will be sure to ask some more questions on your behalf.

Disclaimer:
The sponsors are each paying their share for this non-profit event. We, the delegates, are not paid to attend. Most of us will take some days off from our regular job to attend. What is paid for us is the flight, something to eat and the stay at a hotel. However as stated in the above post, we are not forced to write about anything that happens during the event, or to only write positive things.








%d bloggers like this: