Cisco: Principles of Application Centric Infrastructure

Question-mark Attending Cisco Live 2014 in San Francisco this week?

Well, get ready to hear a lot about ACI (Application Centric Infrastructure). It's almost here and its everywhere. And the acronyms, holy crap, get ready to get hit upside the head with of ton of those that you have never heard before. Cisco is only a close second to VMware when it comes to their love of acronyms.


Here are a couple of them that I have heard for the first time today.

  • APIC
  • VTEP
  • EPG

Anyway – here is Cisco's primer on ACI. Its a good read and might help you wrap your head around a few subjects before you head full bore into your sessions.

Related articles

Announcing Cisco UCS Director 5.0 – with Support for Application Centric Infrastructure
Software Defined Network Services – A Sneak Preview at Cisco Live
HomeLab: Upgrading Cisco IOS Via tftp on RHEL

Cisco UCS Failover Via 6248 CLI

Ucs_6248_lg1Need to fail over Cisco UCS Manager from one 6248 to another. Oh did I mention that you are in a hurry because you are going to have to sift through pages and pages of pdfs to get the information. I found this out the hard way. Anyway, out of the kindness of my heart I have documented this process below.

First log into one of your 6248s and figure out which one is the primary by running the command below.

ucs01-A# show cluster state
Cluster Id: 0x4b05e6042b6111e1-0x8c9e547fee4bbf24


Note that UCS01-B is our primary. So log into UCS-B and issue the command below.

ucs01-B# connect local-mgmt

Then run the command to make UCS-A our primary.

ucs01-B(local-mgmt)# cluster lead a
Cluster Id: 0x4b05e6042b6111e1-0x8c9e547fee4bbf24

Note that you must issue the failover command on the node that is the primary, otherwise this happens.

ucs01-A(local-mgmt)# cluster lead a
Cluster Id: 0x4b05e6042b6111e1-0x8c9e547fee4bbf24
request failed: local node is subordinate

During the failover process you will see the output below when checking the cluster state.

ucs01-B(local-mgmt)# show cluster state

Cluster Id: 0x4b05e6042b6111e1-0x8c9e547fee4bbf24 B: UP, PRIMARY, (Management services: SWITCHOVER IN PROGRESS) A: UP, SUBORDINATE,

(Management services: SWITCHOVER IN PROGRESS) HA NOT READY Management services: switchover in progress on local Fabric Interconnect

How to Reset the Cisco CIMC via the Command Line

General_Ursus_02_by_b_mazeLong ago I wrote a post on how to reset the ILO on a HP Proliant server via the command line, as this was often necessary to get a slow responding ILO responding again.

Cisco has included similar functionality in the CIMC CLI, instructions below.

esx01# scope cimc
esx01 /cimc # reboot
This operation will reboot the CIMC.

The scope command above allows you to attach to a specific module, so in this instance we are attaching to the CIMC module.

Note: The two links below have quite a good bit of information on the CLI



Cisco UCS Visio Stencils

Visio-iconCisco UCS Visio Stencils — Use the link below and scroll down to “Unified Computing and Servers”, and select “Unified Computing System” and download the zip file.

Currently (as of 12/2014) you can cut to the chase and use the direct link below:

Cisco UCS Firmware Best Practices Document


Not sure that I have ever run into a product with as many separate support documents as Cisco UCS – here is another that appears to be useful (the jury is still out on this one). Anyway the link is below

The link below will take you to the Firmware update and install guides — note that Cisco has not upgraded the documentation to reflect an upgrade from an older 2.0 firmware to a newer 2.0 firmware.


How Not to Assign KVM IP Addresses Via Cisco UCS Manager

Boxing-gloveAfter a few hours poking around a newly deployed UCS cluster trying to get some basic profiles created and assigned. I realized that I had actually no idea how the KVM is actually supposed to work inside the UCS cluster. Which is funny as this was a subject that we touched on during my DCUDC class. Apparently we did not touch on it enough.

Anyway, before I get ahead of myself, lets review the gear in this cluster.

2 5108 chassis
7 B200 M2 blades with 2104 IOMs
2 6248s Fabric Interconnects

Now in my network all lights out management ips (ilos, ipmi, etc) are all on one particular vlan, which for the purpose of this post we will call VLAN 100. Non application related infrastructure equipment (servers, virtual hosts) are on another vlan, which we will call VLAN 200. So when the Fabric Interconnets were deployed, I gave them each an ip address on VLAN 200. And once UCS Manager was up and running, I created a KVM ip address pool of unused ip addresses on VLAN 100. Well guess what, this is wrong.

Routing for the KVM addresses is done through the management interfaces on the Fabric Interconects, so unless you are using vlan tagging, your KVM pool must be on the same vlan as the ip addresses assigned to your Fabric Interconnects.

But wait, why is this?

I thought that I could even assign private 192.168.x.x ip addreses to the KVMs as they were only supposed to be managed via the UCS Manager, but this also incorrect.

Navigate to one of your working KVM ip addresses in a web browser and you can access the KVM of the blade outside of UCS Manager. Nice, which is how I actually expected this to work. 


Note that I find it rather dumb to have my KVM management ips and Fabric Interconnects on the same vlan as a rule, however since this is how its supposed to work I am going to have to let that one go.

Now, the fact that you can navigate to a specific KVM IP address via a web browser also makes the idea of using a pool of ip addresses silly. Would you not want to hard code the KVM ip address in the service profile so that you always know which server's console you are logging into? Dunno, I am still working on figuring that one out.


Cisco UCS Manager (2.0) Configuration Guide

688537This is the document that you will want to have access to in order to configure your USC Cluster Manager.

Cisco UCS Manager 2.0 Configuration Guide PDF Download