XenServer Switch Ports Configuration Best Practices

Old_switch
Finally I have found it! Citrix's XenServer switch configuration best practices document.

While everyone in the world has blog posts and documentation regarding how to set up and configure bridged networks in Xen, they hardly ever go into the physical switch configuration required.

This is the document that you will need to pass along to your friendly Network Administrator, as they will more than likely not be familar with networking for Xen as its much different from networking for Vmware ESX.

http://support.citrix.com/article/CTX123158

The contents of the document above are outlined below.

Change the following options on the switches for XenServer ports:

  1. Enable PortFast on XenServer connected ports.
    PortFast allows a switch port running Spanning Tree Protocol (STP) to go directly from blocking to forwarding mode by skipping the learning and listening modes. PortFast should only be enabled on ports connected to a single host. Port must be an 802.1q trunk port if you are using VLANS and the port must be in access mode.
    Ports used for storage should have PortFast enabled.
    Note
    : It is important that you enable PortFast with caution, and only on ports that do not connect to multi-homed devices such as hubs or switches.
  2. Disable Port Security on XenServer connected ports.
    Port security prevents multiple MAC addresses from being presented to the same port. In a virtual environment, you see multiple MAC addresses presented from Virtual Machines to the same port. If you have enabled Port Security, it shuts down the port.
  3. Disable Spanning Tree Protocol on XenServer connected ports.
    Spanning Tree Protocol must be disabled if you are using Bonded or teamed NICs in a virtual environment. Spanning Tree Protocol should be disabled because of the nature of Bonds and NIC teaming, to avoid failover delay issues when using bonding.
  4. Disable BPDU guard on XenServer connected ports.
    BPDU is a protection setting part of the STP that prevents you from attaching a network device to a switch port. When you attach a network device, the port shuts down and has to be enabled by an administrator.
    A PortFast port should never receive configuration BPDUs.
    Note
    : When BPDUs are received by a PortFast port, it indicates another bridge is connected to the port, and it indicates that there is a possibility of a bridging loop formation during the Listening and Learning phases. In a valid PortFast configuration, configuration BPDUs should never be received, so Cisco switches support a feature called PortFast BPDU Guard, which is a feature that shuts down a PortFast-enabled port in the event a BPDU is received. This feature ensures that a bridging loop is not formed, because the switch's shutting down the port removes the possibility of a loop forming.

Forcibly Remove a Host From XenCenter via CLI

PicklePD3Welcome back to another XenServer adventure. Today we are going to review the process of removing an orphaned XenServer host from XenCenter via the command line.

A few days ago one of my Xen hosts died due to an issue with the raid controller. In order to revive it I had to rebuild it, and add it back into the pool. However there was a problem; the original entry for this host in XenCenter appeared in red, which indicated that it was disconnected, and another entry for the same exact host with the same exact hostname appeared next to it in the host list for this particular cluster. I was unable to remove the orphaned host via XenCenter as there was no remove option.

I was in a pickle.

However I was able to find a solution.

First and formost it appears that there are actually two ways to do this; the first one being from the Windows command line where you are running Xencenter, or on the master node in the cluster.

This example is from the Windows CLI. Note that the UUID that is output to the command line is the UUID of the Host that is unreachable — the orphaned host.


C:\Program Files\Citrix\XenCenter>xe -s <Hostname_of_master> -u root -pw <root_password> pool-sync-database
You attempted an operation which involves a host which could not be contacted.
host: 560e233b-0e8b-4c2f-a641-cf1876630a6b (virt04.atlc1)

C:\Program Files\Citrix\XenCenter>xe -s <Hostname_of_master> -u root -pw <root_password> host-foget uuid=560e233b-0e8b-4c2f-a641-cf1876630a6b
WARNING: A host should only be forgotten if it is physically unrecoverable;
WARNING: if possible, Hosts should be 'ejected' from the Pool instead.
WARNING: Once a host has been forgotten it will have to be re-installed.
WARNING: This operation is irreversible.
Type 'yes' to continue
yes

Returning to XenCenter I found that the orphaned host entry had been removed.

 

How to Create A Dedicated Storage NIC in XenServer

Wickenburg-networkIn Vmware ESX, when using NFS storage, you are required to create a separate and additional vmkernel portgroup to access your NFS storage. This way ESX Management traffic travels over one port group and NFS traffic travels over another.

In Xenserver the concept is similar; however executed much differently.

First and foremost you need to be aware of a few limitations in XenServer. While XenServer does allow you to create a dedicated nic (or bond) for NFS traffic, this nic must remain unmanaged by Xenserver. In contrast, a XenServer's management traffic travels over its "Management interface", which obviously has to be an interface that is managed via XenServer.

Allow me sum this up. You cannot share a nic or nics between management traffic and NFS traffic. This means that you are going want to create a bond for management traffic and a bond for NFS traffic you are going to need 4 free interfaces on your Xenserver box.

Now it is possible to use VLAN tagging and route your Management traffic and virtual machine traffic over the same physical interfaces. However its important to know that XenServer does not support VLAN tagging on the Management interface, so whatever VLAN you use for management, it must be the native vlan on the ports configuration.

The information below is from the Admin Guide for XenServer 5.5

Switch ports configured to perform 802.1Q VLAN tagging/untagging, commonly referred to as ports with a native VLAN or as access mode ports, can be used with XenServer management interfaces to place managementtraffic on a desired VLAN. In this case the XenServer host is unaware of any VLAN configuration.

XenServer management interfaces cannot be assigned to a XenServer VLAN via a trunk port.

Bottom line; its probably best to have seperate physical connections for your management traffic and NFS traffic.

Also, according to the Admin Guide, your NFS network should not be routed. See the words below and read them thusly.

"Before dedicating a network interface as a storage interface for use with iSCSI or NFS SRs, you must ensure that the dedicated interface uses a separate IP subnet which is not routable from the main management interface. If this is not enforced, then storage traffic may be directed via the main management interface after a host reboot, due to the order in which network interfaces are initialized."

Ok now that we got all that out of the way, lets actually create our dedicated storage nic.

First we need to get the uuid of the pif (physical interface) that we want to use. Note that this is just an example using a standalone interface.

#xe pif-list host-name-label=<"your_hostname>

Next we reconfigure our pif

#xe pif-reconfigure-ip mode=statc IP=<your-ip-on-nfs-vlan> netmask=<your-mask> uuid=<pif-uuid>

#xe pif-param-set disallow-unplug=true uuid=<pif-uuid>

#xe pif-param-set other-config:ManagementPurpose="Storage" uuid=<pif-uuid>

Alternatively you can use xe pif-forget to remote the interface from Xencenter database and configure it manually via the XenServer Kernel like you would any other interface in Linux, however this could be more confusing in the long run.