If you’ve purchased this: https://www.fs.com/products/48948.html – or something similar on fs.com, than you’ll likely plug this twinax in and you’ll never get the link to come up, even though your listed device is there in the compatibility matrix. There is a resolution: set your duplex and speed manually on both ends.

You’re welcome

I  have seen this around the Internet and a lot of the information is old, and incorrect at the date of this blog post. Thus, what I will link here is the official support matrix Cisco provides for the Cisco Nexus 9000 series switches and the support Cisco Nexus FEX models. One thing to note, if you hover over the “YES” in the cell you choose, it will provide you with the supported connectivity options.

Now, to answer the question above, yes, you can connect the Cisco Nexus 2348UPQ to the Cisco Nexus 93180YC-EX using 40GbE from the FEX uplink ports to the 40GbE ports on the 93180YC-EX. If you are reading the guidelines and limitations where it states something like this “FEX is not supported on 40GbE ALE uplinks” then you need to understand the Cisco Nexus 93180YC-EX does not use the older generation ALE, instead it uses the newer, 2nd generation, Cloud Scale ASIC LSE (Leaf-and-Spine Engine); thus, this limitation does not apply.



If you live in the central Texas region, or like me live in Temple, Texas, you likely live on soil not too well suited for much of any plants, or even grass; however, this does exclude those who’re in the “black lands”, you’re incredibly lucky. For the rest of us, I’ll briefly outline some basic care information here and what you should do to winterize your lawn to keep it green, healthy, and keep the weeds out. I am going to make the following assumptions before continuing

  • You have some form of Bermuda grass, likely Tiff-419 “builders grade” garbage
  • You’re in the Temple, or central Texas region, where you have sandstone about 4-6″ below topsoil
  • You’re planning on continuously re-seeding with a better form of Bermuda, like Pennington Texas Bermuda or Sahara


Before you go spending hundreds, or rather thousands, of dollars on stuff for your lawn I strongly suggest you get soil samples and send them to you Texas A&M for soil analysis, click here for more information or call to get more information. If you’re a homeowner, use the “Urban and Homeowner soil sample information form”. The basic test is currently $10 for standard analysis for basic information on what fertilizer to use; however, I strongly advise against this cheap test as you’re looking for more comprehensive information. The region we live in has wildly varying organic matter levels and this is very important to understand because it’ll determine your top-dressing needs, which also includes adding compost and manures. Finally, because this soil sucks for drainage you’re going to want a texture analysis done, so you can get a detailed understanding of your soil and what methods you’ll need to employ to fix areas of low water intake and too high water retention. When you look at what I write above you can find what you’re looking for, but here are the key words:

  • Fertilizer recommendations
  • Organic Matter analysis
  • Top-dressing with compost and manure
  • Texture analysis

Thus, the test with everything you need is the #12, on the current SU17 form, which runs for $84/sample. This seems like a lot of money; however, quality fertilizer in a 35-50lb bag runs about $60, quality limestone in a 40lb bag runs about $15, and Ironite isn’t cheap, and without a soil analysis you’re going to be throwing money at your lawn at a problem you have no idea how to solve. This soil analysis is the key and this is why I am talking about it first because without it I really cannot help you much and you definitely cannot help yourself.


Alright, you can’t do anything without the proper tools and good tools are not cheap, and cheap tools are not good. I won’t get into too many details here about the items I am recommending, take them or leave them, but always buy quality as it’ll last longer.

  • Lawn Mowers:
    • I recommend Either the Honda Brand lawn mowers that have a bag/mulch option
    • I also recommend the Husqvarna lawn mowers, powered by a Honda mower if possible, with a bag/mulch option. These usually have cleaning ports and 4WD, big pluses!
  • Trimmers/Weed Eaters:
    • I recommend either the Stihl or Husqvarna brands only. I have the 4-stroke Husqvarna and love it!
  • Shovels:
    • Spaded shovel – Used for digging larger holes
    • Transfer shovel – Flat, used for transferring materials
    • Trench shovel with pointed edge – The Razor brand makes an excellent one, at Home Depot I think, I may be incorrect on the brand name
  • Mattock pick – You’ll need this for digging. The pointed edge is what you actually use to dig and the trench shovel is then used to remove the broken materials
  • Drop spreader – This is just my preference, you can use a broadcast spreader, I just prefer a drop
  • Rakes:
    • Standard Metal rake – No need to explain
  • Aerators:
    • Core Aerator – This will be a key in getting your yard where you want it, there are two types:
      • Manual – This is a device with two holes, pointed at the end, and you manually push with your foot to extract plugs. You should have one to do small areas every so often
      • Gas Powered – These run about $1200-$5000, so not cheap. However, you can rent them and I do recommend using one to save time and your back when you’re doing your entire yard, preparing for over-seeding and top-dressing. Average daily rental fee is between $50-90/day.
  • Soil pH tester – I recommend the: Kelway PHD Soil PH Meter. This is an excellent, and very accurate tool, to keep around to inspect your lawn’s pH levels. Helpful if you’re seeing an area with not so great growth or having some growth issues.
  • Pump sprayers:
    • 4-Gal backpack – Get a high quality model, you’ll want this for when you need to spray down larger amounts, saves time and your shoulders from carrying around 2-3 gallon handheld sprays for larger projects
    • 2-3 Gal Hand sprayer – Multiple ones for things like: Fertilizer, Weed Killers, and other chemical types you don’t want to mix. Only use for smaller/quick jobs


You’re going to need a variety of materials, this includes chemicals/fertilizers and other things like:

  • Fertilizers:
    • Synthetic fertilizers for help with starting a new lawn/fixing a bad lawn – This will vary on your soil analysis. Both pelleted and liquid fertilizers
    • Ironite – Once again, based on your soil analysis, will determine if your lawn needs iron, mine did.
    • Organic pelleted fertilizer – They sell this at Lowes, it is kind of pricey; however, I do recommend slowly coming off synthetics and moving to this
  • Compost/Manure/Soil:
    • Garden soil – Generally, garden soil is “thicker”, not really the best thing for a top-dressing operation, all depends on what is in it. You’ll want to buy this as-needed.
    • Top soil – This should be a “finer” product and is used for top-dressing a lawn, buy this as-needed
    • Compost – Very high quality compost only. Basically, it should be as “fine” a product as possible with no larger chunks in it. Buy as-needed
    • Double ground dairy cow manure – This is “the stuff” to use for top-dressing, in my humble opinion. This is usually mixed with top-soil and the manure is double ground, making it an incredible “fine” product that’ll spread evenly and nicely for top-dressing operations while also providing the benefits of adding nutrients for your lawn, requiring less and less synthetic fertilizers
  • Bug control:
    • Diatomaceous Earth – I place this along my entire border of my home, about 2-3 wide line, decent amount of height. This will kill anything which comes into contact with it and it is natural and safe
    • Grub/bug control pellets – I prefer the Bayer Advanced formula, I discovered a really bad infestation of grubs across my entire front lawn. Once I applied this, a week later they were all dead.
      • Bifenthrin – I have a dedicated sprayer for this and use it along all my exterior walls, crevices, shrubs/bushes, and my entire lawn. I apply this a little stronger than recommended and add a little 80/20 surfactant to the mix. This will control almost everything; however, I recommend incredible precaution when using this. Do not spray onto plants which attract pollinators, you can spray it below the plants on the ground to prevent/kill anything from crawling onto the plant, but be careful as we’re not targeting pollinating bugs, which are good for us. This has controlled: mosquitoes, flies, ants, roaches, and fleas.
  • Weed control:
    • Glyphostate 47-50% concentrated liquid – This is the key ingredient in “Round Up”, but you don’t have to buy Round up if you don’t want to, you have options. Even with Round Up, it is recommended to use an 80/20 surfactant when mixing this to ensure the product sticks to the weeds surface and can penetrate the leaf.
    • 80/20 surfactant – You’ll find you’re mixing this with a lot of different chemicals; however, you’ll mostly mix it with Glyphoste at a rate of 2oz per gallon. This helps break the surface tension of the water/chemical mix, which is critical in penetrating the weeds’ green surface. As a rule of thumb, the waxier the plant the higher the strength of surfactant you’ll need. A plant with a matte, non-shiny surface, won’t need as much.
    • 2,4D Amine – This is a general purpose weed control liquid you can use on Bermuda grass lawns to control a wide variety of weeds. When mixed and applied properly, to healthy Bermuda grass, this will not impact the grass.
  • Seeds:
    • Bermuda Grass – I prefer Pennington Texas Bermuda or Pennington Sahara Bermuda grass. Sahara is much more expensive, but if you go that route, you’ll be rewarded with a very drought tolerant grass. Their Texas Bermuda is almost as drought tolerant for a central Texas lawn, and much cheaper
    • Annual Rye-grass – This is your “cool season” grass. You’ll likely want to over-seed this in late October here in central Texas. I do recommend a slight top-dressing for this overseeding, but not entirely necessary, depending on your lawns’ needs.
  • Misc soil and lawn conditioning:
    • Pelleted lime – I recommend the Pennington brand pelleted lime as it works quickly and is very effective while not leaving a “slimy/milky” appearance. You will likely find the “Soil Doctor” brand which is very inexpensive; however, I have found this doesn’t do a great job at helping bust up compacted areas and is hit-or-miss with adjusting the soil pH levels. Sure, it is cheap, but you get what you pay for. If Pennington isn’t available, Sta-Green is also a great choice too.
  • Fish and Seaweed fertilizer – Yes, this stuff does stink, but I promise the benefits of the micro-nutrients and well worth it.
  • Human urine for fertilizer – We’ll discuss this later, but do some research on the topic, you’ll likely be amazed.


First, you need to understand Bermuda grass, of any type, is not a cool-season grass; however, it does grow back in the spring. This being said, during the cool months your lawn will look brown, some grass may die, and this leaves a very open environment for weeds to take over. Thus, to keep both a nice green lawn and maintain an environment which keeps out weeds, you will need to over-seed with Annual Rye-grass. Here is what you need to do for Winterizing your central Texas lawn:

  1. Mow your Bermuda grass to about 1.5-2″ – I highly recommend you bag it so you don’t have to work harder, later.
  2. De-thatch your entire lawn – I would use a gas-powered de-thatcher as this is incredibly hard work.
  3. Core aerate your lawn – The plugs which come up are likely going to be a lot of clay/sandstone plugs REMOVE ALL OF THE PLUGS, DO NOT LEAVE THEM ON YOUR LAWN
  4. Top-dress your lawn – Generally, a 1/8th – 1/4th” thick top-dressing is all you need, but you can take this moment to fill in low spots. Since this is winter, and if you can find double-ground dairy cow manure, I would use that over typical top-soil if available/affordable.
  5. Over-seed – Set your spreader to the appropriate setting for over-seeding and spread your seed
  6. Fertilize – Based on your soil analysis and the recommendations of the Annual Rye-grass seed you have chosen, spread your fertilizers.
  7. Rake over entire yard – You want to ensure the seed is no greater than 1/4th” below the surface. You can take the back end of the rake and just drag it over the lawn a few times, this will ensure good coverage.
  8. Ensure your lawn stays moist – You’ll likely want to water about 3-times per day, do not over-water as this can cause root-rot and soil/nutrient erosion
  9. Use Straw or compost over the top – If you want to ensure optimal moisture retention, lower your water consumption, and provide additional, natural, nutrients to your lawn you can place either high quality compost or straw over the newly over-seeded lawn. For either of these, do not spread it on so it entirely covers everything, you’ll rob the seed and grass of sunlight and not keep in too much moisture, which can introduce mold and fungus, screwing everything up. You’ll just want a very light/thin coating on the top, not you don’t want it to provide 100% coverage. I’d say you should still see about 60% of your original surface.

Allow the seeds to germinate, root, and sprig to a point where you can mow, remember, you don’t want to disturb the surface area so excessive walking, and also mowing, while it is growing can disturb the process. Follow the recommendations on the bag of seed you have chosen to use. Once established you’ll want to dial back your watering to appropriate times/frequency.


More to come later as I update this…

You’ve likely been where I am, and usually there quite often, looking for a specific PDF which contains information you need to reference, but you can’t find the darn PDF! Unless you’re incredibly detail oriented and place everything where you’ll find it 100% of the time, you’re likely to lose it. However, for those on Windows who have CYGWIN installed, or even if you’re using Linux and just want to know how to do the same thing, allow me to enlighten you on how to use two, well three if you think about it, tools for searching your entire machine(s) for that specific file.

On a Windows machine you’ll want the following:

  • Base utilities packages (to ensure you have find and xargs)
  • pdfgrep

So, you know you remember a specific keyword, or phrase, and you just need to search for all of your PDF files and return on the terminal the output with the filename. I recommend using xargs in conjunction with find, instead of exec. Why? Well, exec will, for each file the find command returns, run a separate process; thus, its slow and eats processor time, xargs does not. So, lets say you want to search your entire computer’s C: drive in CYGWIN:

find /cygdrive/c -type f  -name "*.pdf" -print0 | xargs -0 pdfgrep -i "symmetrical"

That is it, it may take a long time, but for any .pdf document it finds, xargs will search for the string “symmetrical”. Your mileage will vary based on your machines resources, but if you can close everything and leave only that running, say at lunch time, it should execute quickly and, hopefully, you’ll find your long lost PDF document.

Never thought I would be writing about how to utilize IPv6 in 2017 because of all the excellent material on the Internet; however, I have discovered a few things:

  1. There are still technologies which have horrible support for IPv6 (including new stuff)
  2. There are people still resistant to implementing it
  3. There is material on the Internet which shows up early in Google searches which references deprecated standards

Without any further delay, I am going to outline a few items you should keep in mind when deploying your IPv6 network:

Subnet mask size

In IPv6, barring a few exceptions like point-to-point links, you should always utilize a /64 for each deployed subnet. Why? Well, if you wanted to use DHCPv6 you’ll find Microsoft’s implementation won’t even allow you to change from a /64 and even a DHCPv6 server in Linux, while it will actually run with a mask larger than a /64, it will only hand out a /64. Also, you’ll find the use of anything larger than a /64 breaks a lot of the auto-discovery mechanisms in the switch/router, namely around EUI-64, and just doesn’t make sense.

What subnet size should I get from ISP/provider/administrator?

If you’re not going to “own” your IPv6 network, that is you’re not getting an assignment with an ASN to advertise, you’re either looking to obtain a public block of addresses for use and/or you’re internal and need your networking administrators to assign you a prefix which you can further subnet yourself. There is a standard most follow to assign prefixes to “customers”.

An ISP, for instance, may have numerous /32’s (or maybe a bit larger) assigned to them for their use to distribute to customers. Lets call them ISP and you work for “company” and you’re an internal IT organization within “company” who uses “ISP”. Your company would request from the ISP an IPv6 block assignment. From one of the ISP’s /32’s you’ll get, lets say, a /48 just for the hell of it. This is how your company can break it down internally for assignment:

  • 65,536 =  /64’s
  • 32,768 = /63’s
  • 16,384 = /62’s
  • 8192 = /61’s
  • 4096 = /60’s
  • 2048 = /59’s
  • 1024 = /58’s
  • 512 = /57’s
  • 256 = /56’s
  • 128 = /55’s
  • 64 = /54’s
  • 32 = /53’s
  • 16 = /52’s
  • 8 = /51’s
  • 4 = /50’s
  • 2 = /49’s

How your company doles these out, is up to them. However, almost no one is going to just directly carve out /64’s from the assigned /48 block, that is stupid. Generally, you’re looking to summarize and aggregate where possible throughout your network and we’ll assume you’re in location “A” at “company”.

We’ll go ahead and assume the company has decided each location is assigned a /58, which gives each location a total of 64 available /64’s to use. As you see, no different than standard IPv4 in the sense of ensuring proper aggregation, except now you’re no longer having to worry about the size of a VLAN’s subnet mask, you’ll always use /64.

What about private IPv6 address space?

If you do not want a Globally Unique IPv6 address you can indeed have what is called a “Unique Local IPv6 address = ULA”. There is a guide on how to properly generate these addresses, which includes a variable which references the time and date, along with other factors to ensure absolute uniqueness.

Why does this matter with private address space? Have you ever been involved with a merger/acquisition, or having to aggregate two offices together which use the same private IPv4 subnet range? I need not say anymore because this can be a PITA! Thus, ULA, when done right, ensures this will never happen; however, there is absolutely nothing stopping you from selecting your own, basic, prefix.

IPv6 ULA uses the FC00::/7 prefix, divided into two groups:

  1. fc00::/8 – The idea for this prefix is to be administered by some authority, but no one can agree to it, so just forget about it
  2. fd00::/8 – Is defined for the generation of /48 prefixes only, using the last 40 bits to generate a random, unique, prefix, according to the algorithm in RFC4193

You will want to use option 2 and you can use online generation tools like those from SiXXs or use a tool from another resource, either way, make sure it generates a proper /48 prefix for you and is, by some degree, RFC4193 compliant.

Finally, your company’s IT department is likely to have this /48 already and is almost very likely to have assigned you a prefix according to the same standards for which they’ll dole out their Globally Unique IPv6 addresses; thus, no additional explanation needed.

Get your DNS infrastructure setup for IPv6 AAAA and PTR-record resolution

I won’t delve into this much more other than you absolutely must make sure your DNS infrastructure is setup for IPv6 AAAA-record and IPv6 PTR-record solution or you WILL have issues!

One area to ponder is the hostnames that’ll resolve when you’re in a dual-stack environment. Do you want the same hostname to return on both a A-record and AAAA-record? Well, some say no, some say yes. Me? I say you should discuss this with your vendor to ensure their solution doesn’t have a problem with this, especially in a dual-stack environment. I was told, by co-workers who know more about Vmware vCenter than I do right now, this is a problem and the returned hostnames must be different when using dual-stack based environments.

Always research and question IPv6 support on your devices

This goes for hardware and software vendors, many have made claims their stuff works with IPv6; however, what, if any, testing was done isn’t known and there are a variety of scenarios to consider. For instance:

  • Does it support native IPv6 from installation-to-operation?
  • Does it support dual-stack, from installation-to-operation?
  • How does it handle DNS requests in dual stack?
    • Does the system start with IPv6 AAAA requests and then fails over to IPv4 A-record requests?
    • If so, what is the timeout if a AAAA record is not available and it must try for an IPv4 A-record?
    • Is the order of DNS resolution preference configurable? (Can you choose to have IPv4 A-records first?)
  • What forms of address configuration are available for IPv6? (SLAAC, static, DHCPv6?)
  • What IPv6 address types are supported? (Globally Unique and/or ULA?)
  • Are there specific “sections” of configuration which cannot support IPv6?
    • For instance, in Cisco NX OS, you cannot reference an IPv6 address for use on a vPC peer keep-alive link.

More questions will come to mind, but these are from experience and I can promise you are a lot of reasons why most IPv6 implementations in the enterprise, and data center, fail. Question all vendors!

This is it for now, hope this clears up some stuff for you out there who’re thinking about their IPv6 implementation

DNSMASQ is both a DNS and DHCP server that is quick and efficient to run on Linux systems and is likely already running on your Linux box. If you’re in need of a quick DHCP server to run your environment to serve multiple DHCP scopes for different subnets in your VLAN, of which we all know the best practice is subnet == VLAN == Broadcast domain, then DNSMASQ is your go to guy and I prefer it over the ISC DHCPD server. This quick tutorial will go over the basics of how to get this setup and running and assumes you’re not going to utilize the DNS service.

Create a directory for your DHCP leases file:

sudo mkdir /opt/dnsmasq

Setup dnsmasq.conf:

#Disable the DNS server
#Setup the server to be your authoritative DHCP server
#Set the DHCP server to hand addresses sequentially
#Enable more detailed logging for DHCP
#Set your DHCP leases file location
#Create different dhcp scopes for each of the three simulated subnets here, using tags for ID
#Format is: dhcp-range=<your_tag_here>,<start_of_scope>,<end_of_scope>,<subnet_mask>,<lease_time>
#Setup different options for each of the unique subnets, since default gateways will be different
#The format for this is: dhcp-options=<your_tags_here>,<option>,<option_value> - 3 is router

Once this is complete, enable your DHCP service to start automatically. You should also check your systems firewall/IPTABLES service(s) to ensure you have created rules to allow UDP traffic over port 67 and port 68, or you can just flush your IPTABLES and/or disable your firewall, your choice, this isn't a security blog so I'll leave the choice to you, the person who knows their environment better.

First, allow me to say these indeed to do exist, the RJ-45 based 10GBaseT SFP+ modules, a company called Methode Electronics manufactures both a SFP+ based module and a X2-RJ-45; however, we’ll only really talk about why a RJ-45 based 10GBaseT SFP+ transceiver still isn’t practical for lengths beyond 30m, with present technology.

The issues
The number one issue we have, with the current technology today in 2017, is the number of transceivers required for distances greater than 30m using 10GBaseT SFP+ modules. The incredible number of transistors will consume an enormous amount of energy per port and the heat generated by the operation of such modules will be monumental, to say the least. Also, with distances greater than 30m, the amount of heat generated needs to be pulled away from the circuitry and will require large heat sinks, which will increase the bulk of the switch itself or careful consideration of airflow characteristics around the SFP+ ports, including higher speed and higher volume fans (which in turn would also consume more energy themselves) further increases the power demands of a switch utilizing SFP+ modules for 10GBaseT SFP+ modules. X2 modules are indeed out there, but X2 is a different form factor to begin with and I won’t be discussing this here.

Why do I reference 30 meters?
Why do I reference distances greater than 30 meters (30m)? Two reasons: 1. When people want to look at Cat6a/7 for long haul connectivity (to somewhat come close to the distance of multi-mode fiber optics on OM4 fiber cables) 2. Current technology at the time of this writing actually permits us to engineer a 10GBaseT SFP+ module for distances of up to 30m using about 2.5W of energy per port. Once again, please look up the company Methode Electronics and their white paper on 10GBaseT SFP+ optics, its pretty cool stuff.

Who wants this?
Now, what audience cares about utilizing such stuff as copper for distances at 100m? In the enterprise market you’ll likely never see anyone think about using copper for spanning distances close to 100m, especially in the Data Center where the copper cross-connect is disappearing in favor of 10/25/40/50/100G fiber cross-connects, because the cost of these optics are dropping fast. When I say 40G here, I am also assuming the use of Cisco 40G BiDi transceivers because they allow you to utilize existing LC based fiber infrastructure. However, service providers are still interested in utilizing copper back haul connections for distances for at 100 meters because, if the SFP+ modules are cheap enough along with the cost of laying the Copper, they’ll want to utilize this. You’ll likely see such things as connections at last mile (rather under a mile, a lot) or between two offices or central offices. Once again, price usually always wins; thus, time will tell. So, now you know, why you’re just not seeing mass produced 10GBaseT SFP+ modules on the market.

If you’re looking to use command line variables for scripting stuff you have some predefined variables in the NX-OS environment to use and you can also create your own. For now, I’ll just show you how to use the most common, the switches hostname. In some environments you’ll have to save the output of a show tech file and later on upload it via SCP. However, if you’re doing this to 2 or more switches, you’ll need unique file names to make your life easier. Instead of going to each one, you can just use the variable SWITCHNAME in the file. So, if you’re using a script or something like cluster-ssh, this makes your job easier.

sh tech all > bootflash:///shtech-$(SWITCHNAME)

I realize there is still some confusion regarding Cisco Nexus FEX as it relates to ToR connected FEX, which is a Cisco Nexus 2K FEX with a Cisco Nexus 5K/7K/9K as a parent switch, and the FEX you find in UCS, which we can refer to as “Blade-FEX”. I am going to outline what ToR (Top of Rack) FEX in this blog post, not Blade-FEX, to help bring some clarity around this still confusing terminology. This is also not meant to bring any additional ambiguity, but it is true you can use certain Cisco Nexus 22XX ToR-FEX and “parent” them to a Cisco UCS Fabric Interconnect; however, I would not classify this as Blade-FEX or ToR-FEX, I’d like to coin it with the term “Fabric-FEX”, you owe me $1.00 every time you use this, send it via paypal :). Thus, moving forward, we’re going to refer to a FEX which parents to a Cisco Nexus switch as a ToR-FEX.

Cisco Nexus FEX works thanks to the Cisco pioneered 802.1BR, click here for more information. Now, you don’t have to worry about configuring the gory details of what is essentially VN-TAG because this is all handled with a few simple commands to get your FEX up and running; however, this is just here do you know how FEX works to communicate with the parent switch underneath the sheets.

The logical representation of FEX is broken down like this:

  • Logical Interfaces (LIF) – This is simple, its the Eth1xx/1/X representation on the switch
  • Network Interfaces (NIF) – These are the physical uplinks connecting the FEX to parent, carrying the VN-TAG
  • Virtual Interface (VIF) – This is the logical interface which correlates, in software, to the physical host interface. We we wil discuss this in a minute about why this makes FEX capable of full swap of a failed FEX without reconfiguring the host ports
  • Host Interface (HIF) – These are the physical ports on the FEX which you connect your hosts to. The parent switch assigned each HIF a unique VN-TAG ID, which is roughly correlated to the above Virtual Interface (VIF) assignment.

Here is some output to take a peek at, taken from a Cisco Nexus 9332PQ switch with 2348TQ and 2348UPQ FEX attached:

slot:36, fab_if:160001f4, p_ind:f4010016, p_numelem:1
dev_inst:0, nif_no:16, hif_no:40, nif_ind:160001f4, hif_ind:1f670a00
Eth104/1/42 0x1f670a40 Down Po501 Po501 NoConf

Take notice, this is Logical port: Eth104/1/42 and there is a plethora of information regarding the port, including the HIF numer and the hif_ind. I haven’t referenced anything with Cisco as of yet, but I would believe the HIF no is the unique number assigned to the port, perhaps the VIF, and the HIF_IND may be an index ID, but I’ll investigate later. For now, just take notice that: Eth[101-199]/1/[1-48] is the LIF, which is attached to a VIF, which correlates to the HIF on the FEX. Because FEX attaches the configuration to a VIF, which is also correlated to the FEX ID, you can have your FEX member, say FEX 104, fail completely and all you need to do is just replace the failed FEX, cable it the same way and when the FEX image is downloaded it’ll reboot and continue operation without the need to rebuild the configurations.

Now, you MUST be diligent in understanding the valid UPLINK topology you can configure your ToR-FEX for, in relation to your parent switch. Always review the configuration guide for your specific model of FEX and parent switch to obtain the valid topology. In my scenarios with the Cisco Nexus 9K switches I do a single-homed, host vPC port-channel uplink topology because we can’t do a more elaborate e-vPC design with the 9K switches and our hosts will be attached with port-channels in an active-active scenario.

Finally, the configuration is simple; however, some Cisco documentation is confusing because the wording in some documents states the UPLINK port-channel is LACP enabled; thus, you would assume you configure your UPLINK as an active LACP member. This is wrong, in fact, the best method, at least from my experience with the 9K switches, is to create the port-channel you’ll be using for the UPLINK, no-shut the interface and nothing more, then move into the physical interfaces that’ll be part of this port-channel, no shut the interfaces and just assign them to the port-channel as static mode. Then, move back into the port-channel configuration mode and build your configuration. Below is the basic configuration you need to get your FEX attached to your 9K switch:

interface po500
no shut
interface eth1/21-24
channel-group 500
no shut
int po500
switchport mode fex-fabric
fex associate
no shut -

A note about setting Jumbo frames on those FEX ports. The FEX host ports will assume the maximum MTU based on the UPLINK port-channels MTU assignment. In our environments we aim to have jumbo frames end-to-end and leave it up to the specific host/OS/application to decide on its optimal packet size. Thus, if you set your MTU on the UPLINK port-channel to 2000, your MTU on will be 2000 on your host interface ports on the FEX.

As an update, and to summarize, here are the FEX types to help clear up confusion, these are not “official” terms, but these will help to clear up confusion, I hope:

  • ToR-FEX: A Nexus 2K FEX attached (Parented) to a Nexus 3K/5K/7K/9K switch for extending ports
  • Blade FEX: These modules are installed into a Cisco UCS chassis
  • Fabric-Interconect FEX: These are the same Nexus 2K FEX used for parenting to 3K/5K/7K/9K; however, now you can attach (Parent) these Nexus 2K FEX to Cisco Fabric Interconnects for the purpose of extending ports for your Fabric Interconnects, or providing a different type/speed of port.

There has been some slight confusion and ambiguity around the “single-connection” configuration statement provided by Cisco switches and routers, including SAN MDS switches. As of this writing, Cisco Nexus 9000 NXOS switches on 7.0.3.I5.1 code do not support single-connection in their tacacs host configuration; however, certain MDS switches do. In either case, if you do find yourself wondering here for the answer, let me elaborate for you.

The purpose of single-connection is to multiplex all of your TACACS authentication requests using a single TCP oriented connection from the switch to the TACACS server. Using tac_plus, an open source TACACS server, you can absolutely set the single-connection bit from say, a Cisco 9706 MDS switch; however, upon packet analysis of any TACACS authentication requests you may discover the single-connection bit is set to 0.

Refer to draft-grant-tacacs-02 and scroll to the FLAGS section for an explanation of where you will, and should, see the single-connection bit set in the TACACS flag. Basically, you’ll only ever find the bit set in the initial setup of the connection so both the TACACS server and the client agree on single-connection TCP. Thus, instead of each and every TACACS request coming through as a unique TCP connection (essentially having to use multiple sockets, sockets being the 4-tuple of SRC IP, DST IP, SRC port, and DST port) the TACACS query and response messages are just carried over the single TCP connection.

If your system supports this, its worth attempting to see if it works as it can save some resources; however, your mileage may vary.