Archive for the ‘Uncategorized’ Category

You’ve likely been where I am, and usually there quite often, looking for a specific PDF which contains information you need to reference, but you can’t find the darn PDF! Unless you’re incredibly detail oriented and place everything where you’ll find it 100% of the time, you’re likely to lose it. However, for those on Windows who have CYGWIN installed, or even if you’re using Linux and just want to know how to do the same thing, allow me to enlighten you on how to use two, well three if you think about it, tools for searching your entire machine(s) for that specific file.

On a Windows machine you’ll want the following:

  • CYGWIN
  • Base utilities packages (to ensure you have find and xargs)
  • pdfgrep

So, you know you remember a specific keyword, or phrase, and you just need to search for all of your PDF files and return on the terminal the output with the filename. I recommend using xargs in conjunction with find, instead of exec. Why? Well, exec will, for each file the find command returns, run a separate process; thus, its slow and eats processor time, xargs does not. So, lets say you want to search your entire computer’s C: drive in CYGWIN:

find /cygdrive/c -type f  -name "*.pdf" -print0 | xargs -0 pdfgrep -i "symmetrical"

That is it, it may take a long time, but for any .pdf document it finds, xargs will search for the string “symmetrical”. Your mileage will vary based on your machines resources, but if you can close everything and leave only that running, say at lunch time, it should execute quickly and, hopefully, you’ll find your long lost PDF document.


I realize there is still some confusion regarding Cisco Nexus FEX as it relates to ToR connected FEX, which is a Cisco Nexus 2K FEX with a Cisco Nexus 5K/7K/9K as a parent switch, and the FEX you find in UCS, which we can refer to as “Blade-FEX”. I am going to outline what ToR (Top of Rack) FEX in this blog post, not Blade-FEX, to help bring some clarity around this still confusing terminology. This is also not meant to bring any additional ambiguity, but it is true you can use certain Cisco Nexus 22XX ToR-FEX and “parent” them to a Cisco UCS Fabric Interconnect; however, I would not classify this as Blade-FEX or ToR-FEX, I’d like to coin it with the term “Fabric-FEX”, you owe me $1.00 every time you use this, send it via paypal :). Thus, moving forward, we’re going to refer to a FEX which parents to a Cisco Nexus switch as a ToR-FEX.

Cisco Nexus FEX works thanks to the Cisco pioneered 802.1BR, click here for more information. Now, you don’t have to worry about configuring the gory details of what is essentially VN-TAG because this is all handled with a few simple commands to get your FEX up and running; however, this is just here do you know how FEX works to communicate with the parent switch underneath the sheets.

The logical representation of FEX is broken down like this:

  • Logical Interfaces (LIF) – This is simple, its the Eth1xx/1/X representation on the switch
  • Network Interfaces (NIF) – These are the physical uplinks connecting the FEX to parent, carrying the VN-TAG
  • Virtual Interface (VIF) – This is the logical interface which correlates, in software, to the physical host interface. We we wil discuss this in a minute about why this makes FEX capable of full swap of a failed FEX without reconfiguring the host ports
  • Host Interface (HIF) – These are the physical ports on the FEX which you connect your hosts to. The parent switch assigned each HIF a unique VN-TAG ID, which is roughly correlated to the above Virtual Interface (VIF) assignment.

Here is some output to take a peek at, taken from a Cisco Nexus 9332PQ switch with 2348TQ and 2348UPQ FEX attached:

slot:36, fab_if:160001f4, p_ind:f4010016, p_numelem:1
dev_inst:0, nif_no:16, hif_no:40, nif_ind:160001f4, hif_ind:1f670a00
Eth104/1/42 0x1f670a40 Down Po501 Po501 NoConf

Take notice, this is Logical port: Eth104/1/42 and there is a plethora of information regarding the port, including the HIF numer and the hif_ind. I haven’t referenced anything with Cisco as of yet, but I would believe the HIF no is the unique number assigned to the port, perhaps the VIF, and the HIF_IND may be an index ID, but I’ll investigate later. For now, just take notice that: Eth[101-199]/1/[1-48] is the LIF, which is attached to a VIF, which correlates to the HIF on the FEX. Because FEX attaches the configuration to a VIF, which is also correlated to the FEX ID, you can have your FEX member, say FEX 104, fail completely and all you need to do is just replace the failed FEX, cable it the same way and when the FEX image is downloaded it’ll reboot and continue operation without the need to rebuild the configurations.

Now, you MUST be diligent in understanding the valid UPLINK topology you can configure your ToR-FEX for, in relation to your parent switch. Always review the configuration guide for your specific model of FEX and parent switch to obtain the valid topology. In my scenarios with the Cisco Nexus 9K switches I do a single-homed, host vPC port-channel uplink topology because we can’t do a more elaborate e-vPC design with the 9K switches and our hosts will be attached with port-channels in an active-active scenario.

Finally, the configuration is simple; however, some Cisco documentation is confusing because the wording in some documents states the UPLINK port-channel is LACP enabled; thus, you would assume you configure your UPLINK as an active LACP member. This is wrong, in fact, the best method, at least from my experience with the 9K switches, is to create the port-channel you’ll be using for the UPLINK, no-shut the interface and nothing more, then move into the physical interfaces that’ll be part of this port-channel, no shut the interfaces and just assign them to the port-channel as static mode. Then, move back into the port-channel configuration mode and build your configuration. Below is the basic configuration you need to get your FEX attached to your 9K switch:


interface po500
no shut
!
interface eth1/21-24
channel-group 500
no shut
!
int po500
switchport
switchport mode fex-fabric
fex associate
mtu
no shut -
!

A note about setting Jumbo frames on those FEX ports. The FEX host ports will assume the maximum MTU based on the UPLINK port-channels MTU assignment. In our environments we aim to have jumbo frames end-to-end and leave it up to the specific host/OS/application to decide on its optimal packet size. Thus, if you set your MTU on the UPLINK port-channel to 2000, your MTU on will be 2000 on your host interface ports on the FEX.

As an update, and to summarize, here are the FEX types to help clear up confusion, these are not “official” terms, but these will help to clear up confusion, I hope:

  • ToR-FEX: A Nexus 2K FEX attached (Parented) to a Nexus 3K/5K/7K/9K switch for extending ports
  • Blade FEX: These modules are installed into a Cisco UCS chassis
  • Fabric-Interconect FEX: These are the same Nexus 2K FEX used for parenting to 3K/5K/7K/9K; however, now you can attach (Parent) these Nexus 2K FEX to Cisco Fabric Interconnects for the purpose of extending ports for your Fabric Interconnects, or providing a different type/speed of port.

There has been some slight confusion and ambiguity around the “single-connection” configuration statement provided by Cisco switches and routers, including SAN MDS switches. As of this writing, Cisco Nexus 9000 NXOS switches on 7.0.3.I5.1 code do not support single-connection in their tacacs host configuration; however, certain MDS switches do. In either case, if you do find yourself wondering here for the answer, let me elaborate for you.

The purpose of single-connection is to multiplex all of your TACACS authentication requests using a single TCP oriented connection from the switch to the TACACS server. Using tac_plus, an open source TACACS server, you can absolutely set the single-connection bit from say, a Cisco 9706 MDS switch; however, upon packet analysis of any TACACS authentication requests you may discover the single-connection bit is set to 0.

Refer to draft-grant-tacacs-02 and scroll to the FLAGS section for an explanation of where you will, and should, see the single-connection bit set in the TACACS flag. Basically, you’ll only ever find the bit set in the initial setup of the connection so both the TACACS server and the client agree on single-connection TCP. Thus, instead of each and every TACACS request coming through as a unique TCP connection (essentially having to use multiple sockets, sockets being the 4-tuple of SRC IP, DST IP, SRC port, and DST port) the TACACS query and response messages are just carried over the single TCP connection.

If your system supports this, its worth attempting to see if it works as it can save some resources; however, your mileage may vary.


Ooook, here is another configuration example for the Cisco implementation for VXLAN using BGP EVPN for distributed control-plane operations. anycast gateway, and unicast head-end replication. I am using Cisco 9396PX devices for leaf switches and Cisco 9508 chassis switches for the spine using iBGP. We’ll explore the basic setup with the leaf switches being vPC enabled, including the Border Leaf switches, while also going over a few scenarios which can blackhole traffic and how to avoid this without a OSPF adjacency between the leaf switches.

This blog will assume you understand the basic setup of BGP EVPN VXLAN by reading the great Cisco documentation already available; thus, I presume you’re coming here for a more in-depth, real-world deployment scenario and for some better explanations and failure scenario testing and outputs

Below, this diagram shows the connectivity in the UNDERLAY network:

Cisco BGP EVPN UNDERLAY

Cisco BGP EVPN UNDERLAY

You can see we have three spine switches, two configured as route reflectors for scalability. Below is the configuration of a single spine switch being used as a route reflector, the other route reflector is setup the same way, with IP addresses being different and such and, of course, the other spine switch not having any iBGP peering relationships with the third spine switch is just runs OSPF, forms adjacencies with all VTEPS for advertisement of VTEP IP reachability.


nv overlay evpn
feature ospf
feature bgp
feature nv overlay

router ospf 1
router-id 172.16.2.253
log-adjacency-changes
passive-interface default

interface Ethernet1/1
description Leaf01-9kA
link debounce time 0
mtu 9216
medium p2p
ip address 172.16.2.1/30
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf 1 area 0.0.0.0
no shutdown

interface loopback0
ip address 1.1.1.10/32
ip router ospf 1 area 0.0.0.0

router bgp 65000
router-id 1.1.1.10
address-family ipv4 unicast
neighbor 1.1.1.40
description VTEP1
password 3 SOMEPASSWORD
update-source loopback0
timers 3 9
address-family ipv4 unicast
address-family l2vpn evpn
send-community both
route-reflector-client
neighbor 1.1.1.41 remote-as 65000
description VTEP2
password 3 SOMEPASSWORD
update-source loopback0
timers 3 9
address-family l2vpn evpn
send-community both
route-reflector-client

The above forms the basis of the Underlay network on the spine and sets up the route-reflectors. We have tuned this for protocol convergence speed; thus, timers are aggressive for BGP and you’ll notice the “link debounce time 0”, which disabled link debounce. In a nutshell, by default, the debounce time is the amount of time after a switchport goes down for which the switchport will wait to notify the supervisor, 100msec by default. Disabling this allows immediate updating to the supervisor on a link failure to start protocol convergence. If you’re worried about an unstable interface, it is quite likely in the event of a link failing/flapping issue, the link-flap detection mechanism will down the port. Finally, we set BOTH the interface medium to p2p and set the OSPF network type to point-to-point. Why? In the event someone misses the command to switch OSPF to point-to-point, since this interface type is broadcast by default, the medium p2p command changes the ports operating mode and OSPF will properly adjust to point-to-point; thus, this is just good extra redundancy.

Now, here is the overlay view, pretend this is an OVERLAY named “Tenant-01”:
VXLAN-OVERLAY

Below is the configuration:


nv overlay evpn
feature ospf
feature bgp
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature vpc
feature nv overlay

fabric forwarding anycast-gateway-mac 0005.0005.0005
fabric forwarding dup-host-ip-addr-detection 5 180

class-map type qos match-any ONE
match cos 1
match dscp 26
class-map type qos match-any TWO
match cos 2
match dscp 16
class-map type qos match-any THREE
match cos 3
match dscp 48
policy-map type qos REST-YOUR-COS-FOR-UCS-FI
class SILVER
set cos 2
class GOLD
set cos 4
class PLATINUM
set cos 6
policy-map type qos FOR-THE-COS-IGNORANT
class class-default
set cos 2
set dscp 16

spanning-tree vlan 1-3967 hello-time 4

vlan 201
name VXLAN-VLAN01
vn-segment 100201
vlan 202
name VXLAN-VLAN02
vn-segment 900202
vlan 203
name VXLAN-VLAN03
vn-segment 900203
vlan 2999
name VLAN-FOR-BRIDGE-DOMAIN
vn-segment 29999

vrf context Tenant01
vni 29999
rd auto
address-family ipv4 unicast
route-target both auto
route-target both auto evpn
address-family ipv6 unicast
route-target both auto
route-target both auto evpn

track 1 ip route 1.1.1.10/32 reachability
track 2 ip route 1.1.1.20/32 reachability
track 10 list boolean and
object 2
object 1
delay up 12

event manager applet spine-down
event track 10 state down
action 1.0 cli vpc domain 50
action 1.1 cli shutdown
action 1.2 cli interface loopback0
action 1.3 cli shutdown
action 1.4 cli interface nve 1
action 1.5 cli shutdown
event manager applet spine-up
event track 10 state down
action 1.0 cli vpc domain 50
action 1.1 cli no shutdown
action 1.2 cli interface loopback0
action 1.3 cli no shutdown
action 1.4 cli interface nve 1
action 1.5 cli no shutdown

hardware access-list tcam region vacl 0
hardware access-list tcam region e-racl 0
hardware access-list tcam region span 0
hardware access-list tcam region redirect 256
hardware access-list tcam region rp-qos 0
hardware access-list tcam region rp-ipv6-qos 0
hardware access-list tcam region rp-mac-qos 0
hardware access-list tcam region e-ipv6-qos 256
hardware access-list tcam region e-qos-lite 256
hardware access-list tcam region arp-ether 256

vpc domain 100
peer-switch
role priority 8192
system-priority 8192
peer-keepalive destination 192.168.1.1 source 192.168.1.2 interval 500 timeout 3
delay restore 5
peer-gateway
auto-recovery
ipv6 nd synchronize
ip arp synchronize

interface Vlan2999
description L3-VXLAN-BD
no shutdown
mtu 9216
vrf member Tenant01
no ip redirects
ip forward
ipv6 forward
no ipv6 redirects

interface Vlan201
description NET01
no shutdown
mtu 9216
no ip redirects
management
vrf member VXLAN
ip address 10.0.0.1/24
no ipv6 nd redirects
fabric forwarding mode anycast-gateway

interface Vlan202
description NET02
no shutdown
mtu 9216
no ip redirects
vrf member Tenant02
ip address 10.0.1.1/24
fabric forwarding mode anycast-gateway

interface Vlan203
description NET03
no shutdown
mtu 9216
no ip redirects
vrf member Tenant01
ip address 10.0.2.1/24
fabric forwarding mode anycast-gateway

interface port-channel50
description To Ethernet Switch B
switchport mode trunk
vpc peer-link

interface port-channel201
description Fabric-Interconnect-A
switchport mode trunk
switchport trunk allowed vlan 201-203
spanning-tree port type edge trunk
mtu 9216
service-policy type qos output REST-YOUR-COS-FOR-UCS-FI
vpc 201

interface port-channel202
description Fabric-Interconnect-B
switchport mode trunk
switchport trunk allowed vlan 201-203
spanning-tree port type edge trunk
mtu 9216
service-policy type qos output REST-YOUR-COS-FOR-UCS-FI
vpc 202

interface nve1
no shutdown
source-interface loopback0
host-reachability protocol bgp
source-interface hold-down-time 120
member vni 29999 associate-vrf
member vni 100201-100202
suppress-arp
ingress-replication protocol bgp

interface Ethernet2/1
switchport mode trunk
channel-group 50 mode active

interface Ethernet2/2
switchport mode trunk
channel-group 50 mode active

interface Ethernet2/3
no switchport
link debounce time 0
medium p2p
mtu 9216
ip address 172.16.2.18/30
no ipv6 redirects
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf 1 area 0.0.0.0
no shutdown

interface Ethernet2/4
no switchport
link debounce time 0
medium p2p
mtu 9216
ip address 172.16.3.22/30
ip ospf network point-to-point
no ip ospf passive-interface
ip router ospf 1 area 0.0.0.0
no shutdown

interface loopback0
description Loopback for NVE VTEP
ip address 1.1.100.44/32
ip address 1.1.1.102/32 secondary
ip router ospf 1 area 0.0.0.0

interface loopback1
description Loopback for BGP update-source
ip address 1.1.1.44/32
ip router ospf 1 area 0.0.0.0

router ospf 1
router-id 172.16.2.18
passive-interface default
log-neigh-adj

router bgp 65000
router-id 1.1.1.44
log-neighbor-changes
address-family ipv4 unicast
maximum-paths ibgp 10
neighbor 1.1.1.10
description spine1
password 3 SOMEPASSWORD
update-source loopback1
timers 3 9
address-family ipv4 unicast
address-family l2vpn evpn
send-community both
neighbor 1.1.1.20
description spine2
password 3 SOMEPASSWORD
update-source loopback1
timers 3 9
address-family ipv4 unicast
address-family l2vpn evpn
send-community both
vrf Tenant01
address-family ipv4 unicast
advertise l2vpn evpn
maximum-paths ibgp 10
address-family ipv6 unicast
advertise l2vpn evpn
maximum-paths ibgp 6
evpn
vni 100201 l2
rd auto
route-target import auto
route-target export auto
vni 100202 l2
rd auto
route-target import auto
route-target export auto
vni 100203 l2
rd auto
route-target import auto
route-target export auto

ip tcp path-mtu-discovery
l2rib dup-host-mac-detection 5 180

A lot to see here, right? This is why I decided to break this into two parts, so this is part 1 and my next post is part 2 for border leafs and failure scenarios! Lets get this initial review over with!

I will just outline all the key points here:

  • policy-map type qos REST-YOUR-COS-FOR-UCS-FI – This is for those of you who utilize the COS in Cisco UCS and want to maintain your COS value AFTER your packet is VXLAN DE-CAPSULATED. With this EVPN VXLAN configuration, the original 802.1Q header is stripped at ingress; thus, no COS value remains, but if you set any DSCP at the virtual switch level it is maintained throughout so we’re assuming you’re marking DSCP at your virtual switch along with COS and you have your own unique mapping from COS to DSCP. So, you create the classes I have above, this is all for example, your mappings will/may be different, and then create a policy-map to match against the DSCP value marked from your virtual switch and set the appropriate COS value. You then set this as a QOS OUTBOUND policy on the port-channel towards your Fabric Interconnects, but you will have to adjust your TCAM entries for this to work. The other one, for the COS-IGNORANT, will be for devices which aren’t smart enough to set either the DSCP or COS value; thus, just apply this to the interface, inbound, and set your values as needed
  • fabric forwarding anycast-gateway-mac 0005.0005.0005 – This is for the anycast gateway mac address. You can get “funny” here, but I like to keep it simple, your choice.
  • fabric forwarding dup-host-ip-addr-detection 5 180 – I set the duplicate host IP detection to 5 moves in 180 seconds for my environment, tune to the values best suited for yours
  • track objects and object list – I set these to look for the BGP neighbor address of the route-reflectors in the routing table and then assign each of those to the track object list for later assignment to the VPC. Part 2 will show and explain why
  • hardware tcam entries – Follow these for success in this configuration, especially if you’re in need of using the outbound QOS service policies
  • VPC peer-keepalive and delay-restore timers – Set to our environment and for specific reasons we’ll explain in part 2
  • NVE source-interface hold-down – This timer is set to 120 seconds, tuned for our environment, from the default of 300 seconds. I will explain the use of this and why I use 120 seconds in part 2
  • Loopback0 – Used ONLY for the NVE VTEP interface
  • Loopback0 secondary address – for vPC enabled VTEPS only, this is the PROXY VTEP address used
  • Loopback1 – Used ONLY for BGP source-updates
  • BGP passwords – This is used for security in the Underlay, you can also utilize OSPF authentication too, for extra security
  • So, like Forest Gump said to all his faithful followers “I’m pretty tired….I think I’ll go home now”. So, see you on Part 2, where the FUN is!!!

    CONTINUE TO PART 2


So, most of you probably got here because you’re probably on your CCIE track and you’re hearing a ton about the 32-bit words in the IPv4 headers and looking for an answer to the topic. It is without question that most may never know exactly what they’re talking about when they say “word” and this can lead to some confusion. First, the definition of a word from Wikipedia is:

“A word is basically a fixed-sized group of digits (binary or decimal) that are handled as a unit by the instruction set or the hardware of the processor. The number of digits in a word (the word size, word width, or word length) is an important characteristic of any specific processor design or computer architecture.”

Essentially, this means each 32 bits, 32 different positions where the values can be 0 or 1 in binary, is a “WORD”. Thus, when they’re referencing the IPv4 header length in a packet capture, you’ll see the size of the header. That header size is calculated by looking at the raw header, generally the next position after the Type, and you’ll find a hexadecimal value, lets say D, which is 13. Thus, you have 13 different 32-bit words.

Now, 13*32=416. Take the 416/8=52 bytes in the IPv4 header. Why 8? There are 8 bits in each byte. So, the next time you hear someone mention there are X number of 32-bit words in an IPv4 header, you now have some idea of what they’re talking about.