Recently I worked on a project that monitor network traffic using “SPAN–Switch Port Analyzer” sessions from Cisco switches. We had to work with a limitation of 2 x 10G port available on Analyzer. So you have to pick two critical switches and define SPAN session destination on those 2 switches.
If you are familiar with configuring SPAN session, then you know it require simply configure soure interface to monitor and then destination interface to send copy of the data of source interface. You can SPAN multiple interfaces to the same destination port if require (as shown below). The main limitation of a SPAN configuration is both source & destination port need to be on the same switch.
monitor session 1 source interface Te1/4 - 5 monitor session 1 destination interface Te2/4
In network troubleshooting, if you want to capture packets from particular switchport, you simply configure a SPAN session and plug a PC running wireshark to the SPAN destination port.
It is always not practical to plug Analyzer to the same switch where you want to capture traffic. In that scenario you have two options
- RSPAN – Remote Switch Port Analyzer
- ERSPAN – Encapsulated Remote Switch Port Analyzer
To use RSPAN, you need to have L2 across your two switches where you configure SPAN source and SPAN destination.Below diagram ( taken from 2016 CiscoLive Presentation – BRKARC-2011 – Overview of Troubleshooting and Packet Capturing Tools) illustrate it
In my case, two devices in two separate sites and across L3 network. In this kind of situations you have to use ERSPAN
Below diagram ( taken from same Ciscolive session) explains what’s happen in a ERSPAN session. It uses GRE encapsulation where source device encapsulate it where as destination device decapsulate prior to send it to Analyzer.
In my setup these switches are 6506-E with Sup2T (running on 15.2-1-SY4). BR-SW is the source of my ERSPAN session. Here is configurations involved.
BR-SW(config)#monitor session 1 type ? capture Packet Capture SPAN Session erspan-destination Encapsulated RSPAN Destination Session erspan-source Encapsulated RSPAN Source Session local Local SPAN Session local-tx Local SPAN Session TX only rspan-destination RSPAN Destination Session rspan-source RSPAN Source Session BR-SW(config)#monitor session 1 type erspan-source BR-SW(config-mon-erspan-src)#source interface g6/2 BR-SW(config-mon-erspan-src)#destination BR-SW(config-mon-erspan-src-dst)#? ERSPAN DST Session Source Commands: erspan-id Enter ERSPAN Identifier exit Exit monitor erspan destination session source property mode ip IP Properties no Negate a command or set its defaults origin Origin IP Properties vrf Enter VRF number BR-SW(config-mon-erspan-src-dst)#ip address y.y.y.3 BR-SW(config-mon-erspan-src-dst)#origin ip address x.x.x.62 BR-SW(config-mon-erspan-src-dst)#erspan-id 62
On HQ-SW, you have to configure monitor session (session number is locally significant, so I used 2) with ERSPAN destination. Source IP address should be the same you use as destination address in the above configuration. You have to use use same “erspan-id” at the ERSPAN destination side.
HQ-SW(config)#monitor session 2 type erspan-destination HQ-SW(config-mon-erspan-dst)#destination interface tenGigabitEthernet 6/8 HQ-SW(config-mon-erspan-dst)#source HQ-SW(config-mon-erspan-dst-src)#ip address y.y.y.3 HQ-SW(config-mon-erspan-dst-src)#erspan-id 62 HQ-SW(config-mon-erspan-dst-src)#exit HQ-SW(config-mon-erspan-dst)#no shut
You can verify configurations using “show monitor session x” command.
HQ-SWCR01#sh monitor session 2 Session 2 --------- Type : ERSPAN Destination Session Status : Admin Enabled Destination Ports : Te6/8 Source IP Address : y.y.y.3 Source ERSPAN ID : 62
With that configuration, your Port Analyzer should be able to see BR-SW G6/2 traffic. Here is a snapshot of that interface traffic from our traffic Analyzer
In troubleshooting, this ERSPAN feature will become really useful. You can directly send SPAN traffic to a PC running on wireshark (even without configuring ERSPAN destination on another switch).
Since ERSPAN use GRE encapsulation, you have to configure a capture filter on Wireshark so you will directly see the traffic you interested. As GRE is IP Protocol 47 (in hex 2f) , you can apply “ip proto 0x2f” as capture filter.
In below example, I have shown how you can configure ERSPAN session on a switch in order to send capture traffic directly to a PC running wireshark.
Here are the basic commands you require to capture traffic on PortChannel 200 interface goes to my WLC.
monitor session 1 type erspan-source source interface Po200 no shut destination erspan-id 18 ip address x.x.33.228 origin ip address x.x.x.18
With above configuration, you should be able to see PortChannel 200 traffic on your PC running wireshark as shown below
If you look at packet in detail, you will see the original packet is encapsulated with GRE (it uses source as ERSPAN source and destination as Wireshark PC IP) . Below example is from a open WLAN client DHCP discover message (AP-10.15.6.98 to WLC – 10.10.0.100)
Everything is looks really cool. But you need to remember this ERSPAN is not supported on all Cisco routers/switches. It is supported on below platforms (below not be a comprehensive list)
- Cisco 6500
- Nexus switches
- Cisco ISR 44xx
- Cisco ASR
- Cisco 3850 ( IOS-XE 16.x)
Note that ERSPAN will add load on to your device CPU. I have configured it on 6506s with Sup2T and Sup6T and would not recommend running it 24×7 for Sup2Ts. Noticed 30-60% CPU hike on 2Ts and did not notice same on Sup6T.
If you do it for a packet capture, that should be ok, anyway monitor your device CPU while it is running. You can see CPU load for 5 min when I ran ERSPAN on Sup2T for a port having 500Mbps traffic.