Tags

, , ,

The ability to take a packet capture easily on Cisco 9800 is one of the coolest features available when it comes to troubleshooting. In AireOS controllers, you have to take it from the switch end (WLC connected trunk port). Not only it is time-consuming but also has too much dependencies when configuring a SPAN session (port availability, 1G/10G, copper port for your laptop connectivity, etc).

Let’s explore PCAP capablity of 9800 WLC in this post. You can configur it using “Troubleshooting > Packet Capture” in GUI. If you are troubleshooting control plane traffic (eg AP troubleshooting) then leaving that “Monitor Control Plane” option checked is useful. If it is a data plane (client traffic), you can uncheck that option. Otherwise, you will see two copies of your traffic (a bit confusing at times). You can very easily filter what you want to capture using extended ACL. Also, you can filter for client MAC addresses when you troubleshoot a particular client issue when you have a WLC with many AP & clients active on it. Below shows a GUI screenshot of PCAP settings.

When you do troubleshooting regularly, It is good to know the CLI commands so that you can enable the same thing without multiple clicks on GUI. Below shows the CLI commands (ACL entries depend on what you try to capture – the example given to filter for a single IP address- can be AP or wired Device). When it comes to wireless client troubleshooting, you have to filter by client_mac_addr as WiFi operates in Layer2 where many activities happen before the client gets an IP address.

### Define ACL to match AP IP_Addr ###
ip access-list extended AP_WLC
 10 permit ip host 192.168.130.102 any
 20 permit ip any host 192.168.130.102
!
### Start the capture on Gig2 interface ###
monitor capture MRN interface g2 both
monitor capture MRN buffer size 20
monitor capture MRN access-list AP_WLC
monitor capture MRN start
.

.
### Stop the capture & Export file ###
monitor capture MRN stop
monitor capture MRN export flash:9130-1791-joinissue.pcap
.
### Delete Capture from 9800 ###
no monitor capture MRN

If you would like you can configure all those monitor capture options in one CLI as given below & start the capture in a single line. You can select physical WLC ports, Port Channel, or Mgt VLAN interface as the interface to capture.

monitor capture MRN interface g2 both buffer size 40 access-list WEB_ADMIN start
.
monitor capture MRN interface vlan 100 both buffer size 100 match any inner mac 14-F6-D8-1D-4B-65 start
.
monitor capture MRN stop
monitor capture MRN export ftp://192.168.99.100/9800-win10ax-tls.pcap
.
no monitor capture MRN

Once you setup your capture, you can verify using “show monitor capture <capture_name>” CLI output as shown below

C9800-1#sh monitor capture MRN

Status Information for Capture MRN
  Target Type: 
 Interface: Vlan100, Direction: BOTH
   Status : Active
  Filter Details: 
    Capture all packets
  Inner Filter Details: 
   Mac: 14f6.d81d.4b65 
  Buffer Details: 
   Buffer Type: LINEAR (default)
   Buffer Size (in MB): 100
  Limit Details: 
   Number of Packets to capture: 0 (no limit)
   Packet Capture duration: 0 (no limit)
   Packet Size to capture: 0 (no limit)
   Maximum number of packets to capture per second: 1000
   Packet sampling rate: 0 (no sampling)

Let’s look at a few examples of packet captures in a different scenario

1. AP Join Troubleshooting
Here is a capture taken when AP tries to register to my 9800 WLC. Once you start the packet capture, I reloaded my AP to get the full registration process captured. 192.168.130.100 was my AP IP address and 192.168.100.10 is 9800 WLC.

### Define ACL using AP IP_Addr & WLC Mgt IP ###
ip access-list extended AP_REG
 10 permit ip host 192.168.130.100 host 192.168.100.10
 20 permit ip host 192.168.100.10 host 192.168.130.100

### Define Monitor Capture with ACL filter ###
no monitor capture MRN
monitor capture MRN interface g2 both buffer size 100 access-list AP_REG start
.
.
### Stop capture & export file ###
monitor capture MRN stop
monitor capture MRN export flash:9800-1-AP-Registration1.pcap
no monitor capture MRN

Here is the AP-Registration PCAP for you to check. Below is the summary of the frame flow during DTLS establishment. This example, it is showing a flow of the working AP. When you have AP join issues, you will see failures occurring DTLS session establishing due to time, country code, trust points, etc.

2. WLC GUI Admin Access
Here is a PCAP taken when I was accessing my 9800 GUI. I noticed that IOS-XE 17.9.1, it is using TLS1.3 and prior versions use TLS1.2 (I will do a separate post on the differences between those two TLS methods).

192.168.100.39 is the PC IP address used to access WLC’s GUI

C9800-1#sh access-lists WEB_ADMIN
Extended IP access list WEB_ADMIN
    10 permit ip host 192.168.100.39 host 192.168.100.10
    20 permit ip host 192.168.100.10 host 192.168.100.39

!
C9800-1#no monitor capture MRN
C9800-1#monitor capture MRN interface g2 both buffer size 100 access-list WEB_ADMIN start
.
<access WLC GUI using PC with 192.168.100.39 IP_Addr>
.
C9800-1#monitor capture MRN stop 
C9800-1#monitor capture MRN export ftp://192.168.99.100/17.6-webadmin-9800cap1.pcap
Writing 17.6-webadmin-9800cap1.pcap 
Exported Successfully

Here is the PCAP taken from 9800. With 17.6.x, even I have installed WebAdmin cert, my browser gave me the warning of cert cannot be trusted.

Here is the capture taken by the PC (1902.168.100.39) using Wireshark with capture filter “host 192.168.100.10 and host 192.168.100.39“. Note that 9800 capture shows 4 bytes extra length which includes 802.1Q as it is captured on 9800 ends configured as a trunk port.

Here is the PCAP taken while doing a similar test with a WLC running on 17.9.1. Noticed it uses TLS 1.3 version where most of the traffic goes encrypted fashion (even certificate details server present, you cannot see it in clear)

3. Client Connectivity Troubleshooting
The most common use case of taking a PCAP is when you have issues with wireless client connectivity. In that case, you have to filter for the client’s MAC address as shown below. Also, I have exported my PCAP file directly to my FTP server (rather than saving it to bootflash)

### Start the capture on Gig2 interface ###
monitor capture MRN interface g2 both buffer size 40 inner mac 32:93:de:03:16:19 match any start 
.
.
### Stop the capture & Export file ###
monitor capture MRN stop
monitor capture MRN export ftp://192.168.99.100/9800-pix3-peap-9117.pcap
.
### Delete Capture from 9800 ###
no monitor capture MRN 

It is a good idea to enable radioactive tracing (refer to my previous post) even if you taking packet capture. Here are the CLI commands to enable that on a 9800

### Start debug conditions using AP Radio MAC & IP_Addr ###
debug platform condition feature wireless mac 32:93:de:03:16:19
debug platform condition start
.
.
### Stop debugging & Export file ###
debug platform condition stop
show logging profile wireless start last 30 minutes to-file bootflash:ratr-pix3-peap-9117-1X-1.txt
.
.
### Clear all debug conditions ###
clear platform condition all

Here is the packet capture taken from the 9800 WLC, if you look at it closely you will see 802.1X traffic is in decrypted fashion. This is one of the key advantages of 9800 PCAP where you can see the inner details of 802.1X frames without much hassle.

Whenever you do 9800 troubleshooting, keep using this PCAP feature and you will find it is a really cool feature and help you to get a deeper understanding of what’s going on at the packet level.

References
1. Collect Logs & Debugs from 9800 WLC