I have a strange issue that I have attempted to research but would welcome input from anyone else that may have encountered anything similar. Specifics: VM Ubuntu 13 server on ESXi 5.0 with 3 NICs (type VMXNET3). (Note: Today I just rebuilt the VM using the new Ubuntu 14 LTS.) Eth0 is for the local network. Eth1 and Eth2 are dedicated to port mirrors from switches/taps. Eth1 only has single VLAN tags in the mirrored traffic. Eth2 has what I consider stacked tags. (I have included an image below.) When I attempt to use a capture filter for VLAN 992 (993, 994, etc), I do not capture any data. I can use a display filter to show the VLAN 992 after it has been captured but this isn't what I desire for troubleshooting purposes due to the high amount of traffic. I can capture filter the second VLANs (811, 812, etc) just fine. I'm not sure where the issue resides. I have experimented with other E1000 NICs in ESXi. I have the virtual switch set properly in ESX for promisc. (If this was set incorrectly I wouldn't have any traffic in the captures.) I have the VM NICs set for promisc (even though they appear not to need it since ESX is handling it). The version of Wireshark is 1.10.6 from the apt repository and whatever pcap is included - I mention this to confirm it's not an issue I created with a custom build/install of source. Although I don't think it is worth going into a great amount of detail, I did install a CentOS 6.5 VM today as well and did perform a custom build/install of pcap and wireshark. Same NIC parameters. When I did this, I was no longer able to see the outer VLAN tag. I'm not sure if that is giving me a clue or not. I know someone smarter than me could likely shed some light on my mystery. Thanks for any assistance! asked 17 Apr '14, 15:26 stjaru edited 17 Apr '14, 15:28 |
2 Answers:
Have you tried filtering on both VLAN ids at the same time? Something like "vlan 992 and vlan 811"? If I remember correctly this is required when filtering on QinQ traffic. answered 17 Apr '14, 15:31 Jasper ♦♦ Thanks for your input Jasper. :) I have tried that without success. Capture filter "vlan 992 or vlan 811" will not collect anything. But I did discover something interesting - Capture filter "vlan 810 or vlan 811" will only collect the first VLAN (810). I would not expect a problem with that capture filter. However since I have never been able to capture filter on the outer tag, I normally capture everything on the interface and then use a display filter on the outer tag. Hence the discovery of the behavior. I don't know if these problems are related but I'm still wrestling with the original question. (17 Apr '14, 15:49) stjaru
I would, but that's because I know the rather kludgy way that the "vlan" capture filter works. "vlan" turns everything to the right of it into a test for traffic under that VLAN, so "vlan 810 or vlan 811" doesn't do what you'd expect. (18 Apr '14, 13:57) Guy Harris ♦♦ |
as @Guy Harris already mentioned the vlan capture filter 'primitive' does some magic behind the curtains, and thus it does not work as you might expect it, based on the behavior of other logical OR operations in capture filters.
So, lets have a look at the BPF code for the following capture filter
(000): Load the location of the ethertype. So far, so good. Now lets check the BPF code of the following capture filter
(000-004): same as before Due to this behavior (call is a bug or not), you cannot capture for several vlan tags in a single capture filter, combined with a logical OR operation. Furthermore, if you use a logical AND operation, you will only see double tagged or QinQ frames (as @Jasper) mentioned. Solution: Run several instances of tcpdump, each with a single vlan capture filter and later merge the capture files with mergecap.
+++ UPDATE +++ I have to correct myself. It is possible to capture multiple (outer) vlan tags in a single capture filter, by doing the vlan tag matching 'manually'.
If you look at the BPF code, you'll see that it is essentially the same as 'vlan 100' combined with 'vlan 200', which is essentially the same as 'vlan 100 or vlan 200'.
(000): Load the location of the ethertype. Regards answered 19 Apr '14, 16:45 Kurt Knochner ♦ edited 20 Apr '14, 14:48
O.K. we need more information. Here are some questions for you:
You say:
O.K. so, either there are no frames with VLAN tag 99x, or something on your system strip the outer VLAN tag before the capturing system gets the frame, which would explain, why you do see the inner tag. However, if the outer tag would have been removed, it does not explain why you see the outer tag in the capture file, with a display filter. But maybe I misunderstand what you did in that case!?! So, can you please add more details about your capturing setup: What do you see in the capture file, if you
BTW: Can you provide a sample capture somewhere (google docs, dropbox, cloudshark.org)? (21 Apr '14, 03:27) Kurt Knochner ♦ |
There are answers below about how to filter for the inner tag, but when I read this question, it sounds more to me like you're unable to filter the outer tag, 992 in this case. To quote:
When I attempt to use a capture filter for VLAN 992 (993, 994, etc), I do not capture any data. I can use a display filter to show the VLAN 992 after it has been captured but this isn't what I desire for troubleshooting purposes due to the high amount of traffic.
So when you apply a capture filter of, "
vlan 992
" you don't capture anything? But if you don't apply any capture filter, then you can later apply a Wireshark display filter of "vlan.id == 992
" to filter the packets of interest? Is that right?well, my answer is actually primarily about the problem of filtering several outer tags in one capture filter statement.
Thank you Guy for participating in the thread. Good stuff!
Kurt, wow man, that was fantastic information! I appreciate you taking the time to give such excellent detail and sourcing! I'm very appreciative of everyone taking some time to help on this topic.
Now that I better understand why my other assumption was incorrect, would anyone have any ideas about why I cannot capture successfully on just the outer tag? That would be the heart of the issue.
I'm still unable to understand why I cannot use capture with "vlan 99x" and see packets (which is the answer to cmaynard's request for clarification/confirmation). Again, I cannot capture using that filter, but I can display to see the VLAN.
Thanks all,