Why does this capture filter not capture my traffic?

asked 2020-07-08 09:27:31 +0000

balderman gravatar image

updated 2020-07-09 08:08:51 +0000

Guy Harris gravatar image

I capture HTTP traffic and build Request/Response pairs.

I am using dumpcap and tshark.

I have a client (python script) that sends HTTP GET to one of 4 HTTP servers.

The client gets 200 OK as a response.

The client sends a 'special' HTTP request header having the name 'X-KINNERET'.

I have one version of BPF where I can "see" my client requests.
Below: 

sudo dumpcap -i eno2 -f '(tcp && ((port 50003 && host 10.36.101.27) || (port 54017 && host 10.36.101.27) || (port 50003 && host 10.36.101.28) || (port 54017 && host 10.36.101.28)))' -B 100 -w - | tshark -r - -Y http -T json -e ip.dst -e tcp.dstport -e tcp.stream -e frame.time_epoch -e http.request -e http.request.method -e http.request.version -e http.request.full_uri -e http.request.uri -e http.request.line -e http.file_data -e http.response -e http.response.version -e http.response.code -e http.response.line | grep X-KINNERET

I have another version of BPF where I can NOT see my client requests.
Below: 

sudo dumpcap -i eno2 -f '(tcp && ((src port 50003 && src host 10.36.101.27) || (dst port 50003 && dst host 10.36.101.27) || (src port 54017 && src host 10.36.101.27) || (dst port 54017 && dst host 10.36.101.27) || (src port 50003 && src host 10.36.101.28) || (dst port 50003 && dst host 10.36.101.28) || (src port 54017 && src host 10.36.101.28) || (dst port 54017 && dst host 10.36.101.28)))' -B 100 -w - | tshark -r - -Y http -T json -e ip.dst -e tcp.dstport -e tcp.stream -e frame.time_epoch -e http.request -e http.request.method -e http.request.version -e http.request.full_uri -e http.request.uri -e http.request.line -e http.file_data -e http.response -e http.response.version -e http.response.code -e http.response.line | grep X-KINNERET

The second BPF does capture traffic but not the traffic coming from my python script.

Any idea why the second 'strict' BPF does not work?

Thanks

edit retag flag offensive close merge delete

Comments

Presumably ports 50003 and 54017 are the destination ports that the HTTP servers are listening on, have you also fixed the source ports your python script is using for the requests or are those using ephemeral ports? If the latter, then your second filter is likely excluding the requests due to the source ports.

Your second filter also restricts the source IP's to .27 & .28, does your request generating script run on a host with both IP's or is that a typo?

grahamb gravatar imagegrahamb ( 2020-07-08 09:56:57 +0000 )edit

Hi Graham
Thanks for your answer.

Below is the list of my "HTTP Servers"

  1. 10.36.101.27:50003
  2. 10.36.101.27:54017
  3. 10.36.101.28:50003
  4. 10.36.101.28:54017

My python client uses ephemeral ports - it does not set the port. The problematic BPF (at least as I currently understand it..) is making sure that one of the "sides" that are using the socket will be one of the servers in the list. Therefore I have

(src port P && src host H) || (dst port P && dst host H) // where P and H are symbol represents an entry in the 4 servers list above.

Can you please try and explain why this filter exclude my client?

Thanks

Avishay

balderman gravatar imagebalderman ( 2020-07-08 10:25:15 +0000 )edit

Could you comment with the output of the following two commands :

sudo dumpcap -i eno2 -d -f '(tcp && ((port 50003 && host 10.36.101.27) || (port 54017 && host 10.36.101.27) || (port 50003 && host 10.36.101.28) || (port 54017 && host 10.36.101.28)))'

sudo dumpcap -i eno2 -d -f '(tcp && ((src port 50003 && src host 10.36.101.27) || (dst port 50003 && dst host 10.36.101.27) || (src port 54017 && src host 10.36.101.27) || (dst port 54017 && dst host 10.36.101.27) || (src port 50003 && src host 10.36.101.28) || (dst port 50003 && dst host 10.36.101.28) || (src port 54017 && src host 10.36.101.28) || (dst port 54017 && dst host 10.36.101.28)))'

(the added -d will dump the BPF code so we can ...(more)

SYN-bit gravatar imageSYN-bit ( 2020-07-08 12:29:53 +0000 )edit

See the answer below (I was not able to use the comment)

Dumpcap (Wireshark) 2.6.10 (Git v2.6.10 packaged as 2.6.10-1~ubuntu18.04.0)

Copyright 1998-2019 Gerald Combs <[email protected]> and contributors.
License GPLv2+: GNU GPL version 2 or later <http://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Compiled (64-bit) with GLib 2.56.4, with zlib 1.2.11, with libpcap, with POSIX
capabilities (Linux), with libnl 3.

Running on Linux 4.15.0-20-generic, with Intel(R) Xeon(R) CPU E5-2680 v4 @
2.40GHz (with SSE4.2), with 257855 MB of physical memory, with locale C, with
libpcap version 1.8.1, with zlib 1.2.11, binary plugins supported (0 loaded).

Built using gcc 7.4.0 ...
(more)
balderman gravatar imagebalderman ( 2020-07-08 12:56:32 +0000 )edit

"Non Strict Filter"

(000) ldh      [12]  
(001) jeq      #0x86dd          jt 41   jf 2  
(002) jeq      #0x800           jt 3    jf 41  
(003) ldb      [23]  
(004) jeq      #0x6             jt 5    jf 41  
(005) ldh      [20]  
(006) jset     #0x1fff          jt 41   jf 7  
(007) ldxb     4*([14]&0xf)  
(008) ldh      [x + 14]  
(009) jeq      #0xc353          jt 12   jf 10  
(010) ldh      [x + 16]  
(011) jeq      #0xc353          jt 12   jf 16  
(012) ld       [26]  
(013) jeq      #0xa24651b       jt 40   jf 14  
(014) ld       [30]  
(015) jeq      #0xa24651b       jt 40   jf 16  
(016) ldh      [x + 14]  
(017) jeq      #0xd301          jt 20   jf 18  
(018) ldh      [x + 16]  
(019) jeq      #0xd301          jt 20   jf 24  
(020) ld       [26]  
(021) jeq      #0xa24651b       jt 40   jf 22  
(022) ld       [30]  
(023) jeq      #0xa24651b       jt 40   jf 24  
(024) ldh      [x + 14]  
(025) jeq      #0xc353          jt 28   jf 26  
(026) ldh      [x + 16]  
(027) jeq      #0xc353          jt 28   jf ...
(more)
balderman gravatar imagebalderman ( 2020-07-08 14:18:49 +0000 )edit
see more comments