Ask Your Question
0

How to Identify TCP Initial receive window from Wireshark trace and what is the exact benefit of it

asked 2019-10-23 11:58:08 +0000

Amit gravatar image

updated 2019-10-28 13:19:06 +0000

grahamb gravatar image

In system configuration, we have set Initial receive window size value as 22* MSS but when checked via wireshark capture we cannot see that in TCP SYN or TCP ACK packets so just want to know how that Initial receive window value is related with Window size value we see in trace.

MSS is set as 1360 so based on that Initial window size should be 29920 but in trace in SYN request i can see window size as 43520, in ACK(in response to SYN-ACK from server) Window size was 45056 and some different value in subsequent ACK/PUSH/ACK packets

So my question is in which packet I can see initial receive window size in wireshark.

Also Can anyone explain me the benefit of high Initial receive window size?

This is the call flow between end-user and Internet server and my product behave as TCP full transparent proxy.

Client ------------> My Product -------> Server

TCP SYN-->
<--SYN/ACK--
--ACK--->      
                                       TCP SYN-->   
                                      <--SYN/ACK--
                                        --ACK--->

From my product point of View towards Client, it must share its initial window size in SYN/ACK message towards the client and towards server-side it should share Initial window size in TCP SYN message. On client-side next message is ACK which doesn't carry any payload while on Server side next message is SYN/ACK which also doesn't carry any payload then how come this Initial window size value is of any use? As after those messages, new ACK message with new window size will be shared and any payload from the Internet towards my product or from my product towards client will follow that new window size to send payload then how this initial window size value is of any use or initial window size is shared after initial TCP handshake?

edit retag flag offensive close merge delete

Comments

Which OS are you running on the client and server?

grahamb gravatar imagegrahamb ( 2019-10-23 12:14:11 +0000 )edit

Hi Grahamb, We are running on Linux 7.1

Regards Amit

Amit gravatar imageAmit ( 2019-10-23 12:16:14 +0000 )edit

Does the server have ethtool loaded?
What is output of ethtool -k <interface name>?

Chuckc gravatar imageChuckc ( 2019-10-23 13:25:31 +0000 )edit

Hi Bubbasnmp,

Please find below ethtool output

Features for bond2:
rx-checksumming: off [fixed]
tx-checksumming: on
        tx-checksum-ipv4: off [fixed]
        tx-checksum-ip-generic: on
        tx-checksum-ipv6: off [fixed]
        tx-checksum-fcoe-crc: off [fixed]
        tx-checksum-sctp: off [fixed]
scatter-gather: on
        tx-scatter-gather: on
        tx-scatter-gather-fraglist: off [requested on]
tcp-segmentation-offload: on
        tx-tcp-segmentation: on
        tx-tcp-ecn-segmentation: on
        tx-tcp6-segmentation: on
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: on
rx-vlan-filter: on
vlan-challenged: off [fixed]
tx-lockless: on [fixed]
netns-local: on [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: on
tx-ipip-segmentation: on
tx-sit-segmentation: on
tx-udp_tnl-segmentation: on
tx-mpls-segmentation: on
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
busy-poll: off [fixed]
tx-sctp-segmentation: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
Amit gravatar imageAmit ( 2019-10-23 18:06:34 +0000 )edit

2 Answers

Sort by ยป oldest newest most voted
0

answered 2019-10-23 22:27:15 +0000

Chuckc gravatar image
tcp-segmentation-offload: on
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off
rx-vlan-offload: on
tx-vlan-offload: on
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]

Offload processing in the NIC might be affecting the window size.
GRO (Generic Receive Offload) is set to ON in the ethtool output.
As a test you might disable GRO and do the capture again.

edit flag offensive delete link more
0

answered 2019-10-23 16:37:03 +0000

cmaynard gravatar image

updated 2019-10-23 16:43:10 +0000

grahamb gravatar image

There's a lot to unpack here.

We are running on Linux 7.1
I guess you meant Red Hat 7.1, so according to https://access.redhat.com/articles/3078, that implies a Linux kernel version of 3.10.0-229.

we have set Initial receive window size value as 22x MSS
How exactly did you attempt to set the initial receive window size value? Presumably, this was done via the ip route command?

So my question is in which packet I can see initial receive window size in wireshark.
The client and server maintain independent receive window sizes. If you want to know the initial recieve window size, you need to look at the SYN packet for the client and the SYN/ACK packet for the server. Note that those packets will indicate the receive window scaling factor; however, the receive window size of those packets are never scaled themselves. All other packets will contiain the calculated window size using the scaling factor learned from the SYN and SYN/ACK packets.

One more to notice was even TCP window scale was 16 still in SYN packet I always see window size and calculated window size as same so not sure why Window scale multiplier doesn't come into the picture for SYN packet.
Yes, that's expected for both SYN and SYN/ACK packets. From section 2.3 of RFC1323:

  *    The window field (SEG.WND) in the header of every incoming
       segment, with the exception of SYN segments, is left-shifted
       by Snd.Wind.Scale bits before updating SND.WND:

          SND.WND = SEG.WND << Snd.Wind.Scale

       (assuming the other conditions of RFC793 are met, and using
       the "C" notation "<<" for left-shift).

Basically, this means that the window size for SYN and SYN/ACK packets are never scaled.

Now, getting to the window size. It would be clearer if a small packet capture file could be uploaded somewhere so we could be sure we're looking at the correct values for the given host. (Again, window sizes and scaling are independent between client and server, so it's important not to confuse them.) For now, I assume the values you provided do correspond to the host of interest.

So, if the MSS is set to 1360 and the intended initial receive window size is 22 x MSS, then indeed one would expect an initial window size of 29,920. Even after looking at the Linux kernel source code, it's unclear to me how exactly the Linux kernel chooses an initial receive window size. For example, on a similar system I have running RHEL7 (7.4) with Linux kernel 3.10.0-957, a TCP connection to this server reveals a window size of 29200, which is 20 x MSS for this system, the MSS being 1460 in this case. However, as I understand from my reading of tcp.h, the default initial receive window size should only be a factor of 10 x MSS, and not 20 x ... (more)

edit flag offensive delete link more

Comments

Hi Cmaynard,

Initial receive window, as well as congestion window both, are set to 22.

default via 10.132.22.129 dev bond1.206 initcwnd 22 initrwnd 22 10.129.148.0/24 via 10.133.191.1 dev bond2.191 initcwnd 22 initrwnd 22 10.129.180.203 via 10.133.191.1 dev bond2.191 initcwnd 22 initrwnd 22 10.129.181.15 via 10.133.191.1 dev bond2.191 initcwnd 22 initrwnd 22

From your explanation, I understand Linux is setting 32 times when we set it to 22 but what would be the usefulness of this initial receive window when Client will not send any data in SYN/ACK and after that amount of data transfer will again depend upon window size mentioned in ACK packet.

So I am not able to see any benefit of setting Initial receive window size as this will be ...(more)

Amit gravatar imageAmit ( 2019-10-23 18:31:37 +0000 )edit

Here's a link to just one of many articles related to TCP tuning that you might find interesting: https://cromwell-intl.com/open-source...

cmaynard gravatar imagecmaynard ( 2019-10-23 18:42:02 +0000 )edit

Thanks Cmaynard, i will take a look but apart from tuning stuff i am a bit more interested in the benefit of TCP initial window parameter and how we can check the exact value from Trace. My Product sits in between Client and Internet server and acts like TCP Transparent proxy so does it means that on server side it advertises TCP initial receive window size in TCP SYN while on Client side it should advertise in TCP SYN-ACK.

Amit gravatar imageAmit ( 2019-10-24 06:45:36 +0000 )edit

Just the opposite. The Client initiates the connection, so the the Client's receive window size is in the SYN packet while the Server's receive window size is in the SYN-ACK packet. If you're looking to know the window size, you can look at 2 fields, tcp.window_size_value, which is the raw value without scaling applied, and tcp.window_size, which is the calculated window size based on the window size value and scaling factor. It may be helpful to add columns for these fields, if that's what you're interested in, but keep in mind that in order for Wireshark to know what the scaling factor is, it must have seen the SYN and SYN-ACK packets, so this usually means you need to start capturing before the TCP connection is initiated.

cmaynard gravatar imagecmaynard ( 2019-10-24 12:42:05 +0000 )edit

thanks, I think you have misunderstood my question let me draw a simple diagram to show where my product sits in the network.

This is the call flow between end-user and Internet server and my product behave as TCP full transparent proxy.

Client ------------> My Product -------> Server

    TCP SYN-->
<--SYN/ACK--
--ACK--->      
                                       TCP SYN-->   
                                      <--SYN/ACK--
                                        --ACK--->

So from my product point of View towards Client, it must share its initial window size in SYN/ACK message towards client and towards server-side it should share Initial window size in TCP SYN message. Now the question is on client-side next message is ACK which doesn't carry any payload while on Server side next message is SYN/ACK which also doesn't carry any payload then how come this Initial window size value is of any use?

As after those messages, new ACK message with new window size ...(more)

Amit gravatar imageAmit ( 2019-10-25 06:49:02 +0000 )edit

As I understand it, the initial window size basically serves as an upper bound for the window size. For my test system then, it advertises an initial receive window size of: (20 * MSS * 512). Why 512? Well, given the MSS of 1460, a factor of 512 is the largest factor allowed without exceeding the system's configured /proc/sys/net/ipv4/tcp_rmem max value of 16777216.

grep . /proc/sys/net/ipv4/tcp_*mem
/proc/sys/net/ipv4/tcp_mem:16777216     16777216        16777216
/proc/sys/net/ipv4/tcp_rmem:4096        524288  16777216
/proc/sys/net/ipv4/tcp_wmem:4096        393216  16777216

As you can see, (20 * 1460) * 512 = 14950400, which is less than 16777216, whereas (20 * 1460) * 1024 = 29900800, and is too big. My question is, "Why the factor of 20 instead of 10, given the value of TCP_DEFAULT_INIT_RCVWND?"

I'm afraid I'll have to defer to other TCP experts on this forum ...(more)

cmaynard gravatar imagecmaynard ( 2019-10-25 13:45:14 +0000 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

Stats

Asked: 2019-10-23 11:58:08 +0000

Seen: 2,838 times

Last updated: Oct 28 '19