Revision history [back]

There's a lot to unpack here.

We are running on Linux 7.1
I guess you meant Red Hat 7.1, so according to https://access.redhat.com/articles/3078, that implies a Linux kernel version of 3.10.0-229.

we have set Initial receive window size value as 22x MSS
How exactly did you attempt to set the initial receive window size value? Presumably, this was done via the ip route command?

So my question is in which packet I can see initial receive window size in wireshark.
The client and server maintain independent receive window sizes. If you want to know the initial recieve window size, you need to look at the SYN packet for the client and the SYN/ACK packet for the server. Note that those packets will indicate the receive window scaling factor; however, the receive window size of those packets are never scaled themselves. All other packets will contiain the calculated window size using the scaling factor learned from the SYN and SYN/ACK packets.

One more to notice was even TCP window scale was 16 still in SYN packet I always see window size and calculated window size as same so not sure why Window scale multiplier doesn't come into the picture for SYN packet.
Yes, that's expected for both SYN and SYN/ACK packets. From section 2.3 of RFC1323:

  *    The window field (SEG.WND) in the header of every incoming
       segment, with the exception of SYN segments, is left-shifted
       by Snd.Wind.Scale bits before updating SND.WND:

          SND.WND = SEG.WND << Snd.Wind.Scale

       (assuming the other conditions of RFC793 are met, and using
       the "C" notation "<<" for left-shift).

Basically, this means that the window size for SYN and SYN/ACK packets are never scaled.

Now, getting to the window size. It would be clearer if a small packet capture file could be uploaded somewhere so we could be sure we're looking at the correct values for the given host. (Again, window sizes and scaling are independent between client and server, so it's important not to confuse them.) For now, I assume the values you provided do correspond to the host of interest.

So, if the MSS is set to 1360 and the intended initial receive window size is 22 x MSS, then indeed one would expect an initial window size of 29,920. Even after looking at the Linux kernel source code, it's unclear to me how exactly the Linux kernel chooses an initial receive window size. For example, on a similar system I have running RHEL7 (7.4) with Linux kernel 3.10.0-957, a TCP connection to this server reveals a window size of 29200, which is 20 x MSS for this system, the MSS being 1460 in this case. However, as I understand from my reading of tcp.h, the default initial receive window size should only be a factor of 10 x MSS, and not 20 x MSS.

In your case, your initial receive window size is 32 x MSS (32 * 1360 = 43520). In both cases - yours and mine - there is an extra 10 x MSS. What happens if you attempt to set the initial receive window size to 12 x MSS? Does the Linux kernel add another 10 x MSS so you end up with the 22 x MSS you desired? If so, then this might explain what's going on; if not, then there's obviously another logical explanation that still alludes us.

in ACK(in response to SYN-ACK from server) Window size was 45056 and some different value in subsequent ACK/PUSH/ACK packets
Yes, that's normal. The receive window size will vary, at least by default. I believe this is controlled by the tcp_moderate_rcvbuf setting. From man tcp:

   tcp_moderate_rcvbuf (Boolean; default: enabled; since Linux
   2.4.17/2.6.7)
          If enabled, TCP performs receive buffer auto-tuning,
          attempting to automatically size the buffer (no greater than
          tcp_rmem[2]) to match the size required by the path for full
          throughput.

There's a lot to unpack here.

We are running on Linux 7.1
I guess you meant Red Hat 7.1, so according to https://access.redhat.com/articles/3078, that implies a Linux kernel version of 3.10.0-229.

we have set Initial receive window size value as 22x MSS
How exactly did you attempt to set the initial receive window size value? Presumably, this was done via the ip route command?

So my question is in which packet I can see initial receive window size in wireshark.
The client and server maintain independent receive window sizes. If you want to know the initial recieve window size, you need to look at the SYN packet for the client and the SYN/ACK packet for the server. Note that those packets will indicate the receive window scaling factor; however, the receive window size of those packets are never scaled themselves. All other packets will contiain the calculated window size using the scaling factor learned from the SYN and SYN/ACK packets.

One more to notice was even TCP window scale was 16 still in SYN packet I always see window size and calculated window size as same so not sure why Window scale multiplier doesn't come into the picture for SYN packet.
Yes, that's expected for both SYN and SYN/ACK packets. From section 2.3 of RFC1323:

  *    The window field (SEG.WND) in the header of every incoming
       segment, with the exception of SYN segments, is left-shifted
       by Snd.Wind.Scale bits before updating SND.WND:

          SND.WND = SEG.WND << Snd.Wind.Scale

       (assuming the other conditions of RFC793 are met, and using
       the "C" notation "<<" for left-shift).

Basically, this means that the window size for SYN and SYN/ACK packets are never scaled.

Now, getting to the window size. It would be clearer if a small packet capture file could be uploaded somewhere so we could be sure we're looking at the correct values for the given host. (Again, window sizes and scaling are independent between client and server, so it's important not to confuse them.) For now, I assume the values you provided do correspond to the host of interest.

So, if the MSS is set to 1360 and the intended initial receive window size is 22 x MSS, then indeed one would expect an initial window size of 29,920. Even after looking at the Linux kernel source code, it's unclear to me how exactly the Linux kernel chooses an initial receive window size. For example, on a similar system I have running RHEL7 (7.4) with Linux kernel 3.10.0-957, a TCP connection to this server reveals a window size of 29200, which is 20 x MSS for this system, the MSS being 1460 in this case. However, as I understand from my reading of tcp.h, the default initial receive window size should only be a factor of 10 x MSS, and not 20 x MSS.

In your case, your initial receive window size is 32 x MSS (32 * 1360 = 43520). In both cases - yours and mine - there is an extra 10 x MSS. What happens if you attempt to set the initial receive window size to 12 x MSS? Does the Linux kernel add another 10 x MSS so you end up with the 22 x MSS you desired? If so, then this might explain what's going on; if not, then there's obviously another logical explanation that still alludes eludes us.

in ACK(in response to SYN-ACK from server) Window size was 45056 and some different value in subsequent ACK/PUSH/ACK packets
Yes, that's normal. The receive window size will vary, at least by default. I believe this is controlled by the tcp_moderate_rcvbuf setting. From man tcp:

   tcp_moderate_rcvbuf (Boolean; default: enabled; since Linux
   2.4.17/2.6.7)
          If enabled, TCP performs receive buffer auto-tuning,
          attempting to automatically size the buffer (no greater than
          tcp_rmem[2]) to match the size required by the path for full
          throughput.