The TCP MSS value is advertised to the peer, it is not negotiated.
When the MSS is not adjusted along the path by an intermediate device, this will result in both sides using the same maximum size for segments. That's because the host that sends the smallest MSS is not capable of sending larger segments and the host receiving the smallest MSS will adjust to the MSS it received.
When the MSS does get adjusted by an intermediate device, then things can go wrong if there is asymmetric routing and the two paths have a different MTU (or tunneling overhead). This is because the MSS being received will be used to send data over the other link. You can solve this by making sure the MSS gets adjusted to the lowest value in both paths of the traffic.
Here is an example of things going wrong with asymmetric routing and MSS adjustments:
- Host A sends a MSS of 1460
- The traffic follows path X in which there is a link with an MTU of 1420, so the MSS is adjusted to 1380
- Host B receives the MSS of 1380
- Host B sends a MSS of 1460
- The traffic follows path Y in which there is a link with an MTU of 1400, so the MSS is adjusted to 1360
- Host A receives the MSS of 1360
- When host A sends a full-size-segment, it uses the MSS of 1360
- The packet takes route X and can be sent without fragmentation (1360 is less than 1380)
- When host B sends a full-size-segment, it uses the MSS of 1380
- The packet takes route Y and can not be sent without fragmentation (1380 is greater than 1360)
- If any device in path X changes the MSS to 1360, then host B will also use segment of max 1360 bytes
No. My understanding is that the smallest will be used because it may not be the host actually contributing to the smaller size. It could be an indication of network capability.