Big SSL Packets Due to List of CAs. Improve SSL Handshake
We are troubleshooting network latency related to SSL Handshake. This doesn't necessarily cause the delay every time but in some cases, Server take 5-10 seconds just to start sending first SSL packet after Acknowledging Client Hello packet. Attached are snapshot of both Client and Server side captures taken at respective application servers. Please see notes and queries below.
Appreciate your help.
- Captures taken at both Client and Server side which are in different location having router and firewalls on the path.
- SSL Handshake type is set to ‘Optional’ at Tomcat 8.x on Windows 2016.
- Client is a Linux Server.
- Client TCP Window is 5840 and Server TCP Window is 8192
- Client to Server, DF (Don’t Fragment) bit is set to 0
- Server to Client, DF (Don’t Fragment) bit is set to 1
- Server’s SSL Segments (Server Hello, Certificate Chain, Server Key, and Certificate Request) is of total 15456 bytes.
- Out of 15456 bytes, lass SSL Segment of Certificate Request contains ‘Distinguished Names’ packet length of 10912 bytes. This packet has list of about 90 Trust Stores in it.
- Server’s SSL Segments is broken into 12 packets (111330 bytes + 1851 bytes) and they are all pushed in one go from Server without waiting for any ACK from Client.
- Server returns Client Hello ACK and that is received immediately at Client side.
- For some reason Server is seeing DUP-ACK of Client Hello from Client Side. At Client Side captures, we do not see this DUP-ACK going out to Server. This is happening 5 times in the TCP session. Server does Fast Re-transmit Client Hello ACK.
- This Re-transmit of Client Hello ACK is not seen at Client who sees first SSL Packet at 0.94th second (After above Re transmit of Client Hello) and starts sending ACK of these 12 SSL Packets as it received.
- At Server ACKs are seen but the Server which is getting ACKs one-by-one, see that it only got ACK of 5th out of 12th packet. This result Server re-transmit of SSL packets from 6th to 12th.
- At Client, it sees re-transmitted SSL Packets (as it arrived first time). It doesn’t see original SSL packet 6th to 12th which was previously sent to Client in One-Go.
Questions:
- How server is able to send SSL packets of 15456 bytes at one go which is more than TCP Window of 8192 bytes?
- Why Server is seeing DUP-ACK of Client Hello from Client Side multiple times but at Client captures we see these DUP-ACKs are not sent?
- Why Tomcat Server is sending list of CAs in ‘Distinguished Name’ for ‘Certificate Request’ payload? This is having 10940 bytes length? Is this due to some Windows Registry setting?
- Can Server not send CAs list in ‘Certificate Request’ and yet SSL Handshake works? Server Certificate is not issued by 3rd party CA but Server’s own CA.
- How to fix this issue of SSL Handshake delay and make this TCP session efficient?
Captures Snapshots:
Sharing the capture file on a publicly accessible site makes analysing this issue so much better.
Captures couldn't be exported out of network. I put the snapshots link of the captures. Hope these details provides extra information.
The snapshots did not get through... Are you allowed to use tracewrangler to anonimize the capture files (replace mac and IP addresses and remove all data above the TCP layer) and then share the anonimized pcaps somewhere?
I think for #1 and #2 you will need to capture "on the wire" to see what the real packets are.
@bubbasnmp Yes, catching the packets on the wire is always preferred over catching them within the endpoints. Some questions can and some questions can't be answered when capturing on one of the endpoints.