Strange timing in SSL handshake

The path is probably going through a firewall infrastructure.
When A SYN packet hits the firewall, it will cause a lookup in the filter rules database, create a connection entry in its stateful connection table etc.
All this processing might be duplicated at the edges of the public network, so it is not surprising that the SYN packets are delayed as they are being inspected for security reasons.
Once the connection is setup, the packets just pass through an 'existing' connection and are simply routed which certainly is faster...
Just a thought that might or might not be the what is happening here ...

Regards Matthias