After chasing this issue with a ARM device only connecting to an email server sometimes finally the people who wrote the TCP/IP stack mentioned that if there is not random delay in between email sends the device would generate identical sequence numbers upon restart (since it uses up-time to seed the pseudo random number generator). I checked in Wireshark and sure enough I had identical sequence numbers in between restarts and the first one always would work and all consecutive tries with the same sequence number failed. Unless I walked away for a minute or two and returned and then restarted the device at which point the sequence number would work the first time and then not work all consecutive times. So is there some timeout period after which the same sequence number can be used on an SMTP server? And consequently is there some period for which the server ignores any connect requests from sequence numbers it has already seen? How long would that period be? asked 20 May '14, 11:32  wes000000 |
One Answer:
Not exactly, see below. (Notes:
I expect that the issue has to do with how TCP works (and nothing to do with SMTP) but is related to a timeout. I'm not an expert on all of this, but I think the following gives the gist: TCP has a concept of "time-wait" being a timeout after a TCP connection "completes" during which a new connection (with the same IP-address/TCP-port endpoints) will not be accepted by a server. So: A question: Are the multiple connection requests using the same source port so that the two IP-address/TCP-port endpoints are identical for each connection ? If so, the "time-wait" timeout is the reason you have to wait a while before a new connection to the server can be established. I believe the default "time-wait" timeout is 2 minutes. Normally a client will increment the TCP source port for each new connection to a server so that this issue is avoided. In your case, you seem to indicate that the client restarts (often ?) and it seems that after a restart that the initial conditions are the same. There's more detail and subtlety to all of this; A web search will provide the gory details. answered 20 May '14, 22:40  Bill Meier ♦♦ showing 5 of 6 show 1 more comments |
Both your assumptions are correct: I did turn off relative TCP sequence numbers in Wireshark and I was referring to TCP sequence numbers.
The client does use the same ports each time and the same IP. And the "time-wait" may be a large part of my problem as yes I am intentionally restarting the client often and after each restart everything goes back to a default condition.
But I am thinking there must be something to do with the sequence numbers too, because if I wait several minutes and turn on the device both emails will send (two different port numbers and two different sequence numbers). But if I restart the device again the first email fails to send (identical port and sequence number to first try) but the second email succeeds (also identical port # but a different sequence number).
Maybe the server only ignores the request if it is the same IP, same port, and the same sequence #...
It's certainly possible. (As I said I'm not an expert on all the details of TCP).
I'm a little confused, however.
I'm understanding you to say that "waiting several minutes before turning on the device" (after sending EMails ?) gives different results than "restarting the device" (not waiting ?).
Based upon your description, I got the impression that the same ports & sequence numbers were used each time after a restart.
Would you be able to share a capture (along with some info about when the device is reset, etc) ?
See standard way people should attach capture fles
You are understanding what I am saying correctly. A device 'turn on' is the same a 'reset' the only difference is whether I reset the device back to back or wait several minutes and then reset (turn off and on) the device.
The same ports are being used after each restart, it always starts at 1024 for the first email and increments up to 1025 for the second email.
The sequence numbers are always the same after each restart as well. If I wait several minutes before resetting I get the same two sequence numbers and port # I do every time, but it works. Form that point forward any requests with the same port, same IP, and same sequence # fail. But requests with the same port # and IP but different sequence #'s succeed.
Here are the before and after reset capture files on Google Drive:
https://drive.google.com/file/d/0B-pYPAmyNVqbTWpLaWNpeWx5MjQ/edit?usp=sharing
https://drive.google.com/file/d/0B-pYPAmyNVqbUHQwMkpZQjNZMk0/edit?usp=sharing
If a server has a connection in the TIME_WAIT state, and it receives a new connection request (SYN) using the same client and server ports, it should drop the old connection and respond to the new connection with a SYN|ACK if the packet sequence number of the new request is higher than that of the old connection.
However, if the sequence of the new connection request is much higher than that of the old connection, the server may respond with an ACK for the previous connection.
An iptrace taken during the proper interaction will show:
TCP: <source port="32869," destination="" port="2050"> TCP: th_seq=f548ee61, th_ack=e24dabe0 TCP: th_off=5, flags<fin |="" ack="">
TCP: <source port="2050," destination="" port="32869"> TCP: th_seq=e24dabe0, th_ack=f548ee62 TCP: th_off=5, flags<ack>
TCP: <source port="32869," destination="" port="2050"> TCP: th_seq=f54bdc5d, th_ack=0 TCP: th_off=10, flags<syn>
TCP: <source port="2050," destination="" port="32869"> TCP: th_seq=f54ae262, th_ack=f54bdc5e TCP: th_off=6, flags<syn |="" ack="">
When the difference in the packet sequence number of the new connection request and that of the previous connection is greater than 2147483647, the server does not drop the old connection.
The following iptrace was taken when the packet sequence number (4074992888) of the new connection request (SYN) is 2233732218 higher than that of the previous connection (1841260670). Here, the server does not drop the old connection:
TCP: <source port="2212," destination="" port="8970"> TCP: th_seq=1841260670, th_ack=200941090 TCP: th_off=5, flags<fin |="" ack="">
TCP: <source port="8970," destination="" port="2212"> TCP: th_seq=200941090, th_ack=1841260671 TCP: th_off=5, flags<ack>
TCP: <source port="2212," destination="" port="8970"> TCP: th_seq=4074992888, th_ack=0 TCP: th_off=7, flags<syn>
TCP: <source port="8970," destination="" port="2212"> TCP: th_seq=200941090, th_ack=1841260671 TCP: th_off=5, flags<ack>
Local fix
If the device were to send a "SYN with source port, destination port, and sequence # all identical" or a "SYN packet with same source and destination port but a seq # greater then 2233732218 from the previous" all the same would the server respond with anything or does it simply ignore the request and send nothing back?
And so from what I'm understanding (correct me if I'm wrong) to get away with using the same ports everytime the new sequence # needs to be greater than the previous # but less than (2233732218 + previous). And if those conditions are true it will work every time?
This problem has been explained and 'solved' in another question: