SSL connection failing

asked 2025-04-08 13:52:36 +0000

Ronnie
1 ●1

updated 2025-04-08 14:40:41 +0000

Help! We converted to a new IBM iSeries (AS400). Since the conversion I can't connect to our EDI trading partner's FTP server using SSL. I sill have access to our old AS400 and it connects fine. However when I attempt to connect with my new server the connection hangs after "234 AUTH TLS successful". In the pcap file, after sending "Client Hello" I get back "TCP Previous segment not captured".

The two AS400 servers are different models, but the OS, user data and configuration was copied and reloaded to the new server. (i.e. a complete scratch load of the OS). So the two system should be identical from an OS perspective.

I asked tech support on the FTP server side to send me a new CA certificate, which I installed. Still didn't help.

Hopefully the links below work.

https://www.dropbox.com/scl/fi/hdas9d...

https://www.dropbox.com/scl/fi/1fxti0...

Comments

To share a capture file, place it on a public file share then update the question with a link to it.

Chuckc ( 2025-04-08 14:22:41 +0000 )edit

thanks Chuckc

Ronnie ( 2025-04-08 14:33:08 +0000 )edit

Can you make a capture on both ends - client and new server?
There seems to be a missing frame(s) between 8 and 9.

Chuckc ( 2025-04-08 15:16:03 +0000 )edit

I have been unable to get the server side support team to do a capture. I've been talking with them for a month and they seem reluctant, or unable to do a packet capture. My guess is they're reselling services from another provider and don't have the technical staff to do a capture themselves. Unfortunately using this EDI VAN service is dictated by our end customer. Otherwise I'd boot them to the curb and use someone else.

Ronnie ( 2025-04-08 15:41:05 +0000 )edit

add a comment

answered 2025-04-09 20:57:50 +0000

SYN-bit

18570 ●9 ●352 ●255 https://SYN-b.it

Looks like the old AS400 was tweaked to use an MTU of 1492 instead of the standard 1500. The new AS400 is using an MTU of 1500.

As the FTP server now assumes it can send 1460 bytes segments, it does so and two of these are missing (there is a gap of 2920 bytes in the TCP stream). As they are not received, they are not ACKed, so they are retransmitted from the server side, but again too big, so these segments never arrive at the (new) AS400.

Have a look at the NIC and TCP settings of the old AS400 and the new AS400 and make sure they match. I expect that only the MTU needs to be lowered to 1492 (1492 is seen on connections that use PPPoE, as the PPPoE header is 8 bytes and will be added to all packets).

edit flag offensive delete link

Comments

Thanks for the info. Where within the pcap file can I see the MTU size? I've selected frame 8 and expanded all the details in the bottom window, but can't seem to find a value of 1500.

Ronnie ( 2025-04-10 10:19:40 +0000 )edit

The MTU size is not directly visible in network packets, as it is a setting off the network interface. But the TCP MSS is derived from the MTU by substracting 20 bytes for the IP header and 20 bytes for the TCP header. This means, an MSS of 1460 indicates an MTU of 1500 and an MSS of 1452 indicates an MTU of 1492.

See the TCP MSS option in the SYN packets of both AS400s

SYN-bit ( 2025-04-10 10:52:11 +0000 )edit

there is a gap of 2920 bytes in the TCP stream

This is a manual calculation (no existing field)?
I'm always looking for MATE and Lua examples and seems like it would be a good one to script. :-)

Chuckc ( 2025-04-10 12:22:53 +0000 )edit

Yup, manual calculation, seems to be something a field we could add to the TCP dissector. But it is tricky when there are multiple gaps, partly filled up, out-of-orders etc. Maybe something like bytes missing since previously seen segment in the stream could make it a bit easier...

SYN-bit ( 2025-04-10 12:35:06 +0000 )edit

A companion to tcp.analysis.bytes_in_flight.

But it is tricky

Code probably not so bad but the RFCs for TCP make it somewhat mysterious how to define things.

Chuckc ( 2025-04-10 13:09:40 +0000 )edit

add a comment

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

SSL connection failing

Comments

1 Answer

Comments

Your Answer

Question Tools

Stats

SSL connection failing edit

Comments

1 Answer

Comments

Your Answer

Question Tools

Stats

SSL connection failing