Ask Your Question
0

Help with a TCP Reset Issue

asked 2018-02-07 12:11:10 +0000

FabsCams gravatar image

updated 2018-02-07 14:02:49 +0000

Hi Guys,

This is my first post on here so please advise if I am leaving anything out, and thank you in advance.

Some background:

We are hosting a Remote Application on a Windows 2012 Server using a Windows 2012 Gateway utilising UDP RDP where possible. We have about 30 customer application servers all using the same Virtual Machine server template with the same Application and Windows Gateway server configuration and all 30 customers share the same proxy server. We use a Cisco ASA 5500-X Firewall - we have approximately 250 concurrent users connected however one customer often experiences RDP disconnects affecting all of their office users

No other customers experience these disconnects so I concluded it must be on their end and asked for them to perform a packet capture on their firewall, they agreed and sent me a pcap showing various 'RST' and 'RST, ACK' originating from my firewall. I checked the TTL value and it's consistent with previous packets so I am certain it did come from my Firewall, however during the disconnect there are a whole bunch of 'FIN, ACK' 'RST ACK' and 'RST' coming from mostly my side.

I can then see their Firewall trying to initiate a 'SYN' but my firewall responding with a 'RST, ACK'. From my limited knowledge I checked the Port number it was trying to connect to - '443' now I am certain this port was open as all other customers were connected and we had no other customers complain.

There is a large block of constant SYN > RST, ACK until finally my Firewall connects and responds with a SYN, ACK.

When their RDP disconnections occurred the first noticeable packet is a 'RST, ACK' sent from my Firewall to their Firewall, then there is a storm of RST, ACK, FIN, ACK and SYN's occur.

Has anyone seen this before, I am sure you have and what further information do you need from me?

Thanks

edit retag flag offensive close merge delete

Comments

Can you provide the packet capture? See https://blog.packet-foo.com/2016/11/t... if you need to sanitize the packets first.

Jasper gravatar imageJasper ( 2018-02-07 13:05:13 +0000 )edit

Hi Jasper,

Thanks for link - I have cleaned up the pcap file but not sure how to trim it down - the disconnect occurred around 15:42:15 - if you scroll down to that time you will begin to see all the interesting packets.

https://drive.google.com/open?id=1nGm...

Just to clarify - this PCAP is from he customers Firewall on their external Interface.

FabsCams gravatar imageFabsCams ( 2018-02-07 14:23:43 +0000 )edit

Hi Jasper,

Thanks for link - I have cleaned up the pcap file but not sure how to trim it down - the disconnect occurred around 15:42:15 - if you scroll down to that time you will begin to see all the interesting packets.

https://drive.google.com/open?id=1nGm...

Just to clarify - this PCAP is from he customers Firewall on their external Interface.

FabsCams gravatar imageFabsCams ( 2018-02-07 16:17:22 +0000 )edit

@FabCan´ms I have converted your answer to a comment. As it is more a comment.

Christian_R gravatar imageChristian_R ( 2018-02-07 19:46:52 +0000 )edit

Thank you Christian_R, you can delete it actually as it's a double post now, you can delete this comment too after just to tidy it up, many thanks.

FabsCams gravatar imageFabsCams ( 2018-02-08 10:02:39 +0000 )edit

1 Answer

Sort by » oldest newest most voted
0

answered 2018-02-09 11:59:29 +0000

FabsCams gravatar image

updated 2018-02-09 12:00:51 +0000

I think I might be getting somewhere.

Yesterday the client experienced a disconnect and luckily I managed to capture my side of the traffic.

This is my current setup:

Windows Database << Cisco Firewall << Windows Application Server Running Remote App << WIndows Gateway Server << Proxy Receiving connections on 443 << Cisco ASA Receiving connections on 443

So after they disconnected I had a look at the capture and it matched up perfectly to the capture the client sent me a few days ago, I could see the same exact pattern, they sent a SYN on port 443 to my firewall and it responded with a RST, ACK - it continues for a little while until it connects successfully.

I know for a fact that port 443 was open on my firewall as it was accepting connections for other clients on the same IP Address, using the same Port, using the same Proxy device and same proxy service, so the issue logically would have to be on the either the Gateway server or the Application server not accepting new connections.

There is nothing in the Windows event logs to suggest an issue at the time of the disconnect but I however I did notice that TCP chimney, AutoTuning, Congestion Provider, Task Offloading and ECN Capability on the application server was not set correctly, these should all have been disabled but they were mostly enabled/automatic.

I have disabled these and enabled RSS on the VMXNET3 adapter and within Windows OS to see if it resolves issue.

I will report back if this has fixed it but this is the only thing I can think of at the moment.

edit flag offensive delete link more

Comments

UPDATE This has not resolved the issue. We are now looking at database slowness as as a possible cause.

FabsCams gravatar imageFabsCams ( 2018-02-14 14:39:23 +0000 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2018-02-07 12:11:10 +0000

Seen: 7,781 times

Last updated: Feb 09 '18