I'm in the process of troubleshooting an issue which is causing some slowness in establishing connections between our Linux servers. I've suspected DNS, but both forward and reverse lookups using dig are extremely fast. As a test I've been attempting to test a web connection between two Linux servers. At a prompt I attempt to telnet to port 80 on the other box. After about a 10 second delay a connection is established, entering GET / returns HTML very quickly. SSH exhibits the same symptoms, after about a 10 second delay in connecting the connection is fast. I've taken some packet dumps using tcpdump and am viewing my findings in Wireshark. I can't explain the output, but it is consistent between multiple DNS servers. When I enter the telnet command to connect to port 80 on the second machine I see the following exchange. Note: In this scenario I was attempting to connect from 10.3.1.171 to academic.luzerne.edu. I have a single DNS server listed in resolv.conf, 10.3.1.166. The command entered was telnet academic.luzerne.edu 80. Source: 10.3.1.171 Dst: 10.3.1.166 Proto: DNS Info: Standard query AAAA academic.luzerne.edu .002 second later the reply: Source: 10.3.1.166 Dst: 10.3.1.171 Proto: DNS Info: Standard query response .0002 seconds later another request: Source: 10.3.1.171 Dst: 10.3.1.166 Proto: DNS Info: Standard query AAAA academic.luzerne.edu.luzerne.edu Over 2 seconds later another request (after timeout?): Source: 10.3.1.171 Dst: 10.3.1.166 Proto: DNS Info: Standard query AAAA academic.luzerne.edu.luzerne.edu Slightly over 5 seconds later another request: Source: 10.3.1.171 Dst: 10.3.1.166 Proto: DNS Info: Standard query AAAA academic.luzerne.edu .0002 seconds later a valid reply: Source: 10.3.1.166 Dst: 10.3.1.171 Proto: DNS Info: Standard query response A 10.3.1.41 At this point the TCP handshake occurs and the connection proceeds. The DNS delay certainly explains the slowness in my connection. I have no idea how to interpret this. Can any give me any suggestions? Thanks, Bob asked 21 Oct '10, 12:03  bobd |
3 Answers:
Well, it looks like your box is asking for an IPv6 DNS record (because you have the query type "AAAA" in your first couple of requests). For an IPv4 address you should have a query type of a single "A" instead. My guess is (and I'm no IPv6 specialist yet) that your box is first trying to get an IPv6 address, doesn't get any reply, retries a couple of times after some delay, and finally tries to ask for an IPv4 Address (as seen in your last Query statement). That one succeeds and the connection is established. My tip: disable IPv6 or at least DNS lookups for IPv6 addresses, and everything should work as quickly as expected. answered 22 Oct '10, 04:43  Jasper ♦♦ |
That 2 seconds pause definitely looks like is a DNS-based timeout Look at the difference in your DNS servers response: First Request --> Info: Standard query AAAA academic.luzerne.edu .002 second later the reply: [...] Info: Standard query response <-- some answer, look into it Second Request --> .0002 seconds later Info: Standard query AAAA academic.luzerne.edu.luzerne.edu You see the "doubled" luzerne.edu -> my tip is that there might be a small problem inside the zone file (Missing "." (dot) after some statement inside maybe, resulting in this strange second request. But that's only a guess... You should definitely look at the DNS response to the first request what the server tells you inside. answered 22 Oct '10, 05:19  Landi |
As Jasper said, disabling IPv6 will likely fix your issue. You are having to wait for the AAAA (IPv6) query to timeout before the box tries a IPv4 AAA query. However is you want to further eliminate DNS from the issue or do some comparison, you can access the server via IP address or put in static host entries to skip the DNS resolve on your test machine. answered 22 Oct '10, 15:31  Peter |
Thanks everyone! The problem was related to the IPv6. I've disabled it on our servers and speed is back to normal.
Thanks again, Bob
very helpful for me , thank you ^^