I am currently investigating an issue with Windows DNS Servers not working when DNSSEC is enabled and the server is configured to use a forwarder (another Windows DNS) for resolving. Windows Event log states, that DNS Server cannot perform an active refresh of the DNSKEY records of the root zone.
To dig into the issue I performed a Wireshark capture and found the requests and responses for the DNSKEY records. I captured the query/response from my DNS server to the forwarder as well as the same communication to a public DNS when the forwarder is disabled. Unfortunately I cannot attach the capture here.
The DNS response from the forwarder server is "malformed" according to the Wireshark packet dissector, which would explain the DNS server event. However it does not state in which way the packet is "malformed". So I manually followed the RFCs to identify and dissect all the fields of the DNS response by hand. I can see only a few differences, none of which look critical or non-RFC-compliant to me:
- "malformed" response does not contain an RRSIG record
- "malformed" response announces a longer UDP payload size in the EDNS OPT Record
- "malformed" response has a different ordering of the returned DNSKEY records
- "malformed" answer records names are a reference 0xC00C instead of the name of the root zone 0x00
I would like to understand, if something is really wrong with the DNS response, which would explain the server problems, or if the response itself is ok and only the DNS dissector of WIreshark is not working correctly.
Wireshark is version 2.6.4 (v2.6.4-0-g29d48ec8) for Windows
