DNS Client -> This Server (this-server) works with DNS-over-TCP but not DNS-over-UDP in 15.4 when using VRFs

[Apachez-]

Version 15.4 fixed some issues regarding selecting device or VRF for "DNS Server Local End Points" which is handy if you use VRFs.

However I assume since neither DNS Server IPV4 Source Addresses or DNS Server IPv6 Source Addresses allows the new syntax of %VRF this means that DNS-over-TCP works with the builtin DNS Client but not DNS-over-UDP.

Current settings:

Settings -> General -> DNS Server Local End Points: 192.168.1.20%PROD:53
Settings -> General -> DNS Server IPv4 Source Addresses: 192.168.1.20
Settings -> General -> DNS Server IPv6 Source Addresses: ::

Above works for incoming DNS queries both regarding DNS-over-UDP and DNS-over-TCP (verified with "dig @192.168.1.20 localhost" aswell as "dig @192.168.1.20 +tcp localhost").

But when using DNS Client through the webgui with "Server" set to "This Server (this-server)" then egressing DNS-over-TCP works but not DNS-over-UDP who gets timeout:

Error! DnsClient failed to resolve the request 'ping.sunet.se. A IN': request timed out for name server [vyos1.example.com (192.168.1.20)]. Connection timed out

Nothing obvious is seen in the log even with stack trace enabled.

Doing a tcpdump I do see the requests leaving and returning to the interface but then nothing:

14:24:29.727360 IP 192.168.1.20.55410 > 192.168.1.20.domain: 62546+ [1au] A? ping.sunet.se. (42)
14:24:29.727956 IP 192.168.1.20.domain > 192.168.1.20.55410: 62546 1/0/1 A 192.36.125.18 (58)
14:24:39.728945 IP 192.168.1.20.52958 > 192.168.1.20.domain: 62546+ A? ping.sunet.se. (31)
14:24:39.729508 IP 192.168.1.20.domain > 192.168.1.20.52958: 62546 1/0/0 A 192.36.125.18 (47)

I also see several "ICMP 192.168.1.20 udp port unreachable" being sent by 192.168.1.20 back to external DNS servers?

[Apachez-]

Changing "DNS Server Local End Points" from "192.168.1.20%PROD:53" to "192.168.1.20%eth1:53" gives the same result regarding "DNS Client".

"DNS Server IPv4 Source Addresses" remained at "192.168.1.20".

That is DNS-over-TCP works but not DNS-over-UDP:

Error! DnsClient failed to resolve the request 'ping.sunet.se. A IN': request timed out for name server [vyos1.example.com (192.168.1.20)]. Connection timed out

I also tried changing "DNS Server IPv4 Source Addresses" to "0.0.0.0" but now getting:

Error! Network is unreachable

Edit:

I can add that changing from "This Server (this-server)" to these (as example) works for both DNS-over-UDP and DNS-over-TCP:

• 192.168.1.20
• 8.8.8.8
• 1.1.1.1
• Recursive Query (recursive-resolver)

But when using "This Server (this-server)" only DNS-over-TCP works while DNS-over-UDP timeouts.

[ShreyasZare]

Thanks for the feedback. Will investigate this and let you know soon.

[ShreyasZare]

Settings -> General -> DNS Server Local End Points: 192.168.1.20%PROD:53

Is that the only local end point configured? If yes then I would suggest that you add 127.0.0.1 and/or 0.0.0.0 in there and see if that makes any difference. Try with the source address option set to 0.0.0.0 too.

Settings -> General -> DNS Server IPv4 Source Addresses: 192.168.1.20

Does your scenario really require using source address?

Just clarifying here that the source address option is not used with the DNS service itself. That is, the inbound request that are answered back do not use the source address feature. The response are sent back using the same socket from where the request was received on.

The source address option is used only for recursive resolution and forwarding. Since this is a global option, it affects the DNS Client code too and the sockets it use for resolving queries.

[Apachez-]

1. I will try and get back with results. 0.0.0.0 is however not an option for me since Im using VRFs. I want the DNS-server to only use VRF PROD and not VRF MGMT for incoming AND outgoing DNS-traffic.

2. See above. I want to lock technitium/dns-server to only use interfaces which belongs to VRF PROD. It should never use VRF MGMT for incoming or outgoing traffic.

[Apachez-]

Adding "0.0.0.0" to "DNS Server IPv4 Source Addresses" like so:

192.168.1.20
0.0.0.0

Didnt change anything, DNS-over-TCP works but DNS-over-UDP doesnt.

So I tried this instead:

192.168.1.20
127.0.0.1

Same result.

It also failed no matter if I used this as "DNS Server Local End Points":

192.168.1.20%PROD:53

Or added to listen at 127.0.0.1:

192.168.1.20%PROD:53
127.0.0.1:53

Here is output from DNS-client when it do work using DNS-over-TCP:

{
  "Metadata": {
    "NameServer": "vyos1.example.com (192.168.1.20)",
    "Protocol": "Tcp",
    "DatagramSize": "58 bytes",
    "RoundTripTime": "1.08 ms"
  },
  "EDNS": {
    "UdpPayloadSize": 1232,
    "ExtendedRCODE": "NoError",
    "Version": 0,
    "Flags": "None",
    "Options": []
  },
  "Identifier": 0,
  "IsResponse": true,
  "OPCODE": "StandardQuery",
  "AuthoritativeAnswer": false,
  "Truncation": false,
  "RecursionDesired": true,
  "RecursionAvailable": true,
  "Z": 0,
  "AuthenticData": false,
  "CheckingDisabled": false,
  "RCODE": "NoError",
  "QDCOUNT": 1,
  "ANCOUNT": 1,
  "NSCOUNT": 0,
  "ARCOUNT": 1,
  "Question": [
    {
      "Name": "ping.sunet.se",
      "Type": "A",
      "Class": "IN"
    }
  ],
  "Answer": [
    {
      "Name": "ping.sunet.se",
      "Type": "A",
      "Class": "IN",
      "TTL": "25668 (7h7m48s)",
      "RDLENGTH": "4 bytes",
      "RDATA": {
        "IPAddress": "192.36.125.18"
      },
      "DnssecStatus": "Disabled"
    }
  ],
  "Authority": [],
  "Additional": [
    {
      "Name": "",
      "Type": "OPT",
      "Class": "1232",
      "TTL": "0 (0s)",
      "RDLENGTH": "0 bytes",
      "RDATA": {
        "Options": []
      },
      "DnssecStatus": "Disabled"
    }
  ]
}

[ShreyasZare]

Thanks for the details.

Adding "0.0.0.0" to "DNS Server IPv4 Source Addresses" like so:

There seems to be some confusion. I wanted to see if adding 0.0.0.0 to DNS Server Local End Points makes any difference.

The "This Server" option selects the IP end point to query based on the local end points that are configured. So, having 0.0.0.0 in there should make some difference when no source address is configured.

[Apachez-]

Yes,

Adding "0.0.0.0:53" ("192.168.1.20%PROD:53" is already present) to "DNS Server Local End Points" did make a difference.

Now both DNS-over-UDP and DNS-over-TCP works:

{
  "Metadata": {
    "NameServer": "vyos1.example.com (127.0.0.1)",
    "Protocol": "Udp",
    "DatagramSize": "58 bytes",
    "RoundTripTime": "1.01 ms"
  },

Odd thing here is that 127.0.0.1 is being used according to the output of above?

So changing "0.0.0.0:53" into "127.0.0.1:53" should work aswell but it doesnt... now TCP-over-DNS works but UDP-over-DNS doesnt!?

Using 0.0.0.0:53 as "DNS Server Local End Points" isnt really an option to me.

But perhaps might nail it down to where the error might be in the code?

I also tried this:

DNS Server Local End Points:

192.168.1.20%PROD:53
127.0.0.1%lo:53
127.0.0.1:53

DNS Server IPv4 Source Addresses

192.168.1.20
127.0.0.1
0.0.0.0

Above is confirmed by "sudo ss -atulpn | grep 53":

udp   UNCONN 0      0              127.0.0.1:53         0.0.0.0:*    users:(("dotnet",pid=51938,fd=285))
udp   UNCONN 0      0           127.0.0.1%lo:53         0.0.0.0:*    users:(("dotnet",pid=51938,fd=252))
udp   UNCONN 0      0      192.168.1.20%PROD:53         0.0.0.0:*    users:(("dotnet",pid=51938,fd=247))
tcp   LISTEN 0      1000           127.0.0.1:53         0.0.0.0:*    users:(("dotnet",pid=51938,fd=286))
tcp   LISTEN 0      1000           127.0.0.1:53         0.0.0.0:*    users:(("dotnet",pid=51938,fd=284))
tcp   LISTEN 0      512          10.99.0.100:53443      0.0.0.0:*    users:(("dotnet",pid=51938,fd=246))
tcp   LISTEN 0      512          10.99.0.100:5380       0.0.0.0:*    users:(("dotnet",pid=51938,fd=245))
tcp   LISTEN 0      1000        192.168.1.20:53         0.0.0.0:*    users:(("dotnet",pid=51938,fd=248))

Where with the original config it looks like this:

udp   UNCONN 0      0      192.168.1.20%PROD:53         0.0.0.0:*    users:(("dotnet",pid=51938,fd=247))
tcp   LISTEN 0      512          10.99.0.100:53443      0.0.0.0:*    users:(("dotnet",pid=51938,fd=246))
tcp   LISTEN 0      512          10.99.0.100:5380       0.0.0.0:*    users:(("dotnet",pid=51938,fd=245))
tcp   LISTEN 0      1000        192.168.1.20:53         0.0.0.0:*    users:(("dotnet",pid=51938,fd=248))

It only seems to be working when adding "0.0.0.0:53" to "DNS Server Local End Points" no matter what the other values are in both "DNS Server Local End Points" and "DNS Server IPv4 Source Addresses".

Which then outputs this from the DNS client for DNS-over-UDP:

{
  "Metadata": {
    "NameServer": "vyos1.example.com (127.0.0.1)",
    "Protocol": "Udp",
    "DatagramSize": "54 bytes",
    "RoundTripTime": "0.54 ms"
  },

[ShreyasZare]

Thanks for the feedback. I think you can try the following values in the given order for DNS Server Local End Points and it should work:

DNS Server Local End Points:

127.0.0.1:53
192.168.1.20%PROD:53

The "This Server" code is picking up the first entry if it does not find 0.0.0.0 or :: in there. Will get that code updated so that the above order to have loopback as the first entry is not necessary to maintain.

[Apachez-]

I tried both ("DNS Server Local End Points"):

127.0.0.1:53
192.168.1.20%PROD:53

and

127.0.0.1%lo:53
192.168.1.20%PROD:53

Still DNS-over-TCP works (says its using "NameServer": "vyos1.example.com (127.0.0.1)", according to the response log?) while DNS-over-UDP timeout.

Replacing 127.0.0.1:53 with 0.0.0.0:53 will make DNS-over-UDP work however it will still say that it is using 127.0.0.1 in the response log?

If I remove 127.0.0.1 and 0.0.0.0 so it will only be (as I began with) just "192.168.1.20%PROD:53" the DNS-over-TCP works but DNS-over-UDP fails. The response log for DNS-over-TCP (which works) now says its using "192.168.1.20" instead of "127.0.0.1".

However when I in DNS-client change "Server" from "This Server {this-server}" to "192.168.1.20" then both DNS-over-TCP and DNS-over-UDP works.

So the order I write "DNS Server Local End Points" doesnt seem to matter?

Summary (DNS-over-TCP works all the time its DNS-over-UDP that timeout):

DNS-client Server set to "This Server {this-server}":

Works (order doesnt seem to matter):

0.0.0.0:53
192.168.1.20%PROD:53

Does not work:

192.168.1.20%PROD:53

127.0.0.1:53
192.168.1.20%PROD:53

127.0.0.1%lo:53
192.168.1.20%PROD:53

DNS-client Server set to "127.0.0.1":

Works:

127.0.0.1:53
192.168.1.20%PROD:53

127.0.0.1%lo:53
192.168.1.20%PROD:53

DNS-client Server set to "192.168.1.20":

Works:

192.168.1.20%PROD:53

So it seems that whatever "This Server {this-server}" is doing is what malfunctions when you have VRFs configured because specifying "192.168.1.20" (or "127.0.0.1" when configured) works as "Server" even when VRFs are being used?