DNS Update Issue

dbeato

@scottalanmiller the only problem with Microsoft Windows DNS Clients when used for authentication that it is so random to choose which DC to login to which makes it so unpredictable. But I know that it out of the main scope of this discussion, but wanted to clarify that.

Obsolesce

@scottalanmiller said in DNS Update Issue:

@Dashrender said in DNS Update Issue:

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

In Windows, apparently, it simply abandones that server until it has no choice but to return.

I don't see any issue there. You're getting DNS either way, what's it matter what it's from if they are the same? If clients are getting DNS from the failover DNS server and you don't want it to, turn off the DNS service on that server then, and clients will fail back... if you even care.

The problem happens when your secondary server isn't part of your internal network (assuming your primary is part of your internal network). When using the secondary you won't get resolution for internal network resources.

That's the BIG problem. But not the only one. Take a common manufacturing plant with one AD at one site, and the other one at a different site. If you can't choose primary or secondary, then failover means slow DNS over a WAN link - potentially for weeks or months at a time. Sometimes for no reason at all, or something as simple as having rebooting the local one.

It's not just wanting to use a public source, that clouds the issue. Lots of people don't want to use public ever, so ignore that. It's bad behaviour regardless.

I've never seen this. We have a DC at one site and a DC at another site. DNS at the computers primary site is always preferred.

Perhaps the whole issue is because AD sites aren't set up properly.

I'll test this now in a properly set up AD environment.

scottalanmiller

@dbeato said in DNS Update Issue:

@scottalanmiller the only problem with Microsoft Windows DNS Clients when used for authentication that it is so random to choose which DC to login to which makes it so unpredictable. But I know that it out of the main scope of this discussion, but wanted to clarify that.

Performance of the DNS itself and traffic on a limited WAN link are also concerns.

scottalanmiller

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

@Dashrender said in DNS Update Issue:

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

In Windows, apparently, it simply abandones that server until it has no choice but to return.

I don't see any issue there. You're getting DNS either way, what's it matter what it's from if they are the same? If clients are getting DNS from the failover DNS server and you don't want it to, turn off the DNS service on that server then, and clients will fail back... if you even care.

The problem happens when your secondary server isn't part of your internal network (assuming your primary is part of your internal network). When using the secondary you won't get resolution for internal network resources.

That's the BIG problem. But not the only one. Take a common manufacturing plant with one AD at one site, and the other one at a different site. If you can't choose primary or secondary, then failover means slow DNS over a WAN link - potentially for weeks or months at a time. Sometimes for no reason at all, or something as simple as having rebooting the local one.

It's not just wanting to use a public source, that clouds the issue. Lots of people don't want to use public ever, so ignore that. It's bad behaviour regardless.

I've never seen this. We have a DC at one site and a DC at another site. DNS at the computers primary site is always preferred.

Perhaps the whole issue is because AD sites aren't set up properly.

I'll test this now in a properly set up AD environment.

I've not seen it either. But it is the reason behind most everyone's decision making around WIndows DNS clients. So my point is that either that reason is false and Windows doesn't actually do this, or it's significant.

Obsolesce

@scottalanmiller said in DNS Update Issue:

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

@Dashrender said in DNS Update Issue:

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

In Windows, apparently, it simply abandones that server until it has no choice but to return.

I don't see any issue there. You're getting DNS either way, what's it matter what it's from if they are the same? If clients are getting DNS from the failover DNS server and you don't want it to, turn off the DNS service on that server then, and clients will fail back... if you even care.

The problem happens when your secondary server isn't part of your internal network (assuming your primary is part of your internal network). When using the secondary you won't get resolution for internal network resources.

That's the BIG problem. But not the only one. Take a common manufacturing plant with one AD at one site, and the other one at a different site. If you can't choose primary or secondary, then failover means slow DNS over a WAN link - potentially for weeks or months at a time. Sometimes for no reason at all, or something as simple as having rebooting the local one.

It's not just wanting to use a public source, that clouds the issue. Lots of people don't want to use public ever, so ignore that. It's bad behaviour regardless.

I've never seen this. We have a DC at one site and a DC at another site. DNS at the computers primary site is always preferred.

Perhaps the whole issue is because AD sites aren't set up properly.

I'll test this now in a properly set up AD environment.

I've not seen it either. But it is the reason behind most everyone's decision making around WIndows DNS clients. So my point is that either that reason is false and Windows doesn't actually do this, or it's significant.

Okay, so... my conclusive results.

I did some testing the only way I could figure out... if there's more I can do, let me know.
(EDIT: I did some additional testing noted below the numbered list)

On a Windows machine, I:

Ping / test nslookup to a server. Make note of DNS server used for NSLookup.
Block IP of primary DC/DNS/Domain server via firewall.
ipconfig /flushdns
Test ping / test nslookup a server. Make note of DNS server used.
UNblock IP of primary DC/DNS/Domain server.
Test ping / test nslookup a server. Make note of DNS server used.

Results:
While the IP of primary DC/DNS/Domain server was blocked, an nslookup resulted in error, that the DNS request timed out, however, I was still able to ping all internal devices by name, as well as external websites. Then I unblocked it, and was still able to ping by name and nslookup was working normally again.

Additional Testing:
To test for real DNS blockage, I added all of our internal DNS servers to the block list and tested pinging stuff by name. Unable to ping any internal device by name.

So now I unblocked, let's call it DC/DNS-2, and left everything else blocked. I was now able to instantly ping things by name, guaranteed to using secondary DC/DNS server.

Upon unblocking the primary DC/DNS server, nslookup worked instantly and I was also able to continue pinging things. Same when blocking the other DNS server again.

Keep in mind that before each test, I performed a ipconfig /flushdns command to be sure.

I wasn't yet convinced, so to completely mitigate any unknowns, as a last path, I enabled Debug Logging on the primary and secondary DC/DNS server filtering stuff from the workstation i'm testing from. I had two powershell windows open side by side, -Tailing the DNS log file from each DC/DNS server.

From there, I continued having the primary DC/DNS server blocked (all DC/DNS servers except the secondary). I removed the primary DC/DNS ip address from being blocked on the workstation, performed a flushdns, and pinged something.

Within a minute I flushed DNS then pinged something internal by name, and it used the primary DNS server, not the secondary one that it was using before I unblocked the primary DNS ip.

This proves it is instant (or nearly instant, as there was about a minute between the time I unblocked the IP and sent out a ping), at least on Windows 10 Pro 1803, in a MS AD Domain environment.

scottalanmiller

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

@Dashrender said in DNS Update Issue:

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

In Windows, apparently, it simply abandones that server until it has no choice but to return.

I don't see any issue there. You're getting DNS either way, what's it matter what it's from if they are the same? If clients are getting DNS from the failover DNS server and you don't want it to, turn off the DNS service on that server then, and clients will fail back... if you even care.

The problem happens when your secondary server isn't part of your internal network (assuming your primary is part of your internal network). When using the secondary you won't get resolution for internal network resources.

That's the BIG problem. But not the only one. Take a common manufacturing plant with one AD at one site, and the other one at a different site. If you can't choose primary or secondary, then failover means slow DNS over a WAN link - potentially for weeks or months at a time. Sometimes for no reason at all, or something as simple as having rebooting the local one.

It's not just wanting to use a public source, that clouds the issue. Lots of people don't want to use public ever, so ignore that. It's bad behaviour regardless.

I've never seen this. We have a DC at one site and a DC at another site. DNS at the computers primary site is always preferred.

Perhaps the whole issue is because AD sites aren't set up properly.

I'll test this now in a properly set up AD environment.

I've not seen it either. But it is the reason behind most everyone's decision making around WIndows DNS clients. So my point is that either that reason is false and Windows doesn't actually do this, or it's significant.

Okay, so... my conclusive results.

I did some testing the only way I could figure out... if there's more I can do, let me know.
(EDIT: I did some additional testing noted below the numbered list)

On a Windows machine, I:

Ping / test nslookup to a server. Make note of DNS server used for NSLookup.

Block IP of primary DC/DNS/Domain server via firewall.

ipconfig /flushdns

Test ping / test nslookup a server. Make note of DNS server used.

UNblock IP of primary DC/DNS/Domain server.

Test ping / test nslookup a server. Make note of DNS server used.

Results:
While the IP of primary DC/DNS/Domain server was blocked, an nslookup resulted in error, that the DNS request timed out, however, I was still able to ping all internal devices by name, as well as external websites. Then I unblocked it, and was still able to ping by name and nslookup was working normally again.

Additional Testing:
To test for real DNS blockage, I added all of our internal DNS servers to the block list and tested pinging stuff by name. Unable to ping any internal device by name.

So now I unblocked, let's call it DC/DNS-2, and left everything else blocked. I was now able to instantly ping things by name, guaranteed to using secondary DC/DNS server.

Upon unblocking the primary DC/DNS server, nslookup worked instantly and I was also able to continue pinging things. Same when blocking the other DNS server again.

Keep in mind that before each test, I performed a ipconfig /flushdns command to be sure.

I wasn't yet convinced, so to completely mitigate any unknowns, as a last path, I enabled Debug Logging on the primary and secondary DC/DNS server filtering stuff from the workstation i'm testing from. I had two powershell windows open side by side, -Tailing the DNS log file from each DC/DNS server.

From there, I continued having the primary DC/DNS server blocked (all DC/DNS servers except the secondary). I removed the primary DC/DNS ip address from being blocked on the workstation, performed a flushdns, and pinged something.

Within a minute I flushed DNS then pinged something internal by name, and it used the primary DNS server, not the secondary one that it was using before I unblocked the primary DNS ip.

This proves it is instant (or nearly instant, as there was about a minute between the time I unblocked the IP and sent out a ping), at least on Windows 10 Pro 1803, in a MS AD Domain environment.

I'm confused, if you used /flushdns, how does it prove anything? No one is questioning what happens when DNS is manually manipulated. What would it do if an actual end user was using it, is the question. From what I can tell, you never tested the scenarios we are talking about.

Try it naturally, without flushing in between changes. Does DNS change instantly, on its own? Delayed, on its own? Never without intervention?

Obsolesce

@scottalanmiller said in DNS Update Issue:

how does it prove anything?

I have to use flushdns because it won't use ANY DNS server if stuff is cached locally.

Obsolesce

Well I suppose I can come up with a list of stuff on the DNS server to prevent... dunno why I didn't think of that.

scottalanmiller

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

how does it prove anything?

I have to use flushdns because it won't use ANY DNS server if stuff is cached locally.

It should, but you are testing incorrectly. You test with nslookup, not ping. nslookup should not use a cache, by definition. If it does, it is broken. Ping is not a useful testing mechanism because it hits the cache.

Obsolesce

@scottalanmiller said in DNS Update Issue:

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

how does it prove anything?

I have to use flushdns because it won't use ANY DNS server if stuff is cached locally.

It should, but you are testing incorrectly. You test with nslookup, not ping. nslookup should not use a cache, by definition. If it does, it is broken. Ping is not a useful testing mechanism because it hits the cache.

I mentioned the word ping like a bunch of times... read it again.

Obsolesce

@scottalanmiller said in DNS Update Issue:

Ping is not a useful testing mechanism because it hits the cache.

Ping does not hit the cache on the computer I'm testing from because I cleared the DNS cache.

scottalanmiller

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

how does it prove anything?

I have to use flushdns because it won't use ANY DNS server if stuff is cached locally.

It should, but you are testing incorrectly. You test with nslookup, not ping. nslookup should not use a cache, by definition. If it does, it is broken. Ping is not a useful testing mechanism because it hits the cache.

I mentioned the word ping like a bunch of times... read it again.

I know you did, that's how I know you used ping, which is what won't work. You need to test with nslookup, which is the DNS testing tool. Ping is for testing ICMP, not DNS.

scottalanmiller

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

Ping is not a useful testing mechanism because it hits the cache.

Ping does not hit the cache on the computer I'm testing from because I cleared the DNS cache.

I know, thereby corrupting the test and making it useless.

scottalanmiller

Just test again with nslookup instead of ping, and no flushing.

Obsolesce

@scottalanmiller said in DNS Update Issue:

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

Ping is not a useful testing mechanism because it hits the cache.

Ping does not hit the cache on the computer I'm testing from because I cleared the DNS cache.

I know, thereby corrupting the test and making it useless.

Dude, i looked at the fucking debug logs on the DNS server itself, i KNOW it used the DNS server when I pinged, every damn time. I still have the logs to prove it. I pinged stuff by NAME, not IP. When you ping stuff by name, it uses DNS to look up the IP....

Also, I was pinging random servers and such, NOT the DNS server.

scottalanmiller

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

Ping is not a useful testing mechanism because it hits the cache.

Ping does not hit the cache on the computer I'm testing from because I cleared the DNS cache.

I know, thereby corrupting the test and making it useless.

Dude, i looked at the fucking debug logs on the DNS server itself, i KNOW it used the DNS server when I pinged, every damn time. I still have the logs to prove it. I pinged stuff by NAME, not IP. When you ping stuff by name, it uses DNS to look up the IP....

Also, I was pinging random servers and such, NOT the DNS server.

And, what does what you said have to do with the problem? Nothing you said here addressed the issue that we are discussing. You clearly stated you used the wrong tool, and clearly stated that you had to flush the DNS and corrupt the test because you used the wrong tool, after that all of your responses are to someone who isn't here, because they are unrelated to what I've been saying.

Just do the test again with the proper tool and without corrupting it. The DNS servers logs are obviously useless here as we already know the test wasn't done correctly so what you see tells us literally nothing.

Obsolesce

@scottalanmiller said in DNS Update Issue:

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

Ping is not a useful testing mechanism because it hits the cache.

Ping does not hit the cache on the computer I'm testing from because I cleared the DNS cache.

I know, thereby corrupting the test and making it useless.

Dude, i looked at the fucking debug logs on the DNS server itself, i KNOW it used the DNS server when I pinged, every damn time. I still have the logs to prove it. I pinged stuff by NAME, not IP. When you ping stuff by name, it uses DNS to look up the IP....

Also, I was pinging random servers and such, NOT the DNS server.

And, what does what you said have to do with the problem? Nothing you said here addressed the issue that we are discussing. You clearly stated you used the wrong tool, and clearly stated that you had to flush the DNS and corrupt the test because you used the wrong tool, after that all of your responses are to someone who isn't here, because they are unrelated to what I've been saying.

Just do the test again with the proper tool and without corrupting it. The DNS servers logs are obviously useless here as we already know the test wasn't done correctly so what you see tells us literally nothing.

The test was to see which DNS server is being used and when.

I did not use the wrong tool. When you ping something by hostname, it WILL do a DNS lookup.

If you had read what I wrote originally, I did say that I used nslookup, which only worked when the primary DNS server was available. But nslookup would fail otherwise, even though other DNS servers were responding to DNS requests. I have to logs to prove everything. You're idea of which tool is correct for the job is incorrect.

scottalanmiller

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

Ping is not a useful testing mechanism because it hits the cache.

Ping does not hit the cache on the computer I'm testing from because I cleared the DNS cache.

I know, thereby corrupting the test and making it useless.

Dude, i looked at the fucking debug logs on the DNS server itself, i KNOW it used the DNS server when I pinged, every damn time. I still have the logs to prove it. I pinged stuff by NAME, not IP. When you ping stuff by name, it uses DNS to look up the IP....

Also, I was pinging random servers and such, NOT the DNS server.

And, what does what you said have to do with the problem? Nothing you said here addressed the issue that we are discussing. You clearly stated you used the wrong tool, and clearly stated that you had to flush the DNS and corrupt the test because you used the wrong tool, after that all of your responses are to someone who isn't here, because they are unrelated to what I've been saying.

Just do the test again with the proper tool and without corrupting it. The DNS servers logs are obviously useless here as we already know the test wasn't done correctly so what you see tells us literally nothing.

The test was to see which DNS server is being used and when.

Sure, but not in a scenario we cared about. So the test was for you, not for this thread. What you tested is not what we discussed. You need one without flushing DNS to test what WE have been discussing. Unless you do that, this test should be its own thread and shows us nothing we didn't already know.

scottalanmiller

@Obsolesce said in DNS Update Issue:

I did not use the wrong tool. When you ping something by hostname, it WILL do a DNS lookup.

Only if it is not cached. As YOU pointed out. So you know this isn't true.

Obsolesce

@scottalanmiller said in DNS Update Issue:

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

@Obsolesce said in DNS Update Issue:

@scottalanmiller said in DNS Update Issue:

Ping is not a useful testing mechanism because it hits the cache.

Ping does not hit the cache on the computer I'm testing from because I cleared the DNS cache.

I know, thereby corrupting the test and making it useless.

Dude, i looked at the fucking debug logs on the DNS server itself, i KNOW it used the DNS server when I pinged, every damn time. I still have the logs to prove it. I pinged stuff by NAME, not IP. When you ping stuff by name, it uses DNS to look up the IP....

Also, I was pinging random servers and such, NOT the DNS server.

And, what does what you said have to do with the problem? Nothing you said here addressed the issue that we are discussing. You clearly stated you used the wrong tool, and clearly stated that you had to flush the DNS and corrupt the test because you used the wrong tool, after that all of your responses are to someone who isn't here, because they are unrelated to what I've been saying.

Just do the test again with the proper tool and without corrupting it. The DNS servers logs are obviously useless here as we already know the test wasn't done correctly so what you see tells us literally nothing.

The test was to see which DNS server is being used and when.

Sure, but not in a scenario we cared about. So the test was for you, not for this thread. What you tested is not what we discussed. You need one without flushing DNS to test what WE have been discussing. Unless you do that, this test should be its own thread and shows us nothing we didn't already know.

Working on a new test and much clearer description of the test and results.