Handling DNS in a Single Active Directory Domain Controller Environment

scottalanmiller

@kelly said in Handling DNS in a Single Active Directory Domain Controller Environment:

Running a second AD server isn't all that expensive. You have a server license and (potentially) hardware, so $1,500 to $2,000 USD if you don't already have a second set of hardware you can run it on. If you run it in core configuration it requires very little in the way of resources.

In the SMB market, $1,500 - $2,500 is anything but small. To most businesses, that's a lot of money. And don't forget that adding a second DC means your IT has to know a lot more about how things work, it's not just double the software and possibly double the hardware, it's also more effort and expertise needed. The system is less "simple."

Compare that to the risk of not having a second AD DC. For the average business, the impact of losing AD temporarily is tiny to zero. Many can have AD down and not even know! As the CEO... how much money is it worth to protect again a "zero impact" outage. I guarantee the answer is... zero. Even if you risk losing $5,000 from an AD outage, that's not half as much as you'd need to justify investing $2,500 up front to protect against a maybe outage someday in the future that small.

Also, in real world terms, if you lose your only AD DC, it means you've likely lost some, most, or even all other AD linked (and even non-AD linked) services. When AD is down, most typically businesses are having issues not because of AD, but because other things running on the same server are down. AD would be the least of their worries.

scottalanmiller

@kelly said in Handling DNS in a Single Active Directory Domain Controller Environment:

Edit: The $1,500 to $2,000 might seem like quite a bit, but those costs are pretty minimal compared to down time and repair costs if you do suffer a catastrophic failure.

One of the problems with this specific discussion is that AD varies wildly from a cost of roughly $0 for one company to close to $3,000 for another. Comes down heavily to the hardware needs, virtualization requirements, how the licensing is handled, etc. For one company, there might already be a spare server, a spare license, and no other use case for them, so the second DC is "free" and they should probably do it without question. But for another, it might be buying a dedicated server and a dedicated Windows license for that one thing, making it rather pricey.

So no way to say what a general case cost will be.

scottalanmiller

@stuartjordan said in Handling DNS in a Single Active Directory Domain Controller Environment:

How passwords are stored and reset is a critical aspect of GDPR compliance. Clients and staff members may legitimately forget or need to reset passwords for a number of reasons. GDPR requirements mean that companies must be able to demonstrate that their password reset processes and procedures are secure. Systems must be in place, for example, to prevent help desk employees that may be involved in resets from directly accessing passwords.

I responded in the GDPR Password thread.

scottalanmiller

@kelly said in Handling DNS in a Single Active Directory Domain Controller Environment:

Given the premise of handling DNS failover scenarios it seems like having two DCs would be better, but I'm open to being convinced.

So there are a few scenarios here. But here are some general approaches to this that I've seen (thanks to @JaredBusch for the big one...)

In really small AD environments, there might be zero internal DNS needs and Windows DNS is for AD only. In which case, point to the router's DNS or a public DNS and you are done, easy peasy. Free, simple, universal.
In environments with internal DNS dependencies, much of the time the impact of losing that is trivial and you just have workers run in a reduced functionality mode until AD DC is fixed. Impact can be so small as to not measure. An option if you are very small and don't need external web access (super rare.)
Jared's solution... point your router to your internal DNS first, then to public DNS. This handles failover and deals with any concerns of Windows DNS stack being flaky. When AD is up, internal DNS works, when it goes down, you transparently fail to public DNS.
Use zone transfer to another DNS server. This could just be to your router. Have your router or some other "free or already included" system keep a copy of Windows' DNS and use this as your secondary. Full DNS redundancy (without active updates) while the AD DC is down. You could build a DNS server just for this, but nearly all networks have one already available. But a Raspberry Pi will do this easily if you really have no other hardware.

scottalanmiller

To be clear, no one is suggesting that two AD DCs is bad. Only that in many cases, only having one is appropriate and when you do have only one, DNS is often a cascaded dependency that might need to be addressed, and that's what we are looking at here - when you have only one AD DC (we are already assuming that you fit that profile) how do you deal with DNS needs.

JaredBusch

@scottalanmiller said in Handling DNS in a Single Active Directory Domain Controller Environment:

Jared's solution... point your router to your internal DNS first, then to public DNS. This handles failover and deals with any concerns of Windows DNS stack being flaky. When AD is up, internal DNS works, when it goes down, you transparently fail to public DNS.

Here is how I handle it in EdgeOS

set service dns forwarding cache-size 150
set service dns forwarding listen-on eth1
set service dns forwarding name-server 10.1.1.show 4
set service dns forwarding name-server 1.1.1.1
set service dns forwarding name-server 8.8.8.8
set service dns forwarding options server=/domain.local/10.1.1.4
set service dns forwarding options server=/domain/10.1.1.4
set system domain-name domain.local 
set system name-server 127.0.0.1

And the few absolute core items that I need to 100% resolve get a static host mapping in EdgeOS.

set system static-host-mapping host-name dc02 inet 10.1.1.4
set system static-host-mapping host-name dc02.domain.local inet 10.1.1.4
set system static-host-mapping host-name domain.local inet 10.1.1.4
set system static-host-mapping host-name pbx.domain.com inet 10.1.1.30

Mike Davis

@jaredbusch said in Handling DNS in a Single Active Directory Domain Controller Environment:

@scottalanmiller said in Handling DNS in a Single Active Directory Domain Controller Environment:

Jared's solution... point your router to your internal DNS first, then to public DNS. This handles failover and deals with any concerns of Windows DNS stack being flaky. When AD is up, internal DNS works, when it goes down, you transparently fail to public DNS.

Here is how I handle it in EdgeOS
set service dns forwarding cache-size 150
set service dns forwarding listen-on eth1
set service dns forwarding name-server 10.1.1.show 4
set service dns forwarding name-server 1.1.1.1
set service dns forwarding name-server 8.8.8.8
set service dns forwarding options server=/domain.local/10.1.1.4
set service dns forwarding options server=/domain/10.1.1.4
set system domain-name domain.local 
set system name-server 127.0.0.1
And the few absolute core items that I need to 100% resolve get a static host mapping in EdgeOS.
set system static-host-mapping host-name dc02 inet 10.1.1.4
set system static-host-mapping host-name dc02.domain.local inet 10.1.1.4
set system static-host-mapping host-name domain.local inet 10.1.1.4
set system static-host-mapping host-name pbx.domain.com inet 10.1.1.30

That's smart thinking if you have an on prem PBX. They can still use the phones even if the DC (and primary DNS) is down.

scottalanmiller

@mike-davis said in Handling DNS in a Single Active Directory Domain Controller Environment:

That's smart thinking if you have an on prem PBX. They can still use the phones even if the DC (and primary DNS) is down.

Exactly. And that's exactly why we do something similar. We want our PCs and phones online even when other things are down.

And honestly, for things like PBX that is off prem, we never use the in house DNS for the phones at all. Point them straight to CloudFlare or Google and if AD fails, no one knows (from a phone perspective.)

Donahue

In our DHCP settings (hosted by the same DC) I currently have 5 DNS servers defined. My two DC's, then the two public DNS from our ISP, then finally 8.8.8.8. But in my experience, if my DC is down, I've got larger issues than just DNS. But when this has happened, I just start up the DHCP server on the router and define 8.8.8.8 as the DNS for everyone while I fix this issue with the DC. I am not sure we have it configured correctly though, because it never seems to actually fail over to the other DC. It could be because its in another subnet at a different location, not sure.

Mike Davis

I have a number of clients where they need a server, but Server Essentials on a small server is enough. Veeam for backup and if the box fails, they are down for an hour or two while we restore to something else. The licensing to go to a second AD server would more than double the cost of the project. (and isn't worth it for them)

scottalanmiller

@mike-davis said in Handling DNS in a Single Active Directory Domain Controller Environment:

I have a number of clients where they need a server, but Server Essentials on a small server is enough. Veeam for backup and if the box fails, they are down for an hour or two while we restore to something else. The licensing to go to a second AD server would more than double the cost of the project. (and isn't worth it for them)

Great example, that's what we see a lot, even with "larger than Essentials" shops. The downtime is still short, getting new hardware is trivial (a big desktop could do it), a single server restore is simple, and the cost to avoid that pain is huge compared to the cost of the downtime.

scottalanmiller

@donahue said in Handling DNS in a Single Active Directory Domain Controller Environment:

In our DHCP settings (hosted by the same DC) I currently have 5 DNS servers defined. My two DC's, then the two public DNS from our ISP, then finally 8.8.8.8. But in my experience, if my DC is down, I've got larger issues than just DNS. But when this has happened, I just start up the DHCP server on the router and define 8.8.8.8 as the DNS for everyone while I fix this issue with the DC. I am not sure we have it configured correctly though, because it never seems to actually fail over to the other DC. It could be because its in another subnet at a different location, not sure.

That might be the cause. Do you have them set up as different sites?

I would avoid using the ISP DNS, that's generally slow and risky. Use Google and CloudFlare for your "other" DNSs on the outside. Fast and secure. And each provides two, so more than enough to replace what you have.

JaredBusch

@donahue said in Handling DNS in a Single Active Directory Domain Controller Environment:

In our DHCP settings (hosted by the same DC) I currently have 5 DNS servers defined. My two DC's, then the two public DNS from our ISP, then finally 8.8.8.8. But in my experience, if my DC is down, I've got larger issues than just DNS.

I would never put public DNS entries in anything related to a client system running a Windows Desktop OS. I have had too much direct experience with Windows' network stack flipping to the secondary DNS server and not flipping back to ever want to deal with that crap. It could be better with Windows 10, but I am not going to change now to find out.

@donahue said in Handling DNS in a Single Active Directory Domain Controller Environment:

But when this has happened, I just start up the DHCP server on the router and define 8.8.8.8 as the DNS for everyone while I fix this issue with the DC. I am not sure we have it configured correctly though, because it never seems to actually fail over to the other DC. It could be because its in another subnet at a different location, not sure.

DHCP is a different but related issue.

Donahue

I dont think I've ever set it all up correctly. I thought I setup the sites correctly, but that was a few years ago and I really had no idea what I was doing. I've been too busy to really worry about it since then and the DC has been pretty reliable. Squeaky wheel and all that.

Obsolesce

@scottalanmiller said in Handling DNS in a Single Active Directory Domain Controller Environment:

@kelly said in Handling DNS in a Single Active Directory Domain Controller Environment:

Given the premise of handling DNS failover scenarios it seems like having two DCs would be better, but I'm open to being convinced.

So there are a few scenarios here. But here are some general approaches to this that I've seen (thanks to @JaredBusch for the big one...)

In really small AD environments, there might be zero internal DNS needs and Windows DNS is for AD only. In which case, point to the router's DNS or a public DNS and you are done, easy peasy. Free, simple, universal.

In environments with internal DNS dependencies, much of the time the impact of losing that is trivial and you just have workers run in a reduced functionality mode until AD DC is fixed. Impact can be so small as to not measure. An option if you are very small and don't need external web access (super rare.)

Jared's solution... point your router to your internal DNS first, then to public DNS. This handles failover and deals with any concerns of Windows DNS stack being flaky. When AD is up, internal DNS works, when it goes down, you transparently fail to public DNS.

Use zone transfer to another DNS server. This could just be to your router. Have your router or some other "free or already included" system keep a copy of Windows' DNS and use this as your secondary. Full DNS redundancy (without active updates) while the AD DC is down. You could build a DNS server just for this, but nearly all networks have one already available. But a Raspberry Pi will do this easily if you really have no other hardware.

This number 3 is the big one. In the time it takes to handle 1, 2, or 4, you can have the DC/DNS VM restored.

scottalanmiller

@obsolesce said in Handling DNS in a Single Active Directory Domain Controller Environment:

@scottalanmiller said in Handling DNS in a Single Active Directory Domain Controller Environment:

@kelly said in Handling DNS in a Single Active Directory Domain Controller Environment:

Given the premise of handling DNS failover scenarios it seems like having two DCs would be better, but I'm open to being convinced.

So there are a few scenarios here. But here are some general approaches to this that I've seen (thanks to @JaredBusch for the big one...)

In really small AD environments, there might be zero internal DNS needs and Windows DNS is for AD only. In which case, point to the router's DNS or a public DNS and you are done, easy peasy. Free, simple, universal.

In environments with internal DNS dependencies, much of the time the impact of losing that is trivial and you just have workers run in a reduced functionality mode until AD DC is fixed. Impact can be so small as to not measure. An option if you are very small and don't need external web access (super rare.)

Jared's solution... point your router to your internal DNS first, then to public DNS. This handles failover and deals with any concerns of Windows DNS stack being flaky. When AD is up, internal DNS works, when it goes down, you transparently fail to public DNS.

Use zone transfer to another DNS server. This could just be to your router. Have your router or some other "free or already included" system keep a copy of Windows' DNS and use this as your secondary. Full DNS redundancy (without active updates) while the AD DC is down. You could build a DNS server just for this, but nearly all networks have one already available. But a Raspberry Pi will do this easily if you really have no other hardware.

This number 3 is the big one. In the time it takes to handle 1, 2, or 4, you can have the DC/DNS VM restored.

Well, but you do them ahead of time

scottalanmiller

@donahue said in Handling DNS in a Single Active Directory Domain Controller Environment:

I dont think I've ever set it all up correctly. I thought I setup the sites correctly, but that was a few years ago and I really had no idea what I was doing. I've been too busy to really worry about it since then and the DC has been pretty reliable. Squeaky wheel and all that.

You need someone to come in and take a look at some of that stuff!

See you in a few hours

Donahue

@scottalanmiller said in Handling DNS in a Single Active Directory Domain Controller Environment:

@donahue said in Handling DNS in a Single Active Directory Domain Controller Environment:

I dont think I've ever set it all up correctly. I thought I setup the sites correctly, but that was a few years ago and I really had no idea what I was doing. I've been too busy to really worry about it since then and the DC has been pretty reliable. Squeaky wheel and all that.

You need someone to come in and take a look at some of that stuff!

See you in a few hours

That's like number 87 on my priority list though.

scottalanmiller

@donahue said in Handling DNS in a Single Active Directory Domain Controller Environment:

@scottalanmiller said in Handling DNS in a Single Active Directory Domain Controller Environment:

@donahue said in Handling DNS in a Single Active Directory Domain Controller Environment:

I dont think I've ever set it all up correctly. I thought I setup the sites correctly, but that was a few years ago and I really had no idea what I was doing. I've been too busy to really worry about it since then and the DC has been pretty reliable. Squeaky wheel and all that.

You need someone to come in and take a look at some of that stuff!

See you in a few hours

That's like number 87 on my priority list though.

More of a Wednesday item

Kelly

@scottalanmiller It doesn't seem like you're interested in discussing my comment in good faith. Your progressive inflation of costs in response to my posts is an interesting rhetorical technique, but not a great way of doing a discussion based on the merits. If you have a basis for the costs you're postulating, then let's discuss them.

Scenario: SMB that depends on AD. Has one VM host. Typical SMB configuration would have DNS and DHCP running on the same Windows install as AD. At that size you could go either way for IT support, but for the sake of discussing costs on the lower side we'll say that they have someone in house with sufficient skill to install and manage Windows, AD, DNS, and DHCP. We'll set their wages at $25/hr. We will also assume that they are currently maxed so that any emergency is an additional cost, and not just absorbed by their daily responsibilities. (Feel free to dispute my assumptions with justifications)

Second DC:
Server license: MSRP $699.99 (probably find it for less, but we'll go with this)
Hardware: $1,000 (If all you need is a failover DC then you could run this on old hardware, a desktop, etc. for easily less than $500, but we'll go for an upper bound)

So, the upper end of costs for a second DC in my scenario is $1,700 and then say, 8 hours of labor (probably less), so another $200. Total cost to purchase and implement is $1,900 at the most.

If we assume that this is a typical SMB, sans the @scottalanmiller touch, then DNS and DHCP are also running on the DC VM. If the hardware suffers a catastrophic failure what is the cost to restore it? We won't factor in any replacement costs because those are the same whether we have a second DC or not.

So, backups. Here is where we may differ a bit. If there is a backup solution that is up to the task of backing up AD and restoring it without any additional issues then I would postulate that the business would have room on that hardware to run a second DC running as a Core install. However, if we are wanting to see the most extreme comparison would be, we will assume that there is a good backup of AD to restore, but not the hardware to run a second DC. This keeps our costs for a second DC still at $1,900.

VM host suffers hardware failure taking down AD. From this point it is speculation. If there aren't sufficient resources to spin up a second DC then there isn't anything to restore the back up to either. This means that there is no AD, DNS, or DHCP until something is put into place. Since every business class edge device can handle the latter two, we'll assume that they get moved over by our on site person. The time to do this is extremely variable. Let's grab an hour as the lower bound. So $25 to get DNS and DHCP up and running so that there is external access, plus employee labor impact. DHCP will continue to be fine during that one hour unless a lease comes up. DNS, however, means that most modern devices will stop functioning right away. A business of 10 employees averaging $15/hr means that there is a cost of $150 for that down time.

At this point, all internal services are down until AD is restored. Another variable that is difficult to account for. The most prevalent one of these would be printers. Impact will vary from business to business. If we say that of those 10 employees, 2 require (whether from felt personal need, or actual professional need it doesn't matter) a printer multiple times per day. How does 5 times sound? There are work arounds. Our enterprising technician goes to each machine to edit their hosts file to allow the users to print. Between getting all the information, figuring out the changes, coordinating with employees, and actually doing the work we'll say it takes an hour, so another $25.

So, our loss of our only DC has cost the company a minimum of $200. These costs can skyrocket depending on the number of employees, their pay rates, and the numbers of internal services. If this is our baseline then almost $2,000 is probably more than a 10 employee company should spend, so I agree with you. If the curves fluctuate much at all (lower priced/repurposed hardware), more employees, more costly employees, more internal service, then the cost/risk ratio approaches 1 very quickly.

Yes, you can defray or reduce almost all of the points of impact, like configuring your DNS scenario the way @JaredBusch suggested, but that is not a common approach. Implementing it and other things to negate potential impacts of down time (scripting system capable of pushing out a new hosts file to clients reducing down time to less than 1 hour, etc.) have their own costs inherent to them, and are also not common in SMB, so I don't think that including them in the comparison of the two approaches.