System Admin - checklist for Don'ts and Important points please!

[openit]

Hi there,

I have been working as IT Administrator for 6 years and you know in SMB, "IT is Jack of All Trades and master of none".

While we get knowledge on wide range of things like managing Server, Antivirus, Backups, Firewall, IT Support, Email Administration and list goes on, but the
downside is, I don't have expertise or not focused position as System Administrator (lets say).

Because of no expertise in something, I don't feel myself more than just some Sr. IT Support.

I'm practicing things to focus as core System Admin, since I wasn't been in real, depth, focus System Admin post, and that too in SMB (you know limitations), other than doing courses and practicing by building LAB, I want to make a checklist of Don'ts and important things to consider from your experience, which are necessary for me to play smooth in next System Admin job.

Following are few examples of Don'ts or important things to consider, please add your point:

1. Not recommended to convert Physical Server which has Domain Controller to Virtual Machine.
2. Need to choose right Generation (1 or 2) type VM on Hyper-V, because later we can't change the generation.
3. Don't set Static IP of some server/machine without consulting Network Team, to avoid conflicts with existing DHCP scope.

Your inputs matters a lot to me, and might help others in community as well.

Thanks!

[IRJ]

You should focus on what to Do instead of Don't. If you think at least didn't do this, you still wont be making the best decisions.

Having a list of DOs is even silly. It you want to know about a particular area like domain controllers, then research about domain controllers. Your 1st point isn't a valid one because people who deal with Active Directory know that there would be zero reason to ever convert a DC from physical to virtual. You would just promote and demote domain controllers. This goes back to Windows 2000 Active Directory (and maybe before) so it isnt really a new concept.

My point is that you study and research in areas where you want expertise. Instead of just following tutorials on how to do a specific thing, you need to understand the whole architecture and reasoning behind implementation of certain things like Active Directory, Hyper-V , or whatever is relevant to your IT infrastructure.

Rome was not built in a day...

[IRJ]

@openit said in System Admin - checklist for Don'ts and Important points please!:

2. Need to choose right Generation (1 or 2) type VM on Hyper-V, because later we can't change the generation.

https://lmddgtfy.net/?q=generation 1 vs 2 hyper-v

See first result

[Obsolesce]

@openit said in System Admin - checklist for Don'ts and Important points please!:

I want to make a checklist of Don'ts and important things to consider from your experience, which are necessary for me to play smooth in next System Admin job.
Following are few examples of Don'ts or important things to consider, please add your point:

Not recommended to convert Physical Server which has Domain Controller to Virtual Machine.
Need to choose right Generation (1 or 2) type VM on Hyper-V, because later we can't change the generation.
Don't set Static IP of some server/machine without consulting Network Team, to avoid conflicts with existing DHCP scope.

Your inputs matters a lot to me, and might help others in community as well.
Thanks!

In addition to not P2Ving a DC:

4. Don't pee on your servers.

I'm not sure where you want to draw the line as far as what not to do...

[IRJ]

@openit said in System Admin - checklist for Don'ts and Important points please!:

3. Don't set Static IP of some server/machine without consulting Network Team, to avoid conflicts with existing DHCP scope.

https://lmddgtfy.net/?q=Should I use static IP addresses%3F

From first result

When Not to Use a Static IP Address
Because a static IP address is assigned manually, it's much less efficient for a network admin to give them out, especially in mobile situations. They have to often visit the device in person to give it an IP address instead of letting DHCP assign the address automatically.

For example, you wouldn't set a static IP address on a smartphone because the moment it reaches another Wi-Fi network, the address might no be supported on that network, meaning that it won't be able to access the internet.

Dynamic addressing would be much more convenient in this situation because it's easy for administrators to set up. DHCP works automatically with minimal intervention needed, allowing mobile devices to seamlessly move between different networks.

[Dashrender]

@IRJ said in System Admin - checklist for Don'ts and Important points please!:

Your 1st point isn't a valid one because people who deal with Active Directory know that there would be zero reason to ever convert a DC from physical to virtual. You would just promote and demote domain controllers. This goes back to Windows 2000 Active Directory (and maybe before) so it isnt really a new concept.

This is along my first thoughts as well, you don't P2V a DC because you just stand up a whole new one, migrate any FSMO roles, etc, then decom the old one.
Doing this helps ensure your AD is in good order as well.

[JaredBusch]

@openit said in System Admin - checklist for Don'ts and Important points please!:

1. Not recommended to convert Physical Server which has Domain Controller to Virtual Machine.

You never need to move a DC. The problem is that sites typically have more than just DC roles on a DC. There are also usually file shares.

But as for a P2V, there is never a technical reason not to do it.

2. Need to choose right Generation (1 or 2) type VM on Hyper-V, because later we can't change the generation.

Always Generation 2. There should never be a reason in 2020 that anyone should use anything else. For the exceptions, they will already know they are exceptions.

3. Don't set Static IP of some server/machine without consulting Network Team, to avoid conflicts with existing DHCP scope.

There are only 4 systems in a network that require a static IP.

1. Router
2. Main hypervisor that runs 3 & 4 below.
   • if 3 & 4 are not on the hypervisor, it has no need for a static address
3. DHCP server
4. DNS Server

A standard Microsoft Active Directory Domain Controller is both a DHCP and DNS server, and thus needs a static IP. But it does not require it without those roles, as long as your DNS works properly and updates smoothly.

[PhlipElder]

@openit said in System Admin - checklist for Don'ts and Important points please!:

1. Not recommended to convert Physical Server which has Domain Controller to Virtual Machine.
2. Need to choose right Generation (1 or 2) type VM on Hyper-V, because later we can't change the generation.
3. Don't set Static IP of some server/machine without consulting Network Team, to avoid conflicts with existing DHCP scope.

Your inputs matters a lot to me, and might help others in community as well.

Thanks!

1: That depends. A DC could be virtualized until such time as one is ready to run a full migration. Server 2019 ADDS requires DFSR. So, existing FRS DCs would need to be migrated to DFSR first. This is an invasive process that requires System State backups of FSMO/PDCe and at least one secondary DC.

2: Always Gen2 unless the OS to be dropped into the VM does not support it. P2V of older workloads for example. Use what is required.

3: The subnet should be documented somewhere. MAC addresses, IP addresses, DHCP scope(s), DHCP settings, and so on. Advanced IP Scanner is free and is a good place to start if none exist. There are other tools out there.

4: Group Policy: Follow best practices. Don't touch the Default Domain and Default Domain Controllers policies. Always set up the OU/GPO structure and settings according to the org's needs.

5: Hyper-V standalone: We don't join the host to the guest's domain. It presents a barrier to a ransomware compromise.

6: Backup: A backup is not considered "Good" until it is fully bare metal/hypervisor restored. Spot file/folder restores are not a verification method.

7: No Remote Desktop Protocol (RDP) port forwards (NAT) from the Internet (alternate port) to 3389 on the intended destination. Ever. Use Remote Desktop Gateway and add DUO or other 2FA to the mix.

[Dashrender]

@PhlipElder said in System Admin - checklist for Don'ts and Important points please!:

7: No Remote Desktop Protocol (RDP) port forwards (NAT) from the Internet (alternate port) to 3389 on the intended destination. Ever. Use Remote Desktop Gateway and add DUO or other 2FA to the mix.

Is this because only RDG supports MFA? not the end clients themselves?

[PhlipElder]

@Dashrender said in System Admin - checklist for Don'ts and Important points please!:

@PhlipElder said in System Admin - checklist for Don'ts and Important points please!:

7: No Remote Desktop Protocol (RDP) port forwards (NAT) from the Internet (alternate port) to 3389 on the intended destination. Ever. Use Remote Desktop Gateway and add DUO or other 2FA to the mix.

Is this because only RDG supports MFA? not the end clients themselves?

Because RDG provides a layer of protection against TSGrinder and its cohort.

It's bad news to publish an RDP listener direct to the Web. Has been for a very long time now.

[JaredBusch]

@PhlipElder said in System Admin - checklist for Don'ts and Important points please!:

@Dashrender said in System Admin - checklist for Don'ts and Important points please!:

@PhlipElder said in System Admin - checklist for Don'ts and Important points please!:

7: No Remote Desktop Protocol (RDP) port forwards (NAT) from the Internet (alternate port) to 3389 on the intended destination. Ever. Use Remote Desktop Gateway and add DUO or other 2FA to the mix.

Is this because only RDG supports MFA? not the end clients themselves?

Because RDG provides a layer of protection against TSGrinder and its cohort.

It's bad news to publish an RDP listener direct to the Web. Has been for a very long time now.

Not any worse than publishing any service to the web without some third party control (fail2ban, etc).

Stupid people using stupid weak passwords is not a failure of the protocol.

[scottalanmiller]

@openit said in System Admin - checklist for Don'ts and Important points please!:

Not recommended to convert Physical Server which has Domain Controller to Virtual Machine.

Better first rule: Don't make "specific" rules. Figure out the underlying technical reasons for things and make rules based on underlying technology.

Example: People say not to P2V a DC. But you only have to say this because people don't understand the basics. So let's break it down and expose the general rule that if people knew, they'd not have to memorize specifics.

DCs are typically distributed databases. DDBs require the full cluster to be coherent. They are clusters requiring the cluster to be treated as a single entity, because it is. You can't isolate a node within the cluster and change its data or it becomes corrupt.

So because we know that a DC is designed to be used in a DDB cluster, and that clusters can't have isolated node manipulation we then know that we can't snapshot, restore, roll back, etc. any of their nodes.

Next we need to understand what a P2V is. It's a snapshot of a machine, then the snap is put into a file based container (like a VHD or QCOW), drivers are injected for the desired platform, and it is then fired up on top of a hypervisor.

So since we know that clusters can't be snapshotted.... we then know that AD DCs can't be P2Vd unless they are standalone or some other mechanism detects that action and addresses it.

Voila, know what a DC is and how it works (which is really trivial stuff) and how databases, clusters, snapshots, etc. work and you don't need to memorize anything and suddenly you find that this same knowledge applies to hundreds of other things you deal with every day.

[scottalanmiller]

@openit said in System Admin - checklist for Don'ts and Important points please!:

Need to choose right Generation (1 or 2) type VM on Hyper-V, because later we can't change the generation.

Okay, but general rule... don't make haphazard decisions without research. Trying to memorize tricky details like this will make you go crazy. The number of these kinds of itty bitty details will go on forever and are 100% dependent on the specific technology that you choose to use.

Why not make the rule that you use KVM instead of Hyper-V, BTW?

[scottalanmiller]

@openit said in System Admin - checklist for Don'ts and Important points please!:

Don't set Static IP of some server/machine without consulting Network Team, to avoid conflicts with existing DHCP scope.

Or use DHCP to do it. Even better in most cases.

[scottalanmiller]

@IRJ said in System Admin - checklist for Don'ts and Important points please!:

You should focus on what to Do instead of Don't. If you think at least didn't do this, you still wont be making the best decisions.

Agreed. What not to do will go on forever. There's no end to what not to do.

[scottalanmiller]

@IRJ said in System Admin - checklist for Don'ts and Important points please!:

Having a list of DOs is even silly. It you want to know about a particular area like domain controllers, then research about domain controllers. Your 1st point isn't a valid one because people who deal with Active Directory know that there would be zero reason to ever convert a DC from physical to virtual. You would just promote and demote domain controllers. This goes back to Windows 2000 Active Directory (and maybe before) so it isnt really a new concept.

IRJ is correct on all points here.

You are approaching IT as a set of checkboxes. It isn't. The only way to do IT well is to learn the fundamentals, then learn the specifics, and apply them. Everything else is both harder, if not impossible, and ineffectual. It might seem like a shortcut... just distill your job to dos and don'ts, or the checkboxes, but IT doesn't and can't work that way. If it did, we'd have no reason to exist because everything would already be automated. IT is hard, it just is. It's complex. But the more you accept that from the beginning, the easier it gets. Learn the basics, and the specifics get easy like in my AD example.

He's also right... the rules about P2V conversions of AD DCs is weird because it's something that should never come up in any situation where you cared about doing things right at all, the only time the thought process even exists is for situations where you weren't following good practices in the first place and aren't following them in the future. And then it only applies when you have more than one DC. And only old versions because recent ones have protection for this (still a bad idea, but it works.) And all of this is stuff you don't need to learn, because just learning what AD is and the high level view of how it works lets you figure it all out automatically anyway.

[scottalanmiller]

@JaredBusch said in System Admin - checklist for Don'ts and Important points please!:

Always Generation 2. There should never be a reason in 2020 that anyone should use anything else. For the exceptions, they will already know they are exceptions.

Right, the pattern here is the "Always, Unless you can't" rule. Meaning, you always want Gen 2, the exclusive reason you'd ever choose Gen 1 is because Gen 2 isn't an option for you. In which case there is no real decision to be mad. The logic is so simple...

Gen 2, unless impossible, then Gen 1. If that also is impossible, stop.

This kind of thing applies as a general pattern over and over again.

[scottalanmiller]

@JaredBusch said in System Admin - checklist for Don'ts and Important points please!:

There are only 4 systems in a network that require a static IP.

Router
Main hypervisor that runs 3 & 4 below.

if 3 & 4 are not on the hypervisor, it has no need for a static address

DHCP server
DNS Server

And even this isn't actually true, it's still assumptions. In reality, the only things that need static are the router and if it is a separate device from the router, the DHCP server.

You never need it for DNS because DNS can get handed out from DHCP. Hypervisors, too. Except in the case where they are the hypervisors for the DHCP VM, in which case they might need a static IP, but not necessarily.

[scottalanmiller]

@PhlipElder said in System Admin - checklist for Don'ts and Important points please!:

3: The subnet should be documented somewhere. MAC addresses, IP addresses, DHCP scope(s), DHCP settings, and so on. Advanced IP Scanner is free and is a good place to start if none exist. There are other tools out there.

This is a great point. The rule about checking with networking assumes that failure has already happened - that you are assigned a networking task that doesn't have documentation. You really should never have a situation where this comes up. Just like the P2V of the AD DC. I realize that you (OpenIT) were just making examples, but all of your examples were of things that shouldn't have rules, because they shouldn't need rules in the first place. They are rules to deal with problems that, if you are taking the time to make rules to fix problems, you should just remove the problems instead.

[scottalanmiller]

@JaredBusch said in System Admin - checklist for Don'ts and Important points please!:

@PhlipElder said in System Admin - checklist for Don'ts and Important points please!:

@Dashrender said in System Admin - checklist for Don'ts and Important points please!:

@PhlipElder said in System Admin - checklist for Don'ts and Important points please!:

7: No Remote Desktop Protocol (RDP) port forwards (NAT) from the Internet (alternate port) to 3389 on the intended destination. Ever. Use Remote Desktop Gateway and add DUO or other 2FA to the mix.

Is this because only RDG supports MFA? not the end clients themselves?

Because RDG provides a layer of protection against TSGrinder and its cohort.

It's bad news to publish an RDP listener direct to the Web. Has been for a very long time now.

Not any worse than publishing any service to the web without some third party control (fail2ban, etc).

Stupid people using stupid weak passwords is not a failure of the protocol.

There is, and again it's a tends to issue, with RDP. And that is that RDP tends to be tied to AD, to a point where it is assumed. Because of the way that RDP gets exposed, and because of the problems of tying it to internal AD, you end up with a need for "exposed security" measures on the RDP that aren't really feasible when you are dealing with an AD account (like locking up after three failed attempts.) If you do that, anyone on the outside can lock anyone on the inside's AD account any time that they want making RDP a really simple DoS attack vector.

RDP itself isn't special. It's highly secure. But people tend to just assume that RDP will be tied to AD, and they are nearly always correct. But RDP itself is just as secure as say SSH. It's just that SSH basically never gets tied into AD and on to internal user accounts in the same way, so the problems aren't the same (that and SSH normally gets Fail2ban solving the issue even further.)

So like everything else here, if we look for general rules we can identify the problem, the potential solutions, etc. Because the issue isn't about RDP, isn't unique to RDP, can be avoided with RDP, and can occur without RDP.

The real rule is a need for security around publicly accessible vectors when tied to internal mechanisms.