Upcoming Job couple thoughts on DC demotion
-
@IRJ said in Upcoming Job couple thoughts on DC demotion:
@prcssupport said in Upcoming Job couple thoughts on DC demotion:
He had about 6 servers spun up(running on no better than server 2003) in that network between the two sets of hardware. They only had the creds for 2, or 3 of them, several were nonfunctional. But the systems were necessary for business function. They ended up "figuring something else out since they didn't know how to fix anything"
But all of the systems were still running and using resources despite the fact they had all mostly failed.This "business" sounds terrible. They definitely need cloud services since they obviously don't give a crap about IT and will only upgrade and fix systems when they have no choice.
Hahaha lol yes. It was a bit if a disaster. But I think we are finally moving forward. And actually it is all forced. The software vendor for the CRM has started EOL on the Hosted CRM deployment. He has alot of bad taste in his mouth for everyone in our field. But we have been able to maintain a good line of trust and communication.
Before I forget...
The CRM solution uses Microsoft access for database storage, And I know somewhere above @scottalanmiller mentioned something about there likely being encryption with the database software. But there is zero encryption whatsoever. I told them we needed to vacate the solution asap. His business insurance carrier agreed! lol
-
@prcssupport said in Upcoming Job couple thoughts on DC demotion:
The CRM solution uses Microsoft access for database storage,
That needs to die in nuclear fire.
-
OK up update on this job...
Today I started this job, it didn't go as planned.
With the owner approving the start.
I started by spooling down the secondary dc.
Then I went to one desktop and started preparing for the disconnection from the domain. I prepped the user profile for the local account. I check the local user names. It looked good. Then I made the disconnect.
Once the system completed the reboot, all of the passwords they had for the local accounts didn't work. We were locked out!
I told the password hint to the owner, he was like oh no issue. Here is that password. Nope. Crap...
I told the user it wasn't a big deal that with the design they could still use any other desktop in the office (because of the AD and roaming profiles)
That's when they said that there was one program they had to use tonight for a required data transfer. And part of their complete transition to the cloud. She hadn't mentioned this, and the owner had forgotten. I asked if she could use it in another system. She said no thats the only PC that can do it.(due to liscensing)Now at this time the others in the office are complaining they can't login, that there is no available domain servers. CRAP! I don't have time for this. I took down the secondary and the primary was still up and running. Not sure why the secondary being down caused that. So I spooled that server back up. Fast forward 5 min and everyone can login again. (Thank you Lord!)
Back to the workstation...
We are locked out of the local admin. Ok, no issue. Cue up a dvdrw, and another system. I created an NT offline Password reset disk.
We attempted to boot to Dvd (sucess). The user described the black screen with txt. Then we get a fatal error on the kernal. Grrr. So now I can't Crack this system quickly. My time is running out, and the heat is on.
We brainstormed more, I attempted a restore to see if I could get it back in time (and back on the domain) to do that we needed the admin password (of course we do!)
So after some thought and over 2 hrs of fussing around. They tried a password and we were in. "Finally"
My time was almost out, once we were in with administration I rejoined the domain and returned it to normal.
I told them before we can work on this project anymore they need to figure out all the usernames and admin passwords for the local accounts.
There were many things that happened just right to cause those issues. I mean how many systems refuse to run NT reset?
Why in the world did the secondary dc being down cause all the systems to not login. I didn't have time to check any of it out this is just the latest info on it.
Any thoughts... (I know dumb question) lol
-
@prcssupport said in Upcoming Job couple thoughts on DC demotion:
Why in the world did the secondary dc being down cause all the systems to not login. I didn't have time to check any of it out this is just the latest info on it.
Any thoughts... (I know dumb question) lol
What roles were on each server?
-
@prcssupport said in Upcoming Job couple thoughts on DC demotion:
We are locked out of the local admin. Ok, no issue. Cue up a dvdrw, and another system. I created an NT offline Password reset disk.
I learned this the hard way just like you did, but if you are in environment that you don't control,You can just do a quick run as command to verify credentials. That way if it doesn't work, just reset the password.
-
Maybe you mentioned this earlier, but why did you remove the workstation from the domain just to re-add it?
-
@IRJ said in Upcoming Job couple thoughts on DC demotion:
@prcssupport said in Upcoming Job couple thoughts on DC demotion:
Why in the world did the secondary dc being down cause all the systems to not login. I didn't have time to check any of it out this is just the latest info on it.
Any thoughts... (I know dumb question) lol
What roles were on each server?
Operations master role is the set to physical server
It maintains all of the 3 roles, the VM dc confirms this...However, I found why it went down. When I was looking to change to the vm dc from the main interface (on the physical server)
It showed the primary dc was unavailable. -
@IRJ said in Upcoming Job couple thoughts on DC demotion:
@prcssupport said in Upcoming Job couple thoughts on DC demotion:
We are locked out of the local admin. Ok, no issue. Cue up a dvdrw, and another system. I created an NT offline Password reset disk.
I learned this the hard way just like you did, but if you are in environment that you don't control,You can just do a quick run as command to verify credentials. That way if it doesn't work, just reset the password.
Good idea I hadn't thought of that before hand. That would have been a great way to check.
This brings me to a question on you're last statement. About resetting the local admin password.
Can you change a local user\admin password as a domain admin? I wasnt sure if they could control at that level. Maybe I have done it in the past, but I'm having a hard time remembering it if I did.
-
@prcssupport said in Upcoming Job couple thoughts on DC demotion:
@IRJ said in Upcoming Job couple thoughts on DC demotion:
@prcssupport said in Upcoming Job couple thoughts on DC demotion:
We are locked out of the local admin. Ok, no issue. Cue up a dvdrw, and another system. I created an NT offline Password reset disk.
I learned this the hard way just like you did, but if you are in environment that you don't control,You can just do a quick run as command to verify credentials. That way if it doesn't work, just reset the password.
Good idea I hadn't thought of that before hand. That would have been a great way to check.
This brings me to a question on you're last statement. About resetting the local admin password.
Can you change a local user\admin password as a domain admin? I wasnt sure if they could control at that level. Maybe I have done it in the past, but I'm having a hard time remembering it if I did.
If the domain admin is also set as a local admin, then yes.
-
@IRJ said in Upcoming Job couple thoughts on DC demotion:
Maybe you mentioned this earlier, but why did you remove the workstation from the domain just to re-add it?
Due to the fact that they had some business critical functions happening tonight I wanted to return it to what was working. The easiest way to help not have calls. I became unavailable after 3pm so I wanted them to be ok for the night.
-
So nothing worked when you powered down the physical server?
What IP are the PCs using for DNS?
What IP is the VM Domain Controller using for DNS?If the PCs are using the turned off server as their primary DNS, they may not ever flip to the secondary (which is hopefully your VM DC)
If the VM DC is not pointing to either 127.0.0.1 or it's own IP for the primary (and we assume there are no other DNS servers on this network for this domain) that would also make everything stop working.
