Unsolved Autoupdates Killed My Windows Server 2008 R2
-
The Windows update log had a ton of these errors:
2017-04-13 14:24:10:022 428 1614 Agent WARNING: Failed to evaluate Installed rule, updateId = {6CFCF2A8-52A0-4DDC-AD90-71A7736E83CE}.200, hr = 80080005 2017-04-13 14:26:10:109 428 1614 Agent WARNING: Failed to evaluate Installable rule, updateId = {6CFCF2A8-52A0-4DDC-AD90-71A7736E83CE}.200, hr = 80080005 2017-04-13 14:30:10:787 428 1614 Agent WARNING: Failed to evaluate Installed rule, updateId = {0C91D9FF-FEE6-421C-ACCD-15582919F140}.200, hr = 80080005 2017-04-13 14:32:10:846 428 1614 Agent WARNING: Failed to evaluate Installable rule, updateId = {0C91D9FF-FEE6-421C-ACCD-15582919F140}.200, hr = 80080005 2017-04-13 14:36:11:259 428 1614 Agent WARNING: Failed to evaluate Installed rule, updateId = {89185EE9-622B-4D77-9DF7-FE3F2E2027EE}.200, hr = 80080005 2017-04-13 14:38:11:326 428 1614 Agent WARNING: Failed to evaluate Installable rule, updateId = {89185EE9-622B-4D77-9DF7-FE3F2E2027EE}.200, hr = 80080005 2017-04-13 14:42:11:656 428 1614 Agent WARNING: Failed to evaluate Installed rule, updateId = {728F10D0-EFA2-494E-B116-FFCACBCF094C}.200, hr = 80080005 2017-04-13 14:44:11:715 428 1614 Agent WARNING: Failed to evaluate Installable rule, updateId = {728F10D0-EFA2-494E-B116-FFCACBCF094C}.200, hr = 80080005 2017-04-13 14:48:13:563 428 1614 Agent WARNING: Failed to evaluate Installed rule, updateId = {67BCDD15-BA51-4465-BED2-6202AD4A9D34}.201, hr = 80080005 2017-04-13 14:50:13:621 428 1614 Agent WARNING: Failed to evaluate Installable rule, updateId = {67BCDD15-BA51-4465-BED2-6202AD4A9D34}.201, hr = 80080005 -
What happens if you disconnect a network cable and then log in?
-
@dafyre said in autoupdates killed my server:
What happens if you disconnect a network cable and then log in?
I haven't tried that. I read about some people having that issue with Windows 7 machines, but didn't think it would apply to the server since it always has a connection to itself. I can give it a rip next time I'm on site.
-
Only other thing I can think of would be to restore it to a hypervisor (and leave it disconnected) to see if that has any bearing on it... and then, you could restore older backups until you find one that works.
-
@dafyre said in autoupdates killed my server:
Only other thing I can think of would be to restore it to a hypervisor (and leave it disconnected) to see if that has any bearing on it... and then, you could restore older backups until you find one that works.
In hind sight I wish I had brought a copy of the backup with me so I could try to restore to something on my bench. That's a good idea.
-
@dafyre said in autoupdates killed my server:
Only other thing I can think of would be to restore it to a hypervisor (and leave it disconnected) to see if that has any bearing on it... and then, you could restore older backups until you find one that works.
But with the old one running you will get conflicts. Power one off, power one on.
Delete the NIC so it doesnt preserve the MAC after the convert. new IP and rename it, then you can have both running. -
@Mike-Davis said in autoupdates killed my server:
I've had auto updates hose stuff before, but this one takes the cake. I took on a client with a Dell T310 server with Server 2008 R2 Standard on it. It's physical, not virtual.
You can sense the impending disaster right away.
-
@scottalanmiller said in Autoupdates Killed My Windows Server 2008 R2:
@Mike-Davis said in autoupdates killed my server:
I've had auto updates hose stuff before, but this one takes the cake. I took on a client with a Dell T310 server with Server 2008 R2 Standard on it. It's physical, not virtual.
You can sense the impending disaster right away.
That is why I always like to check backups before rebooting something that is unkown or not been rebooted in a while.
-
@Texkonc said in autoupdates killed my server:
That is why I always like to check backups before rebooting something that is unkown or not been rebooted in a while.
That's the odd thing. I do have backups and I was able to restore to a point 6 hours before the automatic updates went on. It isn't stuck in a boot loop, but I can't get to the desktop. It's possible that I haven't logged in to the desktop since the last auto update went on. I could try rolling back even further, but it would be better to do that on a test server.
-
@Texkonc said in Autoupdates Killed My Windows Server 2008 R2:
@dafyre said in autoupdates killed my server:
Only other thing I can think of would be to restore it to a hypervisor (and leave it disconnected) to see if that has any bearing on it... and then, you could restore older backups until you find one that works.
But with the old one running you will get conflicts. Power one off, power one on.
Delete the NIC so it doesnt preserve the MAC after the convert. new IP and rename it, then you can have both running.That's why I said leave it disconnected. So the VM doesn't try to take over... at least not right away!
-
Possibly failed update from before that are just sticking around and that is why the last company did not reboot the server? Maybe it is spiraled worse since you took over, but it was left over from the last guy.
-
@Mike-Davis does terminal server connection works?? what about windows offline updater http://download.wsusoffline.net/
-
Have you checked the CBS.log file in %windir%\logs\cbs
It will give more verbose errors usually. -
Update for all those that suggested ideas. I took @dafyre 's idea to restore it to a hyper visor. I went on site and I'm not sure why, but it took like 11 hours to copy the backups 1TB+ to an external USB drive. I brought that back to my office and started the restore. That took about 7 hours each time I did that. The first time I just restored the
drive. After messing around with bcdedit I still couldn't get the thing to boot. Veeam said that the M: drive was a system drive to, so I created another VM and restored the and M: drive. This time I could boot to the Dell system installer setup, but still couldn't boot the OS.Then I decided to restore to another physical Dell server I had on the bench. It booted no problem. Veeam boots you to Directory Services Restore mode and then you have to use msconfig to tell it to do a normal boot and you're good. I did that and it seemed like it was having the same issue where I logged in and it showed the desktop but wouldn't respond to the keyboard. The mouse moved, but wouldn't let me click. I just left it and came back an hour later. At that point it was fine. Not sure what the deal was.
There were a few variables to take in to consideration. Since the NIC was different, none of the network services came up. I also disabled a few things like CrashPlan because when the NIC does come online, I don't want it to try to backup to the cloud since it's a clone of the real server that is still on production.
At this point I'm going to try to P2V the thing.
-
NOT 100% Sure,
but if you go here:
C:\Windows\Installer
Can you try to sorted it by latest modified date, then run the MSI and see if some can be repaired on uninstalled ?
-
We patched over the weekend and had issues with server 2008 R2 on a .net update
It eventually went through but it took 4-5 hours on that one update
-
So I changed the disk layout of the VM I was trying to restore to and it booted like a champ. So now I have the offending server running as a VM. It seems as soon as I log in, CPU goes to 100% and it's all due to one svchost that calls Power, PlugPlay, and DcomLaunch. I can kill the svchost but then the server complains and wants to reboot.
-
I added more CPUs so that the offending process only took up 25% of the total system CPU. Then as I was going through the event viewer I noticed that many of the things that wouldn't start and were timing out had to do with the network. The NIC was still disconnected, so I enabled that. It wasn't recognized in device manager, so I installed the Hyper V integration tools. That fixed the NIC. I still had about 6 devices listed in device manager under "Other devices." Right clicking each one and telling it to update the driver fixed 4 out of 6.
-
@Mike-Davis said in Autoupdates Killed My Windows Server 2008 R2:
I added more CPUs so that the offending process only took up 25% of the total system CPU. Then as I was going through the event viewer I noticed that many of the things that wouldn't start and were timing out had to do with the network. The NIC was still disconnected, so I enabled that. It wasn't recognized in device manager, so I installed the Hyper V integration tools. That fixed the NIC. I still had about 6 devices listed in device manager under "Other devices." Right clicking each one and telling it to update the driver fixed 4 out of 6.
Did this improve the usability of the machine? Or was it still dog slow?
-
@dafyre said in Autoupdates Killed My Windows Server 2008 R2:
Did this improve the usability of the machine? Or was it still dog slow?
Yes, adding CPUs was the difference between being able to use it and it being dog slow.
