XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!
-
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
A lack of a backup and DR strategy was one of the things I was brought in to remediate. No. There are no backups.
Wow. Just.... wow.
-
What storage do you have for the servers? Your san is dead; your xs cant connect to local. Right now the company is nonexistant except for incorporation papers at the state of incorporation.
I would try to do something with that san, power it off, pull it out of the rack and reseat the cards and drives.
Turn it on and watch post and bootup, login to the web interface and see if you actually have hardware failure going on. At least try to get it working long enough to get that data. -
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@NerdyDad On the plus side, this environment was so effed up, I'm not THAT terribly upset about starting from scratch. It'll be a beating to get a lot of that data back, however. Anyone know of any services able to accomplish such a thing?
That's good. And none of it is your fault. Sounds like every inch of the old design was set up for disaster.
-
@NerdyDad said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
Looks like I'm building a bunch of new servers.
I'd say your almost starting from scratch.
Completely from scratch, really.
-
Wow - ug... data recovery sounds like your only option. I've heard of ACE data recovery, never used them though.
good luck!
-
And the owner of the company refuses to accept anything I'm telling him. "There's no way a RAID array can fail. If we need to replace a couple of disk drives then we do that."
So this is fun.
-
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
And the owner of the company refuses to accept anything I'm telling him. "There's no way a RAID array can fail. If we need to replace a couple of disk drives then we do that."
So this is fun.
WHAT? You called him in Europe and he told you that? Ug.. time to look for another job?
-
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
... "There's no way a RAID array can fail. ..."
So, this is going to get interesting. Might be time to brush up that resume...just in case.
-
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
And the owner of the company refuses to accept anything I'm telling him. "There's no way a RAID array can fail. If we need to replace a couple of disk drives then we do that."
So this is fun.
Every piece of that setup was designed to fail. He's clearly incompetent and you now see why such a bad setup exists. I'd look for other work. You just got some serious insight into the cluelessness of this company.
-
@NerdyDad said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
... "There's no way a RAID array can fail. ..."
So, this is going to get interesting. Might be time to brush up that resume...just in case.
Honestly, after a statement like that I'd start job hunting proactively. No IT future in a company with someone so idiotic at the top.
-
If this was a court case, I'd testify that this was designed to lose data. If that owner tries to make an insurance claim, it should be fraud. This was a system that spent extra money and put in extra effort to ensure that data loss was not just possible but extremely likely.
If the owner takes a role in IT decisions then the bad decision making falls to him. Being completely honest, I'd fire an intern for this many mistakes at this level. No future in working for someone that we'd not hire as a high school intern.
-
At this point, I'm even at a loss on trying to figure out where to begin to explain to him how this is possible, as he hasn't seem to accept the premise in the first place.
-
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
At this point, I'm even at a loss on trying to figure out where to begin to explain to him how this is possible, as he hasn't seem to accept the premise in the first place.
Not your job to. If he doesn't believe it, then the system is healthy and there is nothing more for you to do. His system being down is his decision at this point.
-
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
At this point, I'm even at a loss on trying to figure out where to begin to explain to him how this is possible, as he hasn't seem to accept the premise in the first place.
When is he due back in the office? I would try to get on his schedule first thing ASAP, sit down with him, and talk with him about this. Maybe he can't visualize it over the phone, but once you draw it out for him, then he might be able to understand.
If he still refuses, I'd have a sheet of paper that a lawyer has drawn up that takes the blame off of you that he has to sign. CYA all you can here. Document everything and how you found it. I'd also be looking actively for other opportunities as well with a resignation in hand as a last resort.
-
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
At this point, I'm even at a loss on trying to figure out where to begin to explain to him how this is possible, as he hasn't seem to accept the premise in the first place.
Not your job to. If he doesn't believe it, then the system is healthy and there is nothing more for you to do. His system being down is his decision at this point.
If this is truly the case, then the owner is either incompetant in IT, which is typically the case, or he is delusional and needs mental help. I'm not trying to make this a joke either. However, delusional might also show itself in other aspects in his life. I'm not a psychiatrist so don't take this as advice.
-
@NerdyDad said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
At this point, I'm even at a loss on trying to figure out where to begin to explain to him how this is possible, as he hasn't seem to accept the premise in the first place.
Not your job to. If he doesn't believe it, then the system is healthy and there is nothing more for you to do. His system being down is his decision at this point.
If this is truly the case, then the owner is either incompetant in IT, which is typically the case, or he is delusional and needs mental help. I'm not trying to make this a joke either. However, delusional might also show itself in other aspects in his life. I'm not a psychiatrist so don't take this as advice.
And if the owner is getting his hands in IT that makes him foolish in business.
-
@NerdyDad said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
At this point, I'm even at a loss on trying to figure out where to begin to explain to him how this is possible, as he hasn't seem to accept the premise in the first place.
Not your job to. If he doesn't believe it, then the system is healthy and there is nothing more for you to do. His system being down is his decision at this point.
If this is truly the case, then the owner is either incompetant in IT, which is typically the case, or he is delusional and needs mental help. I'm not trying to make this a joke either. However, delusional might also show itself in other aspects in his life. I'm not a psychiatrist so don't take this as advice.
It's true. Saying that rAID 5 can't fail is like saying you don't need brakes. Because. seat belts can't fail.
It's a lacking grasp on reality. That RAID WILL is obvious beyond reason.
-
First, thanks everyone for your help yesterday, here's a brief update, more to come after a couple of phone calls.
I've powered down the SAN and reseated the drives. No joy. About to call Dell and see what options I may have, if any. The good news is there's a second SAN, and some data for the VMs is on there, but few OSes. The good news, is that somehow, I really have no idea how, the Xen servers reattached to the local storage overnight, and to the one virtual disk (Spindle2) and one DC is available. I'll give more details later.I wouldn't blame you for thinking I'm an incompetent tool, but I appreciate your help, anyway.
-
Cool. That the local storage was not attached was weird. That the SAN failed was.... expected. So the combination was making things complicated.
-
The previous guy setup local and SAN storage? lol I mean I guess there might be cases for that that, but still.
Now that you have a working DC, personally I'd grab any ol' desktop, install Hyper-V or XS on it and install Windows server there and get yourself another DC ASAP.
