XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!

scottalanmiller

@CitrixNewbJD okay, no need to look in the xen folder, that's not going to be a useful log

scottalanmiller

I'm discussing conversion options with Scale. Looks like moving the disk images over would not be too hard as long as you can pull them off of the SAN.

scottalanmiller

If you do this...

cd ..

It will take you back to the main logs. What is the output of this...

ls | grep -i xen

scottalanmiller

While you are working on that, this is a good time to discuss triage. In all honesty, this is an old, unmaintained cluster. What is the hardware of the compute nodes?

The storage is poor. Even if we fix things, the system is very old in both software and hardware. So it's only a bandaid. Which is very doable, but it is what it is.

Doing a migration, like right now, might be the logical path forward. Let's get your specs for the current cluster, concerns for a new one and discuss if that's a real possibility. Also, what is the moment to moment impact of the outage?

CitrixNewbJD

@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

If you do this...
cd ..
It will take you back to the main logs. What is the output of this...
ls | grep -i xen

[root@xen2 ~]# ls | grep -i xen
xen2pooldb
[root@xen2 ~]#

scottalanmiller

@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
If you do this...
cd ..
It will take you back to the main logs. What is the output of this...
ls | grep -i xen
[root@xen2 ~]# ls | grep -i xen
xen2pooldb
[root@xen2 ~]#

What does this return...

pwd

CitrixNewbJD

@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

While you are working on that, this is a good time to discuss triage. In all honesty, this is an old, unmaintained cluster. What is the hardware of the compute nodes?

The storage is poor. Even if we fix things, the system is very old in both software and hardware. So it's only a bandaid. Which is very doable, but it is what it is.

Doing a migration, like right now, might be the logical path forward. Let's get your specs for the current cluster, concerns for a new one and discuss if that's a real possibility. Also, what is the moment to moment impact of the outage?

I actually have a quote for a cluster from Scale that I had planned to try to push forward with the owner of the company when he returns from Europe next week. In the interest of properly giving you the information that you need, I want to be sure you're aware of that.

CitrixNewbJD

@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
If you do this...
cd ..
It will take you back to the main logs. What is the output of this...
ls | grep -i xen
[root@xen2 ~]# ls | grep -i xen
xen2pooldb
[root@xen2 ~]#
What does this return...
pwd

/root

Dashrender

@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
If you do this...
cd ..
It will take you back to the main logs. What is the output of this...
ls | grep -i xen
[root@xen2 ~]# ls | grep -i xen
xen2pooldb
[root@xen2 ~]#
What does this return...
pwd
/root

that explains a lot - I think you want to be in /var/logs

 cd /var/logs

scottalanmiller

@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

While you are working on that, this is a good time to discuss triage. In all honesty, this is an old, unmaintained cluster. What is the hardware of the compute nodes?

The storage is poor. Even if we fix things, the system is very old in both software and hardware. So it's only a bandaid. Which is very doable, but it is what it is.

Doing a migration, like right now, might be the logical path forward. Let's get your specs for the current cluster, concerns for a new one and discuss if that's a real possibility. Also, what is the moment to moment impact of the outage?

I actually have a quote for a cluster from Scale that I had planned to try to push forward with the owner of the company when he returns from Europe next week. In the interest of properly giving you the information that you need, I want to be sure you're aware of that.

So the quote is already spec'd and ready to go? Maybe call him in Europe and see if things can be expedited? Would suck to have a major recovery operation happen just to hold off a discussion by one week.

scottalanmiller

@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
If you do this...
cd ..
It will take you back to the main logs. What is the output of this...
ls | grep -i xen
[root@xen2 ~]# ls | grep -i xen
xen2pooldb
[root@xen2 ~]#
What does this return...
pwd
/root

How did you get there?

scottalanmiller

@Dashrender said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
If you do this...
cd ..
It will take you back to the main logs. What is the output of this...
ls | grep -i xen
[root@xen2 ~]# ls | grep -i xen
xen2pooldb
[root@xen2 ~]#
What does this return...
pwd
/root
that explains a lot - I think you want to be in /var/logs
 cd /var/logs

Yes

Dashrender

@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
If you do this...
cd ..
It will take you back to the main logs. What is the output of this...
ls | grep -i xen
[root@xen2 ~]# ls | grep -i xen
xen2pooldb
[root@xen2 ~]#
What does this return...
pwd
/root
How did you get there?

I was wondering that.

CitrixNewbJD

@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

@Dashrender said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
If you do this...
cd ..
It will take you back to the main logs. What is the output of this...
ls | grep -i xen
[root@xen2 ~]# ls | grep -i xen
xen2pooldb
[root@xen2 ~]#
What does this return...
pwd
/root
that explains a lot - I think you want to be in /var/logs
 cd /var/logs
Yes

[root@xen2 log]# ls | grep -i xen
xen
xen-dmesg
xensource.log
xensource.log.1.gz
xensource.log.2.gz
xensource.log.3.gz
xensource.log.4.gz
xenstored-access.log

scottalanmiller

@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@Dashrender said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
If you do this...
cd ..
It will take you back to the main logs. What is the output of this...
ls | grep -i xen
[root@xen2 ~]# ls | grep -i xen
xen2pooldb
[root@xen2 ~]#
What does this return...
pwd
/root
that explains a lot - I think you want to be in /var/logs
 cd /var/logs
Yes
[root@xen2 log]# ls | grep -i xen
xen
xen-dmesg
xensource.log
xensource.log.1.gz
xensource.log.2.gz
xensource.log.3.gz
xensource.log.4.gz
xenstored-access.log

There we go. Let's see what the xen logs thing.

tail -n 100 xen

tail -n 100 xen-dmesg

tail -n 100 xensource.log

Dashrender

Scott could make this a bit easier on you and give you specific paths in the commands

example

 tail -n 100 /var/log/xen
 tail -n 100 /var/log/xen-dmesg
 tail -n 100 /var/log/xensource.log

You can run these from path and they should give you the desired output

CitrixNewbJD

@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@Dashrender said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
If you do this...
cd ..
It will take you back to the main logs. What is the output of this...
ls | grep -i xen
[root@xen2 ~]# ls | grep -i xen
xen2pooldb
[root@xen2 ~]#
What does this return...
pwd
/root
that explains a lot - I think you want to be in /var/logs
 cd /var/logs
Yes
[root@xen2 log]# ls | grep -i xen
xen
xen-dmesg
xensource.log
xensource.log.1.gz
xensource.log.2.gz
xensource.log.3.gz
xensource.log.4.gz
xenstored-access.log
There we go. Let's see what the xen logs thing.
tail -n 100 xen

tail -n 100 xen-dmesg

tail -n 100 xensource.log

[root@xen2 log]# tail -n 100 /var/logs/xen
tail: cannot open `/var/logs/xen' for reading: No such file or directory
[root@xen2 log]# tail -n 100 /var/log/xen
tail: error reading `/var/log/xen': Is a directory
[root@xen2 log]# tail -n 100 xen
tail: error reading `xen': Is a directory
[root@xen2 log]# cd xen
[root@xen2 xen]# tail -n 100 xen
tail: cannot open `xen' for reading: No such file or directory

What's wrong with my syntax?

CitrixNewbJD

@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

While you are working on that, this is a good time to discuss triage. In all honesty, this is an old, unmaintained cluster. What is the hardware of the compute nodes?

The storage is poor. Even if we fix things, the system is very old in both software and hardware. So it's only a bandaid. Which is very doable, but it is what it is.

Doing a migration, like right now, might be the logical path forward. Let's get your specs for the current cluster, concerns for a new one and discuss if that's a real possibility. Also, what is the moment to moment impact of the outage?

I actually have a quote for a cluster from Scale that I had planned to try to push forward with the owner of the company when he returns from Europe next week. In the interest of properly giving you the information that you need, I want to be sure you're aware of that.

So the quote is already spec'd and ready to go? Maybe call him in Europe and see if things can be expedited? Would suck to have a major recovery operation happen just to hold off a discussion by one week.

I'd need to talk to someone from Scale to understand the process and guarantees of it working, or the headaches involved before making that call. Can you tell me with whom you spoke, or have them contact my reps, Blake Dickmeyer and Frank Seal?

scottalanmiller

@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

While you are working on that, this is a good time to discuss triage. In all honesty, this is an old, unmaintained cluster. What is the hardware of the compute nodes?

The storage is poor. Even if we fix things, the system is very old in both software and hardware. So it's only a bandaid. Which is very doable, but it is what it is.

Doing a migration, like right now, might be the logical path forward. Let's get your specs for the current cluster, concerns for a new one and discuss if that's a real possibility. Also, what is the moment to moment impact of the outage?

I actually have a quote for a cluster from Scale that I had planned to try to push forward with the owner of the company when he returns from Europe next week. In the interest of properly giving you the information that you need, I want to be sure you're aware of that.

So the quote is already spec'd and ready to go? Maybe call him in Europe and see if things can be expedited? Would suck to have a major recovery operation happen just to hold off a discussion by one week.

I'd need to talk to someone from Scale to understand the process and guarantees of it working, or the headaches involved before making that call. Can you tell me with whom you spoke, or have them contact my reps, Blake Dickmeyer and Frank Seal?

We know Frank well, he's on here as well. And he attended MangoCon this year (and he was at the Scale Legion event in Dallas a few weeks ago.)