XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!
-
@momurda said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
You might get some useful info from the XenCenter. In one of the earlier posts you had a Screencap of your XC. There were 5700+ notifications listed. I imagine if you click on that and goto Errors and Alerts buttons you will get some useful info. Alerts can be useful for finding the cause, Errors get a bit spammy in XC though they are useful as well.
Generally, the local storage not being connected is probably the first i would look at. That is bad. Then once that is working you need to figure out why your multipathd setup is borked. You said you changed some ip addresses; this is probably a major reason why it isnt working.
Have you tried logging into the switch this is all connected to and seeing if it is ok?It's a few thousand of these.
-
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
We still need to see the logs
[root@xen2 ~]# tail -n 100 messages tail: cannot open `messages' for reading: No such file or directorymight have to cd to the logs folder and try that tail again.
-
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
It's a few thousand of these.
What about the 5 errors under Events?
-
@Dashrender said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
What is the network setup on these servers and the SAN?
One switch, multiple switches? how many connections to each server? etc.
2 switches. I tried reaching out to the guy that set it up 6 years ago, and he said he doesn't even remember how he did it. It was the last time he did a XenServer and PowerVault setup. I can get you the networking info from the XC and from the PV, and maybe we can find info for the switch that way, but I have no documentation for it.
In my defense, I've been lobbying for a replacement for this setup. Hopefully, it's coming due now.
-
@Danp said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
It's a few thousand of these.
What about the 5 errors under Events?
All work I've been trying to do since it went down.
-
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
We still need to see the logs
[root@xen2 ~]# tail -n 100 messages tail: cannot open `messages' for reading: No such file or directoryThat's not good. That means that something really went wrong. Probably the disk had filled up.
-
@Dashrender said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
What is the network setup on these servers and the SAN?
One switch, multiple switches? how many connections to each server? etc.
-
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@Dashrender said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
What is the network setup on these servers and the SAN?
One switch, multiple switches? how many connections to each server? etc.
2 switches. I tried reaching out to the guy that set it up 6 years ago, and he said he doesn't even remember how he did it. It was the last time he did a XenServer and PowerVault setup. I can get you the networking info from the XC and from the PV, and maybe we can find info for the switch that way, but I have no documentation for it.
In my defense, I've been lobbying for a replacement for this setup. Hopefully, it's coming due now.
XenServer is very good, but the version you have is old and not maintained. Having it with the PowerVault was never acceptable. That was a bad setup. XenServer is a very good choice. But if you don't have a XenServer team, you'll want something else. Of course, @scale is a good choice, but you already know that
Moving this to a Scale cluster would be a quick fix that would address both the compute and the storage needs and let you move to a far more reliable setup. -
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@Dashrender said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
What is the network setup on these servers and the SAN?
One switch, multiple switches? how many connections to each server? etc.
2 switches. I tried reaching out to the guy that set it up 6 years ago, and he said he doesn't even remember how he did it. It was the last time he did a XenServer and PowerVault setup. I can get you the networking info from the XC and from the PV, and maybe we can find info for the switch that way, but I have no documentation for it.
In my defense, I've been lobbying for a replacement for this setup. Hopefully, it's coming due now.
XenServer is very good, but the version you have is old and not maintained. Having it with the PowerVault was never acceptable. That was a bad setup. XenServer is a very good choice. But if you don't have a XenServer team, you'll want something else. Of course, @scale is a good choice, but you already know that
Moving this to a Scale cluster would be a quick fix that would address both the compute and the storage needs and let you move to a far more reliable setup.Is it possible to somehow spin up what we have on a Scale cluster as is?
-
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@Dashrender said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
What is the network setup on these servers and the SAN?
One switch, multiple switches? how many connections to each server? etc.
2 switches. I tried reaching out to the guy that set it up 6 years ago, and he said he doesn't even remember how he did it. It was the last time he did a XenServer and PowerVault setup. I can get you the networking info from the XC and from the PV, and maybe we can find info for the switch that way, but I have no documentation for it.
In my defense, I've been lobbying for a replacement for this setup. Hopefully, it's coming due now.
XenServer is very good, but the version you have is old and not maintained. Having it with the PowerVault was never acceptable. That was a bad setup. XenServer is a very good choice. But if you don't have a XenServer team, you'll want something else. Of course, @scale is a good choice, but you already know that
Moving this to a Scale cluster would be a quick fix that would address both the compute and the storage needs and let you move to a far more reliable setup.Is it possible to somehow spin up what we have on a Scale cluster as is?
Not quite, we need a means of grabbing the files off of it. So we either need a utility to do some conversion which I think Scale can help with; or we need to do a P2V process which requires us to get this working (even if just limping) in the mean time.
-
The iSCSI host ports on the SAN are 192.168.130.101-104. No gateway, no reference to a switch that I can find. I'm hoping a guy that's at lunch has some documentation to help out with that.
-
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
The iSCSI host ports on the SAN are 192.168.130.101-104. No gateway, no reference to a switch that I can find. I'm hoping a guy that's at lunch has some documentation to help out with that.
I dont think that that is where we are stuck, so that's good info to have but not in our critical path at the moment (I don't think.)
-
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
We still need to see the logs
[root@xen2 ~]# tail -n 100 messages tail: cannot open `messages' for reading: No such file or directoryThat's not good. That means that something really went wrong. Probably the disk had filled up.
Is there another way to view log entries?
-
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
We still need to see the logs
[root@xen2 ~]# tail -n 100 messages tail: cannot open `messages' for reading: No such file or directoryThat's not good. That means that something really went wrong. Probably the disk had filled up.
Is there another way to view log entries?
Not if they are not there. That's not a log viewer, that's the logs itself. Are you use that you used the cd command to get into the right directory before trying to view the logs?
-
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
We still need to see the logs
[root@xen2 ~]# tail -n 100 messages tail: cannot open `messages' for reading: No such file or directoryThat's not good. That means that something really went wrong. Probably the disk had filled up.
Is there another way to view log entries?
Not if they are not there. That's not a log viewer, that's the logs itself. Are you use that you used the cd command to get into the right directory before trying to view the logs?
I used no cd command. I just tried "cd /var/logs" and was told there was no directory.
-
@scottalanmiller WAIT!!
-
Dec 27 13:04:50 xen2 multipathd: 360024e800070ed06000004314a967309: remaining ac tive paths: 1 Dec 27 13:04:50 xen2 xapi: [ info|xen2|1118516 UNIX /var/xapi/xapi|session.logou t D:8943f3fc0d25|xapi] Session.destroy trackid=08889875145ba350fd61700479bae5a1 Dec 27 13:04:50 xen2 xapi: [ info|xen2|1118517 UNIX /var/xapi/xapi|session.login _with_password D:fc3ccd82666a|xapi] Session.create trackid=f2aea39683d7b6a01a324 e173f3dd8b6 pool=false uname=root is_local_superuser=true auth_user_sid= parent= trackid=9834f5af41c964e225f24279aefe4e49 Dec 27 13:04:51 xen2 kernel: [17525.713536] sd 6:0:0:1: [sdi] Unhandled sense co de Dec 27 13:04:51 xen2 kernel: [17525.713539] sd 6:0:0:1: [sdi] Result: hostbyte=D ID_OK driverbyte=DRIVER_SENSE Dec 27 13:04:51 xen2 kernel: [17525.713541] sd 6:0:0:1: [sdi] Sense Key : Hardwa re Error [current] Dec 27 13:04:51 xen2 kernel: [17525.713544] sd 6:0:0:1: [sdi] <<vendor>> ASC=0x8 4 ASCQ=0x0ASC=0x84 ASCQ=0x0 Dec 27 13:04:51 xen2 kernel: [17525.713548] sd 6:0:0:1: [sdi] CDB: Read(10): 28 00 ae ff ff 80 00 00 08 00 Dec 27 13:04:51 xen2 kernel: [17525.713553] end_request: I/O error, dev sdi, sec tor 2936012672 Dec 27 13:04:51 xen2 kernel: [17525.713569] device-mapper: multipath: Failing pa th 8:128. Dec 27 13:04:51 xen2 xapi: [ info|xen2|1118555 INET 0.0.0.0:80|session.logout D: c734fb0019a4|xapi] Session.destroy trackid=b33614b35799e4591630faf1215fb656 Dec 27 13:04:51 xen2 multipathd: 8:128: mark as failed Dec 27 13:04:51 xen2 multipathd: 360024e800070ed06000004314a967309: Entering rec overy mode: max_retries=15 Dec 27 13:04:51 xen2 multipathd: 360024e800070ed06000004314a967309: remaining ac tive paths: 0 Dec 27 13:04:51 xen2 multipathd: Path event for 360024e800070ed06000004314a96730 9, request call of mpathcount Dec 27 13:04:51 xen2 xapi: [ info|xen2|1118565 UNIX /var/xapi/xapi|session.logou t D:5e8426752b6a|xapi] Session.destroy trackid=f2aea39683d7b6a01a324e173f3dd8b6 Dec 27 13:04:51 xen2 xapi: [ info|xen2|1118566 UNIX /var/xapi/xapi|session.login _with_password D:7bf7776c558a|xapi] Session.create trackid=0b8b2266594c81ed0f491 512ed56da5b pool=false uname=root is_local_superuser=true auth_user_sid= parent= trackid=9834f5af41c964e225f24279aefe4e49 Dec 27 13:04:51 xen2 multipathd: 360024e800070ed06000004314a967309: Entering rec overy mode: max_retries=15 Dec 27 13:04:51 xen2 xapi: [ info|xen2|1118609 UNIX /var/xapi/xapi|session.logou t D:b0f10b9ab800|xapi] Session.destroy trackid=0b8b2266594c81ed0f491512ed56da5b Dec 27 13:04:51 xen2 xapi: [ info|xen2|1118610 UNIX /var/xapi/xapi|session.login _with_password D:83a1d8c13406|xapi] Session.create trackid=57c35017dc0b58cba5006 fab885d6c42 pool=false uname=root is_local_superuser=true auth_user_sid= parent= trackid=9834f5af41c964e225f24279aefe4e49 Dec 27 13:04:52 xen2 xapi: [ info|xen2|1118649 INET 0.0.0.0:80|session.logout D: 80db80fdafff|xapi] Session.destroy trackid=a6e94a7ff69b574b678611e81606d239 Dec 27 13:04:52 xen2 xapi: [ info|xen2|1118654 UNIX /var/xapi/xapi|session.logou t D:29c302273758|xapi] Session.destroy trackid=57c35017dc0b58cba5006fab885d6c42 Dec 27 13:04:52 xen2 multipathd: 360024e800070ed06000004314a967309: sdl - rdac c hecker reports path is ghost Dec 27 13:04:52 xen2 multipathd: Path event for 360024e800070ed06000004314a96730 9, request call of mpathcount Dec 27 13:04:52 xen2 multipathd: 8:176: reinstated Dec 27 13:04:52 xen2 multipathd: 360024e800070ed06000004314a967309: queue_if_no_ path enabled Dec 27 13:04:52 xen2 multipathd: 360024e800070ed06000004314a967309: Recovered to normal mode Dec 27 13:04:52 xen2 multipathd: 360024e800070ed06000004314a967309: remaining ac tive paths: 1 Dec 27 13:04:52 xen2 xapi: [ info|xen2|1118668 UNIX /var/xapi/xapi|session.login _with_password D:8afb60727358|xapi] Session.create trackid=492e0a0a5edb18140a204 dcf801b0eb0 pool=false uname=root is_local_superuser=true auth_user_sid= parent= trackid=9834f5af41c964e225f24279aefe4e49 Dec 27 13:04:52 xen2 multipathd: 360024e800070ed06000004314a967309: switch to pa th group #4 Dec 27 13:04:52 xen2 kernel: [17527.512748] sd 7:0:0:1: rdac: array MDS-Spindle0 1, ctlr 1, queueing MODE_SELECT command Dec 27 13:04:53 xen2 xapi: [ info|xen2|1118702 UNIX /var/xapi/xapi|session.logou t D:d0126e6b3bad|xapi] Session.destroy trackid=492e0a0a5edb18140a204dcf801b0eb0 Dec 27 13:04:53 xen2 kernel: [17528.017140] sd 7:0:0:1: rdac: array MDS-Spindle0 1, ctlr 1, MODE_SELECT returned with sense 05/91/36 Dec 27 13:04:53 xen2 kernel: [17528.017153] sd 7:0:0:1: rdac: array MDS-Spindle0 1, ctlr 1, retrying MODE_SELECT command Dec 27 13:04:53 xen2 kernel: [17528.303402] sd 7:0:0:1: rdac: array MDS-Spindle0 1, ctlr 1, MODE_SELECT completed Dec 27 13:04:53 xen2 multipathd: 360024e800070ed06000004314a967309: sdf - rdac c hecker reports path is up Dec 27 13:04:53 xen2 multipathd: Path event for 360024e800070ed06000004314a96730 9, request call of mpathcount Dec 27 13:04:53 xen2 multipathd: 8:80: reinstated Dec 27 13:04:53 xen2 multipathd: 360024e800070ed06000004314a967309: remaining ac tive paths: 2 Dec 27 13:04:54 xen2 xapi: [ info|xen2|1118703 UNIX /var/xapi/xapi|session.login _with_password D:cd022f51a23b|xapi] Session.create trackid=78096f7ecdee23b2dae33 38911791e6f pool=false uname=root is_local_superuser=true auth_user_sid= parent= trackid=9834f5af41c964e225f24279aefe4e49 Dec 27 13:04:54 xen2 multipathd: 360024e800070ed06000004314a967309: load table [ 0 2936012800 multipath 3 pg_init_retries 50 queue_if_no_path 1 rdac 3 3 round-ro bin 0 1 1 8:128 1000 round-robin 0 1 1 8:32 1000 round-robin 0 2 1 8:80 1000 8:1 76 1000] Dec 27 13:04:54 xen2 multipathd: Path event for 360024e800070ed06000004314a96730 9, request call of mpathcount Dec 27 13:04:54 xen2 multipathd: Path event for 360024e800070ed06000004314a96730 9, request call of mpathcount Dec 27 13:04:54 xen2 kernel: [17528.816240] sd 7:0:0:1: [sdl] Unhandled sense co de Dec 27 13:04:54 xen2 kernel: [17528.816243] sd 7:0:0:1: [sdl] Result: hostbyte=D ID_OK driverbyte=DRIVER_SENSE Dec 27 13:04:54 xen2 kernel: [17528.816245] sd 7:0:0:1: [sdl] Sense Key : Hardwa re Error [current] Dec 27 13:04:54 xen2 kernel: [17528.816248] sd 7:0:0:1: [sdl] <<vendor>> ASC=0x8 4 ASCQ=0x0ASC=0x84 ASCQ=0x0 Dec 27 13:04:54 xen2 kernel: [17528.816252] sd 7:0:0:1: [sdl] CDB: Read(10): 28 00 ae ff ff 80 00 00 08 00 Dec 27 13:04:54 xen2 kernel: [17528.816257] end_request: I/O error, dev sdl, sec tor 2936012672 Dec 27 13:04:54 xen2 kernel: [17528.816274] device-mapper: multipath: Failing pa th 8:176. Dec 27 13:04:54 xen2 kernel: [17528.817236] device-mapper: multipath: Failing pa th 8:128. Dec 27 13:04:54 xen2 kernel: [17528.817315] device-mapper: multipath: Failing pa th 8:32. Dec 27 13:04:54 xen2 xapi: [ info|xen2|1118753 UNIX /var/xapi/xapi|session.logou t D:5eabc2a6317b|xapi] Session.destroy trackid=78096f7ecdee23b2dae3338911791e6f Dec 27 13:04:54 xen2 xapi: [ info|xen2|1118755 UNIX /var/xapi/xapi|session.login _with_password D:1894b4b17ab8|xapi] Session.create trackid=920f0d86a645a2f1f126e d2b0fc12f87 pool=false uname=root is_local_superuser=true auth_user_sid= parent= trackid=9834f5af41c964e225f24279aefe4e49 Dec 27 13:04:54 xen2 xapi: [ info|xen2|1118776 INET 0.0.0.0:80|session.logout D: 3c2cf4cc964b|xapi] Session.destroy trackid=e7b050fd41937a5158bf0b2ca39b1659 Dec 27 13:04:54 xen2 kernel: [17529.332325] sd 5:0:0:1: [sdf] Unhandled sense co de Dec 27 13:04:54 xen2 kernel: [17529.332329] sd 5:0:0:1: [sdf] Result: hostbyte=D ID_OK driverbyte=DRIVER_SENSE Dec 27 13:04:54 xen2 kernel: [17529.332331] sd 5:0:0:1: [sdf] Sense Key : Hardwa re Error [current] Dec 27 13:04:54 xen2 kernel: [17529.332334] sd 5:0:0:1: [sdf] <<vendor>> ASC=0x8 4 ASCQ=0x0ASC=0x84 ASCQ=0x0 Dec 27 13:04:54 xen2 kernel: [17529.332337] sd 5:0:0:1: [sdf] CDB: Read(10): 28 00 ae ff ff 80 00 00 08 00 Dec 27 13:04:54 xen2 kernel: [17529.332343] end_request: I/O error, dev sdf, sec tor 2936012672 Dec 27 13:04:54 xen2 kernel: [17529.332359] device-mapper: multipath: Failing pa th 8:80. Dec 27 13:04:54 xen2 multipathd: 8:80: mark as failed Dec 27 13:04:54 xen2 multipathd: 360024e800070ed06000004314a967309: remaining ac tive paths: 1 Dec 27 13:04:54 xen2 multipathd: Path event for 360024e800070ed06000004314a96730 9, request call of mpathcount Dec 27 13:04:55 xen2 xapi: [ info|xen2|1118805 UNIX /var/xapi/xapi|session.logou t D:41ef5f113fba|xapi] Session.destroy trackid=920f0d86a645a2f1f126ed2b0fc12f87 Dec 27 13:04:55 xen2 xapi: [ info|xen2|1118806 UNIX /var/xapi/xapi|session.login _with_password D:8b9980e8c1ea|xapi] Session.create trackid=8de1347f37c83b6954ea0 9681335db4e pool=false uname=root is_local_superuser=true auth_user_sid= parent= trackid=9834f5af41c964e225f24279aefe4e49 Dec 27 13:04:55 xen2 multipathd: 360024e800070ed06000004314a967309: sdc - rdac c hecker reports path is ghost Dec 27 13:04:55 xen2 multipathd: Path event for 360024e800070ed06000004314a96730 9, request call of mpathcount Dec 27 13:04:55 xen2 multipathd: 8:32: reinstated Dec 27 13:04:55 xen2 multipathd: 360024e800070ed06000004314a967309: remaining ac tive paths: 2 Dec 27 13:04:55 xen2 multipathd: 360024e800070ed06000004314a967309: switch to pa th group #2 Dec 27 13:04:55 xen2 multipathd: 8:176: mark as failed Dec 27 13:04:55 xen2 multipathd: 360024e800070ed06000004314a967309: remaining ac tive paths: 1 Dec 27 13:04:55 xen2 multipathd: Path event for 360024e800070ed06000004314a96730 9, request call of mpathcount Dec 27 13:04:55 xen2 kernel: [17529.844849] sd 7:0:0:1: [sdl] Unhandled sense co de Dec 27 13:04:55 xen2 kernel: [17529.844852] sd 7:0:0:1: [sdl] Result: hostbyte=D ID_OK driverbyte=DRIVER_SENSE Dec 27 13:04:55 xen2 kernel: [17529.844855] sd 7:0:0:1: [sdl] Sense Key : Hardwa re Error [current] Dec 27 13:04:55 xen2 kernel: [17529.844857] sd 7:0:0:1: [sdl] <<vendor>> ASC=0x8 4 ASCQ=0x0ASC=0x84 ASCQ=0x0 Dec 27 13:04:55 xen2 kernel: [17529.844861] sd 7:0:0:1: [sdl] CDB: Read(10): 28 00 ae ff ff 80 00 00 08 00 Dec 27 13:04:55 xen2 kernel: [17529.844866] end_request: I/O error, dev sdl, sec tor 2936012672 Dec 27 13:04:55 xen2 kernel: [17529.844879] device-mapper: multipath: Failing pa th 8:176. Dec 27 13:04:55 xen2 kernel: [17530.081711] sd 4:0:0:1: rdac: array MDS-Spindle0 1, ctlr 0, queueing MODE_SELECT command Dec 27 13:04:55 xen2 xapi: [ info|xen2|1118849 UNIX /var/xapi/xapi|session.logou t D:0494e32893e6|xapi] Session.destroy trackid=8de1347f37c83b6954ea09681335db4e Dec 27 13:04:55 xen2 xapi: [ info|xen2|1118850 UNIX /var/xapi/xapi|session.login _with_password D:6baeffeb13e0|xapi] Session.create trackid=b043dd9f380b982f1e8aa 32967a8de2b pool=false uname=root is_local_superuser=true auth_user_sid= parent= trackid=9834f5af41c964e225f24279aefe4e49 Dec 27 13:04:55 xen2 xapi: [ info|xen2|1118852 INET 0.0.0.0:80|session.logout D: 97bc80b93f75|xapi] Session.destroy trackid=53e2afc20c1a0ab616b48da0c7004030 Dec 27 13:04:55 xen2 kernel: [17530.329388] sd 4:0:0:1: rdac: array MDS-Spindle0 1, ctlr 0, MODE_SELECT completed Dec 27 13:04:55 xen2 xapi: [ info|xen2|1118884 UNIX /var/xapi/xapi|session.logou t D:15d77c89ee0a|xapi] Session.destroy trackid=b043dd9f380b982f1e8aa32967a8de2b Dec 27 13:04:56 xen2 kernel: [17530.841619] sd 4:0:0:1: [sdc] Unhandled sense co de Dec 27 13:04:56 xen2 kernel: [17530.841623] sd 4:0:0:1: [sdc] Result: hostbyte=D ID_OK driverbyte=DRIVER_SENSE Dec 27 13:04:56 xen2 kernel: [17530.841628] sd 4:0:0:1: [sdc] Sense Key : Hardwa re Error [current] Dec 27 13:04:56 xen2 kernel: [17530.841632] sd 4:0:0:1: [sdc] <<vendor>> ASC=0x8 4 ASCQ=0x0ASC=0x84 ASCQ=0x0 Dec 27 13:04:56 xen2 kernel: [17530.841638] sd 4:0:0:1: [sdc] CDB: Read(10): 28 00 ae ff ff 80 00 00 08 00 Dec 27 13:04:56 xen2 kernel: [17530.841648] end_request: I/O error, dev sdc, sec tor 2936012672 Dec 27 13:04:56 xen2 kernel: [17530.841672] device-mapper: multipath: Failing pa th 8:32. Dec 27 13:04:56 xen2 multipathd: 360024e800070ed06000004314a967309: sdi - rdac c hecker reports path is up Dec 27 13:04:56 xen2 multipathd: Path event for 360024e800070ed06000004314a96730 9, request call of mpathcount Dec 27 13:04:56 xen2 multipathd: 8:128: reinstated Dec 27 13:04:56 xen2 multipathd: 360024e800070ed06000004314a967309: remaining ac tive paths: 2 Dec 27 13:04:56 xen2 xapi: [ info|xen2|1118889 UNIX /var/xapi/xapi|session.login _with_password D:f3f2ba7c9b76|xapi] Session.create trackid=1f938a40d5b26b9761de4 6230d454ee5 pool=false uname=root is_local_superuser=true auth_user_sid= parent= trackid=9834f5af41c964e225f24279aefe4e49 -
There you go..
-
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
We still need to see the logs
[root@xen2 ~]# tail -n 100 messages tail: cannot open `messages' for reading: No such file or directoryThat's not good. That means that something really went wrong. Probably the disk had filled up.
Is there another way to view log entries?
Not if they are not there. That's not a log viewer, that's the logs itself. Are you use that you used the cd command to get into the right directory before trying to view the logs?
I used no cd command. I just tried "cd /var/logs" and was told there was no directory.
We are definitely seeing a problem here
-
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
@scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:
We still need to see the logs
[root@xen2 ~]# tail -n 100 messages tail: cannot open `messages' for reading: No such file or directoryThat's not good. That means that something really went wrong. Probably the disk had filled up.
Is there another way to view log entries?
Not if they are not there. That's not a log viewer, that's the logs itself. Are you use that you used the cd command to get into the right directory before trying to view the logs?
I used no cd command. I just tried "cd /var/logs" and was told there was no directory.
We are definitely seeing a problem here
Scott, why is the formatting so messed up?
