ML
    • Recent
    • Categories
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!

    IT Discussion
    xenserver xenserver 6.2 iscsi san
    10
    243
    43.9k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • C
      CitrixNewbJD
      last edited by

      Dec 27 13:04:50 xen2 multipathd: 360024e800070ed06000004314a967309: remaining ac                                                                                                                                                             tive paths: 1
      Dec 27 13:04:50 xen2 xapi: [ info|xen2|1118516 UNIX /var/xapi/xapi|session.logou                                                                                                                                                             t D:8943f3fc0d25|xapi] Session.destroy trackid=08889875145ba350fd61700479bae5a1
      Dec 27 13:04:50 xen2 xapi: [ info|xen2|1118517 UNIX /var/xapi/xapi|session.login                                                                                                                                                             _with_password D:fc3ccd82666a|xapi] Session.create trackid=f2aea39683d7b6a01a324                                                                                                                                                             e173f3dd8b6 pool=false uname=root is_local_superuser=true auth_user_sid= parent=                                                                                                                                                             trackid=9834f5af41c964e225f24279aefe4e49
      Dec 27 13:04:51 xen2 kernel: [17525.713536] sd 6:0:0:1: [sdi] Unhandled sense co                                                                                                                                                             de
      Dec 27 13:04:51 xen2 kernel: [17525.713539] sd 6:0:0:1: [sdi] Result: hostbyte=D                                                                                                                                                             ID_OK driverbyte=DRIVER_SENSE
      Dec 27 13:04:51 xen2 kernel: [17525.713541] sd 6:0:0:1: [sdi] Sense Key : Hardwa                                                                                                                                                             re Error [current]
      Dec 27 13:04:51 xen2 kernel: [17525.713544] sd 6:0:0:1: [sdi] <<vendor>> ASC=0x8                                                                                                                                                             4 ASCQ=0x0ASC=0x84 ASCQ=0x0
      Dec 27 13:04:51 xen2 kernel: [17525.713548] sd 6:0:0:1: [sdi] CDB: Read(10): 28                                                                                                                                                              00 ae ff ff 80 00 00 08 00
      Dec 27 13:04:51 xen2 kernel: [17525.713553] end_request: I/O error, dev sdi, sec                                                                                                                                                             tor 2936012672
      Dec 27 13:04:51 xen2 kernel: [17525.713569] device-mapper: multipath: Failing pa                                                                                                                                                             th 8:128.
      Dec 27 13:04:51 xen2 xapi: [ info|xen2|1118555 INET 0.0.0.0:80|session.logout D:                                                                                                                                                             c734fb0019a4|xapi] Session.destroy trackid=b33614b35799e4591630faf1215fb656
      Dec 27 13:04:51 xen2 multipathd: 8:128: mark as failed
      Dec 27 13:04:51 xen2 multipathd: 360024e800070ed06000004314a967309: Entering rec                                                                                                                                                             overy mode: max_retries=15
      Dec 27 13:04:51 xen2 multipathd: 360024e800070ed06000004314a967309: remaining ac                                                                                                                                                             tive paths: 0
      Dec 27 13:04:51 xen2 multipathd: Path event for 360024e800070ed06000004314a96730                                                                                                                                                             9, request call of mpathcount
      Dec 27 13:04:51 xen2 xapi: [ info|xen2|1118565 UNIX /var/xapi/xapi|session.logou                                                                                                                                                             t D:5e8426752b6a|xapi] Session.destroy trackid=f2aea39683d7b6a01a324e173f3dd8b6
      Dec 27 13:04:51 xen2 xapi: [ info|xen2|1118566 UNIX /var/xapi/xapi|session.login                                                                                                                                                             _with_password D:7bf7776c558a|xapi] Session.create trackid=0b8b2266594c81ed0f491                                                                                                                                                             512ed56da5b pool=false uname=root is_local_superuser=true auth_user_sid= parent=                                                                                                                                                             trackid=9834f5af41c964e225f24279aefe4e49
      Dec 27 13:04:51 xen2 multipathd: 360024e800070ed06000004314a967309: Entering rec                                                                                                                                                             overy mode: max_retries=15
      Dec 27 13:04:51 xen2 xapi: [ info|xen2|1118609 UNIX /var/xapi/xapi|session.logou                                                                                                                                                             t D:b0f10b9ab800|xapi] Session.destroy trackid=0b8b2266594c81ed0f491512ed56da5b
      Dec 27 13:04:51 xen2 xapi: [ info|xen2|1118610 UNIX /var/xapi/xapi|session.login                                                                                                                                                             _with_password D:83a1d8c13406|xapi] Session.create trackid=57c35017dc0b58cba5006                                                                                                                                                             fab885d6c42 pool=false uname=root is_local_superuser=true auth_user_sid= parent=                                                                                                                                                             trackid=9834f5af41c964e225f24279aefe4e49
      Dec 27 13:04:52 xen2 xapi: [ info|xen2|1118649 INET 0.0.0.0:80|session.logout D:                                                                                                                                                             80db80fdafff|xapi] Session.destroy trackid=a6e94a7ff69b574b678611e81606d239
      Dec 27 13:04:52 xen2 xapi: [ info|xen2|1118654 UNIX /var/xapi/xapi|session.logou                                                                                                                                                             t D:29c302273758|xapi] Session.destroy trackid=57c35017dc0b58cba5006fab885d6c42
      Dec 27 13:04:52 xen2 multipathd: 360024e800070ed06000004314a967309: sdl - rdac c                                                                                                                                                             hecker reports path is ghost
      Dec 27 13:04:52 xen2 multipathd: Path event for 360024e800070ed06000004314a96730                                                                                                                                                             9, request call of mpathcount
      Dec 27 13:04:52 xen2 multipathd: 8:176: reinstated
      Dec 27 13:04:52 xen2 multipathd: 360024e800070ed06000004314a967309: queue_if_no_                                                                                                                                                             path enabled
      Dec 27 13:04:52 xen2 multipathd: 360024e800070ed06000004314a967309: Recovered to                                                                                                                                                              normal mode
      Dec 27 13:04:52 xen2 multipathd: 360024e800070ed06000004314a967309: remaining ac                                                                                                                                                             tive paths: 1
      Dec 27 13:04:52 xen2 xapi: [ info|xen2|1118668 UNIX /var/xapi/xapi|session.login                                                                                                                                                             _with_password D:8afb60727358|xapi] Session.create trackid=492e0a0a5edb18140a204                                                                                                                                                             dcf801b0eb0 pool=false uname=root is_local_superuser=true auth_user_sid= parent=                                                                                                                                                             trackid=9834f5af41c964e225f24279aefe4e49
      Dec 27 13:04:52 xen2 multipathd: 360024e800070ed06000004314a967309: switch to pa                                                                                                                                                             th group #4
      Dec 27 13:04:52 xen2 kernel: [17527.512748] sd 7:0:0:1: rdac: array MDS-Spindle0                                                                                                                                                             1, ctlr 1, queueing MODE_SELECT command
      Dec 27 13:04:53 xen2 xapi: [ info|xen2|1118702 UNIX /var/xapi/xapi|session.logou                                                                                                                                                             t D:d0126e6b3bad|xapi] Session.destroy trackid=492e0a0a5edb18140a204dcf801b0eb0
      Dec 27 13:04:53 xen2 kernel: [17528.017140] sd 7:0:0:1: rdac: array MDS-Spindle0                                                                                                                                                             1, ctlr 1, MODE_SELECT returned with sense 05/91/36
      Dec 27 13:04:53 xen2 kernel: [17528.017153] sd 7:0:0:1: rdac: array MDS-Spindle0                                                                                                                                                             1, ctlr 1, retrying MODE_SELECT command
      Dec 27 13:04:53 xen2 kernel: [17528.303402] sd 7:0:0:1: rdac: array MDS-Spindle0                                                                                                                                                             1, ctlr 1, MODE_SELECT completed
      Dec 27 13:04:53 xen2 multipathd: 360024e800070ed06000004314a967309: sdf - rdac c                                                                                                                                                             hecker reports path is up
      Dec 27 13:04:53 xen2 multipathd: Path event for 360024e800070ed06000004314a96730                                                                                                                                                             9, request call of mpathcount
      Dec 27 13:04:53 xen2 multipathd: 8:80: reinstated
      Dec 27 13:04:53 xen2 multipathd: 360024e800070ed06000004314a967309: remaining ac                                                                                                                                                             tive paths: 2
      Dec 27 13:04:54 xen2 xapi: [ info|xen2|1118703 UNIX /var/xapi/xapi|session.login                                                                                                                                                             _with_password D:cd022f51a23b|xapi] Session.create trackid=78096f7ecdee23b2dae33                                                                                                                                                             38911791e6f pool=false uname=root is_local_superuser=true auth_user_sid= parent=                                                                                                                                                             trackid=9834f5af41c964e225f24279aefe4e49
      Dec 27 13:04:54 xen2 multipathd: 360024e800070ed06000004314a967309: load table [                                                                                                                                                             0 2936012800 multipath 3 pg_init_retries 50 queue_if_no_path 1 rdac 3 3 round-ro                                                                                                                                                             bin 0 1 1 8:128 1000 round-robin 0 1 1 8:32 1000 round-robin 0 2 1 8:80 1000 8:1                                                                                                                                                             76 1000]
      Dec 27 13:04:54 xen2 multipathd: Path event for 360024e800070ed06000004314a96730                                                                                                                                                             9, request call of mpathcount
      Dec 27 13:04:54 xen2 multipathd: Path event for 360024e800070ed06000004314a96730                                                                                                                                                             9, request call of mpathcount
      Dec 27 13:04:54 xen2 kernel: [17528.816240] sd 7:0:0:1: [sdl] Unhandled sense co                                                                                                                                                             de
      Dec 27 13:04:54 xen2 kernel: [17528.816243] sd 7:0:0:1: [sdl] Result: hostbyte=D                                                                                                                                                             ID_OK driverbyte=DRIVER_SENSE
      Dec 27 13:04:54 xen2 kernel: [17528.816245] sd 7:0:0:1: [sdl] Sense Key : Hardwa                                                                                                                                                             re Error [current]
      Dec 27 13:04:54 xen2 kernel: [17528.816248] sd 7:0:0:1: [sdl] <<vendor>> ASC=0x8                                                                                                                                                             4 ASCQ=0x0ASC=0x84 ASCQ=0x0
      Dec 27 13:04:54 xen2 kernel: [17528.816252] sd 7:0:0:1: [sdl] CDB: Read(10): 28                                                                                                                                                              00 ae ff ff 80 00 00 08 00
      Dec 27 13:04:54 xen2 kernel: [17528.816257] end_request: I/O error, dev sdl, sec                                                                                                                                                             tor 2936012672
      Dec 27 13:04:54 xen2 kernel: [17528.816274] device-mapper: multipath: Failing pa                                                                                                                                                             th 8:176.
      Dec 27 13:04:54 xen2 kernel: [17528.817236] device-mapper: multipath: Failing pa                                                                                                                                                             th 8:128.
      Dec 27 13:04:54 xen2 kernel: [17528.817315] device-mapper: multipath: Failing pa                                                                                                                                                             th 8:32.
      Dec 27 13:04:54 xen2 xapi: [ info|xen2|1118753 UNIX /var/xapi/xapi|session.logou                                                                                                                                                             t D:5eabc2a6317b|xapi] Session.destroy trackid=78096f7ecdee23b2dae3338911791e6f
      Dec 27 13:04:54 xen2 xapi: [ info|xen2|1118755 UNIX /var/xapi/xapi|session.login                                                                                                                                                             _with_password D:1894b4b17ab8|xapi] Session.create trackid=920f0d86a645a2f1f126e                                                                                                                                                             d2b0fc12f87 pool=false uname=root is_local_superuser=true auth_user_sid= parent=                                                                                                                                                             trackid=9834f5af41c964e225f24279aefe4e49
      Dec 27 13:04:54 xen2 xapi: [ info|xen2|1118776 INET 0.0.0.0:80|session.logout D:                                                                                                                                                             3c2cf4cc964b|xapi] Session.destroy trackid=e7b050fd41937a5158bf0b2ca39b1659
      Dec 27 13:04:54 xen2 kernel: [17529.332325] sd 5:0:0:1: [sdf] Unhandled sense co                                                                                                                                                             de
      Dec 27 13:04:54 xen2 kernel: [17529.332329] sd 5:0:0:1: [sdf] Result: hostbyte=D                                                                                                                                                             ID_OK driverbyte=DRIVER_SENSE
      Dec 27 13:04:54 xen2 kernel: [17529.332331] sd 5:0:0:1: [sdf] Sense Key : Hardwa                                                                                                                                                             re Error [current]
      Dec 27 13:04:54 xen2 kernel: [17529.332334] sd 5:0:0:1: [sdf] <<vendor>> ASC=0x8                                                                                                                                                             4 ASCQ=0x0ASC=0x84 ASCQ=0x0
      Dec 27 13:04:54 xen2 kernel: [17529.332337] sd 5:0:0:1: [sdf] CDB: Read(10): 28                                                                                                                                                              00 ae ff ff 80 00 00 08 00
      Dec 27 13:04:54 xen2 kernel: [17529.332343] end_request: I/O error, dev sdf, sec                                                                                                                                                             tor 2936012672
      Dec 27 13:04:54 xen2 kernel: [17529.332359] device-mapper: multipath: Failing pa                                                                                                                                                             th 8:80.
      Dec 27 13:04:54 xen2 multipathd: 8:80: mark as failed
      Dec 27 13:04:54 xen2 multipathd: 360024e800070ed06000004314a967309: remaining ac                                                                                                                                                             tive paths: 1
      Dec 27 13:04:54 xen2 multipathd: Path event for 360024e800070ed06000004314a96730                                                                                                                                                             9, request call of mpathcount
      Dec 27 13:04:55 xen2 xapi: [ info|xen2|1118805 UNIX /var/xapi/xapi|session.logou                                                                                                                                                             t D:41ef5f113fba|xapi] Session.destroy trackid=920f0d86a645a2f1f126ed2b0fc12f87
      Dec 27 13:04:55 xen2 xapi: [ info|xen2|1118806 UNIX /var/xapi/xapi|session.login                                                                                                                                                             _with_password D:8b9980e8c1ea|xapi] Session.create trackid=8de1347f37c83b6954ea0                                                                                                                                                             9681335db4e pool=false uname=root is_local_superuser=true auth_user_sid= parent=                                                                                                                                                             trackid=9834f5af41c964e225f24279aefe4e49
      Dec 27 13:04:55 xen2 multipathd: 360024e800070ed06000004314a967309: sdc - rdac c                                                                                                                                                             hecker reports path is ghost
      Dec 27 13:04:55 xen2 multipathd: Path event for 360024e800070ed06000004314a96730                                                                                                                                                             9, request call of mpathcount
      Dec 27 13:04:55 xen2 multipathd: 8:32: reinstated
      Dec 27 13:04:55 xen2 multipathd: 360024e800070ed06000004314a967309: remaining ac                                                                                                                                                             tive paths: 2
      Dec 27 13:04:55 xen2 multipathd: 360024e800070ed06000004314a967309: switch to pa                                                                                                                                                             th group #2
      Dec 27 13:04:55 xen2 multipathd: 8:176: mark as failed
      Dec 27 13:04:55 xen2 multipathd: 360024e800070ed06000004314a967309: remaining ac                                                                                                                                                             tive paths: 1
      Dec 27 13:04:55 xen2 multipathd: Path event for 360024e800070ed06000004314a96730                                                                                                                                                             9, request call of mpathcount
      Dec 27 13:04:55 xen2 kernel: [17529.844849] sd 7:0:0:1: [sdl] Unhandled sense co                                                                                                                                                             de
      Dec 27 13:04:55 xen2 kernel: [17529.844852] sd 7:0:0:1: [sdl] Result: hostbyte=D                                                                                                                                                             ID_OK driverbyte=DRIVER_SENSE
      Dec 27 13:04:55 xen2 kernel: [17529.844855] sd 7:0:0:1: [sdl] Sense Key : Hardwa                                                                                                                                                             re Error [current]
      Dec 27 13:04:55 xen2 kernel: [17529.844857] sd 7:0:0:1: [sdl] <<vendor>> ASC=0x8                                                                                                                                                             4 ASCQ=0x0ASC=0x84 ASCQ=0x0
      Dec 27 13:04:55 xen2 kernel: [17529.844861] sd 7:0:0:1: [sdl] CDB: Read(10): 28                                                                                                                                                              00 ae ff ff 80 00 00 08 00
      Dec 27 13:04:55 xen2 kernel: [17529.844866] end_request: I/O error, dev sdl, sec                                                                                                                                                             tor 2936012672
      Dec 27 13:04:55 xen2 kernel: [17529.844879] device-mapper: multipath: Failing pa                                                                                                                                                             th 8:176.
      Dec 27 13:04:55 xen2 kernel: [17530.081711] sd 4:0:0:1: rdac: array MDS-Spindle0                                                                                                                                                             1, ctlr 0, queueing MODE_SELECT command
      Dec 27 13:04:55 xen2 xapi: [ info|xen2|1118849 UNIX /var/xapi/xapi|session.logou                                                                                                                                                             t D:0494e32893e6|xapi] Session.destroy trackid=8de1347f37c83b6954ea09681335db4e
      Dec 27 13:04:55 xen2 xapi: [ info|xen2|1118850 UNIX /var/xapi/xapi|session.login                                                                                                                                                             _with_password D:6baeffeb13e0|xapi] Session.create trackid=b043dd9f380b982f1e8aa                                                                                                                                                             32967a8de2b pool=false uname=root is_local_superuser=true auth_user_sid= parent=                                                                                                                                                             trackid=9834f5af41c964e225f24279aefe4e49
      Dec 27 13:04:55 xen2 xapi: [ info|xen2|1118852 INET 0.0.0.0:80|session.logout D:                                                                                                                                                             97bc80b93f75|xapi] Session.destroy trackid=53e2afc20c1a0ab616b48da0c7004030
      Dec 27 13:04:55 xen2 kernel: [17530.329388] sd 4:0:0:1: rdac: array MDS-Spindle0                                                                                                                                                             1, ctlr 0, MODE_SELECT completed
      Dec 27 13:04:55 xen2 xapi: [ info|xen2|1118884 UNIX /var/xapi/xapi|session.logou                                                                                                                                                             t D:15d77c89ee0a|xapi] Session.destroy trackid=b043dd9f380b982f1e8aa32967a8de2b
      Dec 27 13:04:56 xen2 kernel: [17530.841619] sd 4:0:0:1: [sdc] Unhandled sense co                                                                                                                                                             de
      Dec 27 13:04:56 xen2 kernel: [17530.841623] sd 4:0:0:1: [sdc] Result: hostbyte=D                                                                                                                                                             ID_OK driverbyte=DRIVER_SENSE
      Dec 27 13:04:56 xen2 kernel: [17530.841628] sd 4:0:0:1: [sdc] Sense Key : Hardwa                                                                                                                                                             re Error [current]
      Dec 27 13:04:56 xen2 kernel: [17530.841632] sd 4:0:0:1: [sdc] <<vendor>> ASC=0x8                                                                                                                                                             4 ASCQ=0x0ASC=0x84 ASCQ=0x0
      Dec 27 13:04:56 xen2 kernel: [17530.841638] sd 4:0:0:1: [sdc] CDB: Read(10): 28                                                                                                                                                              00 ae ff ff 80 00 00 08 00
      Dec 27 13:04:56 xen2 kernel: [17530.841648] end_request: I/O error, dev sdc, sec                                                                                                                                                             tor 2936012672
      Dec 27 13:04:56 xen2 kernel: [17530.841672] device-mapper: multipath: Failing pa                                                                                                                                                             th 8:32.
      Dec 27 13:04:56 xen2 multipathd: 360024e800070ed06000004314a967309: sdi - rdac c                                                                                                                                                             hecker reports path is up
      Dec 27 13:04:56 xen2 multipathd: Path event for 360024e800070ed06000004314a96730                                                                                                                                                             9, request call of mpathcount
      Dec 27 13:04:56 xen2 multipathd: 8:128: reinstated
      Dec 27 13:04:56 xen2 multipathd: 360024e800070ed06000004314a967309: remaining ac                                                                                                                                                             tive paths: 2
      Dec 27 13:04:56 xen2 xapi: [ info|xen2|1118889 UNIX /var/xapi/xapi|session.login                                                                                                                                                             _with_password D:f3f2ba7c9b76|xapi] Session.create trackid=1f938a40d5b26b9761de4                                                                                                                                                             6230d454ee5 pool=false uname=root is_local_superuser=true auth_user_sid= parent=                                                                                                                                                             trackid=9834f5af41c964e225f24279aefe4e49
      
      1 Reply Last reply Reply Quote 0
      • DashrenderD
        Dashrender
        last edited by

        There you go.. 🙂

        1 Reply Last reply Reply Quote 0
        • scottalanmillerS
          scottalanmiller @CitrixNewbJD
          last edited by

          @CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

          @scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

          @CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

          @scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

          @CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

          @scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

          We still need to see the logs 🙂

          [root@xen2 ~]# tail -n 100 messages
          tail: cannot open `messages' for reading: No such file or directory
          

          That's not good. That means that something really went wrong. Probably the disk had filled up.

          Is there another way to view log entries?

          Not if they are not there. That's not a log viewer, that's the logs itself. Are you use that you used the cd command to get into the right directory before trying to view the logs?

          I used no cd command. I just tried "cd /var/logs" and was told there was no directory.

          We are definitely seeing a problem here 🙂

          DashrenderD 1 Reply Last reply Reply Quote 1
          • DashrenderD
            Dashrender @scottalanmiller
            last edited by

            @scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

            @CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

            @scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

            @CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

            @scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

            @CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

            @scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

            We still need to see the logs 🙂

            [root@xen2 ~]# tail -n 100 messages
            tail: cannot open `messages' for reading: No such file or directory
            

            That's not good. That means that something really went wrong. Probably the disk had filled up.

            Is there another way to view log entries?

            Not if they are not there. That's not a log viewer, that's the logs itself. Are you use that you used the cd command to get into the right directory before trying to view the logs?

            I used no cd command. I just tried "cd /var/logs" and was told there was no directory.

            We are definitely seeing a problem here 🙂

            Scott, why is the formatting so messed up?

            scottalanmillerS 1 Reply Last reply Reply Quote 0
            • scottalanmillerS
              scottalanmiller @Dashrender
              last edited by

              @Dashrender said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

              @scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

              @CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

              @scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

              @CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

              @scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

              @CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

              @scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

              We still need to see the logs 🙂

              [root@xen2 ~]# tail -n 100 messages
              tail: cannot open `messages' for reading: No such file or directory
              

              That's not good. That means that something really went wrong. Probably the disk had filled up.

              Is there another way to view log entries?

              Not if they are not there. That's not a log viewer, that's the logs itself. Are you use that you used the cd command to get into the right directory before trying to view the logs?

              I used no cd command. I just tried "cd /var/logs" and was told there was no directory.

              We are definitely seeing a problem here 🙂

              Scott, why is the formatting so messed up?

              No idea, but it's not from ML, that's from his cut and paste. Those spaces are all in his original.

              1 Reply Last reply Reply Quote 0
              • C
                CitrixNewbJD
                last edited by

                Dec 27 13:13:16 xen2 last message repeated 2 times
                Dec 27 13:13:16 xen2 kernel: [18030.894547] device-mapper: multipath: Failing path 8:128.
                Dec 27 13:13:16 xen2 kernel: [18030.894934] device-mapper: multipath: Failing path 8:80.
                Dec 27 13:13:16 xen2 kernel: [18030.895002] device-mapper: multipath: Failing path 8:176.
                Dec 27 13:13:16 xen2 xapi: [ info|xen2|1151320 UNIX /var/xapi/xapi|session.logout D:9b0536d5f01c|xapi] Session.destroy trackid=e8dec3c23c8d667dfcfbfd772ecabd9e
                Dec 27 13:13:16 xen2 xapi: [ info|xen2|1151321 UNIX /var/xapi/xapi|session.login_with_password D:72d677979b15|xapi] Session.create trackid=2db03e1031b1b474ab4e3a61a73c4824                                                                   pool=false uname=root is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
                Dec 27 13:13:16 xen2 kernel: [18031.409957] sd 4:0:0:1: [sdc] Unhandled sense code
                Dec 27 13:13:16 xen2 kernel: [18031.409960] sd 4:0:0:1: [sdc] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
                Dec 27 13:13:16 xen2 kernel: [18031.409963] sd 4:0:0:1: [sdc] Sense Key : Hardware Error [current]
                Dec 27 13:13:16 xen2 kernel: [18031.409965] sd 4:0:0:1: [sdc] <<vendor>> ASC=0x84 ASCQ=0x0ASC=0x84 ASCQ=0x0
                Dec 27 13:13:16 xen2 kernel: [18031.409969] sd 4:0:0:1: [sdc] CDB: Read(10): 28 00 ae ff ff 80 00 00 08 00
                Dec 27 13:13:16 xen2 kernel: [18031.409974] end_request: I/O error, dev sdc, sector 2936012672
                Dec 27 13:13:16 xen2 kernel: [18031.409989] device-mapper: multipath: Failing path 8:32.
                Dec 27 13:13:16 xen2 xapi: [ info|xen2|1151359 INET 0.0.0.0:80|session.logout D:6b5f0e1950d3|xapi] Session.destroy trackid=95373089b3a53ad07d348ea483840e40
                Dec 27 13:13:16 xen2 multipathd: 8:32: mark as failed
                Dec 27 13:13:16 xen2 multipathd: 360024e800070ed06000004314a967309: Entering recovery mode: max_retries=15
                Dec 27 13:13:16 xen2 multipathd: 360024e800070ed06000004314a967309: remaining active paths: 0
                Dec 27 13:13:16 xen2 multipathd: Path event for 360024e800070ed06000004314a967309, request call of mpathcount
                Dec 27 13:13:16 xen2 xapi: [ info|xen2|1151365 UNIX /var/xapi/xapi|session.logout D:7808df6ecf1f|xapi] Session.destroy trackid=2db03e1031b1b474ab4e3a61a73c4824
                Dec 27 13:13:16 xen2 xapi: [ info|xen2|1151366 UNIX /var/xapi/xapi|session.login_with_password D:107e4867f954|xapi] Session.create trackid=398819a8505258b63db68cd0d33af4b1                                                                   pool=false uname=root is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
                Dec 27 13:13:17 xen2 multipathd: 360024e800070ed06000004314a967309: sdi - rdac checker reports path is up
                Dec 27 13:13:17 xen2 multipathd: Path event for 360024e800070ed06000004314a967309, request call of mpathcount
                Dec 27 13:13:17 xen2 multipathd: 8:128: reinstated
                Dec 27 13:13:17 xen2 multipathd: 360024e800070ed06000004314a967309: queue_if_no_path enabled
                Dec 27 13:13:17 xen2 multipathd: 360024e800070ed06000004314a967309: Recovered to normal mode
                Dec 27 13:13:17 xen2 multipathd: 360024e800070ed06000004314a967309: remaining active paths: 1
                Dec 27 13:13:17 xen2 xapi: [ info|xen2|1151408 UNIX /var/xapi/xapi|session.logout D:96b0ad24c6c1|xapi] Session.destroy trackid=398819a8505258b63db68cd0d33af4b1
                Dec 27 13:13:17 xen2 xapi: [ info|xen2|1151409 UNIX /var/xapi/xapi|session.login_with_password D:f87fb7e4fce8|xapi] Session.create trackid=a797b1b90698608ddd5f6919804b5484                                                                   pool=false uname=root is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
                Dec 27 13:13:17 xen2 kernel: [18032.440371] sd 6:0:0:1: [sdi] Unhandled sense code
                Dec 27 13:13:17 xen2 kernel: [18032.440374] sd 6:0:0:1: [sdi] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
                Dec 27 13:13:17 xen2 kernel: [18032.440376] sd 6:0:0:1: [sdi] Sense Key : Hardware Error [current]
                Dec 27 13:13:17 xen2 kernel: [18032.440379] sd 6:0:0:1: [sdi] <<vendor>> ASC=0x84 ASCQ=0x0ASC=0x84 ASCQ=0x0
                Dec 27 13:13:17 xen2 kernel: [18032.440382] sd 6:0:0:1: [sdi] CDB: Read(10): 28 00 ae ff ff 80 00 00 08 00
                Dec 27 13:13:17 xen2 kernel: [18032.440387] end_request: I/O error, dev sdi, sector 2936012672
                Dec 27 13:13:17 xen2 kernel: [18032.440404] device-mapper: multipath: Failing path 8:128.
                Dec 27 13:13:17 xen2 multipathd: 8:128: mark as failed
                Dec 27 13:13:17 xen2 multipathd: 360024e800070ed06000004314a967309: Entering recovery mode: max_retries=15
                Dec 27 13:13:17 xen2 multipathd: 360024e800070ed06000004314a967309: remaining active paths: 0
                Dec 27 13:13:17 xen2 multipathd: Path event for 360024e800070ed06000004314a967309, request call of mpathcount
                Dec 27 13:13:17 xen2 xapi: [ info|xen2|1151447 UNIX /var/xapi/xapi|session.logout D:acca552975a2|xapi] Session.destroy trackid=a797b1b90698608ddd5f6919804b5484
                Dec 27 13:13:17 xen2 xapi: [ info|xen2|1151448 UNIX /var/xapi/xapi|session.login_with_password D:5f3e3fccf3d5|xapi] Session.create trackid=53cb15330f2848d21867cbf051db9ed9                                                                   pool=false uname=root is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
                Dec 27 13:13:18 xen2 multipathd: 360024e800070ed06000004314a967309: sdl - rdac checker reports path is ghost
                Dec 27 13:13:18 xen2 multipathd: Path event for 360024e800070ed06000004314a967309, request call of mpathcount
                Dec 27 13:13:18 xen2 multipathd: 8:176: reinstated
                Dec 27 13:13:18 xen2 multipathd: 360024e800070ed06000004314a967309: queue_if_no_path enabled
                Dec 27 13:13:18 xen2 multipathd: 360024e800070ed06000004314a967309: Recovered to normal mode
                Dec 27 13:13:18 xen2 multipathd: 360024e800070ed06000004314a967309: remaining active paths: 1
                Dec 27 13:13:18 xen2 multipathd: 360024e800070ed06000004314a967309: switch to path group #4
                Dec 27 13:13:18 xen2 kernel: [18033.217827] sd 7:0:0:1: rdac: array MDS-Spindle01, ctlr 1, queueing MODE_SELECT command
                Dec 27 13:13:18 xen2 xapi: [ info|xen2|1151482 UNIX /var/xapi/xapi|session.logout D:39dea3d23bb2|xapi] Session.destroy trackid=53cb15330f2848d21867cbf051db9ed9
                Dec 27 13:13:18 xen2 xapi: [ info|xen2|1151483 UNIX /var/xapi/xapi|session.login_with_password D:487d0c440a6f|xapi] Session.create trackid=31c6550a7ad5c485e04e1ce80f3588dd                                                                   pool=false uname=root is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
                Dec 27 13:13:18 xen2 kernel: [18033.500501] sd 7:0:0:1: rdac: array MDS-Spindle01, ctlr 1, MODE_SELECT completed
                Dec 27 13:13:19 xen2 xapi: [ info|xen2|1151517 UNIX /var/xapi/xapi|session.logout D:6b32233bc36f|xapi] Session.destroy trackid=31c6550a7ad5c485e04e1ce80f3588dd
                Dec 27 13:13:19 xen2 kernel: [18034.028373] sd 7:0:0:1: [sdl] Unhandled sense code
                Dec 27 13:13:19 xen2 kernel: [18034.028376] sd 7:0:0:1: [sdl] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
                Dec 27 13:13:19 xen2 kernel: [18034.028379] sd 7:0:0:1: [sdl] Sense Key : Hardware Error [current]
                Dec 27 13:13:19 xen2 kernel: [18034.028381] sd 7:0:0:1: [sdl] <<vendor>> ASC=0x84 ASCQ=0x0ASC=0x84 ASCQ=0x0
                Dec 27 13:13:19 xen2 kernel: [18034.028385] sd 7:0:0:1: [sdl] CDB: Read(10): 28 00 ae ff ff 80 00 00 08 00
                Dec 27 13:13:19 xen2 kernel: [18034.028390] end_request: I/O error, dev sdl, sector 2936012672
                Dec 27 13:13:19 xen2 kernel: [18034.028406] device-mapper: multipath: Failing path 8:176.
                Dec 27 13:13:19 xen2 xapi: [ info|xen2|1151537 INET 0.0.0.0:80|session.logout D:7e74bbbad814|xapi] Session.destroy trackid=57a035032e821d9289ab2ddc9c56ef23
                Dec 27 13:13:19 xen2 multipathd: 8:176: mark as failed
                Dec 27 13:13:19 xen2 multipathd: 360024e800070ed06000004314a967309: Entering recovery mode: max_retries=15
                Dec 27 13:13:19 xen2 multipathd: 360024e800070ed06000004314a967309: remaining active paths: 0
                Dec 27 13:13:19 xen2 multipathd: Path event for 360024e800070ed06000004314a967309, request call of mpathcount
                Dec 27 13:13:19 xen2 xapi: [ info|xen2|1151540 UNIX /var/xapi/xapi|session.login_with_password D:d750b0705d7f|xapi] Session.create trackid=86b83edaef195351a3b0fab7cc6cc21e                                                                   pool=false uname=root is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
                Dec 27 13:13:20 xen2 xapi: [ info|xen2|1151585 UNIX /var/xapi/xapi|session.logout D:3e57cedfef7a|xapi] Session.destroy trackid=86b83edaef195351a3b0fab7cc6cc21e
                Dec 27 13:13:20 xen2 multipathd: 360024e800070ed06000004314a967309: sdf - rdac checker reports path is up
                Dec 27 13:13:20 xen2 multipathd: Path event for 360024e800070ed06000004314a967309, request call of mpathcount
                Dec 27 13:13:20 xen2 multipathd: 8:80: reinstated
                Dec 27 13:13:20 xen2 multipathd: 360024e800070ed06000004314a967309: queue_if_no_path enabled
                Dec 27 13:13:20 xen2 multipathd: 360024e800070ed06000004314a967309: Recovered to normal mode
                Dec 27 13:13:20 xen2 multipathd: 360024e800070ed06000004314a967309: remaining active paths: 1
                Dec 27 13:13:20 xen2 multipathd: 360024e800070ed06000004314a967309: switch to path group #3
                Dec 27 13:13:20 xen2 xapi: [ info|xen2|1151605 UNIX /var/xapi/xapi|session.login_with_password D:1379b8d2955b|xapi] Session.create trackid=3fa7db291a48d0f98ea6e364b1dccd5b                                                                   pool=false uname=root is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
                Dec 27 13:13:20 xen2 xapi: [ info|xen2|1151611 INET 0.0.0.0:80|session.logout D:e6afd1da2cf0|xapi] Session.destroy trackid=a246f22117a6ddf72bd492a7e8505ddd
                Dec 27 13:13:21 xen2 kernel: [18035.743234] sd 5:0:0:1: [sdf] Unhandled sense code
                Dec 27 13:13:21 xen2 kernel: [18035.743237] sd 5:0:0:1: [sdf] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
                Dec 27 13:13:21 xen2 kernel: [18035.743240] sd 5:0:0:1: [sdf] Sense Key : Hardware Error [current]
                Dec 27 13:13:21 xen2 kernel: [18035.743242] sd 5:0:0:1: [sdf] <<vendor>> ASC=0x84 ASCQ=0x0ASC=0x84 ASCQ=0x0
                Dec 27 13:13:21 xen2 kernel: [18035.743246] sd 5:0:0:1: [sdf] CDB: Read(10): 28 00 ae ff ff 80 00 00 08 00
                Dec 27 13:13:21 xen2 kernel: [18035.743251] end_request: I/O error, dev sdf, sector 2936012672
                Dec 27 13:13:21 xen2 kernel: [18035.743266] device-mapper: multipath: Failing path 8:80.
                Dec 27 13:13:21 xen2 multipathd: 8:80: mark as failed
                Dec 27 13:13:21 xen2 multipathd: 360024e800070ed06000004314a967309: Entering recovery mode: max_retries=15
                Dec 27 13:13:21 xen2 multipathd: 360024e800070ed06000004314a967309: remaining active paths: 0
                Dec 27 13:13:21 xen2 multipathd: Path event for 360024e800070ed06000004314a967309, request call of mpathcount
                Dec 27 13:13:21 xen2 xapi: [ info|xen2|1151653 UNIX /var/xapi/xapi|session.logout D:57acea842a91|xapi] Session.destroy trackid=3fa7db291a48d0f98ea6e364b1dccd5b
                Dec 27 13:13:21 xen2 xapi: [ info|xen2|1151654 UNIX /var/xapi/xapi|session.login_with_password D:e547785afb43|xapi] Session.create trackid=8fd799a70bdc34fb2a4d8bd676873ede                                                                   pool=false uname=root is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
                Dec 27 13:13:21 xen2 multipathd: 360024e800070ed06000004314a967309: sdc - rdac checker reports path is ghost
                Dec 27 13:13:21 xen2 multipathd: Path event for 360024e800070ed06000004314a967309, request call of mpathcount
                Dec 27 13:13:21 xen2 multipathd: 8:32: reinstated
                Dec 27 13:13:21 xen2 multipathd: 360024e800070ed06000004314a967309: queue_if_no_path enabled
                Dec 27 13:13:21 xen2 multipathd: 360024e800070ed06000004314a967309: Recovered to normal mode
                Dec 27 13:13:21 xen2 multipathd: 360024e800070ed06000004314a967309: remaining active paths: 1
                Dec 27 13:13:21 xen2 multipathd: 360024e800070ed06000004314a967309: switch to path group #2
                Dec 27 13:13:21 xen2 kernel: [18036.256313] sd 4:0:0:1: rdac: array MDS-Spindle01, ctlr 0, queueing MODE_SELECT command
                Dec 27 13:13:21 xen2 kernel: [18036.525838] sd 4:0:0:1: rdac: array MDS-Spindle01, ctlr 0, MODE_SELECT completed
                Dec 27 13:13:22 xen2 xapi: [ info|xen2|1151678 UNIX /var/xapi/xapi|session.logout D:34466f735dad|xapi] Session.destroy trackid=8fd799a70bdc34fb2a4d8bd676873ede
                Dec 27 13:13:22 xen2 xapi: [ info|xen2|1151679 UNIX /var/xapi/xapi|session.login_with_password D:cd7b037c4106|xapi] Session.create trackid=f258157e7fefe65077464bbd3c912521                                                                   pool=false uname=root is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
                
                1 Reply Last reply Reply Quote 0
                • C
                  CitrixNewbJD
                  last edited by

                  Better?

                  scottalanmillerS 1 Reply Last reply Reply Quote 0
                  • scottalanmillerS
                    scottalanmiller @CitrixNewbJD
                    last edited by

                    @CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

                    Better?

                    Yup, thanks.

                    1 Reply Last reply Reply Quote 0
                    • scottalanmillerS
                      scottalanmiller
                      last edited by

                      We need to see the Xen logs, too. Those are the logs with "xen" in the name.

                      C 1 Reply Last reply Reply Quote 0
                      • C
                        CitrixNewbJD @scottalanmiller
                        last edited by

                        @scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

                        We need to see the Xen logs, too. Those are the logs with "xen" in the name.
                        I can't upload the .gz to here. Can I open the contents in wordpad? Sorry if I missed something.

                        0_1482866277151_Screenshot 2016-12-27 13.17.43.png

                        C 1 Reply Last reply Reply Quote 0
                        • scottalanmillerS
                          scottalanmiller
                          last edited by

                          No current logs at all?

                          C 1 Reply Last reply Reply Quote 0
                          • C
                            CitrixNewbJD @CitrixNewbJD
                            last edited by

                            @CitrixNewbJD

                            It ate my text. How do I display the logs to you? Sorry. A lot of new knowledge in a short period of time, combined with stress.

                            1 Reply Last reply Reply Quote 0
                            • C
                              CitrixNewbJD @scottalanmiller
                              last edited by

                              @scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

                              No current logs at all?

                              0_1482866370090_Screenshot 2016-12-27 13.19.21.png

                              C 1 Reply Last reply Reply Quote 0
                              • C
                                CitrixNewbJD @CitrixNewbJD
                                last edited by

                                @CitrixNewbJD

                                Sorry, ate my text again. Unless I should be looking somewhere else...

                                scottalanmillerS 1 Reply Last reply Reply Quote 0
                                • C
                                  CitrixNewbJD
                                  last edited by

                                  Is there any way to tell why the hosts can't even attach to their own local storage, or why it was unplugged in the first place?

                                  1 Reply Last reply Reply Quote 0
                                  • scottalanmillerS
                                    scottalanmiller @CitrixNewbJD
                                    last edited by

                                    @CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

                                    @CitrixNewbJD

                                    Sorry, ate my text again. Unless I should be looking somewhere else...

                                    Get us the SMlog tail...

                                    tail -n 100 SMlog
                                    
                                    C 1 Reply Last reply Reply Quote 0
                                    • scottalanmillerS
                                      scottalanmiller
                                      last edited by

                                      Then let's see what is in the xen folder, too.

                                      1 Reply Last reply Reply Quote 0
                                      • C
                                        CitrixNewbJD @scottalanmiller
                                        last edited by

                                        @scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

                                        @CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

                                        @CitrixNewbJD

                                        Sorry, ate my text again. Unless I should be looking somewhere else...

                                        Get us the SMlog tail...

                                        tail -n 100 SMlog
                                        
                                        Dec 27 13:24:41 xen2 SM: [6456]   |- 5:0:0:1 sdf 8:80  failed faulty running
                                        Dec 27 13:24:41 xen2 SM: [6456]   `- 7:0:0:1 sdl 8:176 active ready  running
                                        Dec 27 13:24:41 xen2 SM: [6456] multipathd>
                                        Dec 27 13:24:41 xen2 SM: [6456] MPATH:  Set val: [1, 4]
                                        Dec 27 13:24:41 xen2 SM: [6456] Matched SCSIid, updating 360024e800070ed06000007f54dba7bfb
                                        Dec 27 13:24:41 xen2 SM: [6456] MPATH: Updating entry for [360024e800070ed06000007f54dba7bfb], current: [0, 4]
                                        Dec 27 13:24:41 xen2 SM: [6456] MPATH: Update done
                                        Dec 27 13:24:41 xen2 SM: [6456] lock: closed /var/lock/sm/mpathcount2/host
                                        Dec 27 13:24:41 xen2 SM: [6456] lock: released /var/lock/sm/mpathcount1/host
                                        Dec 27 13:24:41 xen2 SM: [6456] lock: closed /var/lock/sm/mpathcount1/host
                                        Dec 27 13:24:41 xen2 SM: [6579] lock: acquired /var/lock/sm/mpathcount1/host
                                        Dec 27 13:24:41 xen2 SM: [6579] lock: released /var/lock/sm/mpathcount2/host
                                        Dec 27 13:24:41 xen2 SM: [6579] MPATH: I get the lock
                                        Dec 27 13:24:41 xen2 SM: [6579] Matched SCSIid, updating 360024e80007b786a000004134a9670f3
                                        Dec 27 13:24:41 xen2 SM: [6579] MPATH: Updating entry for [360024e80007b786a000004134a9670f3], current: [2, 4]
                                        Dec 27 13:24:41 xen2 SM: [6579] Matched SCSIid, updating 360024e800054baef0000ddd550c85082
                                        Dec 27 13:24:41 xen2 SM: [6579] MPATH: Updating entry for [360024e800054baef0000ddd550c85082], current: [4, 4]
                                        Dec 27 13:24:41 xen2 SM: [6579] Matched SCSIid, updating 360024e800070ed06000004314a967309
                                        Dec 27 13:24:41 xen2 SM: [6579] MPATH: Updating entry for [360024e800070ed06000004314a967309], current: [1, 4]
                                        Dec 27 13:24:41 xen2 SM: [6579] mpath cmd: show map 360024e800070ed06000004314a967309 topology
                                        Dec 27 13:24:41 xen2 SM: [6663] MPATH: Trying to acquire the lock
                                        Dec 27 13:24:41 xen2 SM: [6663] lock: tried lock /var/lock/sm/mpathcount1/host, acquired: False (exists: True)
                                        Dec 27 13:24:41 xen2 SM: [6663] lock: tried lock /var/lock/sm/mpathcount2/host, acquired: True (exists: True)
                                        Dec 27 13:24:41 xen2 SM: [6663] Failed to lock /var/lock/sm/mpathcount1/host on first attempt, blocked by PID 6579
                                        Dec 27 13:24:41 xen2 SM: [6666] MPATH: Trying to acquire the lock
                                        Dec 27 13:24:41 xen2 SM: [6666] lock: tried lock /var/lock/sm/mpathcount1/host, acquired: False (exists: True)
                                        Dec 27 13:24:41 xen2 SM: [6666] lock: tried lock /var/lock/sm/mpathcount2/host, acquired: False (exists: True)
                                        Dec 27 13:24:41 xen2 SM: [6666] lock: closed /var/lock/sm/mpathcount2/host
                                        Dec 27 13:24:41 xen2 SM: [6666] lock: closed /var/lock/sm/mpathcount1/host
                                        Dec 27 13:24:41 xen2 SM: [6579] mpath output: multipathd> 360024e800070ed06000004314a967309 dm-0 DELL,MD3000i
                                        Dec 27 13:24:41 xen2 SM: [6579] size=1.4T features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 rdac' wp=rw
                                        Dec 27 13:24:41 xen2 SM: [6579] |-+- policy='round-robin 0' prio=4 status=active
                                        Dec 27 13:24:41 xen2 SM: [6579] | |- 6:0:0:1 sdi 8:128 active ghost  running
                                        Dec 27 13:24:41 xen2 SM: [6579] | `- 4:0:0:1 sdc 8:32  failed faulty running
                                        Dec 27 13:24:41 xen2 SM: [6579] `-+- policy='round-robin 0' prio=3 status=enabled
                                        Dec 27 13:24:41 xen2 SM: [6579]   |- 5:0:0:1 sdf 8:80  failed faulty running
                                        Dec 27 13:24:41 xen2 SM: [6579]   `- 7:0:0:1 sdl 8:176 active ready  running
                                        Dec 27 13:24:41 xen2 SM: [6579] multipathd>
                                        Dec 27 13:24:41 xen2 SM: [6579] mpath cmd: show map 360024e800070ed06000004314a967309 topology
                                        Dec 27 13:24:41 xen2 SM: [6579] mpath output: multipathd> 360024e800070ed06000004314a967309 dm-0 DELL,MD3000i
                                        Dec 27 13:24:41 xen2 SM: [6579] size=1.4T features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 rdac' wp=rw
                                        Dec 27 13:24:41 xen2 SM: [6579] |-+- policy='round-robin 0' prio=4 status=active
                                        Dec 27 13:24:41 xen2 SM: [6579] | |- 6:0:0:1 sdi 8:128 active ghost  running
                                        Dec 27 13:24:41 xen2 SM: [6579] | `- 4:0:0:1 sdc 8:32  failed faulty running
                                        Dec 27 13:24:41 xen2 SM: [6579] `-+- policy='round-robin 0' prio=3 status=enabled
                                        Dec 27 13:24:41 xen2 SM: [6579]   |- 5:0:0:1 sdf 8:80  failed faulty running
                                        Dec 27 13:24:41 xen2 SM: [6579]   `- 7:0:0:1 sdl 8:176 active ready  running
                                        Dec 27 13:24:41 xen2 SM: [6579] multipathd>
                                        Dec 27 13:24:41 xen2 SM: [6579] MPATH:  Set val: [2, 4]
                                        Dec 27 13:24:41 xen2 SM: [6579] Matched SCSIid, updating 360024e800070ed06000007f54dba7bfb
                                        Dec 27 13:24:41 xen2 SM: [6579] MPATH: Updating entry for [360024e800070ed06000007f54dba7bfb], current: [0, 4]
                                        Dec 27 13:24:41 xen2 SM: [6579] MPATH: Update done
                                        Dec 27 13:24:41 xen2 SM: [6579] lock: closed /var/lock/sm/mpathcount2/host
                                        Dec 27 13:24:41 xen2 SM: [6579] lock: released /var/lock/sm/mpathcount1/host
                                        Dec 27 13:24:41 xen2 SM: [6579] lock: closed /var/lock/sm/mpathcount1/host
                                        Dec 27 13:24:41 xen2 SM: [6663] lock: acquired /var/lock/sm/mpathcount1/host
                                        Dec 27 13:24:41 xen2 SM: [6663] lock: released /var/lock/sm/mpathcount2/host
                                        Dec 27 13:24:41 xen2 SM: [6663] MPATH: I get the lock
                                        Dec 27 13:24:41 xen2 SM: [6663] Matched SCSIid, updating 360024e80007b786a000004134a9670f3
                                        Dec 27 13:24:41 xen2 SM: [6663] MPATH: Updating entry for [360024e80007b786a000004134a9670f3], current: [2, 4]
                                        Dec 27 13:24:41 xen2 SM: [6663] Matched SCSIid, updating 360024e800054baef0000ddd550c85082
                                        Dec 27 13:24:41 xen2 SM: [6663] MPATH: Updating entry for [360024e800054baef0000ddd550c85082], current: [4, 4]
                                        Dec 27 13:24:41 xen2 SM: [6663] Matched SCSIid, updating 360024e800070ed06000004314a967309
                                        Dec 27 13:24:41 xen2 SM: [6663] MPATH: Updating entry for [360024e800070ed06000004314a967309], current: [2, 4]
                                        Dec 27 13:24:41 xen2 SM: [6663] mpath cmd: show map 360024e800070ed06000004314a967309 topology
                                        Dec 27 13:24:41 xen2 SM: [6663] mpath output: multipathd> 360024e800070ed06000004314a967309 dm-0 DELL,MD3000i
                                        Dec 27 13:24:41 xen2 SM: [6663] size=1.4T features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 rdac' wp=rw
                                        Dec 27 13:24:41 xen2 SM: [6663] |-+- policy='round-robin 0' prio=4 status=active
                                        Dec 27 13:24:41 xen2 SM: [6663] | |- 6:0:0:1 sdi 8:128 active ghost  running
                                        Dec 27 13:24:41 xen2 SM: [6663] | `- 4:0:0:1 sdc 8:32  failed faulty running
                                        Dec 27 13:24:41 xen2 SM: [6663] `-+- policy='round-robin 0' prio=3 status=enabled
                                        Dec 27 13:24:41 xen2 SM: [6663]   |- 5:0:0:1 sdf 8:80  failed faulty running
                                        Dec 27 13:24:41 xen2 SM: [6663]   `- 7:0:0:1 sdl 8:176 active ready  running
                                        Dec 27 13:24:41 xen2 SM: [6663] multipathd>
                                        Dec 27 13:24:41 xen2 SM: [6663] mpath cmd: show map 360024e800070ed06000004314a967309 topology
                                        Dec 27 13:24:42 xen2 SM: [6663] mpath output: multipathd> 360024e800070ed06000004314a967309 dm-0 DELL,MD3000i
                                        Dec 27 13:24:42 xen2 SM: [6663] size=1.4T features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 rdac' wp=rw
                                        Dec 27 13:24:42 xen2 SM: [6663] |-+- policy='round-robin 0' prio=4 status=active
                                        Dec 27 13:24:42 xen2 SM: [6663] | |- 6:0:0:1 sdi 8:128 active ghost  running
                                        Dec 27 13:24:42 xen2 SM: [6663] | `- 4:0:0:1 sdc 8:32  failed faulty running
                                        Dec 27 13:24:42 xen2 SM: [6663] `-+- policy='round-robin 0' prio=3 status=enabled
                                        Dec 27 13:24:42 xen2 SM: [6663]   |- 5:0:0:1 sdf 8:80  failed faulty running
                                        Dec 27 13:24:42 xen2 SM: [6663]   `- 7:0:0:1 sdl 8:176 active ready  running
                                        Dec 27 13:24:42 xen2 SM: [6663] multipathd>
                                        Dec 27 13:24:42 xen2 SM: [6663] Matched SCSIid, updating 360024e800070ed06000007f54dba7bfb
                                        Dec 27 13:24:42 xen2 SM: [6663] MPATH: Updating entry for [360024e800070ed06000007f54dba7bfb], current: [0, 4]
                                        Dec 27 13:24:42 xen2 SM: [6663] MPATH: Update done
                                        Dec 27 13:24:42 xen2 SM: [6663] lock: closed /var/lock/sm/mpathcount2/host
                                        Dec 27 13:24:42 xen2 SM: [6663] lock: released /var/lock/sm/mpathcount1/host
                                        Dec 27 13:24:42 xen2 SM: [6663] lock: closed /var/lock/sm/mpathcount1/host
                                        Dec 27 13:24:42 xen2 SM: [6728] MPATH: Trying to acquire the lock
                                        Dec 27 13:24:42 xen2 SM: [6728] lock: tried lock /var/lock/sm/mpathcount1/host, acquired: True (exists: True)
                                        Dec 27 13:24:42 xen2 SM: [6728] MPATH: I get the lock
                                        Dec 27 13:24:42 xen2 SM: [6728] Matched SCSIid, updating 360024e80007b786a000004134a9670f3
                                        Dec 27 13:24:42 xen2 SM: [6728] MPATH: Updating entry for [360024e80007b786a000004134a9670f3], current: [2, 4]
                                        Dec 27 13:24:42 xen2 SM: [6728] Matched SCSIid, updating 360024e800054baef0000ddd550c85082
                                        Dec 27 13:24:42 xen2 SM: [6728] MPATH: Updating entry for [360024e800054baef0000ddd550c85082], current: [4, 4]
                                        Dec 27 13:24:42 xen2 SM: [6728] Matched SCSIid, updating 360024e800070ed06000004314a967309
                                        Dec 27 13:24:42 xen2 SM: [6728] MPATH: Updating entry for [360024e800070ed06000004314a967309], current: [2, 4]
                                        Dec 27 13:24:42 xen2 SM: [6728] mpath cmd: show map 360024e800070ed06000004314a967309 topology
                                        
                                        1 Reply Last reply Reply Quote 0
                                        • C
                                          CitrixNewbJD
                                          last edited by

                                          0_1482866750909_Screenshot 2016-12-27 13.25.40.png

                                          scottalanmillerS 1 Reply Last reply Reply Quote 0
                                          • scottalanmillerS
                                            scottalanmiller @CitrixNewbJD
                                            last edited by

                                            @CitrixNewbJD okay, no need to look in the xen folder, that's not going to be a useful log 🙂

                                            1 Reply Last reply Reply Quote 0
                                            • 1
                                            • 2
                                            • 3
                                            • 4
                                            • 5
                                            • 6
                                            • 7
                                            • 8
                                            • 12
                                            • 13
                                            • 6 / 13
                                            • First post
                                              Last post