ML
    • Recent
    • Categories
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!

    IT Discussion
    xenserver xenserver 6.2 iscsi san
    10
    243
    46.0k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      momurda @CitrixNewbJD
      last edited by

      @CitrixNewbJD
      No i dont think so.
      You got some work to do though
      Probably need to do:
      log into the san and change the creds for iscsi, disable ad authentication in xc, and then probably forget/reattach the sr

      S 1 Reply Last reply Reply Quote 0
      • N
        NerdyDad
        last edited by

        I always try to make sure that I have an administrative local user to every system, just in case I cannot get to AD.

        S 1 Reply Last reply Reply Quote 0
        • S
          scottalanmiller @momurda
          last edited by

          @momurda said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

          @CitrixNewbJD
          No i dont think so.
          You got some work to do though
          Probably need to do:
          log into the san and change the creds for iscsi, disable ad authentication in xc, and then probably forget/reattach the sr

          I didn't see it on the iSCSI, darn it. Still fixable, but more work.

          1 Reply Last reply Reply Quote 0
          • S
            scottalanmiller @NerdyDad
            last edited by

            @NerdyDad said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

            I always try to make sure that I have an administrative local user to every system, just in case I cannot get to AD.

            That, and never have AD on top of something that depends on it. That's like locking your keys in your car.

            N 1 Reply Last reply Reply Quote 3
            • N
              NerdyDad @scottalanmiller
              last edited by

              @scottalanmiller said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

              @NerdyDad said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

              I always try to make sure that I have an administrative local user to every system, just in case I cannot get to AD.

              That, and never have AD on top of something that depends on it. That's like locking your keys in your car.

              Yup. Our SAN is isolated from the rest of the house network physically. All authentication between the SAN and the host should never even touch AD just in case of things like this.

              Sometimes playing what-if is a good exercise to keep bad things from happening as far as security, reliability, and recovery.

              1 Reply Last reply Reply Quote 1
              • M
                momurda @scottalanmiller
                last edited by

                @scottalanmiller
                Yes the cifs repos can be forgotten for now. I doubt there is anything important there in the iso repo.
                @CitrixNewbJD
                Before you do anything else, make sure you have the root pw for your xs servers as disabling AD integration will force you to use them.

                M C 2 Replies Last reply Reply Quote 3
                • M
                  momurda @momurda
                  last edited by

                  Unless you have a DC that you can get working, ie one not attached to your shared storage. Perhaps you have a physical dc somewhere? Unlikely i know.

                  C 1 Reply Last reply Reply Quote 1
                  • S
                    seal
                    last edited by

                    @CitrixNewbJD This is Frank. I hope Scott and the gang here can help get you back operational quickly. To touch on a topic that was mentioned earlier, VMs from Xen can be imported and converted on the fly. As long as the Scale nodes can browse to the VM files (.vhd) on the storage, the XenServer functional state doesn't matter.

                    C 1 Reply Last reply Reply Quote 1
                    • C
                      CitrixNewbJD @momurda
                      last edited by

                      @momurda

                      Having been through this once before, and learning the hard way, I do normally have a physical DC. Despite my warnings, because I know that we do not currently have one here, I was told to bring it all down. And here we are. We do not have a physical DC.

                      S 1 Reply Last reply Reply Quote 0
                      • C
                        CitrixNewbJD @momurda
                        last edited by

                        @momurda

                        I've been using the root authentication for everything.

                        S 1 Reply Last reply Reply Quote 0
                        • C
                          CitrixNewbJD @seal
                          last edited by

                          @seal

                          Frank, can we speak on the phone for a minute so I can be sure I can intelligently talk to the guy when demanding his money?

                          1 Reply Last reply Reply Quote 0
                          • S
                            scottalanmiller @CitrixNewbJD
                            last edited by

                            @CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

                            @momurda

                            Having been through this once before, and learning the hard way, I do normally have a physical DC.

                            This is absolutely the wrong response. You should never have a physical DC, ever. There is zero issues here with virtualization. There are two problems....

                            • Zero AD redundancy
                            • An inverted pyramid of doom (single storage for all systems)

                            Fixing either of those anti-practices would have saved you. Physical would have zero benefit and is the polar opposite of the reaction that you should have.

                            D 1 Reply Last reply Reply Quote 1
                            • S
                              scottalanmiller @CitrixNewbJD
                              last edited by

                              @CitrixNewbJD said in XenServer 6.2 servers down. I have no Xen skill. Most likely networking? Help!:

                              @momurda

                              I've been using the root authentication for everything.

                              So we are safe there.

                              1 Reply Last reply Reply Quote 0
                              • S
                                scottalanmiller
                                last edited by

                                More on the IPOD: http://www.smbitjournal.com/2013/06/the-inverted-pyramid-of-doom/

                                And in video form from MangoCon:

                                Youtube Video

                                1 Reply Last reply Reply Quote 1
                                • C
                                  CitrixNewbJD
                                  last edited by

                                  So, when looking for places to turn off AD integration, I see this...

                                  0_1482873847039_Screenshot 2016-12-27 15.23.59.png

                                  1 Reply Last reply Reply Quote 0
                                  • S
                                    scottalanmiller
                                    last edited by

                                    It's not pool integration that is the issue, it's SAN integration. Check the SAN (PowerVault) interface instead.

                                    1 Reply Last reply Reply Quote 2
                                    • C
                                      CitrixNewbJD
                                      last edited by

                                      @seal Just came across these two items on the SAN interface. Dental_Data, Spindlemedia, are critical and it looks like those VDs failed.

                                      PROFILE FOR STORAGE ARRAY: MDS-Spindle01 (12/27/16 3:28:58 PM) 
                                       
                                       
                                      STANDARD VIRTUAL DISKS------------------------------ 
                                       
                                      SUMMARY 
                                       
                                         Number of standard virtual disks: 3 
                                       
                                         See other Virtual Disks sub-tabs for premium feature information. 
                                       
                                       
                                         NAME          STATUS  CAPACITY  RAID LEVEL  DISK GROUP  DRIVE TYPE   
                                         Dental_Data   Failed  1.495 TB  5           0           SAS          
                                         SpindleMedia  Failed  2.862 TB  5           0           SAS          
                                         Virtual       Failed  1.367 TB  5           0           SAS          
                                       
                                      DETAILS 
                                       
                                       
                                         Virtual Disk name:                      Dental_Data                                       
                                                                                                                                   
                                            Virtual Disk status:                 Failed                                            
                                                                                                                                   
                                            Capacity:                            1.495 TB                                          
                                            Virtual Disk world-wide identifier:  60:02:4e:80:00:7b:78:6a:00:00:04:13:4a:96:70:f3   
                                            Subsystem ID (SSID):                 1                                                 
                                            Associated disk group:               0                                                 
                                            RAID level:                          5                                                 
                                                                                                                                   
                                            Physical Disk type:                  Serial Attached SCSI (SAS)                        
                                            Enclosure loss protection:           No                                                
                                                                                                                                   
                                            Preferred owner:                     RAID Controller Module in slot 1                  
                                            Current owner:                       RAID Controller Module in slot 1                  
                                       
                                       
                                            Segment size:                                       128 KB     
                                            Capacity reserved for future segment size changes:  Yes        
                                            Maximum future segment size:                        2,048 KB   
                                            Modification priority:                              High       
                                       
                                       
                                            Read cache:                            Enabled    
                                            Write cache:                           Enabled    
                                               Write cache without batteries:      Disabled   
                                               Write cache with mirroring:         Enabled    
                                            Flush write cache after (in seconds):  10.00      
                                            Dynamic cache read prefetch:           Enabled    
                                                                                              
                                            Enable background media scan:          Enabled    
                                            Media scan with consistency check:     Enabled    
                                                                                              
                                            Pre-Read consistency check:            Disabled   
                                       
                                       
                                         Virtual Disk name:                      SpindleMedia                                      
                                                                                                                                   
                                            Virtual Disk status:                 Failed                                            
                                                                                                                                   
                                            Capacity:                            2.862 TB                                          
                                            Virtual Disk world-wide identifier:  60:02:4e:80:00:70:ed:06:00:00:07:f5:4d:ba:7b:fb   
                                            Subsystem ID (SSID):                 2                                                 
                                            Associated disk group:               0                                                 
                                            RAID level:                          5                                                 
                                                                                                                                   
                                            Physical Disk type:                  Serial Attached SCSI (SAS)                        
                                            Enclosure loss protection:           No                                                
                                                                                                                                   
                                            Preferred owner:                     RAID Controller Module in slot 0                  
                                            Current owner:                       RAID Controller Module in slot 1                  
                                       
                                       
                                            Segment size:                                       128 KB     
                                            Capacity reserved for future segment size changes:  Yes        
                                            Maximum future segment size:                        2,048 KB   
                                            Modification priority:                              High       
                                       
                                       
                                            Read cache:                            Enabled    
                                            Write cache:                           Enabled    
                                               Write cache without batteries:      Disabled   
                                               Write cache with mirroring:         Enabled    
                                            Flush write cache after (in seconds):  10.00      
                                            Dynamic cache read prefetch:           Enabled    
                                                                                              
                                            Enable background media scan:          Enabled    
                                            Media scan with consistency check:     Enabled    
                                                                                              
                                            Pre-Read consistency check:            Disabled   
                                       
                                       
                                         Virtual Disk name:                      Virtual                                           
                                                                                                                                   
                                            Virtual Disk status:                 Failed                                            
                                                                                                                                   
                                            Capacity:                            1.367 TB                                          
                                            Virtual Disk world-wide identifier:  60:02:4e:80:00:70:ed:06:00:00:04:31:4a:96:73:09   
                                            Subsystem ID (SSID):                 0                                                 
                                            Associated disk group:               0                                                 
                                            RAID level:                          5                                                 
                                                                                                                                   
                                            Physical Disk type:                  Serial Attached SCSI (SAS)                        
                                            Enclosure loss protection:           No                                                
                                                                                                                                   
                                            Preferred owner:                     RAID Controller Module in slot 0                  
                                            Current owner:                       RAID Controller Module in slot 1                  
                                       
                                       
                                            Segment size:                                       128 KB     
                                            Capacity reserved for future segment size changes:  Yes        
                                            Maximum future segment size:                        2,048 KB   
                                            Modification priority:                              High       
                                       
                                       
                                            Read cache:                            Enabled    
                                            Write cache:                           Enabled    
                                               Write cache without batteries:      Disabled   
                                               Write cache with mirroring:         Enabled    
                                            Flush write cache after (in seconds):  10.00      
                                            Dynamic cache read prefetch:           Enabled    
                                                                                              
                                            Enable background media scan:          Enabled    
                                            Media scan with consistency check:     Enabled    
                                                                                              
                                            Pre-Read consistency check:            Disabled   
                                      

                                      0_1482874726865_Screenshot 2016-12-27 15.38.36.png

                                      1 Reply Last reply Reply Quote 0
                                      • S
                                        scottalanmiller
                                        last edited by

                                        Oh look, on top of everything else, they left you with RAID 5, too. Figures. Whoever set this up really set you up for failure.

                                        1 Reply Last reply Reply Quote 2
                                        • S
                                          scottalanmiller
                                          last edited by

                                          Your predecessor definitely pulled this on you: https://mangolassi.it/topic/11852/why-it-builds-a-house-of-cards

                                          1 Reply Last reply Reply Quote 1
                                          • S
                                            scottalanmiller
                                            last edited by

                                            Looks like, on top of other problems, the SAN has died. It's hard to tell from this, but it looks like those are the LUNs that hold all of your VMs?

                                            1 Reply Last reply Reply Quote 2
                                            • 1
                                            • 2
                                            • 3
                                            • 4
                                            • 5
                                            • 12
                                            • 13
                                            • 3 / 13
                                            • First post
                                              Last post