Fixing Software RAID on XenServer

Dashrender

You're running a MDADM RAID 10, yes?

Kelly

@Dashrender Yes

DustinB3403

This is why you replicate your Boot devices to a backup.

Something I plan on doing this weekend (to avoid this very same issue)

DustinB3403

As for trying to recover, can you not mount the array in XenServer without it wiping the array?

Kelly

@DustinB3403 said:

As for trying to recover, can you not mount the array in XenServer without it wiping the array?

I can't access the array in dom0 right now.

I tried doing mdadm --assemble /dev/md0 /dev/sd[bcef]. It says that md0 already exists. When I run cat /proc/mdstat it shows md0 as inactive with only sdb as a member.

coliver

This link has lots of good info on MD RAID recovery.

https://raid.wiki.kernel.org/index.php/RAID_Recovery

Kelly

So the RAID is rebuilding. I ended up having to run this to get the drives reconnected to each other and dom0:

mdadm --create /dev/md0 /dev/sd[bcef]

It is still resyncing, so I'm not sure if it worked or not, but md0 appears to be happy.

travisdh1

@Kelly said:

So the RAID is rebuilding. I ended up having to run this to get the drives reconnected to each other and dom0:
mdadm --create /dev/md0 /dev/sd[bcef]
It is still resyncing, so I'm not sure if it worked or not, but md0 appears to be happy.

It should be fine. Just assuming that none of the drives got mounted and written too, and even if 1 did, the RAID should recover from a single corrupt drive.

scottalanmiller

@Dashrender said:

You're running a MDADM RAID 10, yes?

MD RAID. MDADM is the administration utility for MD RAID. There is no such thing as MDADM RAID.

scottalanmiller

Did it rebuild successfully?

Kelly

@scottalanmiller said:

Did it rebuild successfully?

I don't know. I ended up setting up the necessary VMs on other hosts and started rebuilding this one since it has been having stability issues in the last few weeks.

scottalanmiller

@Kelly said:

@scottalanmiller said:

Did it rebuild successfully?

I don't know. I ended up setting up the necessary VMs on other hosts and started rebuilding this one since it has been having stability issues in the last few weeks.

Ah okay, probably best but it is nice when you can figure out what happened, just to know what it was