Adding tape drive
-
We currently use veeam in a VM on ESXi, but I am changing my whole setup soon. I am currently backing up to disk, mostly to a few synology boxes that are presenting their storage to ESXi to distribute to the VM's. In the role of offsite, which is where the tapes would come in, I am doing weekly backups to a external SSD via usb3 and then taking this offsite. But the size of these backups are very limited and they take forever because of the way my network is configured (also fixing this). I am not currently using veeam for these offsite backups, I am forced to do file level copies because of my limited storage space on the usb drives. But inside of veeam, I am using the GFS scheme to keep a limited number of backups on disk at our two physical locations.
I still need to decide if I want to do the GFS scheme with tape, or with onsite disk. I originally had the idea to use disk for those, but Scott tried to talk me into using the tapes for this.
I use reverse incremental in veeam, and I dont ever have to take "fulls".
-
I am also going to be adding a lot more storage space to this host, and I've got a raid card in it now that will support 8 drives. I go with a 7 drive raid 6 (HDD's) and have an open spot for the tape drive, or I can stick with 8 drives in raid 10 and just buy another HBA.
-
between the raid 6 option and the raid 10 option, both will cover my storage needs just fine. But I want to be able to backup quickly and restore quickly, with the later being my priority. That is why I had thought raid 10 was the way to go, and it still might be.
-
@Donahue said in Adding tape drive:
between the raid 6 option and the raid 10 option, both will cover my storage needs just fine. But I want to be able to backup quickly and restore quickly, with the later being my priority. That is why I had thought raid 10 was the way to go, and it still might be.
Not really, speed doesn't vary in a meaningful way to you. What factor do you feel is driving you to RAID 10?
-
@Donahue said in Adding tape drive:
between the raid 6 option and the raid 10 option, both will cover my storage needs just fine. But I want to be able to backup quickly and restore quickly, with the later being my priority. That is why I had thought raid 10 was the way to go, and it still might be.
If the backup repository is over the network, it doesn't matter how fast your storage array is. The maximum data you'll be able to pull from your backup will be 50-110 MB/s over a regular network in this type of setup.
[[storage] + [backup server]] --> Network --> [restore location/server]
-
@Obsolesce said in Adding tape drive:
@Donahue said in Adding tape drive:
between the raid 6 option and the raid 10 option, both will cover my storage needs just fine. But I want to be able to backup quickly and restore quickly, with the later being my priority. That is why I had thought raid 10 was the way to go, and it still might be.
If the backup repository is over the network, it doesn't matter how fast your storage array is. The maximum data you'll be able to pull from your backup will be 50-110 MB/s over a regular network in this type of setup.
[[storage] + [backup server]] --> Network --> [restore location/server]
10Gb/s, but still it'll saturate everything else before that array.
-
@scottalanmiller said in Adding tape drive:
@Donahue said in Adding tape drive:
between the raid 6 option and the raid 10 option, both will cover my storage needs just fine. But I want to be able to backup quickly and restore quickly, with the later being my priority. That is why I had thought raid 10 was the way to go, and it still might be.
Not really, speed doesn't vary in a meaningful way to you. What factor do you feel is driving you to RAID 10?
you shouting from the mountain tops that OBR10 is the gold standard.
-
@Obsolesce said in Adding tape drive:
@Donahue said in Adding tape drive:
between the raid 6 option and the raid 10 option, both will cover my storage needs just fine. But I want to be able to backup quickly and restore quickly, with the later being my priority. That is why I had thought raid 10 was the way to go, and it still might be.
If the backup repository is over the network, it doesn't matter how fast your storage array is. The maximum data you'll be able to pull from your backup will be 50-110 MB/s over a regular network in this type of setup.
[[storage] + [backup server]] --> Network --> [restore location/server]
the backup server will have onboard storage, but will be backing up from and restoring to a 10G network.
-
@Donahue said in Adding tape drive:
@scottalanmiller said in Adding tape drive:
@Donahue said in Adding tape drive:
between the raid 6 option and the raid 10 option, both will cover my storage needs just fine. But I want to be able to backup quickly and restore quickly, with the later being my priority. That is why I had thought raid 10 was the way to go, and it still might be.
Not really, speed doesn't vary in a meaningful way to you. What factor do you feel is driving you to RAID 10?
you shouting from the mountain tops that OBR10 is the gold standard.
No, it's the starting point and safe fallback for spinners. I never stated anything more than that.
-
@Donahue said in Adding tape drive:
@Obsolesce said in Adding tape drive:
@Donahue said in Adding tape drive:
between the raid 6 option and the raid 10 option, both will cover my storage needs just fine. But I want to be able to backup quickly and restore quickly, with the later being my priority. That is why I had thought raid 10 was the way to go, and it still might be.
If the backup repository is over the network, it doesn't matter how fast your storage array is. The maximum data you'll be able to pull from your backup will be 50-110 MB/s over a regular network in this type of setup.
[[storage] + [backup server]] --> Network --> [restore location/server]
the backup server will have onboard storage, but will be backing up from and restoring to a 10G network.
Right, so very likely the speed of the backup system's drives and networking will be the bottleneck.
-
@scottalanmiller said in Adding tape drive:
@Donahue said in Adding tape drive:
@Obsolesce said in Adding tape drive:
@Donahue said in Adding tape drive:
between the raid 6 option and the raid 10 option, both will cover my storage needs just fine. But I want to be able to backup quickly and restore quickly, with the later being my priority. That is why I had thought raid 10 was the way to go, and it still might be.
If the backup repository is over the network, it doesn't matter how fast your storage array is. The maximum data you'll be able to pull from your backup will be 50-110 MB/s over a regular network in this type of setup.
[[storage] + [backup server]] --> Network --> [restore location/server]
the backup server will have onboard storage, but will be backing up from and restoring to a 10G network.
Right, so very likely the speed of the backup system's drives and networking will be the bottleneck.
Specifically the drives themselves, and not the array?
-
@scottalanmiller said in Adding tape drive:
@Obsolesce said in Adding tape drive:
@Donahue said in Adding tape drive:
between the raid 6 option and the raid 10 option, both will cover my storage needs just fine. But I want to be able to backup quickly and restore quickly, with the later being my priority. That is why I had thought raid 10 was the way to go, and it still might be.
If the backup repository is over the network, it doesn't matter how fast your storage array is. The maximum data you'll be able to pull from your backup will be 50-110 MB/s over a regular network in this type of setup.
[[storage] + [backup server]] --> Network --> [restore location/server]
10Gb/s, but still it'll saturate everything else before that array.
Oh didn't see it was 10GB. But yeah, IOPS matters more for internal storage processing... like busy databases, a bunch of running VMs on the same storage array, etc. Not for pulling/retrieving large files.
-
@Donahue said in Adding tape drive:
@scottalanmiller said in Adding tape drive:
@Donahue said in Adding tape drive:
@Obsolesce said in Adding tape drive:
@Donahue said in Adding tape drive:
between the raid 6 option and the raid 10 option, both will cover my storage needs just fine. But I want to be able to backup quickly and restore quickly, with the later being my priority. That is why I had thought raid 10 was the way to go, and it still might be.
If the backup repository is over the network, it doesn't matter how fast your storage array is. The maximum data you'll be able to pull from your backup will be 50-110 MB/s over a regular network in this type of setup.
[[storage] + [backup server]] --> Network --> [restore location/server]
the backup server will have onboard storage, but will be backing up from and restoring to a 10G network.
Right, so very likely the speed of the backup system's drives and networking will be the bottleneck.
Specifically the drives themselves, and not the array?
Everything will be a factor.
-
ok, If I understand this, Scott's championing OBR10 as the starting point for a production load, but it may not be the starting point, or in this case the end point on a backup storage, especially if IOPS are not the primary concern? All this seems like just doing 7 drives in raid 6 and using the tape drive on the extra channel is the right way to go.
Its very easy for someone like me to read some of the stuff that Scott has written as "the most prolific raid author in history" as authoritative and think of it as the gold standard.
-
@Donahue said in Adding tape drive:
ok, If I understand this, Scott's championing OBR10 as the starting point for a production load, but it may not be the starting point, or in this case the end point on a backup storage, especially if IOPS are not the primary concern? All this seems like just doing 7 drives in raid 6 and using the tape drive on the extra channel is the right way to go.
Its very easy for someone like me to read some of the stuff that Scott has written as "the most prolific raid author in history" as authoritative and think of it as the gold standard.
OBR10 for spinning rust as "a safe starting point", before any considerations or environmental factors.
-
@Donahue said in Adding tape drive:
ok, If I understand this, Scott's championing OBR10 as the starting point for a production load...
OBR10 when using spinners, as a starting point for decision making. No RAID should be chosen without clear planning, every situation is unique. But if you can't find something that mathematically is clearly better for you when using spinners, you fail to OBR10 because it is the safest and fastest and without clear financial benefit, you don't want something else.
-
@scottalanmiller said in Adding tape drive:
@Donahue said in Adding tape drive:
ok, If I understand this, Scott's championing OBR10 as the starting point for a production load...
OBR10 when using spinners, as a starting point for decision making. No RAID should be chosen without clear planning, every situation is unique. But if you can't find something that mathematically is clearly better for you when using spinners, you fail to OBR10 because it is the safest and fastest and without clear financial benefit, you don't want something else.
I interpret all that as OBR10 = gold standard, or as you would put it, "best practice". I have probably been guilty of over simplifying for the sake of
lazinessefficiency. -
@Donahue said in Adding tape drive:
@scottalanmiller said in Adding tape drive:
@Donahue said in Adding tape drive:
ok, If I understand this, Scott's championing OBR10 as the starting point for a production load...
OBR10 when using spinners, as a starting point for decision making. No RAID should be chosen without clear planning, every situation is unique. But if you can't find something that mathematically is clearly better for you when using spinners, you fail to OBR10 because it is the safest and fastest and without clear financial benefit, you don't want something else.
I interpret all that as OBR10 = gold standard, or as you would put it, "best practice". I have probably been guilty of over simplifying for the sake of
lazinessefficiency.Right, a lot of people have repeated "starting point" as "best practice" which are wholly different things. A BP means you have no discussion, there is only one way to do it. Starting point means the opposite, there is no BP and no given answer, you have to figure it out.
Think of it as a starting point for choosing a car. "Well, you should look at the dealer nearest your home first, since you know it is conveniently located and you have nothing else to go on before test driving some different cars." Buying whatever car is closest to your home is hardly a best practice, but it is a reasonable way to start a process of test driving different models.
-
@scottalanmiller said in Adding tape drive:
@Donahue said in Adding tape drive:
@scottalanmiller said in Adding tape drive:
@Donahue said in Adding tape drive:
ok, If I understand this, Scott's championing OBR10 as the starting point for a production load...
OBR10 when using spinners, as a starting point for decision making. No RAID should be chosen without clear planning, every situation is unique. But if you can't find something that mathematically is clearly better for you when using spinners, you fail to OBR10 because it is the safest and fastest and without clear financial benefit, you don't want something else.
I interpret all that as OBR10 = gold standard, or as you would put it, "best practice". I have probably been guilty of over simplifying for the sake of
lazinessefficiency.Right, a lot of people have repeated "starting point" as "best practice" which are wholly different things. A BP means you have no discussion, there is only one way to do it. Starting point means the opposite, there is no BP and no given answer, you have to figure it out.
Think of it as a starting point for choosing a car. "Well, you should look at the dealer nearest your home first, since you know it is conveniently located and you have nothing else to go on before test driving some different cars." Buying whatever car is closest to your home is hardly a best practice, but it is a reasonable way to start a process of test driving different models.
I loosely define "standard" to mean the same thing that you say when you say "best practice". I would call "starting point" the same as "default"
-
@Donahue said in Adding tape drive:
@scottalanmiller said in Adding tape drive:
@Donahue said in Adding tape drive:
@scottalanmiller said in Adding tape drive:
@Donahue said in Adding tape drive:
ok, If I understand this, Scott's championing OBR10 as the starting point for a production load...
OBR10 when using spinners, as a starting point for decision making. No RAID should be chosen without clear planning, every situation is unique. But if you can't find something that mathematically is clearly better for you when using spinners, you fail to OBR10 because it is the safest and fastest and without clear financial benefit, you don't want something else.
I interpret all that as OBR10 = gold standard, or as you would put it, "best practice". I have probably been guilty of over simplifying for the sake of
lazinessefficiency.Right, a lot of people have repeated "starting point" as "best practice" which are wholly different things. A BP means you have no discussion, there is only one way to do it. Starting point means the opposite, there is no BP and no given answer, you have to figure it out.
Think of it as a starting point for choosing a car. "Well, you should look at the dealer nearest your home first, since you know it is conveniently located and you have nothing else to go on before test driving some different cars." Buying whatever car is closest to your home is hardly a best practice, but it is a reasonable way to start a process of test driving different models.
I loosely define "standard" to mean the same thing that you say when you say "best practice". I would call "starting point" the same as "default"
I wouldn't generally. Gold standard implies it is better. Starting point doesn't, it implies that it is a proper place to begin to encourage good decision making. Gold standard implies that if you deviate, you get something inferior. Starting point does not.
Example, which sounds weird...
Manager: "John, I hear you didn't use the gold standard for our industry, explain yourself?"
or...
Manager: "John, I hear you didn't just go with the starting point option for our industry, explain yourself?"
One is a criticism, one is a praise.
