Gluster and RAID question

scottalanmiller

@biggen said in Gluster and RAID question:

Sharing it out via SAMBA seems a little complicated since you also need to layer CTBD.

Why do you need that?

scottalanmiller

@biggen said in Gluster and RAID question:

Is that the standard way to share it out to Windows clients?

No, that would not be common. The common way is to have Samba in a VM that uses Gluster as a backing share.

biggen

@scottalanmiller Ok, it seems most of the tutorials show it being done with CTBD. I’ve found a couple that just create a standard samba share and export it. I’ll play with that route.

So would samba be installed on each node and then shared out? To which samba node do the clients connect to?

Dashrender

@scottalanmiller it sounds like explaining the whole stack might be in order, and where Gluster/etc fall in that stack.

biggen

Creating a two node Gluster volume was real easy. Its the sharing that I'm having an issue with.

Do you install Samba on both nodes and create identical smb.conf file in order to share out the volume? To which nodes are the Samba clients supposed to connect with? Does it matter?

JaredBusch

@biggen said in Gluster and RAID question:

Creating a two node Gluster volume was real easy. Its the sharing that I'm having an issue with.

Do you install Samba on both nodes and create identical smb.conf file in order to share out the volume? To which nodes are the Samba clients supposed to connect with? Does it matter?

If I am understanding WTF you are trying to do, n o you create the Gluster volume and then go into your hypervisor and attach that volume as the datastore, just like you would do for a RAID array.

biggen

@JaredBusch Once the volume is up and running how the heck does one share it out? That what I'm trying to do. I have a successful two node system running:

joe@glusternode1:/mnt$ sudo gluster volume info

Volume Name: gv0
Type: Replicate
Volume ID: ab19d123-eb34-4186-8a03-316a3fc790e3
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: glusternode1:/data/xvdb1/brick
Brick2: glusternode2:/data/xvdb1/brick
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off

That volume must now be mounted "somewhere" to access it. How do I mount it so Windows clients can access it? Do I simply mount the share in one of the nodes under /mnt/big_ole_gluster_space and then share out that mount point via Samba from that same Gluster node?

stacksofplates

@biggen said in Gluster and RAID question:

@JaredBusch Once the volume is up and running how the heck does one share it out? That what I'm trying to do. I have a successful two node system running:
joe@glusternode1:/mnt$ sudo gluster volume info

Volume Name: gv0
Type: Replicate
Volume ID: ab19d123-eb34-4186-8a03-316a3fc790e3
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: glusternode1:/data/xvdb1/brick
Brick2: glusternode2:/data/xvdb1/brick
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
That volume must now be mounted "somewhere" to access it. How do I mount it so Windows clients can access it? Do I simply mount the share in one of the nodes under /mnt/big_ole_gluster_space and then share out that mount point via Samba from that same Gluster node?

The preferred way is to use the GlusterFS FUSE client. Last I knew it's the only one that automatically handles failover and HA.

travisdh1

@biggen said in Gluster and RAID question:

@JaredBusch Once the volume is up and running how the heck does one share it out? That what I'm trying to do. I have a successful two node system running:
joe@glusternode1:/mnt$ sudo gluster volume info

Volume Name: gv0
Type: Replicate
Volume ID: ab19d123-eb34-4186-8a03-316a3fc790e3
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: glusternode1:/data/xvdb1/brick
Brick2: glusternode2:/data/xvdb1/brick
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
That volume must now be mounted "somewhere" to access it. How do I mount it so Windows clients can access it? Do I simply mount the share in one of the nodes under /mnt and then share out that mount point via Samba?

If you want to experiment with it properly, you'll want to follow https://docs.gluster.org/en/latest/Administrator Guide/Accessing Gluster from Windows/

Creating the storage is just the first piece, if you want to share the storage and have it be fault tolerant, then there is a whole lot of other hoops to jump through. Which is also why everyone is saying to just mount it on one of the gluster boxes and create a normal samba share.

biggen

This was the piece of the puzzle I was missing. It explains at the bottom how to configure a simple Samba share.

When one types in "samba gluster" in Google, this unwieldy page is the very first hit. And since its from the official Gluster docs, it makes it seems that is the RIGHT way to do it. That was my confusion when I asked earlier about CTDB.

If one doesn't want to mess with CTDB then sharing out a simple Samba share on one of the Gluster nodes is real easy as I just found out. There is no fault tolerance as far as Samba goes since you are only dealing with a single Samba connection however.

scottalanmiller

@biggen said in Gluster and RAID question:

And since its from the official Gluster docs, it makes it seems that is the RIGHT way to do it.

It's a bit of a conceptual break. Gluster is the wrong place to be looking. That's a filesystem. In no other circumstance, ever, do you look at filesystem documentation (NTFS, XFS, EXT4, etc.) to ask about SMB networking.

So looking at Gluster in this way will be confusing because it doesn't really make any sense. Gluster is a filesystem. Samba is an SMB server. It just reads Gluster the same as any other filesystem if you want.

How do you share out from XFS, ZFS, NTFS, etc.? You do the same way with Gluster. However you answer the first part, is how you will normally answer the second part.

scottalanmiller

@biggen said in Gluster and RAID question:

There is no fault tolerance as far as Samba goes since you are only dealing with a single Samba connection however.

That's because you are "being weird" and acting like Gluster is replacing your hypervisors and virtualization. Since when do we build file servers without virtualizing them? Virtualize Samba and you solve it that way at the platform level. Or make Samba failover the way that Samba normally does.

Basically you are acting like Gluster is a special case, but it is not. Ignore that Gluster is the mechanism that you are using and everything gets really simple. Get fixated and Gluster, and you'll be looking for Gluster-specific answers to all the normal problems.

It's a bit like looking for a guide on how to drive a Ford. But you'll never find one. You'll just find guides to driving cars. The brand of car just doesn't matter, it's all the same. If you are convinced that you need a guide that is specific to steering a Ford, you'll be forever lost and confused thinking that it can't be done when, in reality, it's so simple that no guide exists outside of basic steering guides.

Dashrender

@scottalanmiller that's why I suggested that you make a post about the entire stack :

Hardware
hypervisor
storage (or vice versa with hypervisor)
VMs
storage inside VM
share from inside VM

biggen

I appreciate the explanation guys. Not being in the IT field (directly) for some time means I'm playing catch up with a lot of the stuff.

Lets say as a hypothetical one wanted to build out a 500TB Gluster cluster to be used as a backup target for VMs. It looks like you need at least 3 nodes to build out the Gluster Cluster. Then, of course, you need an additional node for the hypervisor - so 4 nodes minimum.

On the three Gluster nodes, would you be installing a Linux OS directly to them (bare metal)? I know from reading here physical servers have fallen out of style. Is this a use case where a physical server still serves a purpose?

Once the Gluster volume is up and running, you could then connect the hypervisor to the cluster assuming the hypervisor had Gluster Client support and then you have the massive cluster attached to the hypervisor as a SR to be used appropriately.

I'm just wondering if something like this would work.

travisdh1

@biggen said in Gluster and RAID question:

I appreciate the explanation guys. Not being in the IT field (directly) for some time means I'm playing catch up with a lot of the stuff.

Lets say as a hypothetical one wanted to build out a 500TB Gluster cluster to be used as a backup target for VMs. It looks like you need at least 3 nodes to build out the Gluster Cluster. Then, of course, you need an additional node for the hypervisor - so 4 nodes minimum.

On the three Gluster nodes, would you be installing a Linux OS directly to them (bare metal)? I know from reading here physical servers have fallen out of style. Is this a use case where a physical server still serves a purpose?

Once the Gluster volume is up and running, you could then connect the hypervisor to the cluster assuming the hypervisor had Gluster Client support and then you have the massive cluster attached to the hypervisor as a SR to be used appropriately.

I'm just wondering if something like this would work.

Would it work, of course. It wouldn't be very efficient tho.

Gluster would make more sense as the storage for VMs. No matter what size, you really don't need 3 boxes of drives just for backups till your environment is absolutely gigantic!

biggen

@travisdh1 Great thanks for that info. When you say storage for VMs are you speaking of a SAN? So your VMs are running off the Gluster?

Yeah I thought 3 nodes of storage + the hypervisor node sounded like a ton of equipment. I know you can buy single boxes that have 2 - 4 nodes inside of them to reduce the footprint.

travisdh1

@biggen said in Gluster and RAID question:

@travisdh1 Great thanks for that info. When you say storage for VMs are you speaking of a SAN? So your VMs are running off the Gluster?

Yeah I thought 3 nodes of storage + the hypervisor node sounded like a ton of equipment. I know you can buy single boxes that have 2 - 4 nodes inside of them to reduce the footprint.

Something like that. Basically Gluster would replace the SAN.

Those 2-4 node in a boxes are just horrible solutions if you want fault-tolerance. Basically, you still have a single point of failure, but now it takes down all 3 nodes instead of a single node.

Dashrender

@biggen said in Gluster and RAID question:

I appreciate the explanation guys. Not being in the IT field (directly) for some time means I'm playing catch up with a lot of the stuff.

Lets say as a hypothetical one wanted to build out a 500TB Gluster cluster to be used as a backup target for VMs. It looks like you need at least 3 nodes to build out the Gluster Cluster. Then, of course, you need an additional node for the hypervisor - so 4 nodes minimum.

On the three Gluster nodes, would you be installing a Linux OS directly to them (bare metal)? I know from reading here physical servers have fallen out of style. Is this a use case where a physical server still serves a purpose?

Once the Gluster volume is up and running, you could then connect the hypervisor to the cluster assuming the hypervisor had Gluster Client support and then you have the massive cluster attached to the hypervisor as a SR to be used appropriately.

I'm just wondering if something like this would work.

Why would you need a fault tolerant storage solution for your backups? i would think if it was that important - you'd more likely go to tapes as part of your backups D2D2T.

Dashrender

Question for those in the know - Can Gluster run on the same boxes as the hypervisor like in a hyperconveraged setup? It seems crazy to have a solution as @biggen is suggesting - 3 Gluster nodes and a single VM host using that Gluster cluster - i.e. SPOF in that one VM host.
And as he mentioned, that's a ton of hardware.

travisdh1

@Dashrender said in Gluster and RAID question:

Question for those in the know - Can Gluster run on the same boxes as the hypervisor like in a hyperconveraged setup? It seems crazy to have a solution as @biggen is suggesting - 3 Gluster nodes and a single VM host using that Gluster cluster - i.e. SPOF in that one VM host.
And as he mentioned, that's a ton of hardware.

Yes. Really easy if using a linux based KVM. Just create your Gluster storage and mount it as your VM config and storage directory. I've not done a setup like this myself, so I'm probably missing some high-points, but that's the basic idea.