ML
    • Recent
    • Categories
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    How would you build a File server with 170TB of Usable Storage?

    IT Discussion
    storage storage planning storage infrastructure
    10
    51
    3.1k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      scottalanmiller @DustinB3403
      last edited by

      @DustinB3403 said in How would you build a File server with 170TB of Usable Storage?:

      Which also raises the question is the 170TB 1 full backup, or some combination.

      Not backup, the OP was file storage.

      J 1 Reply Last reply Reply Quote 0
      • S
        scottalanmiller @dafyre
        last edited by

        @dafyre said in How would you build a File server with 170TB of Usable Storage?:

        Dual Power Supplies, No Setup / Configuration, and 24 x 14TB WD Purple Drives ~= $18,000.

        You would not use Purple drives, either. Those are for streaming, not for file server usage. You'd want other kinds of drives for file server use.

        1 Reply Last reply Reply Quote 0
        • J
          JaredBusch @scottalanmiller
          last edited by

          @scottalanmiller said in How would you build a File server with 170TB of Usable Storage?:

          @DustinB3403 said in How would you build a File server with 170TB of Usable Storage?:

          Which also raises the question is the 170TB 1 full backup, or some combination.

          Not backup, the OP was file storage.

          But @DustinB3403 was trying to drag this back to the original post, instead of reply appropriately to the call out of this topic.

          1 Reply Last reply Reply Quote 0
          • O
            Obsolesce @dafyre
            last edited by Obsolesce

            @dafyre said in How would you build a File server with 170TB of Usable Storage?:

            As a response to the RAID10 Must use... thread, it got me to thinking.

            [This topic isn't to rehash things stated in the other thread, so much as to hear how others would put something at this scale into production].

            Business Goal: Build a production level storage system with 170TB of usable storage for SMB / NFS file shares for regular office workers up to power users and video editors.

            • No NAS Products (Synology, QNAP), please.
            • No Budget Constraints.
            • RAIN Storage / multiple nodes can also be considered (CEPH, GLUSTER, etc).

            My Build would use some flavor of Linux and based on the Storinator Q30 Enhanced and configured for RAID10.

            Dual Power Supplies, No Setup / Configuration, and 24 x 14TB WD Purple Drives ~= $18,000.

            (as we all know, the most significant cost will be the drives).

            What would yours look like?

            How much of the 170TB is video files, how much if it is hot? How big are the video files?

            What are backup requirements?

            Do you need triple the storage considerations for onprem backups plus a tape archive solution for off-site?

            How quickly does that 170TB grow. You said 170TB of data, that tells me zero free space, so you already need more. How do you want to scale?

            No budget? That makes no sense at all, Impossible.

            It depends on many things.

            Could it all live in the cloud with an onprem cache for hot or sticky data?

            It's hard to give a real answer when the full scope and picture is not there.

            1 Reply Last reply Reply Quote 0
            • O
              Obsolesce
              last edited by

              Misread. So how much of the 170TB is used immediately and what's the projected growth rate?

              D J 2 Replies Last reply Reply Quote 0
              • D
                dafyre @scottalanmiller
                last edited by

                @scottalanmiller said in How would you build a File server with 170TB of Usable Storage?:

                @dafyre said in How would you build a File server with 170TB of Usable Storage?:

                My Build would use some flavor of Linux and based on the Storinator Q30 Enhanced and configured for RAID10.

                This I would absolutely avoid. The Storinator has no place in a production SMB environment. It's exclusively designed for use as a RAIN node as part of an N+X cluster. It's disposable / ephemeral only. Any attempt to use it in another way, which would include any configuration with RAID, is misusing it by intent and will lead to support issues. It is not hot swap, it is not designed to be reliable. It's not meant to be serviced. All top considerations for a storage node. It's actually, other than capacity, the exact opposite of what you would seek in a storage device.

                @scottalanmiller -- How would you build it? That is specifically my question or discussion point... How would you go about building a storage platform for 170TB usable data?

                T S 2 Replies Last reply Reply Quote 0
                • D
                  dafyre @Obsolesce
                  last edited by

                  @Obsolesce said in How would you build a File server with 170TB of Usable Storage?:

                  Misread. So how much of the 170TB is used immediately and what's the projected growth rate?

                  40% Currently Used. 6% /year growth rate.

                  1 Reply Last reply Reply Quote 0
                  • J
                    JaredBusch @Obsolesce
                    last edited by

                    @Obsolesce said in How would you build a File server with 170TB of Usable Storage?:

                    Misread. So how much of the 170TB is used immediately and what's the projected growth rate?

                    Not relevant. The concept is that we need 170TB usable. Usable implies available now.

                    You are asking a different question to resolve a different problem.

                    O V 2 Replies Last reply Reply Quote 0
                    • T
                      travisdh1 @dafyre
                      last edited by

                      @dafyre said in How would you build a File server with 170TB of Usable Storage?:

                      @scottalanmiller said in How would you build a File server with 170TB of Usable Storage?:

                      @dafyre said in How would you build a File server with 170TB of Usable Storage?:

                      My Build would use some flavor of Linux and based on the Storinator Q30 Enhanced and configured for RAID10.

                      This I would absolutely avoid. The Storinator has no place in a production SMB environment. It's exclusively designed for use as a RAIN node as part of an N+X cluster. It's disposable / ephemeral only. Any attempt to use it in another way, which would include any configuration with RAID, is misusing it by intent and will lead to support issues. It is not hot swap, it is not designed to be reliable. It's not meant to be serviced. All top considerations for a storage node. It's actually, other than capacity, the exact opposite of what you would seek in a storage device.

                      @scottalanmiller -- How would you build it? That is specifically my question or discussion point... How would you go about building a storage platform for 170TB usable data?

                      Assuming a single chassis, pick a linux distribution, and setup software raid, LVM, and XFS. My other two preferences would be ZFS on BSD, or hardware RAID.

                      Now if we start talking performance, backup or failover, it quickly becomes more difficult.

                      D 2 Replies Last reply Reply Quote 0
                      • D
                        dafyre @travisdh1
                        last edited by

                        @travisdh1 said in How would you build a File server with 170TB of Usable Storage?:

                        Now if we start talking performance, backup or failover, it quickly becomes more difficult.

                        I'm specifically avoiding those topics for the sake of this post. I'm mainly curious as to what the hardware would look like... but more detailed than "buy a server and put hard drives in it" lol.

                        O 1 Reply Last reply Reply Quote 1
                        • D
                          dafyre @travisdh1
                          last edited by

                          @travisdh1 said in How would you build a File server with 170TB of Usable Storage?:

                          @dafyre said in How would you build a File server with 170TB of Usable Storage?:

                          @scottalanmiller said in How would you build a File server with 170TB of Usable Storage?:

                          @dafyre said in How would you build a File server with 170TB of Usable Storage?:

                          My Build would use some flavor of Linux and based on the Storinator Q30 Enhanced and configured for RAID10.

                          This I would absolutely avoid. The Storinator has no place in a production SMB environment. It's exclusively designed for use as a RAIN node as part of an N+X cluster. It's disposable / ephemeral only. Any attempt to use it in another way, which would include any configuration with RAID, is misusing it by intent and will lead to support issues. It is not hot swap, it is not designed to be reliable. It's not meant to be serviced. All top considerations for a storage node. It's actually, other than capacity, the exact opposite of what you would seek in a storage device.

                          @scottalanmiller -- How would you build it? That is specifically my question or discussion point... How would you go about building a storage platform for 170TB usable data?

                          Assuming a single chassis, pick a linux distribution, and setup software raid, LVM, and XFS. My other two preferences would be ZFS on BSD, or hardware RAID.

                          Now if we start talking performance, backup or failover, it quickly becomes more difficult.

                          I'm not familiar with ZFS. I will stay far FAR FAAR FAAAAAAARRRRRRRR away from BTRFS. I've seen too many problems with it at my day job, lol.

                          I'd likely go with Linux + XFS for the file system.

                          S 1 Reply Last reply Reply Quote 2
                          • black3dynamiteB
                            black3dynamite
                            last edited by

                            LVM Thin Pool and XFS for a file server?

                            S 2 Replies Last reply Reply Quote 1
                            • O
                              Obsolesce @dafyre
                              last edited by

                              @dafyre said in How would you build a File server with 170TB of Usable Storage?:

                              @travisdh1 said in How would you build a File server with 170TB of Usable Storage?:

                              Now if we start talking performance, backup or failover, it quickly becomes more difficult.

                              I'm specifically avoiding those topics for the sake of this post. I'm mainly curious as to what the hardware would look like... but more detailed than "buy a server and put hard drives in it" lol.

                              Dumb topic them.

                              DustinB3403D 1 Reply Last reply Reply Quote 0
                              • DustinB3403D
                                DustinB3403 @Obsolesce
                                last edited by

                                @Obsolesce said in How would you build a File server with 170TB of Usable Storage?:

                                Dumb topic them.

                                :thumbs_up:

                                1 Reply Last reply Reply Quote 0
                                • O
                                  Obsolesce @JaredBusch
                                  last edited by Obsolesce

                                  @JaredBusch said in How would you build a File server with 170TB of Usable Storage?:

                                  @Obsolesce said in How would you build a File server with 170TB of Usable Storage?:

                                  Misread. So how much of the 170TB is used immediately and what's the projected growth rate?

                                  Not relevant. The concept is that we need 170TB usable. Usable implies available now.

                                  You are asking a different question to resolve a different problem.

                                  It is relevant because I have no idea what was put into the plan. For all I know, they only have 35TB of data that will only grow to 46TB over the next 5 years. If that's the case, it could change every aspect of the best solution.

                                  If you are saying to just take the 170TB that they just want that amount and tjays that... Then I don't care to put any thought into this thread and will just say buy a server, some DAS, and move on.

                                  J 1 Reply Last reply Reply Quote 0
                                  • J
                                    JaredBusch @Obsolesce
                                    last edited by

                                    @Obsolesce said in How would you build a File server with 170TB of Usable Storage?:

                                    @JaredBusch said in How would you build a File server with 170TB of Usable Storage?:

                                    @Obsolesce said in How would you build a File server with 170TB of Usable Storage?:

                                    Misread. So how much of the 170TB is used immediately and what's the projected growth rate?

                                    Not relevant. The concept is that we need 170TB usable. Usable implies available now.

                                    You are asking a different question to resolve a different problem.

                                    It is relevant because I have no idea what was put into the plan. For all I know, they only have 35TB of data that will only grow to 46TB over the next 5 years. If that's the case, it could change every aspect of the solution.

                                    If you are saying to just take the 170TB that they just want that amount and tjays that... Then I don't care to put any thought into this thread and will just say buy a server, some DAS, and move on.

                                    That is the point of the thread.. what hardware to buy to do that..

                                    The point of the thread is not to analyze some made up needs. The original thread does that.

                                    This thread is simply to say, I have this valid need for 170TB of storage. How do I actually build it with what hardware.

                                    O 1 Reply Last reply Reply Quote 0
                                    • O
                                      Obsolesce @JaredBusch
                                      last edited by

                                      @JaredBusch said in How would you build a File server with 170TB of Usable Storage?:

                                      @Obsolesce said in How would you build a File server with 170TB of Usable Storage?:

                                      @JaredBusch said in How would you build a File server with 170TB of Usable Storage?:

                                      @Obsolesce said in How would you build a File server with 170TB of Usable Storage?:

                                      Misread. So how much of the 170TB is used immediately and what's the projected growth rate?

                                      Not relevant. The concept is that we need 170TB usable. Usable implies available now.

                                      You are asking a different question to resolve a different problem.

                                      It is relevant because I have no idea what was put into the plan. For all I know, they only have 35TB of data that will only grow to 46TB over the next 5 years. If that's the case, it could change every aspect of the solution.

                                      If you are saying to just take the 170TB that they just want that amount and tjays that... Then I don't care to put any thought into this thread and will just say buy a server, some DAS, and move on.

                                      That is the point of the thread.. what hardware to buy to do that..

                                      The point of the thread is not to analyze some made up needs. The original thread does that.

                                      This thread is simply to say, I have this valid need for 170TB of storage. How do I actually build it with what hardware.

                                      I see.

                                      1 Reply Last reply Reply Quote 0
                                      • S
                                        scottalanmiller @black3dynamite
                                        last edited by

                                        @dafyre said in How would you build a File server with 170TB of Usable Storage?:

                                        No Setup / Configuration

                                        Not sure what you mean. Storinator doesn't ship any production ready configuration last I knew. ALL of the work has to be done by you. I don't think that they have any RAIN offerings or even support. Not that they would, they are a chassis vendor, not a storage vendor. But it would be a lot of work to do correctly.

                                        1 Reply Last reply Reply Quote 0
                                        • S
                                          scottalanmiller @dafyre
                                          last edited by

                                          @dafyre said in How would you build a File server with 170TB of Usable Storage?:

                                          @scottalanmiller said in How would you build a File server with 170TB of Usable Storage?:

                                          @dafyre said in How would you build a File server with 170TB of Usable Storage?:

                                          My Build would use some flavor of Linux and based on the Storinator Q30 Enhanced and configured for RAID10.

                                          This I would absolutely avoid. The Storinator has no place in a production SMB environment. It's exclusively designed for use as a RAIN node as part of an N+X cluster. It's disposable / ephemeral only. Any attempt to use it in another way, which would include any configuration with RAID, is misusing it by intent and will lead to support issues. It is not hot swap, it is not designed to be reliable. It's not meant to be serviced. All top considerations for a storage node. It's actually, other than capacity, the exact opposite of what you would seek in a storage device.

                                          @scottalanmiller -- How would you build it? That is specifically my question or discussion point... How would you go about building a storage platform for 170TB usable data?

                                          I would use a real server platform, no matter what the product need is, I use production gear. That's a starting point, even when you need it cheap, you still have to support it.

                                          For 170TB, if it were me, I'd have a RAIN cluster. You didn't go into all of your theoretical details, but assuming pretty common needs, Gluster might be a good choice across a handful of chassis.

                                          But you can do a single chassis, if I was doing that I would use RAID 10. Vendors like SuperMicro and Huawei make servers for this purpose (Storinator is a grid node, not a server on its own, it's a component of a larger system only) that act as servers with server class components and things like hot swap on the drives.

                                          D 2 Replies Last reply Reply Quote 0
                                          • S
                                            scottalanmiller @dafyre
                                            last edited by

                                            @dafyre said in How would you build a File server with 170TB of Usable Storage?:

                                            I'm not familiar with ZFS.

                                            ZFS is specifically the filesystem made by SUN for exactly this purpose. Before there was Storinator, before there were BackBlaze Pods, there was the SUN Thumper - an enterprise class, top loading, customer engineering storage server made to hold an insane number of drives and keep them safe and operational in a single chassis. All those others are attempts to replicate it without the enterprise components. The difference is, Thumper was designed to be a server, the Pods were not, but rather nodes.

                                            The ZFS / Thumper team is who worked with me to design the first SAM-SD as well.

                                            ZFS was super important to making Thumper work as it added RAID 7 (aka RAID 5-3) as an option, and all kinds of cache control, and a file system that could scale as big as Thumper could go.

                                            1 Reply Last reply Reply Quote 2
                                            • 1
                                            • 2
                                            • 3
                                            • 1 / 3
                                            • First post
                                              Last post