Building your own lab
-
@openit said in Building your own lab:
True, I was not aware of multi-node servers.
They are called blades. This is a tiny blade unit. Very common in 2, 4, 8 and 16 node varieties.
-
@gjacobse said in Building your own lab:
Not to be a downer - in these times- who has an extra $1,300? I sincerely hope that all have kept their employment as I have - $1,300 will go a long way in repairs on a vehicle I can’t afford to replace.
Having a home lab is a great tool but just doesn’t seem practical to invest in right now.
It's always good to invest. In fact, investing when times are tough often leads to the lowest investment cost and the highest returns. Holding back investments due to uncertainty is often a cause for the greatest losses.
It is specifically because times are uncertain that embracing growth and preparing to be as ready as possible for the job market matters more than normal.
-
@openit said in Building your own lab:
Whether those 4 servers are individual, except cooling system and server chassis?
Yes, the servers are physically discrete logically. They share power and chassis, but they can't "see" each other.
-
@openit said in Building your own lab:
Why there are only two power supplies for 4 nodes?
Because that's where the big savings come from - not needing eight power supplies to get redundancy, you only need two. That reduces power consumption and cost and space, dramatically. If it wasn't for the shared power and chassis, blades would have no purpose to exist.
-
@openit said in Building your own lab:
How it would be if require any maintenance on 1 node? you can manage individual or impact other 3 nodes in this process?
That depends on the specific unit, but typically nodes are hot swap.
-
@VoIP_n00b said in Building your own lab:
Enjoy your $500+ power bill
Labs aren't cheap, but they are a huge, huge deal in career advancement.
-
@VoIP_n00b said in Building your own lab:
Also how do you plan to handle cooling?
If you have an air conditioned house normal people won't have cooling issues with lab at home. It's not that big and you can power down when you have to.
If you live in a climate with a basement, then it's even easier.
-
I think most people separate multi-node servers from blade servers because there are some minor differences. Blades are multi-node too of course but also shares management, sometimes networking and other things in the blade chassis that the multi-node servers do not.
Supermicro for example puts these products in two different categories.
I think there is also a difference in how you buy them. Multi-node servers comes with all nodes while blade servers are separated into chassis and blades.
-
What node has access to the disks or is each node responsible for 3 drives?
Can you assign one node all the storage and the remaining 3 be just compute? -
@scottalanmiller said in Building your own lab:
@gjacobse said in Building your own lab:
Not to be a downer - in these times- who has an extra $1,300? I sincerely hope that all have kept their employment as I have - $1,300 will go a long way in repairs on a vehicle I can’t afford to replace.
Having a home lab is a great tool but just doesn’t seem practical to invest in right now.
It's always good to invest. In fact, investing when times are tough often leads to the lowest investment cost and the highest returns. Holding back investments due to uncertainty is often a cause for the greatest losses.
It is specifically because times are uncertain that embracing growth and preparing to be as ready as possible for the job market matters more than normal.
I can understand this principle, however tell that to my mechanic.
Truck needs a new power steering pump and a boat load of other things. Price paid for having to drive for work and having a fifteen year old vehicle.
-
@EddieJennings said in Building your own lab:
How loud is such a setup? I have a single Dell T420 for my lab in my apartment, since it's generally quiet. I've been thinking about how I could expand to having more physical hosts.
It's less noise than four individual rack servers. But these servers are common in HPC clusters where you are running jobs that have to complete as soon as possible. So they have the capacity to cool themselves when all 8 CPUs are running at 100% for hours and days at end. So they have high speed fans in the chassis and a very straight air flow path and are capable of making lots of noise.
I haven't used that particular model however the ones I've used sounds like any other rack server. A lot of noise when booting up but then it settles down.
Noise is an effect of cooling need though so I would make smart choices in populating the nodes. Lower power consumption means less noise.
-
@VoIP_n00b said in Building your own lab:
Enjoy your $500+ power bill
Depends on what you pay per kWh and what you put in the nodes.
We have similar servers at work and they draw about 55W per node with VMs running but almost no load. I think that would be a typical power profile in a lab setup.
So let's say 250W for four servers. That's 6 kWh per day and 180 kWh per month. If you pay 14 cents per kWh that's $25 per month in electricity.
And if you have it at home maybe you don't need all nodes running all the time.
-
@Pete-S said in Building your own lab:
I think most people separate multi-node servers from blade servers because there are some minor differences. Blades are multi-node too of course but also shares management, sometimes networking and other things in the blade chassis that the multi-node servers do not.
They sometimes do those things, but by definition, both are blades.
-
@Pete-S said in Building your own lab:
I think there is also a difference in how you buy them. Multi-node servers comes with all nodes while blade servers are separated into chassis and blades.
Both are sold both ways. The one is smaller, so more likely to be pre-populated. But that is really all. You can buy the nodes separately in both cases.
Multi-node and blade are synonymous terms. All blades are multi-node, all multi-node are blades.
It's just a new marketing thing as a response to the negativity of blades to make these small units that tend to serve different purposes due to their size to call them something different.
Mostly blades are used for farms. These 2U units are used for single app clusters. Same tech, different use cases at different scales.
-
@gjacobse said in Building your own lab:
@scottalanmiller said in Building your own lab:
@gjacobse said in Building your own lab:
Not to be a downer - in these times- who has an extra $1,300? I sincerely hope that all have kept their employment as I have - $1,300 will go a long way in repairs on a vehicle I can’t afford to replace.
Having a home lab is a great tool but just doesn’t seem practical to invest in right now.
It's always good to invest. In fact, investing when times are tough often leads to the lowest investment cost and the highest returns. Holding back investments due to uncertainty is often a cause for the greatest losses.
It is specifically because times are uncertain that embracing growth and preparing to be as ready as possible for the job market matters more than normal.
I can understand this principle, however tell that to my mechanic.
Truck needs a new power steering pump and a boat load of other things. Price paid for having to drive for work and having a fifteen year old vehicle.
Right, maybe time to invest in a more cost effective vehicle.
-
@Pete-S said in Building your own lab:
@VoIP_n00b said in Building your own lab:
Enjoy your $500+ power bill
Depends on what you pay per kWh and what you put in the nodes.
We have similar servers at work and they draw about 55W per node with VMs running but almost no load. I think that would be a typical power profile in a lab setup.
So let's say 250W for four servers. That's 6 kWh per day and 180 kWh per month. If you pay 14 cents per kWh that's $25 per month in electricity.
And if you have it at home maybe you don't need all nodes running all the time.
Right, you can easily use just two, but get the experience of the hardware.
-
@scottalanmiller said in Building your own lab:
@openit said in Building your own lab:
Why there are only two power supplies for 4 nodes?
Because that's where the big savings come from - not needing eight power supplies to get redundancy, you only need two. That reduces power consumption and cost and space, dramatically. If it wasn't for the shared power and chassis, blades would have no purpose to exist.
Yes, it's better use of resources.
The power supplies are also more effective and have less power loss than individual servers because they operate at a higher current.
-
@Pete-S said in Building your own lab:
@scottalanmiller said in Building your own lab:
@openit said in Building your own lab:
Why there are only two power supplies for 4 nodes?
Because that's where the big savings come from - not needing eight power supplies to get redundancy, you only need two. That reduces power consumption and cost and space, dramatically. If it wasn't for the shared power and chassis, blades would have no purpose to exist.
Yes, it's better use of resources.
The power supplies are also more effective and have less power loss than individual servers because they operate at a higher current.
Yeah, a small scale version of what some giant datacenters do.... convert to DC when the power enters the building and power the whole building on 12V DC so you just need one (or two) insanely high power, high quality converters. Then all the mobos get straight 12V / 5V feeds.
Big blades (16 units) do this at a smaller scale.
These 4 node units do it at a still smaller scale. But it's all the same theory, reduce the number of power transactions for efficiency.
-
@manxam said in Building your own lab:
What node has access to the disks or is each node responsible for 3 drives?
Can you assign one node all the storage and the remaining 3 be just compute?Each node is hardwired to the drives so in this particular case you have 3 x 3.5" bays for each node. The ones with 2.5" bays have 6x2.5" per node.
There are also SATA ports and USB ports inside for boot drives.
-
@scottalanmiller said in Building your own lab:
@Pete-S said in Building your own lab:
@VoIP_n00b said in Building your own lab:
Enjoy your $500+ power bill
Depends on what you pay per kWh and what you put in the nodes.
We have similar servers at work and they draw about 55W per node with VMs running but almost no load. I think that would be a typical power profile in a lab setup.
So let's say 250W for four servers. That's 6 kWh per day and 180 kWh per month. If you pay 14 cents per kWh that's $25 per month in electricity.
And if you have it at home maybe you don't need all nodes running all the time.
Right, you can easily use just two, but get the experience of the hardware.
That's a good point. You could potentially populate just two nodes if you wanted for starters. Then when you need more, you could pick up some more CPUs and memory and populate the others.
A minimum config could be two nodes with just 1 CPU and 32GB of RAM each. That would be...$700.
