Tonight's Platform Update
-
@Dashrender said:
Sure, at what point does a colo make more sense than rending VM space from these vendors? Of course the problem with that is a single server, single point of failure.
When we start getting to the point that having 4-6 physical CPUs and 96GB+ of RAM are needed per site for performance. That is a long, long way off. The platform that we use is so efficient that we are handling close to 100,000 views a day and were only just starting to run into memory constraints on the old system and mostly because the system has grown over the last two years to have so much content that keeping stuff in memory was bogging things down.
The leap that we have made likely will carry us for more than another two years, every aspect of the system is 300% or more faster or bigger than we had before. We have 300% more cores now, but also 15% more speed per core, that adds up. That's like a 345% total speed increase. That is a lot. And that wasn't our bottleneck. And we have heard numbers up to 300% increase from the database update. And 300% faster disk IO!!! These things all add up. The CPU waits on the disks less, the database requires less of all of the resources, more things are cached in memory - if we were capping out at 100,000 views a day (before people could notice some minor lag) we are guessing that the new system can handle a million or more.
The amount of growth that we are prepared to handle with zero to trivial effort is pretty enormous. That we will need to consider anything else for a very long time is unlikely.
Chances are in two years we will want to revisit the architecture and, at that time, there is a very good chance that getting a 100% boost in memory size will be obvious and simple, that per core CPU speeds will have increased, etc. The platform naturally gets faster underneath us in many ways. So the need for moving to a completely different approach is much farther off than you would think. Because of the type of site that we are, the scale that can be handled from the current design is pretty extreme.
-
@Dashrender said:
Of course the problem with that is a single server, single point of failure.
With good database backups, we could mitigate that pretty easily. MongoDB does an export, compress, transport, decompress, restore with blinding speed. We just did it last night and it was insane how fast the entire site was able to be transported around.
-
@scottalanmiller said:
Chances are in two years we will want to revisit the architecture and, at that time, there is a very good chance that getting a 100% boost in memory size will be obvious and simple, that per core CPU speeds will have increased, etc. The platform naturally gets faster underneath us in many ways. So the need for moving to a completely different approach is much farther off than you would think. Because of the type of site that we are, the scale that can be handled from the current design is pretty extreme.
Actually, this is exactly what I would expect. As the platform under the VMs becomes better as old hardware is replaced, I would expect there to be less need to migrate to something new. Sure you might need to assign more RAM, something that the underlying hardware might have more of now because it was upgraded, but the VM won't because, well that's not how that works. But when the CPU and disk are improved, the VM just gets those gains because everything on the system get those gains - not taking tiered storage into account.
-
@Dashrender said:
@scottalanmiller said:
Chances are in two years we will want to revisit the architecture and, at that time, there is a very good chance that getting a 100% boost in memory size will be obvious and simple, that per core CPU speeds will have increased, etc. The platform naturally gets faster underneath us in many ways. So the need for moving to a completely different approach is much farther off than you would think. Because of the type of site that we are, the scale that can be handled from the current design is pretty extreme.
Actually, this is exactly what I would expect. As the platform under the VMs becomes better as old hardware is replaced, I would expect there to be less need to migrate to something new. Sure you might need to assign more RAM, something that the underlying hardware might have more of now because it was upgraded, but the VM won't because, well that's not how that works. But when the CPU and disk are improved, the VM just gets those gains because everything on the system get those gains - not taking tiered storage into account.
Yup, and realistically we grow at a rather steady pace not like we get twenty fold increase in a month. So the underlying hardware tends to grow with us pretty steadily.
-
NodeBB 1.0.2 is out. Might as well upgrade
It has been a day of upheaval already. -
That was ridiculously fast, lol.
-
We did a full update in under a minute!!
-
Huzzah! What did it fix/change?
-
-
@scottalanmiller said:
@nadnerB said:
Huzzah! What did it fix/change?
No idea
There's this thing called Changelog... Generally one should read it before applying said updates...
-
A month in and Linode is still rocking it!
