Is this server strategy reckless and/or insane?
-
@dustinb3403 said in Is this server strategy reckless and/or insane?:
@dashrender said in Is this server strategy reckless and/or insane?:
Copying to the other server is not a backup, FYI.
Not a good one, that's for sure. As there is no way to be certain that the copy is functional.
What do you mean? It's a live cluster. That is definitely being testing constantly. It's mirrored, live copies. THe backup itself, that he has to test offline.
-
How are you confirming the second server is hosting access to this site?
So this is a active/active setup, correct?
-
@dustinb3403 said in Is this server strategy reckless and/or insane?:
How are you confirming the second server is hosting access to this site?
So this is a active/active setup, correct?
Yes, it is live/live.
-
Has anyone mentioned going OBR5 instead of split arrays yet?
Also, I'd spend the little extra money for the Pro edition of the Samsung 850 drives if you want to use commodity parts rather than Dell supplied ones.
-
@travisdh1 said in Is this server strategy reckless and/or insane?:
Has anyone mentioned going OBR5 instead of split arrays yet?
Also, I'd spend the little extra money for the Pro edition of the Samsung 850 drives if you want to use commodity parts rather than Dell supplied ones.
People did suggest OBR5, yep. The benchmarks I ran ( see the large Crystal DiskMark grid below ) made me feel like I'm going to be giving up a lot of performance for not that much additional peace of mind w/ a 5 given my set up and the ability of either server to temporarily take over duties in a pinch. My overarching goal is for most requests to be as close to perceptibly instant as possible most of the time, w/ some downtime being acceptable.
The drives are all Pros, good tip, thanks.
-
@creayt said in Is this server strategy reckless and/or insane?:
@travisdh1 said in Is this server strategy reckless and/or insane?:
Has anyone mentioned going OBR5 instead of split arrays yet?
Also, I'd spend the little extra money for the Pro edition of the Samsung 850 drives if you want to use commodity parts rather than Dell supplied ones.
People did suggest OBR5, yep. The benchmarks I ran ( see the large Crystal DiskMark grid below ) made me feel like I'm going to be giving up a lot of performance for not that much additional peace of mind w/ a 5 given my set up and the ability of either server to temporarily take over duties in a pinch. My overarching goal is for most requests to be as close to perceptibly instant as possible most of the time, w/ some downtime being acceptable.
Sounds to me like you've thought everything through and are aware of all the pitfalls at least, even if I do think you're worrying overmuch about the performance piece. I'm more focused on that for whatever reason, than that you have an effective network level RAID1.
-
@creayt said in Is this server strategy reckless and/or insane?:
@travisdh1 said in Is this server strategy reckless and/or insane?:
Has anyone mentioned going OBR5 instead of split arrays yet?
Also, I'd spend the little extra money for the Pro edition of the Samsung 850 drives if you want to use commodity parts rather than Dell supplied ones.
People did suggest OBR5, yep. The benchmarks I ran ( see the large Crystal DiskMark grid below ) made me feel like I'm going to be giving up a lot of performance for not that much additional peace of mind w/ a 5 given my set up and the ability of either server to temporarily take over duties in a pinch. My overarching goal is for most requests to be as close to perceptibly instant as possible most of the time, w/ some downtime being acceptable.
The drives are all Pros, good tip, thanks.
The big question is... is it performance that affects the application? Benchmarks and raw numbers don't matter all that much. What matters is how the app is impacted. That's why people are asking about the WAN and other components. Getting that kind of performance on such a small web app typically is all throw away performance. Not necessarily, but often.
-
@scottalanmiller said in Is this server strategy reckless and/or insane?:
@creayt said in Is this server strategy reckless and/or insane?:
@travisdh1 said in Is this server strategy reckless and/or insane?:
Has anyone mentioned going OBR5 instead of split arrays yet?
Also, I'd spend the little extra money for the Pro edition of the Samsung 850 drives if you want to use commodity parts rather than Dell supplied ones.
People did suggest OBR5, yep. The benchmarks I ran ( see the large Crystal DiskMark grid below ) made me feel like I'm going to be giving up a lot of performance for not that much additional peace of mind w/ a 5 given my set up and the ability of either server to temporarily take over duties in a pinch. My overarching goal is for most requests to be as close to perceptibly instant as possible most of the time, w/ some downtime being acceptable.
The drives are all Pros, good tip, thanks.
The big question is... is it performance that affects the application? Benchmarks and raw numbers don't matter all that much. What matters is how the app is impacted. That's why people are asking about the WAN and other components. Getting that kind of performance on such a small web app typically is all throw away performance. Not necessarily, but often.
It's heavily realtime-oriented, by which I mean I'm going to be attempting to stream the presence and actions of users to other users in real time and let them see what the other is doing Google Docs style. The ability to retrieve a good handful of information from MySQL per request in as close to 0 ms as possible is very important for the effect to work correctly, hence wanting to keep the app server and database on the same machine for example. Every little ms counts.
-
@creayt Are you making a real-time adult entertainment system?
I'm just not seeing what web application needs to be built from the ground up for your case based on what has been discussed thus far.
-
I'll just throw this out there to prove what a benefit virtualization is (and Veeam for that matter).
2 days ago, I upgraded my WDS/MDT server so I could image Windows 10 (1703). I uninstalled Windows ADK and installed the updated version. Then I upgraded MDT to 8443 and upgraded my deployment share. I did a test image of my new setup using a machine type that I had already had in MDT and it worked just great.
Yesterday, I had to add a new machine with drivers, task sequence and selection profiles. When I went to update the deployment share, it failed. It couldn't find the boot wim. It was looking for a new directory in the ADK installation path and when I checked the previous location of where the wim file was it was gone. I don't know what happened to it. I was running short on time because this was a new laptop that had to be sent priority overnight to be there today.
I used Veeam instant recovery to spin up a backup of my WDS MDT server from earlier that day (it only took a minute or two) sans network connection and verified that everything was where I thought it should be on the old one. I tried doing a side-by-side comparison on changing the settings on the 'live' server but I couldn't get it working. I then decided to restore the server back in place of the live server and that took only 15 minutes. I was able to add the new drivers, task sequence and selection profiles and update the deployment share successfully and image the laptop to get it out to the guy for this morning.
I would not have that flexibility to do even half of that stuff with the budget of an SMB without virtualization. I have virtualized everything in the past 6 years. I only have 2 physical servers left. A terminal server and my Exchange server. I will virtualize the first and with Exchange, I am planning on migrating to Office 365 in the next 6 months.
But first, I need to figure out why my WDS/MDT upgrade went tits up.
-
@dustinb3403 said in Is this server strategy reckless and/or insane?:
@creayt Are you making a real-time adult entertainment system?
I'm just not seeing what web application needs to be built from the ground up for your case based on what has been discussed thus far.
I'm sorry, but the "I'm just not seeing" sentence didn't make sense to me. Were you asking what I'm building or why I have the requirements I do or what? Definitely nothing to do w/ "adult entertainment" hahaha.
-
@dashrender said in Is this server strategy reckless and/or insane?:
UREs are probably pretty low on these SSDs, but not zero, so something else to consider, what are the chances of a URE killing your RAID 0? (now Scott will educate me that these don't matter
- seriously don't know if do or not)The failure mode that you should be afraid of isn't a URE, but data loss on power loss that is OUT of order with acknowledged writes. This breaks standard data loss recovery you get from a Journal Log on MySQL and other database apps.
If it would just cleanly loose the last write that would be fantastic, sadly it's how the Samsung consumer drives tend to recover lost data. Normally they are used in laptops that have a giant battery attached, are not running RAID (Which will see this out of order recovery as a failed drive when it fails their ECC check). This can/will catastrophically fail with multiple drives dropping on something as simple as a controller or host failure or hard reboot.
-
@creayt said in Is this server strategy reckless and/or insane?:
Ideally more than that, but it'll be a gradual climb. Right now it's in private alpha w/ ~ 100 users and they post stuff all the time. Once I make it public I imagine the content volume will skyrocket.
Why not use Cloud/PaaS? There are some systems where you pay by the transaction so you're not out capital for hardware that will not scale where you need to go a long time, and you will not waste money on hardware if this project goes nowhere.
-
@storageninja said in Is this server strategy reckless and/or insane?:
@creayt said in Is this server strategy reckless and/or insane?:
Ideally more than that, but it'll be a gradual climb. Right now it's in private alpha w/ ~ 100 users and they post stuff all the time. Once I make it public I imagine the content volume will skyrocket.
Why not use Cloud/PaaS? There are some systems where you pay by the transaction so you're not out capital for hardware that will not scale where you need to go a long time, and you will not waste money on hardware if this project goes nowhere.
Pricing out equivalent horsepower on Amazon I think came to something like $50k a month, this whole set up cost me under $10k I believe. By the time I exhaust the capabilities of this hardware/investment, I hope, I'll be at the venture capital phase and and can redeploy into a fully cloud strategy, grinning shit-eatingly at how well that original $10k investment served me.
Will also mention that colocation where I live is a dirt-effing-cheap $55-per-U/month.
-
@creayt said in Is this server strategy reckless and/or insane?:
@scottalanmiller said in Is this server strategy reckless and/or insane?:
@creayt said in Is this server strategy reckless and/or insane?:
@travisdh1 said in Is this server strategy reckless and/or insane?:
Has anyone mentioned going OBR5 instead of split arrays yet?
Also, I'd spend the little extra money for the Pro edition of the Samsung 850 drives if you want to use commodity parts rather than Dell supplied ones.
People did suggest OBR5, yep. The benchmarks I ran ( see the large Crystal DiskMark grid below ) made me feel like I'm going to be giving up a lot of performance for not that much additional peace of mind w/ a 5 given my set up and the ability of either server to temporarily take over duties in a pinch. My overarching goal is for most requests to be as close to perceptibly instant as possible most of the time, w/ some downtime being acceptable.
The drives are all Pros, good tip, thanks.
The big question is... is it performance that affects the application? Benchmarks and raw numbers don't matter all that much. What matters is how the app is impacted. That's why people are asking about the WAN and other components. Getting that kind of performance on such a small web app typically is all throw away performance. Not necessarily, but often.
It's heavily realtime-oriented, by which I mean I'm going to be attempting to stream the presence and actions of users to other users in real time and let them see what the other is doing Google Docs style. The ability to retrieve a good handful of information from MySQL per request in as close to 0 ms as possible is very important for the effect to work correctly, hence wanting to keep the app server and database on the same machine for example. Every little ms counts.
This is where it feels to be like MySQL was a bad choice. I don't know your details, but MySQL seems at odds with all of your other requirements.
-
@scottalanmiller said in Is this server strategy reckless and/or insane?:
@creayt said in Is this server strategy reckless and/or insane?:
@scottalanmiller said in Is this server strategy reckless and/or insane?:
@creayt said in Is this server strategy reckless and/or insane?:
@travisdh1 said in Is this server strategy reckless and/or insane?:
Has anyone mentioned going OBR5 instead of split arrays yet?
Also, I'd spend the little extra money for the Pro edition of the Samsung 850 drives if you want to use commodity parts rather than Dell supplied ones.
People did suggest OBR5, yep. The benchmarks I ran ( see the large Crystal DiskMark grid below ) made me feel like I'm going to be giving up a lot of performance for not that much additional peace of mind w/ a 5 given my set up and the ability of either server to temporarily take over duties in a pinch. My overarching goal is for most requests to be as close to perceptibly instant as possible most of the time, w/ some downtime being acceptable.
The drives are all Pros, good tip, thanks.
The big question is... is it performance that affects the application? Benchmarks and raw numbers don't matter all that much. What matters is how the app is impacted. That's why people are asking about the WAN and other components. Getting that kind of performance on such a small web app typically is all throw away performance. Not necessarily, but often.
It's heavily realtime-oriented, by which I mean I'm going to be attempting to stream the presence and actions of users to other users in real time and let them see what the other is doing Google Docs style. The ability to retrieve a good handful of information from MySQL per request in as close to 0 ms as possible is very important for the effect to work correctly, hence wanting to keep the app server and database on the same machine for example. Every little ms counts.
This is where it feels to be like MySQL was a bad choice. I don't know your details, but MySQL seems at odds with all of your other requirements.
Can you elaborate a little bit?
-
@creayt said in Is this server strategy reckless and/or insane?:
@storageninja said in Is this server strategy reckless and/or insane?:
@creayt said in Is this server strategy reckless and/or insane?:
Ideally more than that, but it'll be a gradual climb. Right now it's in private alpha w/ ~ 100 users and they post stuff all the time. Once I make it public I imagine the content volume will skyrocket.
Why not use Cloud/PaaS? There are some systems where you pay by the transaction so you're not out capital for hardware that will not scale where you need to go a long time, and you will not waste money on hardware if this project goes nowhere.
Pricing out equivalent horsepower on Amazon I think came to something like $50k a month, this whole set up cost me under $10k I believe. By the time I exhaust the capabilities of this hardware/investment, I hope, I'll be at the venture capital phase and and can redeploy into a fully cloud strategy, grinning shit-eatingly at how well that original $10k investment served me.
Will also mention that colocation where I live is a dirt-effing-cheap $55-per-U/month.
There are far cheaper IaaS providers than Amazon (I assume you are looking at EC2, when you should be looking at RDS if you're doing AWS). I'm partial to Softlayer these days, but to each their own.
Deploying and managing your own infrastructure for a startup is a nightmare as if/when your product "Blows up" and goes from 100, to 100K users it will implode and crash on the weekend before you can get new hardware in and scale it, or refactor for a platform with real scalability. If your worried about cloud lock-in use abstraction systems that allow for multi-cloud strategies (although honestly in the early phase I'd just accept the lockin as that's easier to refactor than trying to refactor the platform AND scale at the same time).
If you can't maintain growth and have large hiccups in engineering VC gets spooked easily.
Also If you're really looking to scale one thing is trying to limit your dependency on RDMS in general. 9/10 times I see a startup using one, they should have used object storage or a No-SQL system.
Then again, I'm just a Palo Alto Serf working for "the man" and not feeling the wind in my hair of founding the next big thing in the garage.
-
@storageninja said in Is this server strategy reckless and/or insane?:
@creayt said in Is this server strategy reckless and/or insane?:
@storageninja said in Is this server strategy reckless and/or insane?:
@creayt said in Is this server strategy reckless and/or insane?:
Ideally more than that, but it'll be a gradual climb. Right now it's in private alpha w/ ~ 100 users and they post stuff all the time. Once I make it public I imagine the content volume will skyrocket.
Why not use Cloud/PaaS? There are some systems where you pay by the transaction so you're not out capital for hardware that will not scale where you need to go a long time, and you will not waste money on hardware if this project goes nowhere.
Pricing out equivalent horsepower on Amazon I think came to something like $50k a month, this whole set up cost me under $10k I believe. By the time I exhaust the capabilities of this hardware/investment, I hope, I'll be at the venture capital phase and and can redeploy into a fully cloud strategy, grinning shit-eatingly at how well that original $10k investment served me.
Will also mention that colocation where I live is a dirt-effing-cheap $55-per-U/month.
There are far cheaper IaaS providers than Amazon (I assume you are looking at EC2, when you should be looking at RDS if you're doing AWS). I'm partial to Softlayer these days, but to each their own.
Deploying and managing your own infrastructure for a startup is a nightmare as if/when your product "Blows up" and goes from 100, to 100K users it will implode and crash on the weekend before you can get new hardware in and scale it, or refactor for a platform with real scalability. If your worried about cloud lock-in use abstraction systems that allow for multi-cloud strategies (although honestly in the early phase I'd just accept the lockin as that's easier to refactor than trying to refactor the platform AND scale at the same time).
If you can't maintain growth and have large hiccups in engineering VC gets spooked easily.
Also If you're really looking to scale one thing is trying to limit your dependency on RDMS in general. 9/10 times I see a startup using one, they should have used object storage or a No-SQL system.
Then again, I'm just a Palo Alto Serf working for "the man" and not feeling the wind in my hair of founding the next big thing in the garage.
Thanks Dad
-
@scottalanmiller said in Is this server strategy reckless and/or insane?:
@creayt said in Is this server strategy reckless and/or insane?:
@scottalanmiller said in Is this server strategy reckless and/or insane?:
@creayt said in Is this server strategy reckless and/or insane?:
@travisdh1 said in Is this server strategy reckless and/or insane?:
Has anyone mentioned going OBR5 instead of split arrays yet?
Also, I'd spend the little extra money for the Pro edition of the Samsung 850 drives if you want to use commodity parts rather than Dell supplied ones.
People did suggest OBR5, yep. The benchmarks I ran ( see the large Crystal DiskMark grid below ) made me feel like I'm going to be giving up a lot of performance for not that much additional peace of mind w/ a 5 given my set up and the ability of either server to temporarily take over duties in a pinch. My overarching goal is for most requests to be as close to perceptibly instant as possible most of the time, w/ some downtime being acceptable.
The drives are all Pros, good tip, thanks.
The big question is... is it performance that affects the application? Benchmarks and raw numbers don't matter all that much. What matters is how the app is impacted. That's why people are asking about the WAN and other components. Getting that kind of performance on such a small web app typically is all throw away performance. Not necessarily, but often.
It's heavily realtime-oriented, by which I mean I'm going to be attempting to stream the presence and actions of users to other users in real time and let them see what the other is doing Google Docs style. The ability to retrieve a good handful of information from MySQL per request in as close to 0 ms as possible is very important for the effect to work correctly, hence wanting to keep the app server and database on the same machine for example. Every little ms counts.
This is where it feels to be like MySQL was a bad choice. I don't know your details, but MySQL seems at odds with all of your other requirements.
Yah, you don't need consistency isn't needed and mySQL will scale like shit no matter how much you shard it.
-
@storageninja said in Is this server strategy reckless and/or insane?:
@scottalanmiller said in Is this server strategy reckless and/or insane?:
@creayt said in Is this server strategy reckless and/or insane?:
@scottalanmiller said in Is this server strategy reckless and/or insane?:
@creayt said in Is this server strategy reckless and/or insane?:
@travisdh1 said in Is this server strategy reckless and/or insane?:
Has anyone mentioned going OBR5 instead of split arrays yet?
Also, I'd spend the little extra money for the Pro edition of the Samsung 850 drives if you want to use commodity parts rather than Dell supplied ones.
People did suggest OBR5, yep. The benchmarks I ran ( see the large Crystal DiskMark grid below ) made me feel like I'm going to be giving up a lot of performance for not that much additional peace of mind w/ a 5 given my set up and the ability of either server to temporarily take over duties in a pinch. My overarching goal is for most requests to be as close to perceptibly instant as possible most of the time, w/ some downtime being acceptable.
The drives are all Pros, good tip, thanks.
The big question is... is it performance that affects the application? Benchmarks and raw numbers don't matter all that much. What matters is how the app is impacted. That's why people are asking about the WAN and other components. Getting that kind of performance on such a small web app typically is all throw away performance. Not necessarily, but often.
It's heavily realtime-oriented, by which I mean I'm going to be attempting to stream the presence and actions of users to other users in real time and let them see what the other is doing Google Docs style. The ability to retrieve a good handful of information from MySQL per request in as close to 0 ms as possible is very important for the effect to work correctly, hence wanting to keep the app server and database on the same machine for example. Every little ms counts.
This is where it feels to be like MySQL was a bad choice. I don't know your details, but MySQL seems at odds with all of your other requirements.
Yah, you don't need consistency isn't needed and mySQL will scale like shit no matter how much you shard it.
Really? What scale does that come into play on? Just ran a sample query against a table on my local system running MySQL w/ no tuning ( out of the box developer install -- low ram ), table has 77 million plus rows and a few simple indexes, and returns what I want in .001 seconds. I've always found MySQL to be super duper duper fast when your schema is well-designed and your queries are strategically indexed.
