On prem Exchange hardware questions.
-
@JasGot said in On prem Exchange hardware questions.:
I need to provide an on prem exchange solution for a company with 100 users and about 135GB in mailbox database use.
I am interested if anything has changed in the last few years concerning Exchange on a VM and Spinning v. SSD drives?
If Exchange were going to be the ONLY VM on a host, would you still go VM?
Are SSDs overkill for Exchange?
I know there will be lots of questions, and thoughts, but I this is what is on my mind right now, so I thought it would be a good place to start.
I will also be proposing Microsoft 365, but I want to have a solid onprem plan if they choose to stay on prem.
Exchange should have SSD and on a VM, that is all the remaining Exchange Onprem severs I have. That said make sure to Size the backup well and plan for Database Growth and Email Archiving and Spam Filtering. If you are wanting to setup DKIM and DMARC look at DKIM-Exchange
https://github.com/Pro/dkim-exchange/releasesAlso Setup LEt's Encrypt with the Certify the Web on the Exchange Server
https://certifytheweb.com/
https://docs.certifytheweb.com/docs/script-hooks#example-enable-certificate-for-exchange-2013--2016-services-on-local-server -
@travisdh1 said in On prem Exchange hardware questions.:
@Dashrender said in On prem Exchange hardware questions.:
@scottalanmiller said in On prem Exchange hardware questions.:
@JasGot said in On prem Exchange hardware questions.:
Are SSDs overkill for Exchange?
SSDs aren't overkill for a desktop. Unless you've got an extreme edge case, spinning drives haven't had a place in servers in several years now.
I want to agree with this, but the prices are still kinda high for enterprise drives... you have to look at the financial aspect in my opinion... it's quite possible that SSDs will be the way to go.
I.e. HDD = RAID 10 or RAID 6 (min)
135 GB of Exchange data is nothing Two 500 GB HDDs in RAID 1 will get you there, but will it have the performance you want, not likely, but two SSDs in RAID 10 will.
But perhaps 4 or 6 HDDs in RAID 10 will be 'good enough' and possibly less expensive than 2 SSDs? it's math plus requirements.
The price is only true if you are looking at OEM drives. Enterprise class SSD, and especially NVMe can be found relatively cheaply outside of the crazy OEM pricing.
Sure I suppose, but that often has it's own issues. If you're gong Tier one vendor (HPE/DELL) I've heard of a few failures... and you're toss a lot out the window with the Tier one hardware by not buying all their parts.
move to Tier 2, (Supermicro) then sure...
-
@Dashrender said in On prem Exchange hardware questions.:
135 GB of Exchange data is nothing Two 500 GB HDDs in RAID 1 will get you there, but will it have the performance you want, not likely, but two SSDs in RAID 10 will.
English much????
-
@scottalanmiller said in On prem Exchange hardware questions.:
What's driving them to only MS solutions?
Their primary software has important features that require it.
-
@scottalanmiller said in On prem Exchange hardware questions.:
SSDs... yes, unless you have a special case with extreme amounts of storage (like 30+ TB) combined with little need for performance, you should just use SSD and never have spinners even on the table.
Read Intensive? Write Intensive? or Mixed use?
-
@JasGot said in On prem Exchange hardware questions.:
@scottalanmiller said in On prem Exchange hardware questions.:
SSDs... yes, unless you have a special case with extreme amounts of storage (like 30+ TB) combined with little need for performance, you should just use SSD and never have spinners even on the table.
Read Intensive? Write Intensive? or Mixed use?
Write intensive, that is what Exchange does most of the time.
-
@JasGot said in On prem Exchange hardware questions.:
@scottalanmiller said in On prem Exchange hardware questions.:
SSDs... yes, unless you have a special case with extreme amounts of storage (like 30+ TB) combined with little need for performance, you should just use SSD and never have spinners even on the table.
Read Intensive? Write Intensive? or Mixed use?
None of those factor in for SSD vs. spinners really makes a difference. SSD is so much faster for both reads and writes that it doesn't really matter what your blend is. Only when you don't need any performance at all, then spinners aren't such a negative.
-
@JasGot said in On prem Exchange hardware questions.:
I need to provide an on prem exchange solution for a company with 100 users and about 135GB in mailbox database use.
I am interested if anything has changed in the last few years concerning Exchange on a VM and Spinning v. SSD drives?
If Exchange were going to be the ONLY VM on a host, would you still go VM?
Are SSDs overkill for Exchange?
I know there will be lots of questions, and thoughts, but I this is what is on my mind right now, so I thought it would be a good place to start.
I will also be proposing Microsoft 365, but I want to have a solid on prem plan if they choose to stay on prem.
After the big outage that still going to be a go as far as M365/O365?
Exchange via SPLA is a dollar or two a month per SAL. Cheap like Borscht.
100 users with a 135GB database is tiny.
Depending on user's reliance on search we would set up as follows:
- Virtual Machine with 4 vCPUs or 6 vCPUs depending on underlying setup.
- 24GB vRAM to 32GB vRAM to start
** Tuning for search post install - Exchange needs RAM not I/O - VHDX/VMDK 0: Operating System
- VHDX/VMDK 1: Exchange install
- VHDX/VMDK 2: Database(s)
- VHDX/VMDK 3: Logs
We install into Windows Server 2019 Core with the latest updates slipstreamed into the image.
So far, we've done quite well.
As an FYI: Almost all of our clients are on-premises for their workloads including Exchange.
EDIT: Which was great because we weren't being hit by a bus load of calls when O365/M365 went down.
-
@PhlipElder said in On prem Exchange hardware questions.:
After the big outage that still going to be a go as far as M365/O365?
That really wasn't a very long outage. Nothing compared to the normal outages from on-prem solutions. Par for the course outages are already assumed in any planning and that's all that the "big" outage a few days ago was. It's not outside of the standard operating of O365, everyone using O365 should have already been expecting something like that. And that's not condemning MS, it's just how it is. Hosted isn't perfect, and O365 isn't balanced for maximum uptime, that's not its goal. This wasn't a huge outage in time, nor were there a lot of other outages recently.
-
@PhlipElder said in On prem Exchange hardware questions.:
Exchange via SPLA is a dollar or two a month per SAL. Cheap like Borscht.
That's the licensing cost. That doesn't include the Windows cost, the hardware cost, the IT costs... that stuff all adds up. I'm not saying that on premises never makes sense, just that you have to compare apples to apples.
-
@scottalanmiller said in On prem Exchange hardware questions.:
@PhlipElder said in On prem Exchange hardware questions.:
After the big outage that still going to be a go as far as M365/O365?
That really wasn't a very long outage. Nothing compared to the normal outages from on-prem solutions. Par for the course outages are already assumed in any planning and that's all that the "big" outage a few days ago was. It's not outside of the standard operating of O365, everyone using O365 should have already been expecting something like that. And that's not condemning MS, it's just how it is. Hosted isn't perfect, and O365 isn't balanced for maximum uptime, that's not its goal. This wasn't a huge outage in time, nor were there a lot of other outages recently.
It was for those that depended on it.
Our on-premises solutions are 100% up-time with the exception of one due to environmental issues.
-
@scottalanmiller said in On prem Exchange hardware questions.:
@PhlipElder said in On prem Exchange hardware questions.:
Exchange via SPLA is a dollar or two a month per SAL. Cheap like Borscht.
That's the licensing cost. That doesn't include the Windows cost, the hardware cost, the IT costs... that stuff all adds up. I'm not saying that on premises never makes sense, just that you have to compare apples to apples.
We won a competition against cloud. Our solution set all-in was less than the O365 competition.
Cloud is never cheaper.
-
@PhlipElder said in On prem Exchange hardware questions.:
EDIT: Which was great because we weren't being hit by a bus load of calls when O365/M365 went down.
Sure, but that's like driving without a seatbelt in one car, and having someone in another car with a seatbelt. Then saying "ha, by being in the car without a seatbelt, we weren't hurt when the other car had an accident." It sounds like you are doing something safer, but you aren't, it's just presented in an emotionally misleading way.
The real advantage to lost of on-prem hosted systems is that outages tend to be temporally isolated - each outage has no connection to another. So you don't get swamped with outage calls all at once, even though your overall downtime is likely many, many times higher and requires way more engineering effort.
Supporting both, I know the difference is huge. On prem outages means we have to dedicate engineering time, generally billable, and do all kinds of customer management. O365 outages our service center can just point customers to the DownDetector page and explain that the service is down until MS corrects it. Even with loads of O365 customers calling in at once, it's less effort to deal with 100 customers on O365 during an outage than one on prem that we have to actually fix.
Again, on prem makes lots of sense at the right times. Just saying that presenting the recent outage as if it would affect the decision of any logical IT shop doing its evaluation properly is misleading. It's an emotional plea, but someone using proper risk assessment would understand that it's just part of any system and the fact that it was recent is not relevant and doesn't affect future risk assessment.
-
@PhlipElder said in On prem Exchange hardware questions.:
Our on-premises solutions are 100% up-time with the exception of one due to environmental issues.
No one is 100% uptime. All systems are 100% uptime in between outages. You might beat O365 in uptime, that's relatively easy to do, but you can't get to 100% uptime.
What I mean is.... I had a system that went just a few weeks short of ten years, zero downtime. It was as reliable as you could reasonably get. We decommissioned it without ever experiencing an outage. We had "observed 100% uptime", yes. But the system wasn't a 100% uptime system, we just got lucky that we had operated it successfully during an uptime window and shut it down before the downtime hit. But if we ran it again, our risk isn't zero. Maybe close to zero, definitely really low, but not zero.
-
@scottalanmiller said in On prem Exchange hardware questions.:
@PhlipElder said in On prem Exchange hardware questions.:
EDIT: Which was great because we weren't being hit by a bus load of calls when O365/M365 went down.
Sure, but that's like driving without a seatbelt in one car, and having someone in another car with a seatbelt. Then saying "ha, by being in the car without a seatbelt, we weren't hurt when the other car had an accident." It sounds like you are doing something safer, but you aren't, it's just presented in an emotionally misleading way.
The real advantage to lost of on-prem hosted systems is that outages tend to be temporally isolated - each outage has no connection to another. So you don't get swamped with outage calls all at once, even though your overall downtime is likely many, many times higher and requires way more engineering effort.
Supporting both, I know the difference is huge. On prem outages means we have to dedicate engineering time, generally billable, and do all kinds of customer management. O365 outages our service center can just point customers to the DownDetector page and explain that the service is down until MS corrects it. Even with loads of O365 customers calling in at once, it's less effort to deal with 100 customers on O365 during an outage than one on prem that we have to actually fix.
Again, on prem makes lots of sense at the right times. Just saying that presenting the recent outage as if it would affect the decision of any logical IT shop doing its evaluation properly is misleading. It's an emotional plea, but someone using proper risk assessment would understand that it's just part of any system and the fact that it was recent is not relevant and doesn't affect future risk assessment.
Not going down this road with you again Scott.
TTFN
-
@PhlipElder said in On prem Exchange hardware questions.:
@scottalanmiller said in On prem Exchange hardware questions.:
@PhlipElder said in On prem Exchange hardware questions.:
Exchange via SPLA is a dollar or two a month per SAL. Cheap like Borscht.
That's the licensing cost. That doesn't include the Windows cost, the hardware cost, the IT costs... that stuff all adds up. I'm not saying that on premises never makes sense, just that you have to compare apples to apples.
We won a competition against cloud. Our solution set all-in was less than the O365 competition.
Cloud is never cheaper.
I agree that cloud is almost never cheaper, but that's dependent on the specific cloud and products. In a situation like this, both sides are primarily just a determination by Microsoft as to how much either will cost and the cost of the cloud solution and the cost of the on prem aren't artefacts of cloud or on prem, but rather artefacts of the pricing model.
I know lots of on prem that is way more expensive than cloud. Like dramatically so. Now, your argument is that it shouldn't be because they are doing it wrong. Sure, and that's valid. But you can't always make the same argument about the cloud side, too, that it costs too much only because you are picking the wrong one.
In your pricing case, you are providing a flat price that includes everything with power, HVAC, all support, licensing, hardware, software, virtualization, backups, etc. and coming out below $4/mailbox for the installation over time?
-
@PhlipElder said in On prem Exchange hardware questions.:
@scottalanmiller said in On prem Exchange hardware questions.:
@PhlipElder said in On prem Exchange hardware questions.:
EDIT: Which was great because we weren't being hit by a bus load of calls when O365/M365 went down.
Sure, but that's like driving without a seatbelt in one car, and having someone in another car with a seatbelt. Then saying "ha, by being in the car without a seatbelt, we weren't hurt when the other car had an accident." It sounds like you are doing something safer, but you aren't, it's just presented in an emotionally misleading way.
The real advantage to lost of on-prem hosted systems is that outages tend to be temporally isolated - each outage has no connection to another. So you don't get swamped with outage calls all at once, even though your overall downtime is likely many, many times higher and requires way more engineering effort.
Supporting both, I know the difference is huge. On prem outages means we have to dedicate engineering time, generally billable, and do all kinds of customer management. O365 outages our service center can just point customers to the DownDetector page and explain that the service is down until MS corrects it. Even with loads of O365 customers calling in at once, it's less effort to deal with 100 customers on O365 during an outage than one on prem that we have to actually fix.
Again, on prem makes lots of sense at the right times. Just saying that presenting the recent outage as if it would affect the decision of any logical IT shop doing its evaluation properly is misleading. It's an emotional plea, but someone using proper risk assessment would understand that it's just part of any system and the fact that it was recent is not relevant and doesn't affect future risk assessment.
Not going down this road with you again Scott.
TTFN
You went down the road of making the statement. We can't let an emotional plea go as if it was a true statement. You can't decide not to go down the road, you are the one that went down it and force us to have to explain to readers why it's an emotional plea. Don't make emotional pleas if you don't want us pointing them out.
-
@PhlipElder said in On prem Exchange hardware questions.:
So far, we've done quite well.
This is the problem with all on prem solutions.... measuring it. The reality is you can deploy almost any solution, in almost any way, and run quite well, for quite some time. The small risk pool for analysis is terribly small so it is super easy to get lucky and have few, or even no, outages.
Compare it to using no RAID. Just a single hard drive, no backups. Sounds crazy, right? But now let's deploy servers this way. Let's say a dozen of them. Let's run for four years like this. Assuming no one drops a server or gets water on it... chances are quite likely that we will see zero data loss. None at all.
On one hand, we all know that the setup with no RAID and no backups is insanely risky. But if we do the analysis, we also know that someone using anecdotal evidence is likely to be able to say that data protection was 100%. Both things are true... it was observed to be totally reliable, and the risk is insanely high.
Things like O365 feel like they have way more downtime because the downtime is far more visible and nearly every customer experiences some, sometimes. But what isn't as visible is how often others have it, and how big their outages are.
It's like T1 vs. cable. Cable tends to fail a little all the time. T1s tend to fail dramatically, once in a great while. Overall, I've never seen any T1 come close to the worst cable in terms of reliability, yet everyone presents T1s as being super reliable. Why? Because they can select observation windows that are 100% up quite easily. It's easy to miss outages that can last days or even months. Yet, try to find a T1 customer that hasn't had an outage of more than a week, it's actually hard.
Now, of course, you can invest more, design a great environment, and potentially get on premises to be better in reliability than O365. It's not really all that hard. But the average on prem Exchange is not going to beat it, not over the long haul whether because of networking, software, bugs, or whatever. But likely any individual on prem outage will be less frequent, but longer in duration. That's just the nature of how these kinds of outages work.
-
@dbeato said in On prem Exchange hardware questions.:
@JasGot said in On prem Exchange hardware questions.:
@scottalanmiller said in On prem Exchange hardware questions.:
SSDs... yes, unless you have a special case with extreme amounts of storage (like 30+ TB) combined with little need for performance, you should just use SSD and never have spinners even on the table.
Read Intensive? Write Intensive? or Mixed use?
Write intensive, that is what Exchange does most of the time.
No, read intensive is what you want. It's Dell's description of the write endurance of the drive, not the workload per se.
Read intensive SSDs normally have about 1 DWPD for 5 years. DWPD is drive writes per day so if the drive is 480GB you can write 480GB of data each day for 5 years straight.
Put another way, you can write 876 TB of data to the drive during it's 5 year lifespan.
Mixed use is usually 3 DWPD and write intensive 5-10 DWPD.
A mail server will never need anything more than read intensive drives.
2 drives in RAID1 will do the job in this case - without breaking a sweat. -
@Dashrender said in On prem Exchange hardware questions.:
@scottalanmiller said in On prem Exchange hardware questions.:
@JasGot said in On prem Exchange hardware questions.:
Are SSDs overkill for Exchange?
SSDs aren't overkill for a desktop. Unless you've got an extreme edge case, spinning drives haven't had a place in servers in several years now.
I want to agree with this, but the prices are still kinda high for enterprise drives... you have to look at the financial aspect in my opinion... it's quite possible that SSDs will be the way to go.
I.e. HDD = RAID 10 or RAID 6 (min)
135 GB of Exchange data is nothing Two 500 GB HDDs in RAID 1 will get you there, but will it have the performance you want, not likely, but two SSDs in RAID 10 will.
But perhaps 4 or 6 HDDs in RAID 10 will be 'good enough' and possibly less expensive than 2 SSDs? it's math plus requirements.
Enterprise 2.5" drives are not cheap either.
A 600GB 2.5" 10K drive is slightly more expensive than a 480GB enterprise SSD SATA drive. And the SSD will of course outperform the 10K drive easily.
If Dell has it the other way around they are trying to get rid of old stock or something.
