Burned by Eschewing Best Practices
-
@Dashrender said:
I'm mostly going with @dafyre definition of an outage.
Sure, if my building has no power, we have a technical Exchange outage. But, do I care? Nope! why not? because my whole business is down.
So while in the truest sense of the term, yes it's and Exchange outage, but I don't count it against myself or Exchange because the rest of my business is down as well, and I can't work. Furthermore, unlike other businesses, we can easily live without email for 2-3 days if we had to, so it's even that much less of an issue for us personally - And this may also be tenable for most small businesses as well.
Now that said, we don't appear down to the outside world. We are using a hosted Spam/AV scanning solution (AppRiver). The email flows to them and they hold it until we come back online. The outside world never notices a thing.
But that's a problem that you are applying different criteria to the same service. If YOUR ISP goes down, an in house Exchange system is useless - no email can flow. If YOUR ISP goes down and you have hosted Exchange, Exchange is still up and usable, you have simple workarounds and the company can choose to keep using the service.
But even though the outage is with you, not Exchange, we generally look at the opposite which is very misleading, I feel.
-
@Dashrender said:
The only possible savings I could possibly see is if they would have only used one server instead of two. then all of the disk would be local to that one host, and they wouldn't have spent money on a second host or the NAS - is that what you mean?
Right, they bought three servers and got less value than buying only one. Now you couldn't quite get all of the cost gone from the three so you can't realistically save 67% of the cost. But 50-60% is easy. Two of the three devices were useless and actively hurt them by increasing risk, power consumption, IT effort, cabling, HVAC, space, etc.
So not only did the project fail to deliver on goals (HA and general protection) it also did not do so in a cost effective way. It lost a huge amount of money while additionally not successfully meeting goals.
-
Ok.. all of that makes sense - I'm wondering why you haven't brought that up in his thread? Instead he was more or less punted to Scale or XenServer with HA-Lizard (Hyper-v while mentioned was quickly dismissed it appeared).
I did this morning ask the question, why not just go to one single super awesome box... the answer 'HA.'
-
If you hire someone to get you a cost effective and simple means to move thousands of moving boxes from Philly to Seattle and you expect a large moving truck and he returns with a thousand Ferraris, was he successful?
The goal was to be cost effective. And to be simple.
Anyone can move boxes, we assume that the ability to actually move the boxes is below the line of success. Success is not in moving the boxes, because it's so simple that it is assumed success. What we needed from someone giving advice is cost effectiveness and simplicity.
Is driving a thousand Ferraris, each with one moving box strapped into the passenger seat, simpler than having a single box truck take all of the boxes at once in a single load? Is it more cost effective?
At what point is this a failure or a success? I would deem this a failure - this is about the biggest waste of resources in cost while not providing a superior solution. It's all bad. Each piece - simplicity, speed, cost are all below a reasonable baseline.
-
@Dashrender said:
I did this morning ask the question, why not just go to one single super awesome box... the answer 'HA.'
Pretty sure that HA was listed as a requirement. We definitely covered what I said above several times in stating that the NAS was not delivering any benefit and not meeting goals which is all it takes to get all that is stated above. Once the NAS is known to not have delivered HA, we know that all of the money around that was wasted.
-
@scottalanmiller said:
@Dashrender said:
I'm mostly going with @dafyre definition of an outage.
Sure, if my building has no power, we have a technical Exchange outage. But, do I care? Nope! why not? because my whole business is down.
So while in the truest sense of the term, yes it's and Exchange outage, but I don't count it against myself or Exchange because the rest of my business is down as well, and I can't work. Furthermore, unlike other businesses, we can easily live without email for 2-3 days if we had to, so it's even that much less of an issue for us personally - And this may also be tenable for most small businesses as well.
Now that said, we don't appear down to the outside world. We are using a hosted Spam/AV scanning solution (AppRiver). The email flows to them and they hold it until we come back online. The outside world never notices a thing.
But that's a problem that you are applying different criteria to the same service. If YOUR ISP goes down, an in house Exchange system is useless - no email can flow. If YOUR ISP goes down and you have hosted Exchange, Exchange is still up and usable, you have simple workarounds and the company can choose to keep using the service.
But even though the outage is with you, not Exchange, we generally look at the opposite which is very misleading, I feel.
I think a big difference in the comparisons comes from what you can see versus what you can't. If you're just an O365 user, you can't see the difference between an ISP outage/Azure outage/Exchange outage. To you, it's just an outage.
When you control all the pieces, local Exchange, ISP, Firewall, etc - you're able to more specifically understand the failure.
Plus with local services, assuming the main service itself or the local network itself is up, you can often get local access. That's never the case with hosted options. If the hosted solution is having an outage, it's because something is preventing the whole thing from working, even if it's only the ISP is down, the fact it no one can use it because everyone/everything is on the outside.
-
@Dashrender said:
I think a big difference in the comparisons comes from what you can see versus what you can't. If you're just an O365 user, you can't see the difference between an ISP outage/Azure outage/Exchange outage. To you, it's just an outage.
There is a reason that I don't like this, though. And that is that we define "Exchange is down" based on factors unrelated to Exchange. See the problem?
How often is Exchange expected to fail? Well that depends on the price of raisins on Tuesday. Huh?
Having Exchange deemed "up or down" based on factors unrelated to Exchange or the service seems like a really bad, and totally useless, idea. If we start doing that, why use the term outage at all because what value does it have for us?
-
@scottalanmiller said:
@Dashrender said:
I did this morning ask the question, why not just go to one single super awesome box... the answer 'HA.'
Pretty sure that HA was listed as a requirement. We definitely covered what I said above several times in stating that the NAS was not delivering any benefit and not meeting goals which is all it takes to get all that is stated above. Once the NAS is known to not have delivered HA, we know that all of the money around that was wasted.
The Problem that most people have with @scottalanmiller 's definition of HA, is that he uses a much more deeply defined idea of HA than what most folks do. Most people really mean Redundancy when they say HA.... To answer the question of "Will it stay up if Server A explodes?"
If their response is, "Yes, it will stay up, because it will automatically fail over with little-to-no downtime to Server B"
Then a lot of people think they have High Availability, when what they really have is Redundancy.
-
@Dashrender said:
When you control all the pieces, local Exchange, ISP, Firewall, etc - you're able to more specifically understand the failure.
True, it assists in placing blame.
-
@scottalanmiller said:
@Dashrender said:
When you control all the pieces, local Exchange, ISP, Firewall, etc - you're able to more specifically understand the failure.
True, it assists in placing blame.
Not my fault!
-
@dafyre said:
@scottalanmiller said:
@Dashrender said:
I did this morning ask the question, why not just go to one single super awesome box... the answer 'HA.'
Pretty sure that HA was listed as a requirement. We definitely covered what I said above several times in stating that the NAS was not delivering any benefit and not meeting goals which is all it takes to get all that is stated above. Once the NAS is known to not have delivered HA, we know that all of the money around that was wasted.
The Problem that most people have with @scottalanmiller 's definition of HA, is that he uses a much more deeply defined idea of HA than what most folks do. Most people really mean Redundancy when they say HA.... To answer the question of "Will it stay up if Server A explodes?"
If their response is, "Yes, it will stay up, because it will automatically fail over with little-to-no downtime to Server B"
Then a lot of people think they have High Availability, when what they really have is Redundancy.
Even in the way that they use it, though, it's all about how you ask the question. They often state it as "if the server fails" but if you ask them "is it about keeping the services online" they would answer "yes." They all know what HA is, but they can't figure out how to get to it.
-
I think that it's more useful for even small shops to know that they have a "loss of Internet access" rather than "Exchange is down." Because in one, it makes it sound like Exchange has failed. Yes the service might not be available to the end users but it is still working to customers. Is that an outage? Is acting like it is an outage useful?
If Exchange is not accessible to end users, even if they can't tell that it is still up to others, it's far more useful to define an outage by what is actually down rather than what can't be accessed.
-
So with Exchange, you can kind of get this "but our users can't use it" perspective. Let's change that up and say your company website. If your ISP goes down and internal users cannot access the company website, do you say that your website is down or offline or has an outage? Of course not. If your customers looks, the site is up and normal. We all know that calling that an outage would be ridiculous and just being obnoxious.
But how would hosted web and hosted Exchange really be different in this case? What makes it okay to call Exchange down even when it is sending and receiving emails from the outside and not okay to call the website down under identical circumstances?
-
@scottalanmiller said:
@Dashrender said:
I did this morning ask the question, why not just go to one single super awesome box... the answer 'HA.'
Pretty sure that HA was listed as a requirement. We definitely covered what I said above several times in stating that the NAS was not delivering any benefit and not meeting goals which is all it takes to get all that is stated above. Once the NAS is known to not have delivered HA, we know that all of the money around that was wasted.
Sure you said that the NAS didn't provide HA, but you didn't spell out why - most of us, myself included when I go back 3 or so years would never read the reasons you say NAS didn't get us HA and poof we'd understand the rest automatically. You had to write that stuff out 3-5 times and me reading it before I really started to understand the situation.
-
In general, although I assume that there are exceptions, when a bunch of stuff is bought for one purpose and actually provide less of that purpose than if they had not been bought, all of the associated money was wasted
-
@scottalanmiller Again, that makes sense, but most of use don't see that just because you tell us the first part. We need time to get up to your speed.
In the case of the person in question, they have stated that they haven't been in IT all that long.
-
@scottalanmiller said:
So with Exchange, you can kind of get this "but our users can't use it" perspective. Let's change that up and say your company website. If your ISP goes down and internal users cannot access the company website, do you say that your website is down or offline or has an outage? Of course not. If your customers looks, the site is up and normal. We all know that calling that an outage would be ridiculous and just being obnoxious.
But how would hosted web and hosted Exchange really be different in this case? What makes it okay to call Exchange down even when it is sending and receiving emails from the outside and not okay to call the website down under identical circumstances?
You've lost me here.
If hosted Exchange is down then it's down. If my local ISP is down, I would never tell me users we were having an Exchange outage. -
@Dashrender said:
@scottalanmiller said:
So with Exchange, you can kind of get this "but our users can't use it" perspective. Let's change that up and say your company website. If your ISP goes down and internal users cannot access the company website, do you say that your website is down or offline or has an outage? Of course not. If your customers looks, the site is up and normal. We all know that calling that an outage would be ridiculous and just being obnoxious.
But how would hosted web and hosted Exchange really be different in this case? What makes it okay to call Exchange down even when it is sending and receiving emails from the outside and not okay to call the website down under identical circumstances?
You've lost me here.
If hosted Exchange is down then it's down. If my local ISP is down, I would never tell me users we were having an Exchange outage.But the whole question is... how do we define that the hosted Exchange is down?
-
@scottalanmiller said:
@Dashrender said:
@scottalanmiller said:
So with Exchange, you can kind of get this "but our users can't use it" perspective. Let's change that up and say your company website. If your ISP goes down and internal users cannot access the company website, do you say that your website is down or offline or has an outage? Of course not. If your customers looks, the site is up and normal. We all know that calling that an outage would be ridiculous and just being obnoxious.
But how would hosted web and hosted Exchange really be different in this case? What makes it okay to call Exchange down even when it is sending and receiving emails from the outside and not okay to call the website down under identical circumstances?
You've lost me here.
If hosted Exchange is down then it's down. If my local ISP is down, I would never tell me users we were having an Exchange outage.But the whole question is... how do we define that the hosted Exchange is down?
How do I define it? how do 'we' define it?
Great question - I'm guessing that MS doesn't tell us the lowly users where the actual fault is, therefore the only thing we can say is... it's an Exchange outage if we can't use the service.
IF MS says.. hey our ISP is out, but Exchange itself is fine... OK great, I'm personally no longer pissed at MS (other than why don't you have 2+ ISPs with auto failover), I'm now pissed at their ISP.
Same goes if the outage is caused by Azure... I'm happy to put the blame where it belongs.. but rarely do cloud service providers tell us that information.
-
@Dashrender said:
Great question - I'm guessing that MS doesn't tell us the lowly users where the actual fault is, therefore the only thing we can say is... it's an Exchange outage if we can't use the service.
That's the issue, though, do we define it as "when Exchange is down"? Or "when the vendor encapsulated service is unavailable to everyone?" Or "when the vendor service is unavailable to an account, region, country, product?" Or "when it is down in a specific place?"
To me, you can call it an Office 365 outage when the customer does not have an option of a workaround. Which is why our Azure outage, to me, was as clear as any outage can be to being an outage. The fault was Azure, Azure was unavailable to people with whom there was on possible workaround. Only the Azure support team had any ability to bring it back up.
