Exchange 365 Down?
-
Internal domain email works fine, I've received several emails from staff.
Something is up... Maybe it's just "us"
-
@DustinB3403 said:
Internal domain email works fine, I've received several emails from staff.
Something is up... Maybe it's just "us"
multiple people have told you there was a warning in the admin console. How the f[moderated] does that equate to "just us"?
-
Not that issue, I have that here as well.
I was specifically referring to our inability to send or receive emails from off of our domain.
How would you come to that conclusion from my reply @JaredBusch ?
-
@JaredBusch said:
@DustinB3403 said:
My org is unable to send and receive emails to / from the outside.
Anyone else having issues?
When I was in the admin portal earlier, it stated there was an Exchange issue. I did not look into it.
And we've been seeing portal issues, and issues with Azure VMs. So I'm guessing a networking issue or something else generic.
-
Without wishing to take the thread off-topic, it feels like about 10% of the threads on ML relate to problems with Microsoft cloud services. At the same time, I get called an idiot for still running on-premise Exchange and Sharepoint (which to date have chugged away in the corner with zero downtime and minimal maintenance).
It's all anecdotal, but my confidence in Microsoft keeps getting eroded and I worry that their infrastructure is still too immature to be the obvious solution versus continuing with on-premise for another few years (ie another major Exchange/Sharepoint on-premise upgrade will keep us sweet for another 5 years).
-
@Carnival-Boy said:
Without wishing to take the thread off-topic, it feels like about 10% of the threads on ML relate to problems with Microsoft cloud services. At the same time, I get called an idiot for still running on-premise Exchange and Sharepoint (which to date have chugged away in the corner with zero downtime and minimal maintenance).
I feel the "every time we mention problems, someone is going to mention this" concern with every thread as well.
Here is my take on it... the issues with O365 services are rare and often very isolated. They do have problems, no question. But, for example, the issues listed here don't affect most of us and email is only partially impacted for the few that are hit. It is hard to quantify what the impact is - how many affected, how badly, etc. You still have to look at the cost, the overall impact, the services, etc. Yes, your Exchange has a good track record as do my servers (generally a decade or more without failures) but it is about average risk. If you look at big businesses running their own Exchange or the average across SMBs, the outages are far higher and far more costly. That's why the push for hosted - support, cost and risk.
Azure, on the other hand, I am not going to say is bad but I certainly do not recommend. It is not on par at all with its competitors. We've found it to be more costly by a little but unsatisfactorily unstable and poorly maintained and, the worst part, is that MS is dishonest about it. @Minion-Queen recently attended a conference where MS touted uptime and the majority of the crowd complained about massive Azure failures - clearly uptime wasn't even something they could bluff about. Now no one expects perfects, but Azure is really bad in comparison to whom they are up against.
Azure does have some nice and unique services, it's not all bad. We are certainly leaving them and nothing new goes there. We had one engineer that pushed hard and sneaked a lot of services there without oversight and we are paying for that (literally and figuratively) now. But having been a customer of at least six or seven major cloud providers, none is concerning and disappointing like Azure is.
-
To be fair, if Azure was the only game in town we'd be raving about how well it works. It is because Amazon, Rackspace, Digital Ocean and the like are so good that it makes Azure seem silly.
-
We got the issue resolved.
The low-down.
We have Exchange 2010 in a Hybrid environment. AD onsite and MS manages Exchange.
In the EMC our Office 365 Exchange management was disconnected, we could no longer manage Exchange Online from our on premise server.
Submitted a ticket into MS to determine why, and the technician recommends running the Hybrid configuration wizard and confirms the details of what needs to be in the "Wizard".
What this Wizard does is put a new connector onto your Office365 Exchange account, which in our case change the route for our email to flow.
Rather than straight from us to MS to the outside, it was going from Us to MS back to Us and then attempting to go out.
A second ticket with Microsoft reporting the outage found the issue after about 4 hours of downtime, and the MS team even admitting that the tech who was trying to help fix the initial issue needs further training.
Summary, MS updated something on O365 Exchange that is no longer compatible with Exchange 2010 on premise, using the wizard broke our mail flow connector. I not being an exchange admin (nor office 365 admin) would have known where this issue was or how to fix it.
-
Interesting, so in this case, while an MS tech was involved (we've had a lot of issues tied to MS support screwing things up) the issue was on premises Exchange causing the outage, not Office 365? So this actually supports why hosted email is important rather than the opposite.
And supports what we keep saying that SMBs rarely have highly trained Exchange admins, you need to be doing Exchange all day, every day to really be good at it.
-
Our on premise server only acts as an Active Directory relay for our User / Room accounts that need email.
Being in a hybrid environment is what messed this up, and the tech not knowing what the hell he was doing.
-
@DustinB3403 said:
Being in a hybrid environment is what messed this up, and the tech not knowing what the hell he was doing.
That was my point, it's the on premises stuff that created the risk.
-
@scottalanmiller You could also look at it as 365 was the problem too
That being said, I will be migrating from on-premise to 365 within the next year.
-
@wrx7m said:
@scottalanmiller You could also look at it as 365 was the problem too
In what way could you, though?
-
@scottalanmiller If he didn't have 365, the problem wouldn't have existed because MS introduced a change to the 365 side that broke his hybrid setup. Either, full hosted or full on-premise would not have been impacted.
-
@wrx7m said:
@scottalanmiller If he didn't have 365, the problem wouldn't have existed because MS introduced a change to the 365 side that broke his hybrid setup. Either, full hosted or full on-premise would not have been impacted.
Sort of, if he didn't have O365 he'd have no email at all
That the change was made on the O365 side doesn't mean that the break wasn't on the local side, though. No one said it was a mistake on the hosted side. -
@DustinB3403 said:
Summary,** MS updated something on O365 Exchange **that is no longer compatible with Exchange 2010 on premise, using the wizard broke our mail flow connector. I not being an exchange admin (nor office 365 admin) would have known where this issue was or how to fix it.
-
@wrx7m said:
@DustinB3403 said:
Summary, MS updated something on O365 Exchange that is no longer compatible with Exchange 2010 on premise, using the wizard broke our mail flow connector. I not being an exchange admin (nor office 365 admin) would have known where this issue was or how to fix it.
Exactly.... using the wizard broke our mail flow connector. The on site wizard broke it, not the change to O365.
-
@scottalanmiller Ha! But first MS updated something on O365 Exchange that was incompatible with Exchange 2010. It was then followed by the tech running the wizard, which broke the on-premise side. Insult to injury performed by MS.
-
@wrx7m said:
@scottalanmiller Ha! But first MS updated something on O365 Exchange that was incompatible with Exchange 2010.
You have no proof of that, because the correct troubleshooting steps were not taken. It could easily have been restarting the services or properly reauthenticating. We will never know because the tech broke it immediately.
-
@JaredBusch said:
@wrx7m said:
@scottalanmiller Ha! But first MS updated something on O365 Exchange that was incompatible with Exchange 2010.
You have no proof of that, because the correct troubleshooting steps were not taken. It could easily have been restarting the services or properly reauthenticating. We will never know because the tech broke it immediately.
Right, the only thing that we are nearly sure of is that the tech broke something on the local side. Almost totally certain that this is all MS's fault based on what we know, but the fault of MS support (which we've had terrible experiences with in general ourselves so this follows on what we've seen first hand, but that's pretty anecdotal) but nothing here suggests any reason to think that the issue is in any way on the hosted side. Is there the possibility that it is? Yes, it is possible. Do we have any indicator to make us pin any blame there? No, not without more information and forensics. All of the known issues are on the local side (and caused by an MS tech working locally.)
