Massive Azure Outage and No Support
-
At three this morning, Microsoft arbitrarily decided to turn off all of our Azure resources. No billing issues, no problems, no communications.... just turned them off. Everywhere, all regions. Everything failed. Multi-region protection failed, everything. Backups gone, everything. Every feature that Microsoft touts as protecting you on Azure - failed.
Seven hours later, we are still struggling just to get someone who knows what Azure is (no kidding!!) on the phone. We've had a Microsoft concierge on with us for a while and even he cannot get anything approximating support.
This is an epic outage. This isn't a datacenter blip that MS is working on. This is a massive, total outage for us as a customer that no one is working to fix. We can't sit back and wait for it to get resolved, we can't even find a way to get someone to understand that we are down.
This is the biggest lack of competence and of support that I have ever seen from a vendor like this. This is absolutely ridiculous. We have no idea how to get someone to provide support or get this working. Apparently Azure is too fragile and MS has lost the ability to provide support to use. Even with Rackspace's support issues, things like this never happen.
We are getting to wits' end here. Has Microsoft abandoned Azure? Why is there no support channel for it? Why is anyone allowed to just shut off systems?
-
It's not just you who is down, right?
-
Holy sh*t snacks, that is seriously bad news.... I guess they're all too busy going to Spiceworld this week to remember to make Azure work.
-
Eight hours, still down.
-
@Dashrender said:
It's not just you who is down, right?
Just us. 100% down. Epic Azure failure. Took out everything in every region associated with us.
-
@RojoLoco said:
Holy sh*t snacks, that is seriously bad news.... I guess they're all too busy going to Spiceworld this week to remember to make Azure work.
They didn't even remember to leave a support department behind!
-
@scottalanmiller said:
@Dashrender said:
It's not just you who is down, right?
Just us. 100% down. Epic Azure failure. Took out everything in every region associated with us.
Sounds like they just took your account and switched it off. Does this include Office365? Or is that not hosted on Azure?
-
@scottalanmiller said:
@Dashrender said:
It's not just you who is down, right?
Just us. 100% down. Epic Azure failure. Took out everything in every region associated with us.
OK.. that explains why you're hounding on it so.. and rightly so - since it's only you affected.
-
@coliver said:
@scottalanmiller said:
@Dashrender said:
It's not just you who is down, right?
Just us. 100% down. Epic Azure failure. Took out everything in every region associated with us.
Sounds like they just took your account and switched it off. Does this include Office365? Or is that not hosted on Azure?
They did, but only Azure. O365 is fine. They made a mistake and gave us three accounts, didn't merge them like they were supposed to, shut off one or two of them so our account is still up and fine, but all of the services on it are gone. We never made more than one account. It's total MS screw up end to end.
Same issues caused an O365 outage two weeks ago and email data loss. I still am missing most of my email from the last two years.
-
So our account is on and "fine". But they associated our Azure stuff with another account that they turned off. So it gets really complicated. The core issue is that they associated our Azure systems with the wrong account.
-
@scottalanmiller said:
So our account is on and "fine". But they associated our Azure stuff with another account that they turned off. So it gets really complicated. The core issue is that they associated our Azure systems with the wrong account.
Oh man... that isn't good.
-
Wow ,..... Just,....wow.
-
What's worse is I've been warning that they were going to screw this up for months. For about six months we've been trying to get them to fix their account issues because we knew that they were getting this all messed up and were going to do something stupid. Their account managers kept saying that everything was "fixed" or everything was "fine."
Now we've been down for over eight hours and they refused to even talk to the technical people until the account was fixed. The account that was "fixed" for the last six months?
I am so not happy. That they didn't fix the account before this when I've been warning and warning them that this was risky makes this impossible to be an accident. They decided to play fast and loose with our account because they were lazy or didn't care. Everyone who works there leaves so quickly they probably thought (and were probably right) that it would be "someone else's problem" when it fell apart.
-
Did you have some customers hosted on Azure or was this exclusively NTG stuff?
-
@coliver said:
Did you have some customers hosted on Azure or was this exclusively NTG stuff?
Customers too.
-
@scottalanmiller said:
@coliver said:
Did you have some customers hosted on Azure or was this exclusively NTG stuff?
Customers too.
That really isn't good.
-
I'm told that they have decided that it is okay to turn them back on now. We will see.
-
@scottalanmiller said:
I'm told that they have decided that it is okay to turn them back on now. We will see.
Waiting,....with baited breath.
-
Stuff like this makes me glad we have (most) of our junk internal. So much trust to put in another company!
Edit: advantages to cloud are obvious, don't get me wrong.
-
I've run into the multiple accounts on a vendors backend problem before.
Hell Disney has this problem too. If you register at their streaming movies website it never asks you to create a username, of course this leads you to the assumption that your email address is your username. But if you setup an account on store.disney.com they ask for a user or an email address.
This combined with the fact that their authentication process actually all goes through GO.COM left me without access to my store account for 4 days. Additionally since I use LastPass, Lastpass wasn't able to log me in correctly because of the whole damned GO.COM problem, because not all logon prompts provide the correct information as to where the credentials are going, so LP didn't offer me the needed information.
