Azure Outage... Again
-
@dafyre said in Azure Outage... Again:
What about the AzureAD? Is that down as well?
Don't know. Not likely as that is handled by a different functional system.
-
@scottalanmiller Then why haven't you migrated already?
-
@aaronstuder said in Azure Outage... Again:
I just created a VM - no problem seen here....
What we've learned from rooms full of people who see Azure outages all the time is that nearly all outages are very localized. They seem to have their systems set up in such a way that even platform level issues only appear to certain blocks of people. Probably people running on different servers or whatever (not for the VMs, obviously, but for the console.) This lets MS claim zero outages even in the face of hundreds of pissed off customers in a single room talking about how they often have outages.
-
@aaronstuder said in Azure Outage... Again:
@scottalanmiller Then why haven't you migrated already?
Did you miss that we aren't the decision makers?
-
@scottalanmiller sounds like it's time to fire the client...
-
@aaronstuder said in Azure Outage... Again:
@scottalanmiller sounds like it's time to fire the client...
That seems a bit drastic. They aren't complaining about it, they just let us know and we let MS know and MS says that they will look at the ticket in eight hours. Based on what factor would you fire the client? Would you also fire every client that uses Windows on the same logic, for example? What's the reason for firing the client?
-
@scottalanmiller said in Azure Outage... Again:
@aaronstuder said in Azure Outage... Again:
I just created a VM - no problem seen here....
What we've learned from rooms full of people who see Azure outages all the time is that nearly all outages are very localized. They seem to have their systems set up in such a way that even platform level issues only appear to certain blocks of people. Probably people running on different servers or whatever (not for the VMs, obviously, but for the console.) This lets MS claim zero outages even in the face of hundreds of pissed off customers in a single room talking about how they often have outages.
Literally paying for a service that brings your entire company (more or less) to a standstill. I hope the client isn't a production company.
-
@aaronstuder said in Azure Outage... Again:
I just created a VM - no problem seen here....
Ah - but which site are you running from?
-
@gjacobse East US
-
@aaronstuder said in Azure Outage... Again:
@gjacobse East US
How can you tell? I would guess that that is the one that we are on too. I know what datacenters the VMs were in, but not which datacenter is providing the console.
-
We can't even open a ticket about our VMs being down. MS Support is offline for us, too. Their outage with Azure goes so deep in their infrastructure that it disables their technical problem reporting system. So we can't file an outage, because Azure is down that hard (for us.)
This is how most Azure outages have been, both for us and for people that we have spoken to about it. This seems to be very common, they don't just lose the VMs, they lose everything including all visibility and communications channels.
-
@aaronstuder said in Azure Outage... Again:
@scottalanmiller said in Azure Outage... Again:
Microsoft says: That they will respond to the outage in eight hours.
Eight hours for Azure being 100% down (within our visible scope?) That's a pretty awful SLA for just checking their tickets to see that their platform has gone offline (within some scope.)
For all Internet facing Virtual Machines that have two or more instances deployed in the same Availability Set, we guarantee you will have external connectivity at least 99.95% of the time. - http://uptime.is/99.95
Problem there is that MS lies through their teeth about not having outages and refuses to acknowledge them. And so far, almost all outages that we've seen remove:
- Ability to open tickets
- Ability to report outages
- Ability to track downtime
- All Availability Sets and Failover
So those SLAs are totally fake. They don't even have systems for working with them. Paying for extra support would be crazy as they don't respect the support systems that they have.
-
@scottalanmiller How you tried from the US?
-
@aaronstuder said in Azure Outage... Again:
@scottalanmiller How you tried from the US?
Yup, NY and KY.
-
And the client sees it down from PA and MD.
-
This is what we are showing.. and we have I believe four systems running under Azure..
-
Way more than four.
-
What the status of your subscription?
-
@aaronstuder said in Azure Outage... Again:
What the status of your subscription?
Can't check on it. The outage has taken out the system that shows it.
-
Which is what we see with most outages... they lose some core database that reports subscriptions, this cascades to the console and on to the VMs. It's, and this is just me guessing, probably a database instance that handles the subscription data or some data that builds the subscription that has failed and then all of the other outages are likely from dependencies on that system. We've see that or almost exactly that a few times and tons of other companies (hundreds) that we have interfaced with (mostly via MS conferences) have reported the exact same problem as what they see most often.
