At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange
-
@Dashrender said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@Obsolesce said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@JaredBusch said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@Obsolesce said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
Wow that's a lot of companies. If they aren't keeping Exchange Server updated of all things, image all the other software they are likely running out of date too!!!
This hack had nothing to do with servers up-to-date. It was a zero day. There was no patch prior to March 2.
It said thousands of servers are still being compromised daily since the patch was released. I imagined that companies not patching something as serious as this likely have so much else not patched, because of either not caring or lack of awareness of this kind of thing.
All the more reason to move to hosted/cloud services.
The fact that an on-premise solution had a 0-day vulnerability doesn't mean that hosted/cloud services don't have unknown issues of their own.
While I get what you're saying the platform type is unrelated to the discovery of a 0-day vulnerability.
-
@DustinB3403 said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@Dashrender said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@Obsolesce said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@JaredBusch said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@Obsolesce said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
Wow that's a lot of companies. If they aren't keeping Exchange Server updated of all things, image all the other software they are likely running out of date too!!!
This hack had nothing to do with servers up-to-date. It was a zero day. There was no patch prior to March 2.
It said thousands of servers are still being compromised daily since the patch was released. I imagined that companies not patching something as serious as this likely have so much else not patched, because of either not caring or lack of awareness of this kind of thing.
All the more reason to move to hosted/cloud services.
The fact that an on-premise solution had a 0-day vulnerability doesn't mean that hosted/cloud services don't have unknown issues of their own.
While I get what you're saying the platform type is unrelated to the discovery of a 0-day vulnerability.
It's less about that and more in response to @Obsolesce post of lack of patching. Moving to hosted/cloud solutions takes patching out of the companies hands in most cases.
-
@Dashrender said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
All the more reason to move to hosted/cloud services.
True. In a normal on-prem scenario, the Admin to Company ratio is usually 1 to 1.
In a hosted/Cloud environment (MS365), the odds are very high that when an admin patches the server, it will be a 1 to MANY (and perhaps hundreds or thousands of MANYs!)
No less risk in a hosted/Cloud environment, but certainly more customer mail systems get patched faster.
-
@Dashrender said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@DustinB3403 said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@Dashrender said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@Obsolesce said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@JaredBusch said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@Obsolesce said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
Wow that's a lot of companies. If they aren't keeping Exchange Server updated of all things, image all the other software they are likely running out of date too!!!
This hack had nothing to do with servers up-to-date. It was a zero day. There was no patch prior to March 2.
It said thousands of servers are still being compromised daily since the patch was released. I imagined that companies not patching something as serious as this likely have so much else not patched, because of either not caring or lack of awareness of this kind of thing.
All the more reason to move to hosted/cloud services.
The fact that an on-premise solution had a 0-day vulnerability doesn't mean that hosted/cloud services don't have unknown issues of their own.
While I get what you're saying the platform type is unrelated to the discovery of a 0-day vulnerability.
It's less about that and more in response to @Obsolesce post of lack of patching. Moving to hosted/cloud solutions takes patching out of the companies hands in most cases.
Even in that case, a lack of patching or moving to the cloud would indicate other issues at play. Your statement makes sense at a general level, but a company that is refusing or just not considering the ramifications of "not patching" clearly has other priorities or as @scottalanmiller would say "is playing at being in business".
Moving to the cloud would fly in the face of everything that "business" has done up until now and would likely pose some major Personnel type issues within the business leadership "why do we need this", "it's been working fine for decades", "what value am I getting out of this". etc.
-
@JasGot said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@Dashrender said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
All the more reason to move to hosted/cloud services.
True. In a normal on-prem scenario, the Admin to Company ratio is usually 1 to 1.
In a hosted/Cloud environment (MS365), the odds are very high that when an admin patches the server, it will be a 1 to MANY (and perhaps hundreds or thousands of MANYs!)
No less risk in a hosted/Cloud environment, but certainly more customer mail systems get patched faster.
Oh I disagree, I think the risk is likely significantly lower. Two reasons 1) as you mentioned, more people doing that dedicated work 3) the cloud vendor likely has a significantly better in depth security model than those 1 to 1 Admin to business shops you mentioned.
Is it zero - no of course not. But it's likely to be caught sooner and remediated sooner, etc. -
@DustinB3403 said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@Dashrender said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@DustinB3403 said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@Dashrender said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@Obsolesce said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@JaredBusch said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@Obsolesce said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
Wow that's a lot of companies. If they aren't keeping Exchange Server updated of all things, image all the other software they are likely running out of date too!!!
This hack had nothing to do with servers up-to-date. It was a zero day. There was no patch prior to March 2.
It said thousands of servers are still being compromised daily since the patch was released. I imagined that companies not patching something as serious as this likely have so much else not patched, because of either not caring or lack of awareness of this kind of thing.
All the more reason to move to hosted/cloud services.
The fact that an on-premise solution had a 0-day vulnerability doesn't mean that hosted/cloud services don't have unknown issues of their own.
While I get what you're saying the platform type is unrelated to the discovery of a 0-day vulnerability.
It's less about that and more in response to @Obsolesce post of lack of patching. Moving to hosted/cloud solutions takes patching out of the companies hands in most cases.
Even in that case, a lack of patching or moving to the cloud would indicate other issues at play. Your statement makes sense at a general level, but a company that is refusing or just not considering the ramifications of "not patching" clearly has other priorities or as @scottalanmiller would say "is playing at being in business".
Moving to the cloud would fly in the face of everything that "business" has done up until now and would likely pose some major Personnel type issues within the business leadership "why do we need this", "it's been working fine for decades", "what value am I getting out of this". etc.
Sure, I'm fighting this battle at a small client of mine. Though, once an issue comes to light, they are much more apt to move - or fold up shop.
-
@Dashrender said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
My comments come from the "Zero Day" standpoint. Cloud providers have the same level of Risk for Zero Day exploits.
-
@JasGot said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@Dashrender said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
My comments come from the "Zero Day" standpoint. Cloud providers have the same level of Risk for Zero Day exploits.
mmmm... I'll say maybe. I'm guessing that many of these cloud providers have heuristic protections looking for some of these flaws and preventing them... sure, the 1 to 1 guys could too, but likely don't/can't afford them.
But you're right - everyone suffers Zero Days. -
I patched one and since the patch, cannot connect with Exchange Powershell. I didn't realize how dependent I have become on powershell until became unusable on this server.
Been pulling my hair out over this for days.....
-
As @Dashrender mentioned, patching and security will be better with a major cloud provider. There are so many things we could talk about like automation, compliance, role separation, physical security, etc that clouds are going to do better. Not to mention internal actors are the number one threat to companies. Poorly configured exchange server could lead to denial of service, extended outages, data loss, etc.
All of that is great and fine, but the real reason companies choose hosted email is for cost savings. Whether it's exchange online or zoho. Hosting email on premise in 2021 is not cost effective in any way.
-
I generally agree with that statement @IRJ except that the long term cost of hosting isn't cost effective as the vendor can price jack the rates any time that they want.
At a prior position they went full tilt "O365/SSO everything" and while it all worked with a LOT of effort the monthly cost was insane per user, something like $42/U/Month for just our 1 location of 160 people.
Globally they had over 9000, that's a huge burden.
Now I do agree that attempting to maintain that sort of a system (all 9000) would be difficult but most of the organization was separately run under, but owned by one umbrella.
-
@DustinB3403 said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
I generally agree with that statement @IRJ except that the long term cost of hosting isn't cost effective as the vendor can price jack the rates any time that they want.
At a prior position they went full tilt "O365/SSO everything" and while it all worked with a LOT of effort the monthly cost was insane per user, something like $42/U/Month for just our 1 location of 160 people.
Globally they had over 9000, that's a huge burden.
Now I do agree that attempting to maintain that sort of a system (all 9000) would be difficult but most of the organization was separately run under, but owned by one umbrella.
Yeah, this is where I constantly come back to.
Sure, We've all seen Scott's cost analysis of running Exchange in his house, cost of a single server, cost of licensing, cost of power, cost of AC, cost of internet, possible cost of IT support - and how of those things add up to cheaper to use hosted email ($4/user/month on O365)
But this leaves the file still housed on a server - so you still need that server in house for those files, or you move to O365. So that moves you to $5/u/m minimum.
Oh yeah, but who doesn't have that one inhouse app that they use?
In my case we have three - and old EMR, old billing system, and current corp financial system.
So we'll need a server no matter what for a while. now maybe we could save some money by putting it in a hosted DC (doubt it because we'd likely need/want a higher speed connection between us, and that will easily eat up any savings versus managing it onsite).
But then, we definitely need to look at the reality of the things people need to do their jobs, and the expenses involved in getting those things.
We recently replaced our phone system. We could have gone old school - paid out $60K for a system, then monthly support costs, etc... or we could go hosted for $8/u/m (plus the devices) Devices are cheap, and toss away/replace... no up front fees, much more modern than most on-prem systems.. I choose hosted. So now we have this $8/u/m forever... will it cost us more than our last phone system? oh you bet your ass it will, that thing was 30+ years old, almost never updated, etc - but it also WASN'T on the internet! That's huge!
we needed a system that was highly mobile, that means usable anywhere, which means internet will play a major part in it. So updates and maintenance are a must! -
@Dashrender Yeah so what's the math on 8 * # of Users * ∞= ????
It gets expensive damn quick.
-
@DustinB3403 said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@Dashrender Yeah so what's the math on 8 * # of Users * ∞= ????
It gets expensive damn quick.
He has like 90 users. So $720 a month. To hit the cost of his old phone system will take 7 years. But they never updated the old one. So if you include updates to the system then it's even further out. If you include features that are introduced to the SaaS system that you would have to pay for in the old system it's even more. It makes sense financially to pay per user for most things.
-
@DustinB3403 said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@Dashrender Yeah so what's the math on 8 * # of Users * ∞= ????
It gets expensive damn quick.
So you have to look at it a few different ways.
Sure the old system lasted 30 years - will a current one? probably not, they don't make things like they used to.
Also - the old system was nearly completely self contained - it's only connections where phonelines to the LEC. That's not the case with our new needs. We need nearly anywhere access to the phone system now, so we need regular updates.This new requirement really changes the cost of the system.
The old system cost around $80K with no maintenance. over 70 users (old number) over 30 years = $3.17/u/m.
Considering no mobility, and no reporting, and no maintenance $8/u/m really isn't that bad.Now that said, I could likely have gotten a similar system to the one I'm on now, and paid hourly for maintenance as needed, but I would have had a large upfront setup fee, and my monthly maintenance still would have been $5/u/m, not including updates/upgrades, software licenses (yep, even FOSS PBX still have add-on license fees).
-
@DustinB3403 said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
I generally agree with that statement @IRJ except that the long term cost of hosting isn't cost effective as the vendor can price jack the rates any time that they want.
The market will dictate the price in the long term. Microsoft is not the only hosted email provider. They alone cannot control pricing. Unless they want to price themselves away from many customers including existing ones.
-
@IRJ said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@DustinB3403 said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
I generally agree with that statement @IRJ except that the long term cost of hosting isn't cost effective as the vendor can price jack the rates any time that they want.
The market will dictate the price in the long term. Microsoft is not the only hosted email provider. They alone cannot control pricing. Unless they want to price themselves away from many customers including existing ones.
Zoho Mail is $1/u/m for instance.
-
@DustinB3403 said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
I generally agree with that statement @IRJ except that the long term cost of hosting isn't cost effective as the vendor can price jack the rates any time that they want.
They can't for us. We signed a EA and have fixed price terms for the length of the contract.
-
@DustinB3403 said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
At a prior position they went full tilt "O365/SSO everything" and while it all worked with a LOT of effort the monthly cost was insane per user, something like $42/U/Month for just our 1 location of 160 people.
Globally they had over 9000, that's a huge burden.Except it's not.
- It's opex not capex, so it's not dragging down RIOC ratio's for wall street. (big in Mfg and some industries).
- It's just dumped into the fully burdened cost of an employee. If your average employee is paid 50K they probably cost another 20K in benefits, training, taxes, office space, utilities and other overhead a year. Paying $42 a user per month at that scale gets you out of:
- "owning" versions of Office Suite is great until you end up with 4 different versions of office in the office. Then it becomes a nightmare
- Managing Exchange and Sharepoint etc at scale is a full-time job. paying someone else to manage it wins vs. hiring people to do that.
- Again it's $42 per user per month. We were spending more than that per employee on drinks and snacks before COVID hit. stocking 14 flavors of le croix, and the thousands of pounds of M&M's and "the good nuts" adds up. For a company with 9000 users, something that people are spending hours a day in, that's just cheap.
-
@StorageNinja said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
@DustinB3403 said in At Least 30,000 U.S. Organizations Newly Hacked Via Holes in Microsoft’s Exchange:
At a prior position they went full tilt "O365/SSO everything" and while it all worked with a LOT of effort the monthly cost was insane per user, something like $42/U/Month for just our 1 location of 160 people.
Globally they had over 9000, that's a huge burden.Except it's not.
- It's opex not capex, so it's not dragging down RIOC ratio's for wall street. (big in Mfg and some industries).
- It's just dumped into the fully burdened cost of an employee. If your average employee is paid 50K they probably cost another 20K in benefits, training, taxes, office space, utilities and other overhead a year. Paying $42 a user per month at that scale gets you out of:
- "owning" versions of Office Suite is great until you end up with 4 different versions of office in the office. Then it becomes a nightmare
- Managing Exchange and Sharepoint etc at scale is a full-time job. paying someone else to manage it wins vs. hiring people to do that.
- Again it's $42 per user per month. We were spending more than that per employee on drinks and snacks before COVID hit. stocking 14 flavors of le croix, and the thousands of pounds of M&M's and "the good nuts" adds up. For a company with 9000 users, something that people are spending hours a day in, that's just cheap.
All that easily makes sense - at that scale.
I know I don't live in anywhere near that scale, and I don't believe Dustin does either.
2/3's of our employees get paid $20K/y.
It can, and likely does still make sense to do it, spend the $42/u/m, but as is contantly talked about on these forums... most small businesses play at business, they don't really run one.
