Centralized Log Management
-
Loki is another popular open source solution.
-
Scott pretty much nailed it. Although collecting and preserving logs centrally is a good idea, analysing them anything but superficially would normally require a dedicated IT security team. There are (expensive) solutions like SIEM that make this job easier but even those can hardly be managed by a typical SMB/SME IT depts on their own. If the OP's organisation needs to be ISO 27001 certified or compliant with PCI, HIPAA etc. yet small enough, looking at MDR, MSSP or managed SIEM providers might be an alternative.
-
@taurex said in Centralized Log Management:
Scott pretty much nailed it. Although collecting and preserving logs centrally is a good idea, analysing them anything but superficially would normally require a dedicated IT security team. There are (expensive) solutions like SIEM that make this job easier but even those can hardly be managed by a typical SMB/SME IT depts on their own. If the OP's organisation needs to be ISO 27001 certified or compliant with PCI, HIPAA etc. yet small enough, looking at MDR, MSSP or managed SIEM providers might be an alternative.
Opendistro/opensearch is an SIEM, same with Loki/Grafana or Graylog (even with the crappy licensing).
They don't have to be expensive and they can be relatively easy to set up reasonable alerting for events you need to know about.
-
@braswelljay said in Centralized Log Management:
@scottalanmiller said in Centralized Log Management:
@braswelljay said in Centralized Log Management:
Does not collect server, application and network logs sufficiently to respond to and investigate a cybersecurity incident
This is not a bad thing. Collecting logs is good, centrally is best. But only if you have a team that can use them. If you had that, likely you'd already be doing this. So the question is... before doing this, do you have a team ready to leverage it? Or is this just a way to potentially spend more money with the "cyber security" guys because there's no better way to make money than getting paid to read logs.
No there is no team that would be able to leverage. This is a very good point and definitely something for us to keep in mind.
From your perspective is there any value in collecting them centrally so that if we were to be breached it might help other experts that we would almost certainly have to engage? That is even if we don't really have the inhouse expertise to use them effectively on a day to day basis, is it still worth keeping and protecting those logs in the event that they are needed due to an actual compromise of some kind?
Only if that is something that you would truly do. Ask management to do a few things...
-
Write a policy and procedure as to what to do in case of a breach. Does this document include "hire an outside team to look at logs"? If yes, okay. If not, then obviously collecting has no value. Make them determine this now, you can't determine it later without having already spent or not spent a lot of the money.
-
Get a budget, both for the log collection AND for the log reading.
-
Compare the expected cost against value. How much will it cost to collect logs if nothing bad happens (and you have no one to leverage them in a healthy environment?) How much will you gain from someone paid to read them who isn't familiar with your logs already (a silly proposition from the start.) Does spending that money reflect good security policy (saves money) or reckless business mistake (losing money.)
-
-
@pete-s said in Centralized Log Management:
@scottalanmiller said in Centralized Log Management:
@braswelljay said in Centralized Log Management:
Does not collect server, application and network logs sufficiently to respond to and investigate a cybersecurity incident
This is not a bad thing. Collecting logs is good, centrally is best. But only if you have a team that can use them. If you had that, likely you'd already be doing this. So the question is... before doing this, do you have a team ready to leverage it? Or is this just a way to potentially spend more money with the "cyber security" guys because there's no better way to make money than getting paid to read logs.
Standards such as ISO 27001 have requirement that logs are protected. If an intruder gains root privileges on a server the only way to protect the logs is to have them stored somewhere else. So having central logging might be a compliance issues in some cases.
Not many companies need to follow that. And that's one of the reasons that those certifications are so often considered bad.... they make core business decisions a requirement when they rarely make sense.
-
@dashrender said in Centralized Log Management:
He was audited by what we assume is a third party likely strickly pointing out best practices. These pointed out items likely don't take the specific company and their current setup into mind at all.
Not possible. In order to be a best practice is MUST apply to all businesses. If you have to evaluate the efficacy of something, then it's a "rule of thumb" at best.
If they presented this as a best practice, they are a security risk and we know that they are being dishonest. It is a good security practice, from an isolated "security without business concern" perspective. But in business, security like IT, is a business function. All security has to be seen in respect to its business protection value. If the security costs more than it protects, it itself is the failure.
-
@dashrender said in Centralized Log Management:
While it's not bad for the audit to show these things - the OP and their management should be asking - OK - logs - do we need them?
Exactly. The audio SHOULD mention it, as it is really important. But it needs to be "We found no logging, and no log centralization, and no security around logs. These are powerful security tools and should be evaluated for efficacy in the given environment."
-
@stacksofplates said in Centralized Log Management:
@taurex said in Centralized Log Management:
Scott pretty much nailed it. Although collecting and preserving logs centrally is a good idea, analysing them anything but superficially would normally require a dedicated IT security team. There are (expensive) solutions like SIEM that make this job easier but even those can hardly be managed by a typical SMB/SME IT depts on their own. If the OP's organisation needs to be ISO 27001 certified or compliant with PCI, HIPAA etc. yet small enough, looking at MDR, MSSP or managed SIEM providers might be an alternative.
Opendistro/opensearch is an SIEM, same with Loki/Grafana or Graylog (even with the crappy licensing).
They don't have to be expensive and they can be relatively easy to set up reasonable alerting for events you need to know about.
Can be, for sure. But you need someone to install, configure, maintain those systems. They don't currently have an IT team that is ready to read the logs as it is. While we could certainly discuss the issues with having a more general IT gap (log reading is more for troubleshooting than for security, so this gap could be pretty large) we'd be forced to look at the cost in a larger context that could get pretty big.
-
@scottalanmiller said in Centralized Log Management:
should be evaluated for efficacy in the given environment."
That is how exactly zero "audits" work.
-
@jaredbusch said in Centralized Log Management:
@scottalanmiller said in Centralized Log Management:
should be evaluated for efficacy in the given environment."
That is how exactly zero "audits" work.
It's how ALL honest audits work. The problem is, like most MSPs who are secretly scam VARs, almost all audits, especially those hired outside of IT by incompetent managers, bring in scammers with no knowledge, qualifications, or honesty who just seek to defraud and are, themselves, a security risk.
We do audits, however, and we'd never present that way. Real auditors are out there. But people don't like to hire them because they can't produce checklists and shopping lists.
-
@scottalanmiller said in Centralized Log Management:
OpenSearch from Amazon. They took the ELK stack, made it 100% open source, and back it by Amazon. It is so good both in technical product and in licensing, that essentially it is the only game in town now.
Interesting take from ELK side
https://www.elastic.co/what-is/opensearchOur products remain free and open, but Amazon can no longer freely use Elasticsearch and Kibana products without collaborating with us. Rather than collaborate with us and contribute back, Amazon created its own forked projects, which are less mature, not ready for production use, and provide inferior capabilities compared to Elasticsearch and Kibana.
-
@hobbit666 said in Centralized Log Management:
@scottalanmiller said in Centralized Log Management:
OpenSearch from Amazon. They took the ELK stack, made it 100% open source, and back it by Amazon. It is so good both in technical product and in licensing, that essentially it is the only game in town now.
Interesting take from ELK side
https://www.elastic.co/what-is/opensearchOur products remain free and open, but Amazon can no longer freely use Elasticsearch and Kibana products without collaborating with us. Rather than collaborate with us and contribute back, Amazon created its own forked projects, which are less mature, not ready for production use, and provide inferior capabilities compared to Elasticsearch and Kibana.
LOL - someone sounds like they are just complaining that their toy was taken.
-
@dashrender said in Centralized Log Management:
@hobbit666 said in Centralized Log Management:
@scottalanmiller said in Centralized Log Management:
OpenSearch from Amazon. They took the ELK stack, made it 100% open source, and back it by Amazon. It is so good both in technical product and in licensing, that essentially it is the only game in town now.
Interesting take from ELK side
https://www.elastic.co/what-is/opensearchOur products remain free and open, but Amazon can no longer freely use Elasticsearch and Kibana products without collaborating with us. Rather than collaborate with us and contribute back, Amazon created its own forked projects, which are less mature, not ready for production use, and provide inferior capabilities compared to Elasticsearch and Kibana.
LOL - someone sounds like they are just complaining that their toy was taken.
True, but they are probably right. Amazon and other providers bastardize open source projects because the licences doesn't require them to share their changes with the open source community.
-
@pete-s said in Centralized Log Management:
@dashrender said in Centralized Log Management:
@hobbit666 said in Centralized Log Management:
@scottalanmiller said in Centralized Log Management:
OpenSearch from Amazon. They took the ELK stack, made it 100% open source, and back it by Amazon. It is so good both in technical product and in licensing, that essentially it is the only game in town now.
Interesting take from ELK side
https://www.elastic.co/what-is/opensearchOur products remain free and open, but Amazon can no longer freely use Elasticsearch and Kibana products without collaborating with us. Rather than collaborate with us and contribute back, Amazon created its own forked projects, which are less mature, not ready for production use, and provide inferior capabilities compared to Elasticsearch and Kibana.
LOL - someone sounds like they are just complaining that their toy was taken.
True, but they are probably right. Amazon and other providers bastardize open source projects because the licences doesn't require them to share their changes with the open source community.
I guess I need to learn more about those licenses - I thought if you partook of those open source licenses - then made code changes and then made the code availalble outside of yourself - you had to give all new cold along with all the old - is that not so?
i.e. I fork ES - I update it with my own code - call it "ES of Mine" I publish ES of Mine - don't I have to give all of my new code away because I used ES as the base?
-
@dashrender said in Centralized Log Management:
@pete-s said in Centralized Log Management:
@dashrender said in Centralized Log Management:
@hobbit666 said in Centralized Log Management:
@scottalanmiller said in Centralized Log Management:
OpenSearch from Amazon. They took the ELK stack, made it 100% open source, and back it by Amazon. It is so good both in technical product and in licensing, that essentially it is the only game in town now.
Interesting take from ELK side
https://www.elastic.co/what-is/opensearchOur products remain free and open, but Amazon can no longer freely use Elasticsearch and Kibana products without collaborating with us. Rather than collaborate with us and contribute back, Amazon created its own forked projects, which are less mature, not ready for production use, and provide inferior capabilities compared to Elasticsearch and Kibana.
LOL - someone sounds like they are just complaining that their toy was taken.
True, but they are probably right. Amazon and other providers bastardize open source projects because the licences doesn't require them to share their changes with the open source community.
I guess I need to learn more about those licenses - I thought if you partook of those open source licenses - then made code changes and then made the code availalble outside of yourself - you had to give all new cold along with all the old - is that not so?
i.e. I fork ES - I update it with my own code - call it "ES of Mine" I publish ES of Mine - don't I have to give all of my new code away because I used ES as the base?
It depends on the exact license. There are so many ways that licenses work. I'd say Elasticsearch used the wrong license originally and threw a hissy fit about it.
-
@dashrender said in Centralized Log Management:
i.e. I fork ES - I update it with my own code - call it "ES of Mine" I publish ES of Mine - don't I have to give all of my new code away because I used ES as the base?
No, you don't when you're not providing the software. Amazon is providing the service, not the software. So they don't need to adhere to GPL and similar licenses.
That's why Richard Stallman (open source guru) calls SaaS, service as a software substitute.
-
@hobbit666 said in Centralized Log Management:
@scottalanmiller said in Centralized Log Management:
OpenSearch from Amazon. They took the ELK stack, made it 100% open source, and back it by Amazon. It is so good both in technical product and in licensing, that essentially it is the only game in town now.
Interesting take from ELK side
https://www.elastic.co/what-is/opensearchOur products remain free and open, but Amazon can no longer freely use Elasticsearch and Kibana products without collaborating with us. Rather than collaborate with us and contribute back, Amazon created its own forked projects, which are less mature, not ready for production use, and provide inferior capabilities compared to Elasticsearch and Kibana.
That's BS. It's that you can't just use their products. It affects everyone, Amazon only did it because their customers needed it. Not because Amazon needed it. ELK is full of crap.
-
@pete-s said in Centralized Log Management:
@dashrender said in Centralized Log Management:
@hobbit666 said in Centralized Log Management:
@scottalanmiller said in Centralized Log Management:
OpenSearch from Amazon. They took the ELK stack, made it 100% open source, and back it by Amazon. It is so good both in technical product and in licensing, that essentially it is the only game in town now.
Interesting take from ELK side
https://www.elastic.co/what-is/opensearchOur products remain free and open, but Amazon can no longer freely use Elasticsearch and Kibana products without collaborating with us. Rather than collaborate with us and contribute back, Amazon created its own forked projects, which are less mature, not ready for production use, and provide inferior capabilities compared to Elasticsearch and Kibana.
LOL - someone sounds like they are just complaining that their toy was taken.
True, but they are probably right. Amazon and other providers bastardize open source projects because the licences doesn't require them to share their changes with the open source community.
That's not what the issue was. It was that they were allowed to USE the software without paying for it. It had nothing to do with being open source, and everything to do with having been free (to use.)
-
@pete-s said in Centralized Log Management:
Amazon and other providers bastardize open source projects because the licences doesn't require them to share their changes with the open source community.
Almost all licenses DO require that. BSD does not, but most do. In fact, that you must give back is often considered a requirement of being open source (but not always.)
-
@dashrender said in Centralized Log Management:
I guess I need to learn more about those licenses - I thought if you partook of those open source licenses - then made code changes and then made the code availalble outside of yourself - you had to give all new cold along with all the old - is that not so?
That's by license. Open source is simply a giant family of licenses that aren't closed. The variety is huge and you could go make your own license right now that is different from everything else. For most people, OS only means that the right to view the code exists.
That remains with ELK. It is still open to audit and compile. but it is no longer fully open to USE.
