3rd Party InfoSec Testing Center
-
Re: [Cylance Unbelievable Tour Lives Up to Name](Can Cylance Be Trusted?)
From my point of view, the best way to answer this question would be with a 3rd party independent non-profit organization. The organization would have to have subscribers that are IT pros and IT Sec pros in order to find out who is the best and worse in the industry. Reviews would have to be fully open to subscribers as to how tests are conducted, signatures are collected, and tools are used. Businesses in the industry would not be allowed to financially influence the organization for a better report of their own or worse report of competitors.
This is an ongoing document. Please feel free to add or edit what you think is necessary.
-
It's not a bad idea. I've been kicking around some ideas around something semi-related for a while, but not sure how to make it happen.
-
Question is...how do we fund it as a start-up? Is this where kickstarter comes in?
-
@NerdyDad said in 3rd Party InfoSec Testing Center:
Question is...how do we fund it as a start-up? Is this where kickstarter comes in?
That's the toughest part. What is the profile of someone that is going to donate money to a non-profit lab of this nature? And can you qualify for non-profit status for a research lab?
-
Call it the "First Unified Church of Software Testology" or something... pretty easy to form a church and not pay taxes, look how many there are already. And this one might actually be useful for something (besides not paying taxes).
-
@RojoLoco said in 3rd Party InfoSec Testing Center:
Call it the "First Unified Church of Software Testology" or something... pretty easy to form a church and not pay taxes, look how many there are already. And this one might actually be useful for something (besides not paying taxes).
Hail root
-
Before getting too far into it, the testing methods must be standardized and PUBLISHED and made available to third parties.
My question is why would a vendor like Cyclance or McAffee or MalwareBytes, et al not be willing to pay to have an outside entity actually test their products?
By accepting payments for actual work, does that risk it becoming more like the Magic Quadrant people?
Edit for clarity.
-
More importantly, how do you show trust and accountability.
I don't care if you are a non profit or a commercial entity, neither proves that you are a trusted source. Either can be bought and paid for.
Also, what is the definition of a good test. I've seen tests where Symantec has won consistently and I know the product sucked at the time.
-
@Breffni-Potter said in 3rd Party InfoSec Testing Center:
More importantly, how do you show trust and accountability.
I don't care if you are a non profit or a commercial entity, neither proves that you are a trusted source. Either can be bought and paid for.
That's why my comment about the testing methods being published and repeatable. If an outside entity can't duplicate our results, then it's not a good test.
-
Best way I can say it is to have public policies about gifts/bribes towards the company and employees of the company. If that policy is violated, then the public would need to know somehow that the company has to follow its policy.
-
@dafyre said in 3rd Party InfoSec Testing Center:
@Breffni-Potter said in 3rd Party InfoSec Testing Center:
More importantly, how do you show trust and accountability.
I don't care if you are a non profit or a commercial entity, neither proves that you are a trusted source. Either can be bought and paid for.
That's why my comment about the testing methods being published and repeatable. If an outside entity can't duplicate our results, then it's not a good test.
Probably a bad example, but the best one that we have is the scientific community. If somebody at a university says that they discovered x, y, and z and you can prove it with this test, can another university at another independent location reproduce the same discoveries with a test that was setup the same way?
-
The tests need to be about real world usage. Screen recorded, with log dumps published.
I.e go to freemusic4u.com
-
How about a consortium from universities? Universities fund the project, AVs gets tested, knowledge of the tests goes into updating curriculum of IT & cyber-security courses.
-
Universities? Those incredibly slow to react to change organisations delivering up to date security data?
-
Yeh, you may have a point there.
-
@NerdyDad said in 3rd Party InfoSec Testing Center:
How about a consortium from universities? Universities fund the project, AVs gets tested, knowledge of the tests goes into updating curriculum of IT & cyber-security courses.
LOL. If universities cared they would have already done this.
-
@scottalanmiller said in 3rd Party InfoSec Testing Center:
@NerdyDad said in 3rd Party InfoSec Testing Center:
How about a consortium from universities? Universities fund the project, AVs gets tested, knowledge of the tests goes into updating curriculum of IT & cyber-security courses.
LOL. If universities cared they would have already done this.
Yeah. I think a Kickstarter or Non-Profit would be probably the best way to go about something like this.
-
I doubt that Kickstarter would work. A non-profit is needed, almost certainly, but is a big pain to run as Republic of IT found out and very hard to get people to commit to donations.
-
@scottalanmiller said in 3rd Party InfoSec Testing Center:
I doubt that Kickstarter would work. A non-profit is needed, almost certainly, but is a big pain to run as Republic of IT found out and very hard to get people to commit to donations.
Not to mention this is a fairly niche thing to be testing. Everyone needs it but fewer actually care about the results.
-
@coliver said in 3rd Party InfoSec Testing Center:
@scottalanmiller said in 3rd Party InfoSec Testing Center:
I doubt that Kickstarter would work. A non-profit is needed, almost certainly, but is a big pain to run as Republic of IT found out and very hard to get people to commit to donations.
Not to mention this is a fairly niche thing to be testing. Everyone needs it but fewer actually care about the results.
Or understand them, or trust them. And they change constantly.
