What is crowdsourced pen testing?

What is crowdsourced pen testing?

If the cybersecurity industry leads the world in anything, it’s probably in the use of jargon. There are some many acronyms and industry-specific jargon, that even professionals with years of experience have a hard time keeping up.

If you know anything about information security, you’ve probably heard about penetration testing.

But there’s a new term going around recently – crowdsourced penetration testing.

What is crowdsourced penetration testing?

Crowdsourced pen testing is a unique approach where testers are drawn from a large pool of professionals rather than a company-specific team. Crowdsourced pen tests can be organized around a testing timeframe like traditional pen tests or they can be open ended and pay the testers per vulnerability they discover.

There are benefits to each approach that organizations may find appealing.

Before we can talk about the various approaches to crowdsourced pen testing, or the difference in bug bounty, you must first understand why it came to be.

Table of Contents

What's broken about penetration testing?

How is crowdsourced pen testing different?

Various approaches to crowdsourced testing

How is it different than bug bounty programs?

Conclusion

What's broken about penetration testing?

The practice of pen testing has been around for several decades at this point.

Auditors and compliance bodies have propelled it’s popularity even further by strongly recommending and in some cases even mandating that organizations within certain industries undergo pen testing as a prerequisite to their audit certification.

SOC2, ISO27001, and PCI-DSS are just a couple of examples.

There are three main defects in the traditional pen testing approach that the crowdsourced method resolves.

Modern development cycles – most organizations work in development scrums, short bursts where the team focuses on a few core items to resolve or release. Within that burst, they will plan, develop, test, and publish the items they are working on. In some cases, scrums are happening daily, in others, they run for two weeks. That means that every two weeks, a company could be releasing vulnerabilities into their platform. Yet most companies only hire a pen testing team to inspect their systems annually. As you can guess, exploitable vulnerabilities could be in the wild for months before they are identified.
Limited by the testers knowledge – Every heard that two heads are better than one? Well, what if you can add dozens of heads? While the majority of penetration testers are quite skilled, they are still, in fact, limited by their own knowledge and experience. They will have a unique methodology they have perfected and a tool set that they prefer. This is fine, but it limits the assessment they perform to their own method. If you bring on another tester with a different methodology and toolset, he’ll find other vulnerabilities. Do this a dozen more times, and you have a pretty thorough assessment, right? That is exactly what crowdsourced testing focuses on.
Pen tester syndrome – “The sky is falling” - that’s the impression you get from these pen testers. While it’s unfortunate, there are some pen testers who write up every vulnerability as critical or high risk, even when the sensitivity of the systems is low and the difficulty is very high. When you have multiple testers with varying backgrounds during a crowdsourced assessment, you are more likely to get an honest assessment of how serious a vulnerability is. They aren’t trying to make their own company look good by findings a bunch of “critical” vulnerabilities.

How is crowdsourced pen testing different?

The main ways that a crowdsourced approach is different is in skill variety, improved technology, and incentivization.

Skill variety – a traditional pen test is performed by an agency with a set number of security consultants. Because these consultants have worked together for years, they have developed methodologies and habits. With the crowd-based approach, testers are taken from all over the world – greatly expanding the experience of testers.
Improved technology – most of the companies that provide crowdsourced testing offer better integrations with your technical systems. Integrations with your ticketing systems, as well as live communication improve the overall testing experience and remediation process. There’s also usually a dashboard where you view results from the assessment and track them through remediation is confirmed.
Incentivization – Depending on the testing approach, crowdsourced testers can be more motivated to perform well than traditional penetration testers. If they are paid per verified vulnerability, for example, they will likely spend their time differently than if they were paid to spend a bulk about of hours on an assessment.

Overwatch PTaaS improves on many aspects of crowdsourced pen testing and the deficiencies of traditional testing methods.

Because our testers have a wide range of backgrounds, our customers are guaranteed to get an assessment by the most skilled professionals possible.

Furthermore, our focus on improved communication and unlimited remediation assistance mean that our customers finish an engagement much more securely than when they start.

Learn More about Overwatch PTaaS

Various approaches to crowdsourced testing

As I mentioned earlier, there a several ways that companies can approach crowdsourced pen testing.

The two most common approaches are pay-per-vulnerability and pay-per-engagement.

With the pay-per-vulnerability approach, the crowdsourced pen testing platform provides a pool of testers who can test your system whenever they want.

You may get testers from all over the world with varying backgrounds.

When they find a vulnerability, they will submit it for verification and compensation.

There is a downside to this though – your pen test request may not garner enough attention; you may end up with no one interested in testing your system.

The other common approach to crowdsourced pen testing is an engagement approach.

With the engagement approach, the platform providing the testers will typically look through their pool of testers to find a few that are a good match for the system you want tested.

Those few testers will then assess your system for a pre-determined amount of time.

While this may sound nearly identical to traditional pen testing, the big difference is the large pool of testers they can draw from.

How is it different than bug bounty programs?

The big difference in bug bounty and crowdsourced penetration testing is that the crowdsourced approach offers a point in time assessment that can be used to meet compliance obligations.

Bug bounties are basically open-ended invitations for security researches to practice responsible disclosure and receive some kind of compensation for doing so.

Crowdsourced pen testing still provides some kind of time limiting and a more methodical assessment.

This is important for companies getting tested in order to comply with frameworks like SOC and ISO.

Conclusion

So while crowdsourced penetration testing may not be the solution for every organization, there are definitely a lot of benefits that should be considered.

The wider experience base, more realistic vulnerability identification, and ongoing nature are just a few examples.

Crowdsourced testing is one way that traditional security assessments have evolved to meet the needs of the ever changing cybersecurity industry.