I like the idea of a marketplace, but I don't think background checks and references are the way to build a credible list of the world's best pentesters.
I think what patio11 is doing with Starfighters.io is orders of magnitude better. Run developers through a gambit of supremely difficult tests via a fun CTF-type game and pair the best hackers with the highest enterprise bidder. Works not just for pentesters, but all devs really.
Also, I know where to get the best pentesters because they're listed on all the top companies' bug bounty pages. It's proof of skill I'm after, not some Gartner-esque gatekeeper telling me who's best because they've "background checked" them.
Give me a system more like StackOverflow or Starfighters where I can see the work. Not something subjective like eBay or Yelp, which can be easily gamed.
I hear you and I get it. What you are describing are security-focused pen testers, mission-focused red-teams.
They are absolutely welcome, and they are a subset of the pen-tester universe. They are not a good fit for someone needing to get a PCI pen test, but they do incredible work in other areas.
It highlights the point in the blog that the landscape of finding the 'right' pen-test team is not easy. Some are brilliant at one thing, others at many, but even an elite group may not be the right fit for the task at hand.
We are taking the feedback system seriously and are slowly testing it out. An easily gamed system is useless for everyone.
But why must demonstration of skill be limited to elite red-team style pentesting? You could devise challenges geared at demonstrating all sorts of knowledge (HIPAA, PCI, websec) basic or advanced.
If you've seen the sad state of PCI audits in particular these days, you'll get my drift. I think there's a huge opportunity here to raise the quality bar with your marketplace.
We're hoping to be less "hoops" and more "a fun experience which competes with someone's Starcraft/Instagram/Game of Thrones/etc time" that also happens to be really useful the next time you're in the market for a job.
Take a look at the leaderboard for Microcorruption some time. It's public. (SF's are not, as a considered design decision for the moment.) If you do and cannot understand the claim I am making, that's cool, but I feel no particular need to elaborate.
More important in the long term than the names you will recognize are the names you will not.
I think what patio11 is doing with Starfighters.io is orders of magnitude better. Run developers through a gambit of supremely difficult tests via a fun CTF-type game and pair the best hackers with the highest enterprise bidder. Works not just for pentesters, but all devs really.
Also, I know where to get the best pentesters because they're listed on all the top companies' bug bounty pages. It's proof of skill I'm after, not some Gartner-esque gatekeeper telling me who's best because they've "background checked" them.
Give me a system more like StackOverflow or Starfighters where I can see the work. Not something subjective like eBay or Yelp, which can be easily gamed.