Hey job seekers. I'm level 45 in starfighter with a power VI blaster in C++ and IX in Python. $100 to get you to level 30, x1.5 per level above that (though obviously I can't get you above 45, but I know a guy so I can subcontract up to 52.) You just have to set up a port forward from your machine and I'll take care of the rest. Contact info is in my profile.
Don't get suckered into using a bot. They may be cheaper in the short run but you know you're going to get lifebanned. My prices are nothing compared to your future earnings.
I don't think you understand their business. Contingency recruiters' fees can reach 30% or more of an engineer's first year of salary, which could work out to $50k or more per engineer. You can bet they will be looking closely at each candidate's solutions, plus automated code plagiarism detection is pretty easy.
It also depends on the type of problem. If you just need one weird trick to pass the level (e.x. use this SQL injection) then that's easy to game. If you have to write a bunch of distributed systems code (e.x. Stripe's last CTF) that's a lot harder to cheat.
Sorry, I don't see the connection. Yes, a successful candidate is worth a lot of money to Kalzumeus, and they are incented to make sure that the solutions are real and not plagiarized. But that salary is worth even more to that successful candidate, and hence they would be willing to pay handsomely for someone to write customized, non-plagiarized solutions for them if it gets them in. (The mythical level 45 mentioned would cost $44K. If I'd done the math, I probably would've made it more. I guess the level 52 would be nearly $750K...)
For now, I'm assuming they have good protection against automated solutions. If it were just a matter of detecting plagiarism, you're right that it would be trivial. (Heck, I even cofounded a company that did code plagiarism detection, so I have a very good sense for how easy or hard it is. tl;dr: it's straightforward to defeat a plagiarism detector if you are sufficiently motivated and knowledgeable and willing to put in some time to go through all the code, but people/companies who plagiarize are typically none of those.)
Detecting whether someone else is doing the candidate's work, on the other hand, is impossible. Unless you bend the rules. Fortunately, the rules seem pretty pliable to me -- when a candidate arrives at a company's interview, spot-check them on their understanding of a few of "their" trickier solutions. It's a logistical nuisance, to be sure.
Sure, but if Kalzumeus can't verify the candidate's identity while playing the game, how could they verify it afterwards? Until someone physically walks through your door, you really have no guaranteed way of knowing who you're talking to.
I don't think they'll really need to worry about these things for quite a while, though. Good problems to have, I guess.
They won't look closely. As you seem to see yourself, they would rather spend $50k per engineer to a 3rd party service than perform proper recruiting by themselves in house.
You seem to be trolling, but to clarify I was suggesting Starfighter would be looking closely before referring candidates to clients. Their reputation depends on the quality of the candidates they refer.
I'm not sure why you're being downvoted. Obviously, if they reference Danny the Word programmer as a ninja rockstar to a Fortune 500 on a regular basis, their business is going to take a nosedive.
I like this idea, but I also like the one proposed here http://ortask.com/a-better-way-to-hire-developers-and-tester....
I fear that starfighter will overlook goood candidates simply because they are not good at gaming, which makes the second approach by ortask more interesting and maybe balanced.
As long as starfighter isn't the only way technical recruiting happens, I think that's not really a problem. There will always be lots of ways to demonstrate talent :)
The "overlooking" problem comes inherently with most heuristics. There should just be an option to forgo this system of measurement and use another one.
I'm not sure that cheating on this would be unethical. Programming is about getting things done. As a hiring metric, nothing can beat open source.
The open source portfolio really is kind of a golden standard. It is real work that you have done, that has (hopefully) advanced the commons, because you love programming and have something unique to add to the world. Some fake video game programming may indicate something, but is simply not as strong an indicator.
Who are you going to hire- TJ Holowaychuk, who wrote Express, or the guy who placed 3rd on TopCoder in 2012?
I think it is wrong to except everyone to contribute to open-source and I have seen that even the very smart programmers are not interested in making personal projects or contributing to existing ones.
Honestly, after mind-taxing eight hour job, you want to go home, watch a movie, read a book or some other relaxing activity. Not to mention that for people with families, it is nearly impossible to find the time to contribute. The rockstar-ninjas with thousand-starred repos on Github are very rare.
That's awesome if you're the guy who wrote Expressjs, and less awesome if you're an extremely talented systems programmer nobody's ever heard of who hasn't hit on a project that happens to capture the industry's attention.
Exactly. I know some amazing programmers who happen to also like mountain biking and building robots with their kids, who happen to work for companies that don't publish their code to github♯. The idea that all the good programmers work with open source in their own time and spend every waking moment pushing to github is a dangerous and fruitless one, because you're missing out on great talent with those blinkers on.
♯And let's be fair, 99% of what's "open source" is junk anyway, just because you wrote an 80% done static blog generator doesn't mean you're "advancing the commons"
Oh awesome, all I have to do is pay you a bunch of money, then I can get a job I'm not qualified for, fail, get fired, and maybe have my reputation ruined? :)
The scam you've described may be possible, but it's not very rational for someone to use it, IMHO.
Meh. The port-forwarding bit is probably unnecessary. And yes, there's a good chance you'll end up hacked and still poor, but there's still a decent chance that you'll end up with a large salary and not hacked. I suspect the black market is adequately efficient at self-policing when there's money involved.
Though that's where my scam would be most likely to fall apart. Recruiters typically only get their cut if the candidate doesn't get fired in the first N months. So if they purely cheated their way in, then the hiring company is only out a couple months of useless work. (Which could be very costly, to be sure, but at least they don't have to fork over the $50K. And there are many other reasons why a new employee doesn't pan out.)
> I suspect the black market is adequately efficient at self-policing when there's money involved.
I suspect that too, and this is something I'd really want to read more about - how does self-policing work in black markets and how effective is it. If anyone has any good sources, I'd be grateful for sharing.
RE fraud prevention, there were two interesting lines in the Starfighter's announcement:
> We assess for skill first, passively as players play our games and then actively. Our founders — talented technologists — personally reconstruct candidates’ solutions and evaluate them.
> We follow-up with players to ask if they have any interest in a no-obligation chat about career options. If they’re interested, we have an honest geek-to-geek conversation.
I suspect that between the first and the second there will be a place for Starfighter team to verify that the candidate has the skills his account shows. After all, if they want to maintain the access "deep into the hiring funnel", they have a strong incentive not to send bad candidates and waste their clients' time.
I'm all, involved? and stuff? You can, like, ask us stuff, I guess.
Let me answer the most frequent questions we're getting right here:
"CTF" stands for "Capture The Flag". Conventionally, it's a contest with a collection of "flags" each of which is guarded by a programming puzzle; teams of people compete to collect flags. What we're doing is not a conventional CTF, but if you want to get the flavor of what we're doing (without the whole game dynamic), check out MICROCORRUPTION.COM, which is a more conventional CTF we ran last year.
Security is one of two problem domains we're starting with. But this isn't a "security recruiting" service, and our take on security uses it as a venue for systems and network programming, not for the minutia of SQL query quoting rules.
Thank you for running a long-term CTF ! I've always been interested in those "improve your coding skills by playing" challenges but from my inexistent experience most of them fall in a combination of:
- They are timeboxed
- They are mostly (if not only) about cracking security
- They are targeted towards low-level languages
You've already stated that it won't be timeboxed (which totally makes sense as a hiring "middleman": you're interested in applicants at all times). You already said that security will not be the single domain. Now, will I be able to use Starfighter as an excuse to finally stop procrastinating and learn that shiny new language I've had on my to-learn list for far too long ? (Patrick more or less hinted it shouldn't be the case, but I'd like to know)
I'm really looking forward to this. On the overall point of breaking the interview standard we have, I'd like to say a huge "Thank you". The points you've made in your blog post really resonate with what I can see (the interview process is a joke if you want to hire actual programming engineers). I really hope we can move towards a model where applicants can show skills through a portfolio, of which Starfighter should be a part if I understand things correctly.
If you haven't seen it before, check out https://projecteuler.net/ which is a series of (very) loosely connected programming challenges.
The challenges start mostly bite-sized and typical programming problems (think fizzbuzz or "what is the 100th prime") and grow into some really interesting areas. You use whatever language(s) you wish, with no time limit, and are free to skip around to whichever questions interest you (or just do them in order... whatever you like)
Euler is more like math problems that can be solved with programming than programming challenges. Very neat, but decreasingly useful for programming practice past the first 20ish.
I always ask for the easier Project Euler tests (or give them a couple) to do; the reason they are good test (IMO) is that they are too small a task to leverage frameworks and can demonstrate things like good approaches to engineering and programming practice (even if the questions are mathematical)
It is possible to demonstrate good approaches to engineering with your solution. But if you're cranking out PE problems, you really are better off just throwing out a bunch of one-off unmaintainable scripts. It is a good test of math and cleverness. It is not a good exercise for maintainable software.
If you are interested in a long-running CTF, we launched https://backdoor.sdslabs.co a few days back (with >30 challenges as of now). We also tend to have more beginner level challenges.
I'm also really looking forward to this, especially from the pov of Hiring. Ours is an academic initiative, with no plans to monetize ever and its cool to see patrick and tptacek picking up the mantle on such a task.
As someone who plays in a lot of "conventional" CTFs (e.g. DEFCON, Codegate, GitS, etc.), this sounds very cool, even if my participation in those already gets me some job offers :)
One question: for a less "just for fun" site like this, with no time barriers, how do you plan on dealing with people just following write-ups? You can randomly generate the flags, but people could still follow the steps. You can say "don't write write-ups," but write-ups will still be written ;)
This is (a) a real challenge, (b) something we're addressing in part by not letting the content get stale, (c) something we were surprised by with the crypto challenges --- if you ask people to follow the honor system, they tend to do it! --- and (d) something that the game model will also somewhat address.
It's hard for me to go into more detail on (d) without revealing a whole bunch of stuff about the game I don't want to talk about yet; I should be more comfortable talking about it, but until we announce it officially I have a lot more leeway to slip rev1 features. :)
The shortest simplest answer though is: we're a firm whose whole purpose is to make fun, interesting CTF-style games (well, one game; we're the Blizzard of CTFs, and we're building our WoW), so we can address a lot of these kinds of problems with brute force, because this isn't a spare-time thing.
If it helps to understand where we're coming from:
Chris Eagle, the author of The IDA Pro book, published an IDA Pro plugin for the bizarro-MSP430 that Microcorruption (our last CTF) emulates. There are tools with "microcorruption mode" in them because of all the little ways we broke MSP430. Someone wrote a symbolic execution engine to solve the Hollywood level on Microcorruption and posted it to Github. There is still a #uctf channel on Freenode for Microcorruption.
This. Is. Awesome. It is my favorite thing about Microcorruption.
We did practically nothing at all to foster a community for Microcorruption, apart from Erin starting the IRC channel. That's not an opportunity we are going to miss this time; in fact, doing better on the community and sharing side is part of the thesis of the company.
I reallllllllllllly hope people share code and tools and stuff to make progress in the game. How cool will that be?
> "... if you ask people to follow the honor system, they tend to do it!"
This is great but I think the stakes are somewhat different if it's ostensibly about jobs/hiring. I don't recall the crypto challenges being promoted as a hiring mechanism (though they may have been useful for that).
I think we plan on making minimal demands of our users, and none of them involve grooming them for prospective employers.
Look, the reality is, most of the people who participate aren't going to be looking for a job when they do. So all our incentives are to make the experience itself rewarding to participants.
I'm weird about typing those words because very very soon we're going to actually ship the first rev and levels of this thing, and as anyone who ships software knows: right now, at this point in the release calendar, my instincts are to be LOWERING the bar, not raising it. :)
> The crypto challenges were how we (for instance) hired Alex
But that's the use from your (or Matasano's) perspective. I joined the crypto-challenges not at all because I want a job in security, but because I continuously heard people be super-enthusiastic about it (both the participants, as well as yourself, tptacek :) ), because it reminded me of the old Malattia+ 3564020356 puzzles (level 6!), because it seemed more fun than the Euler Project puzzles (which I did enjoy, but you can only solve so many palindrome prime puzzles before it gets tedious) and of course because I would learn things about practical crypto.
Unfortunately I only got halfway the first set of the Matasano challenges, but that was more because I did it in Python and at some point got frustrated by its lack of speed :) (even using NumPy) I did make a rather elegant English-text MLE detector using a log-probability frequency table of only 256 bytes :) I thought that was pretty cool. I might have another go at it and this time use Java instead.
>I think we plan on making minimal demands of our users, and none of them involve grooming them for prospective employers.
>most of the people who participate aren't going to be looking for a job when they do.
This is weird, and I'm sensing some miscommunication between you and patio.
I'm only hearing about this as a tool for proving my worth. Your major marketing (as far as I'll likely ever be aware) has CAREER CAREER CAREER stamped all over it.
People won't follow the honor system if the stakes are at the 'career' level. You're losing the 'fun and free' culture of Microcorruption that makes people spend their free time building fun tools.
Your understanding of the incentives differ from the incentives that have been communicated to me about Starfighter.
Just giving feedback as someone that's theoretically right square in the middle of your target audience. I really want to take this 'college course' and take advantage of what I see as your opportunity as a security expert to make your knowledge replicable.
But I'll play along. I'll be taking Starfighter in a new direction. Most notably, we'll be reorganizing how we react to online discussion. Anonymous critics will be summarily executed unless we cannot identify them, in which case they shall merely be barred from Starfighter for life, which, if you believe our marketing department (AND AS CEO I DO IN FACT I'M SORRY I IMPLIED IT WAS POSSIBLE TO DOUBT THEIR CLAIMS), will make it very difficult for them to find work in the hiring utopia that is the post-Starfighter process.
While this might indeed help employers find better employees, I am concerned about hidden (or not so hidden) bias. Games like this (and the way this one is described confirms it) select for competitive people. Other talented programmers might prefer expressing their creativity and talent in less competitive ways. While competitiveness might be a desirable quality, sometimes it is not. It might also appeal to men more than women, and so might be inadvertently sexist. I think you should very carefully analyze player demographics to make sure you're not making matters worse in terms of diversity.
When you design a test -- any test -- you should make sure that 1/ score correlates with success (i.e. full bivariate correlation), and 2/ if the test is voluntary, that there are no prior biases which would select candidates prior to taking the test.
Best answer I can give was that we've heard that concern from a bunch of different sources, we share it, and we're designing against it to some extent.
As a reminder: this is an outreach strategy we deployed at Matasano to enormous success. We had an English professor finish the crypto challenges. We didn't solve all our diversity problems, but we made a palpable dent in them, and we did that by coming up with something that surfaced aptitude that wasn't held hostage to the biases of random human interviewers.
There is, to my mind, no hope for the "random human interviewer" hiring strategy. It's hard to make the problem we have now worse. But we're watching out for it.
I would love to hear more thoughts on how we can address this at the level of game design! We're at a "finishing touches on infrastructure, just starting with level design" place in our design right now.
Somewhat relatedly, as a person who's discussed these issues with you in the past, it sounds like you'll be gathering new evidence. How will your beliefs change?
For example, suppose $GROUP is over/underrepresented among top starfighter performers. What will be your new belief about $GROUP's technical talent and the causes of over/underrepresentation in tech in general?
I don't think my beliefs about the facts are any different from yours; we disagree on the causes of the facts and their mutability. Therefore, my beliefs will change when the dynamics are tested. But this kind of data provides evidence for our disagreement no more than a still image of a flying arrow can settle the one between Aristotle's and Newton's theories of motion.
Somewhat relatedly I'd like to add that while the result of an experiment in dynamics will obviously change my theory, it will in no way change my values[1]. Privilege based on an immutable characteristic such as intelligence (assuming such a result were to be obtained) is no more arbitrary than one based on bloodline. A smart person, though she will obtain it, deserves no more power than the average bloke, just as a nobleman, though he will obtain it, deserves no more power than a commoner. This is one reason the original meaning of "meritocracy" is satirical[2], as it does not change anything other than for the worse, by making the wielders of power believe that they actually deserve it (like the nobility in ancient times but unlike more recent ruling classes such as the American WASPs).
So far I have found that people who believe that the variance of some traits such as intelligence between population groups is dominantly the result of genetics do so because they think it provides a moral justification to the social order: things are as they ought to be because nature dictates so. I see no connection between the two. Nature (if it is, in fact, at play) has little bearing on ethics[3], and thus can, at most, explain but never justify an unfair distribution of power.
Tom and I disagree on how much of the underrepresentation of women in tech is due to skill deficits. His new platform can answer that question quite effectively.
My disagreement with you stems from the fact that I don't believe in any privileged population groups - I only believe in individual rights. I may or may not disagree with you about "power" but so far you've yet to provide a clear definition of it. (I did read your wikipedia links, but they provided multiple disparate and unclear definitions.)
> His new platform can answer that question quite effectively.
Assuming his game tests for skill and skill alone. Though even if so, it wouldn't explain the difference -- just report it.
> I don't believe in any privileged population groups - I only believe in individual rights
I don't understand. One of these things is normative (individual rights) and the other is positive (privileged population groups). The existence of privileged population groups is a matter of fact[1] -- no one thinks they should (normatively) exist. As to individual rights -- everybody believes in them, too. The question is what would be their nature. For example, I believe that if the wealthy were allowed to wield their power (money) over the poor unhindered, then the poor should be allowed to wield their power (numbers) unhindered over the rich as well. The point is that power, by definition (see next paragraph), means restricting in some way the freedom of others, so to obtain freedom you must either restrict all power or unloose all power.
> they provided multiple disparate and unclear definitions.
Perhaps, but not different enough or unclear enough to preclude study or reasoning. The gist of it is, power = the ability to bend (or sway) others to your will. Power is measured by how many people you can sway, and to what degree you can sway them.
[1]: For example, that white men are more privileged in America than black men is a fact.
The existence of privileged population groups is a matter of fact[1] -- no one thinks they should (normatively) exist.
As a person who cares only about individual rights, I don't care if membership in some particular group is correlated with lack of privilege.
If you want to argue that lack of privilege is an individual injustice, fine - but then you need to stop discussing race since there are plenty of privileged blacks/women and underprivileged asians/males.
...not different enough or unclear enough to preclude study or reasoning.
You assert that white men are more privileged than black men. The definitions you've provided are insufficient for me to concretely state a test we could run to to disprove that.
For example, being black will sway college admissions officers for you but police against you. How do any of these definitions allow me to say that on balance, these things are negative? At what magnitude would the balance become positive?
Anyway, this is completely tangential to starfighter.
> I don't care if membership in some particular group is correlated with lack of privilege.
And what if lack of privilege is caused by association with the group?
> If you want to argue that lack of privilege is an individual injustice
You may believe in individual rights, but you can't deny group injustice. Blacks were made slaves not due to any individual selection.
> For example, being black will sway college admissions officers for you but police against you. How do any of these definitions allow me to say that on balance, these things are negative? At what magnitude would the balance become positive?
You can't possibly be serious. But just in case you are, there are clear tests to measure power: money, positions of control in the private and public sectors, positions of control in the media. QED
I don't deny group injustice. I don't believe injustice based on group membership is worse than any other kind - slavery is an injustice even if it's equal opportunity. Insofar as having bad schools or bad parents is not an injustice, it's also not an injustice if some group has more of them than another.
And insofar as such a thing is an injustice, the fact that it's correlated with some group is irrelevant to me.
If the only way power can be measured is via outcomes (in this case, I guess a high developer salary?), then I'm not sure why we need a new word to describe it. I also don't get what your point is. I guess you are arguing that smart people don't "deserve" the money/control that comes with a developer job any more than dumb people, and therefore starfighter is a bad thing?
Well, group injustice is just individual injustice done to many individuals. And, I think, you would not consider it unfair to demand a robber to pay back the money he's stolen. The problem with underprivileged groups is not the reality of the offense against them, nor the identity of the victim, but the identity of the perpetrator, and this is where study and of power comes in. Because the reality is that human society, like gas molecules, performs many acts not as individuals but as a group (even collective property preceded the invention the first private property), and therefore the perpetrator in the offenses of underprivilege is society as a whole. The actions under discussion are not as acts of nature but acts like for men, acting as a group (and collective action is a positive fact). Assigning individual responsibility is as futile as assigning individual responsibilty to specific gas molecules in expanding a balloon; doing so is just an ineffective model. And what society has stolen as a group (from individuals!) it must pay back as a group. Identifying the victims as a group is just a matter of statistical convenience as well as an aid in the description of the dynamics.
That an electron is measured through its effect does not mean that we don't need to describe the electron itself. In fact, it is crucial that we do in order to understand its action. Same thing for power: you almost never observe it directly, but its study, and the term itself, are required in order to understand the workings of society.
And as to this game, I don't think it's inherently bad at all, but if its role in society is not studied it might become an unwilling accomplice to injustice. And, if, one day it is somehow discovered that abilities that convey power are dominantly genetic, I do not think that we should give certain jobs to people unsuited to them, but our society is judged not by the achievements of those born to privilege, but by how it takes care of those who lack it. Exactly how this moral obligation should be carried out is a complex matter in itself, and far beyond what I can write here.
That an electron is measured through its effect does not mean that we don't need to describe the electron itself.
We certainly do. So look at what physicists did. First they came up with a clear definition - "a discrete and indivisible negatively charged particle". Second, they went out of their way to distinguish concepts like an electron from charge as a fluid and other models. They didn't simply declare "well, electric current proves electrons exist", they went to crazy lengths like Millikan's oil drops to distinguish these concepts.
When they can't actually distinguish these concepts intrinsically (as was the case with Maxwell's equations vs Aether), they tend to drop the more complex theory.
In contrast, you seem quite resistant to doing any of these things. I don't quite understand why.
Resistant to what? I just gave you a clear and concise definition of power (the ability to bend others to your will). The social sciences did all of the required work as well, I'm just unable to write a whole sociology book in HN comments. There have been countless studies in sociology, history, anthropology and psychology examining the different forms of power and how it works. But bear in mind that these sciences are much more complicated than physics; in fact they're intractable. The simplest social mechanism is more complicated than the gravitational interactions of 1 million bodies.
Actually the example I’ve give of the electron is a rather weak one, as the electron is a very specific thing, while power is the most fundamental concept in all the social sciences except psychology (i.e. history, sociology, anthropology and political science). A better example — the obvious one, in fact — is energy. Energy can also only be measured through its effects, and yet it is a very fundamental concept. Describing it not as one thing but as disparate manifestations would take away its explaining power and many of our most useful models (I should really write a book called “social science for physicists”).
I’ve also given more thought to your focus on individual rights. Individual rights are an obvious “good thing”, but here’s where they get complicated: When I think of individual rights I imagine a universe composed of mass but no force. Such a universe will be no more than a cloud of plasma. But in our universe, mass gives rise to force, and force creates the interesting interactions that have, in turn, created our world. Likewise, human society is not made of humans, but of human interaction, interaction gives rise to power, and power, by definition, restricts freedom. Now, this is not a bad thing necessarily, as, if you think about it, all cooperation is basically the voluntary yielding of freedom in order to concentrate power for some common goal that wouldn’t have otherwise been achievable. But that doesn’t change the fact that whether they like it or not, just as mass-ful particles induce force, humans induce power, and both force and power take away freedom from others. So saying something like “everyone should be free” makes little sense, as that is only possible in a plasma cloud. In order to grant freedom, freedom has to be compromised — based on some values — and then managed somehow. Obviously, different people will prefer different compromises. Personally, I’d either like to see all power restricted and controlled, or all power unrestricted (including physical violence).
This is why I think that American libertarianism is either hypocritical or ignorant. It is either ignorant of the fact that there is no freedom without power (and hence, coercion), or hypocritical in calling for unrestrained use of certain forms of power alone (money) and not others (physical violence, preferably mediated by a democratic government that restrains the use of money).
I was therefore delighted (intellectually, that is) to learn recently of a fringe Silicon Valley movement called neoreactionism or “Dark Enlightenment”, funded by Peter Thiel. These guys (few women would join that openly sexist, openly autistic movement) are probably all former libertarians that have discovered that there is no such thing as freedom from power, and now openly call for a tyranny. As someone who’d studied medieval history in graduate school (though I have never obtained my doctorate) I was delighted to see the movement’s leader, a programmer by the name of Curtis Yarvin, analyze some historical document and call for the return of feudalism (he complains that those documents are not studied by historians, which is true for the simple reason that they are false accounts).
The problem with the return to feudalism — even if you were to believe the false accounts of how life was good then (it by no means was) — is that the power structure back then was at least held in check by technology, that is, lack thereof. With the invention of mass media and fast transportation, power can be more concentrated than ever before, which is why the greatest invention of Western civilization was the central government, which rose to contain and manage power (of course, this only made conflicts among those governments more violent than ever before, but that fear of annihilation reduced the number of conflicts considerably). BTW, the modern academic definition of politics is, not surprisingly, the management of power in society.
Feudalism combined with modern technology has only been tried — to the best of my knowledge — once, in nineteenth century America. The US at the time had a very weak federal government with almost no regulation. The result was a period of extremely fast economic expansion but at great social cost: a large portion of the American population was enslaved in all by name by a very small number of slave-owners-in-all-by-name known as the Robber Barons (Rockefeller, Carnegie, Mellon, Stanford, JP Morgan, Frick, et al.). All options were taken from them — they couldn’t migrate (they were sometimes paid in company-issued currency, that was useless anywhere else) and couldn’t organize to concentrate power to improve their lot (in fact, they did organize, but the robber barons had private armies that killed the rabble rousers and intimidated everyone else). The people cried for help, and Theodore Roosevelt rescued them by creating federal regulation.
Energy is also well defined - for any particular case you'll be able to write down a precise formula defining it. For example, a system of colliding particles has E=sum(p^2/2m) (p is momentum, m is mass).
If you want to make a claim that one system has more energy than another, testing that claim will be easy - just apply the formula. If you wanted me to take it seriously, you should have applied the power formula to figure out whether influencing college admissions officers > influencing police - all you did was scoff.
I assert that your use of the word "power" is pointless. It transmits no information about the world, much like how you use privilege. Why do you insist on engaging in long discussions advocating the use of words that mean merely "any probabilistic cause of social outcomes"?
Information can most certainly be conveyed outside the use of precise formulas, lest you think that entire fields of human endeavor are bunk (merely "long discussions"): art; music; aspects of philosophy, anthropology and sociology; etc. Nonetheless, I would posit that a number of quantifiable metrics contribute to pron's definition of power (the ability to bend others to your will), such as net worth, yearly earnings, spending patterns, education levels, employment statistics, and social network models, especially when these are considered over time.
These "long discussions" that you show disdain for are attempting to introduce a semantic base on which "well defined" terms can be further developed and evaluated in light of new social experiences (e.g., the latest claims of * -ism in SV). pron has set out an extended metaphor, say, which is akin to describing how one would perform an oil drop experiment. That is the opposite of "resistance". Your assertion that "power" and "privilege" are semantically empty is completely without grounding. Just because the metaphoric equivalent of "applying the power formula" has been left as an exercise to the reader, does not mean that the terms of the equation are void of meaning.
Power is also well defined (I have provided a definition which is not "any probabilistic cause of social outcomes") and in cases like this -- easy to measure. You may assert the last forty years of research are pointless. That it's definition applies in many cases (though not all -- an earthquake leveling a city isn't directly related to power) does not make it any less useful (just like energy is always involved in any interaction of particles). If you apply the definition, you will see that influencing college admissions officers << influencing police in this case.
The fact that intractable interactions lend themselves less easily to formulations (let alone closed expressions) does not take away their reality or invalidate the model. Much of the work in applied mathematics (non-linear equations) is qualitative, as well. I think it is you who are resisting to admit that the past decades of research have taught us a lot about how society works.
If your definition is not simply equivalent to social outcomes, there should be an experiment that can potentially measure power absent social outcomes. If so, what is it?
I.e., suppose a group has good outcomes but low power or vice versa. How can I find out?
You can't measure power without its effects just like you can't measure energy (at the very least, it will have to affect your measurement device), but you can often tell whether the effect was due to power or not. Sometimes -- like with the case of potential energy -- you might be able to deduce its existence once you've learned how it works for a while. But, you can often see powerful people fail (say, a billionaire dies in a plane crash, or loses all his money when the market crashes), and powerless people thrive (a homeless man wins the lottery). But power, like force in physics, is the main thing driving human interaction. It is certainly the only mechanism of any interesting social dynamics, but it is not trivial, as power -- like energy -- takes many forms.
So, to me, your question sounds a bit like, "how do we know when the planets in the solar system move by gravity, and when they move by something else?", to which the answer is that the planets almost always move by gravity, except very rarely, when, say, hit by a particularly large asteroid; how doe we know when that happens? We look. Same here, if a group has good outcomes and low power and vice versa -- while very rare (as power is at the core of the mechanism), you can either study the case carefully (which is what historians do), or compare it with power's known outcomes to see if it's one of those flukes. But, you'll say, I can isolate gravity and test it in a lab to make sure I'm certain this is how the planets move. Well, experiments like that are harder in the social sciences, but they are done quite regularly. Two very famous experiments in power are the Milgram experiment (testing authority power) and the Stanford prison experiment (testing authority power as well as its effect on those who have it). Many dictator games are experiments in other forms of power.
Besides, I don't see what exactly you're driving at. Thousands of studies have uncovered some mechanisms at the very core of human society. The mechanisms behave similarly enough to warrant a name (kinetic energy, potential energy etc.), and that concept seems to be at the heart of what drives most of society. Not only that, it induces a quantifiable (if sometimes only roughly, or even in theory) property. That mechanism, along with its quantifiable trait is called power. It was found to be roughly "the ability to bend others to your will", and has produced interesting, useful models (qualitative -- not quantitative). You want to give it another name? Fine, call it X. But 100 years ago we did not know about X as much as we do now. If you want to identify X with something that you think has been known for a long time -- you'll be wrong; if you want to identify X with something you think is still a complete mystery -- you'll be wrong again. You want to argue with scientists about the names they choose and then quote someone who says arguing about names is futile -- great. What is it that you're saying?
The concept of power conveys a lot of knowledge that has been gathered over decades. Your responses seem to be like those of someone who's just heard of energy, and says, "If energy is what moving things possess, why not just call it speed? Oh, a ball at the top of the hill also has energy, why not just call it height? Oh, fire has energy too? So energy is everywhere, and if it's everywhere then it doesn't mean anything!" Either that someone decides to learn basic physics, or decides to stay ignorant. But if he decides to stay ignorant, I think you would agree it would be foolish of him to continue arguing.
I turned the page. The answer was, for the wind-up toy, "Energy makes it go." And for the boy on the bicycle, "Energy makes it go." For everything, "Energy makes it go." Now that doesn't mean anything. What they should have done is to look at the wind-up toy, see that there are springs inside, learn about springs, learn about wheels, and never mind "energy."...Now that doesn't mean anything. Suppose it's "Wakalixes." That's the general principle: "Wakalixes makes it go."...It's also not even true that "energy makes it go," because if it stops, you could say, "energy makes it stop" just as well.
I claim that this critique applies equally well to your use of the word "power".
And your comparisons to real sciences are quite inapt - again, as I've pointed out to you before, a discussion with tptacek on crypto or kasey_junk on high speed trading results in the aforementioned posters being very specific while their critics are vague. Kind of the opposite of what is happening here.
Note that you still haven't actually provided an experiment or measurement that could identify a successful yet powerless group (if such a thing existed), or vice versa.
You keep saying "your use of the word power" as if power is not a well known, well studied concept. It is not "my use of the word power" but simply power. You did the same thing when we discussed sexism, which is an academic term invented relatively recently by feminist scholars, which you insisted on treating as some obscure, ill-defined notion. This tone shows a misplaced contempt to a vast scholarly endeavor.
In any case, I don't see how that critique applies to power at all, because, yet again, some of the mechanics involving power are well known and well documented. Nobody says "power makes it so". It's just that explaining the power dynamics of racial neighborhood segregation or sexism in tech would take dozens of pages.
It is not simply that "power is what drives women participation in tech down". I can trace a process -- some documented and some hypothesized -- starting with "classical" gender roles, through the massive transition in gender roles and general separation between the sexes that occurred in Victorian times (they had rooms in houses meant to serve men and rooms for women) and shapes society to this day, through the history of women in computing (starting with the transition of switchboards from being seen as a job for women to one for men), with the more general association of which jobs are for men and for women. That would take me about 50 pages, I guess. But power is the central mechanism. I'm not saying "power did it"; I can show how. Just not here.
> And your comparisons to real sciences are quite inapt
Well, I've been using metaphors, naturally. The intractable sciences are much more complex than physics, chemistry and even biology. There are no closed-form formulas in the social sciences; at least not yet.
> a discussion with tptacek on crypto or kasey_junk on high speed trading
Maybe they're just better communicators than me, and maybe HFT is more amenable to discussion in HN comments than the history of gender roles and the evolution of power in human society. However, if you have specific questions (and they would have to be more specific than "how come there are fewer women in tech") I could try to answer succinctly if it is at all possible. The problem is that these are things that are never even taught to first-year social sciences students (some are only taught in grad school), and unlike with HFT, I don't think you even have the basics.
For example, I don't know if you're at all familiar with the techniques used to study history or sociology, how historical documents are analyzed, how different societies are compared etc., and I really can't lay out an intro to social studies here (BTW, that Curtis Yarvin guy I told you about suffers from the same problem, except he considers himself knowledgable for some reason. His writings read like an Aristotelian scholar discussing quantum mechanics; he's completely ill equipped to handle the materials he's using, which is why he draws such ridiculous conclusions. Of the months spent teaching students simply how to approach reading documents, he doesn't even apply the very first lesson: classifying the genre of the document and identifying the intended audience and purpose)
Now, I'm sure that there are some introductory materials to gender studies that skip the basics of social science, but I doubt you'll find them convincing if you're not familiar with the methodology. If you are interested, I could try to find some online course in history or sociology that seems good, but my guess is that they won't get to gender roles in an intro course (and if they do, it will be by skipping the groundwork, which, again, will make it seem less convincing).
> Note that you still haven't actually provided an experiment or measurement that could identify a successful yet powerless group (if such a thing existed), or vice versa.
You still haven't provided an example of a planetary system whose planets revolve around a star due to a force other than gravity! Gravity is what makes planets revolve around a star, and power is the mechanism by which groups (and individuals) obtain success. Once in a while there are aberrations, to which I have provided examples: winning the lottery. Or, if California is covered by the ocean, then the very powerful people who live their might become extremely unsuccessful. Of course because that population is powerful, various disasters would probably be addressed by the government faster and with more rigor than in other parts of the US, but that may still happen.
You still haven't provided an example of a planetary system whose planets revolve around a star due to force other than gravity!
I asked for a measurement which could identify such a group, not a measurement that would. I can easily tell you experiments to test this in physics - solve Newton's law of motion and find a celestial body with motion that doesn't agree with it.
If I were advocating for the invisible roller coaster track theory of celestial motion, I couldn't provide such an experiment. The invisible roller coaster tracks are observable only by celestial motion - whichever way the moon moves, that's where the track is.
The only way to refute the theory would be via an alternate method of observing the position of the tracks and then observing whether the moon actually followed that track. If someone didn't provide that alternate method, I'd say he was not even wrong.
Well, there are numerous examples of groups that had little power that achieved success, but remember that there's a feedback loop here, as once you achieve success you obtain power. But everywhere you see social mobility, those are cases where people with little power slowly obtained success, which then turned to power.
Examples from the middle ages include grants of knighthood[1] as payment for some unusual service. While usually a knight would only come from wealthy or noble families (or at least a family with good connections) -- hence, from a position of some power -- sometimes knighthood was granted to brave foot soldiers -- i.e. people with little power. Sometimes, the title came with land (and the serfs that worked it, of course).
In non feudal societies, social mobility was usually achieved through money, although some classes were barred from obtaining any money whatsoever (slaves). You can see groups of immigrants, provided the host society did not block their steps too much, slowly gain money, and later recognition and connections. This process would often take several generations.
Analyzing those processes is helped by the fact that often you can observe power directly. Money and nobility titles are very conspicuous forms of power, easily measurable directly. More hidden forms of power such as connections can also be traced directly (a boy of low background would be taken to the home of a merchant as a gift to his parents in recognition of some service; this lets you trace connections across classes); charisma (which in the middle ages was a great way to attain power in religious circles) could be seen in some extraordinary ascetic acts[2] or visions[3]. The latter was one of the few ways women could rise to positions of power in medieval societies (see Joan of Arc), although others would be marrying, and surviving, a man of power. While it was often expected of widows to remarry, some medieval societies were surprisingly relatively accepting widows, recognized their independence, and allowed them to transact on their own.
>I was therefore delighted (intellectually, that is) to learn recently of a fringe Silicon Valley movement called neoreactionism or “Dark Enlightenment”, funded by Peter Thiel.
AFAIK Thiel's sole, extremely tenuous connection to the Dark Enlightenment is thinking libertarianism and democracy are incompatible. He certainly hasn't funded any of its leading lights. This is less accurate than the belief that the Koch brothers control the Tea Party. (They did have a large impact on its early growth.)
>These guys (few women would join that openly sexist, openly autistic movement)
Ableist. If you want an example of a woman who's been involved (more than periphally but not as an identified adherent) look up Justine Tunney. If you're transphobic then she doesn't count as a woman, ableist.
>are probably all former libertarians that have discovered that there is no such thing as freedom from power, and now openly call for a tyranny.
That's one branch of the trichotomy in case you're interested.
>As someone who’d studied medieval history in graduate school (though I have never obtained my doctorate) I was delighted to see the movement’s leader, a programmer by the name of Curtis Yarvin, analyze some historical document and call for the return of feudalism (he complains that those documents are not studied by historians, which is true for the simple reason that they are false accounts).
This is a massive misreading of Yarvin, a.ka. Mencius Moldbug. He's got a hard on for absolute monarchy, not for feudalism. The two are very, very different. Feudalism was basically a Western European phenomenom, Ottoman depotism, Russian autocracy or France during the reign of the Sun King are more his thing.
What false accounts are you referring to? Could you provide some links to the deceptive documents among Yarvin's output?
>The problem with the return to feudalism — even if you were to believe the false accounts of how life was good then (it by no means was) — is that the power structure back then was at least held in check by technology, that is, lack thereof. With the invention of mass media and fast transportation, power can be more concentrated than ever before, which is why the greatest invention of Western civilization was the central government, which rose to contain and manage power (of course, this only made conflicts among those governments more violent than ever before, but that fear of annihilation reduced the number of conflicts considerably).
Autocracy, not feudalism.
> Feudalism combined with modern technology has only been tried — to the best of my knowledge — once, in nineteenth century America.
This is so ridiculous that it makes me question your claims of having studied history at a graduate level. Have you ever heard of the Bolsheviks? They had a successful revolution in the Russian Empire in 1917 and founded and ruled the Soviet Union until its dissolution. They arose in a state that attempted to combine autocratic government with modern technology.
>The US at the time had a very weak federal government with almost no regulation. The result was a period of extremely fast economic expansion but at great social cost: a large portion of the American population was enslaved in all by name by a very small number of slave-owners-in-all-by-name known as the Robber Barons (Rockefeller, Carnegie, Mellon, Stanford, JP Morgan, Frick, et al.).
Are you aware that the USA has been among the richest societies on Earth for its entire existence? People were poor because there was so little to go around, not because people were hoarding for the sake of it. You occasionally say lucid and intelligent things but the US economy was growing insanely fast by any historic standards more or less from settlement by Europeans to around 1970. It's still growing insanely fast but the trend has slowed down. economic growth in North America was labour limited for a long, long time. Things were much, much better in North America than anywhere else on Earth, all this while having large inflows of migrants from much poorer nations, i.e. the entire rest of the planet.
>All options were taken from them — they couldn’t migrate (they were sometimes paid in company-issued currency, that was useless anywhere else) and couldn’t organize to concentrate power to improve their lot (in fact, they did organize, but the robber barons had private armies that killed the rabble rousers and intimidated everyone else). The people cried for help, and Theodore Roosevelt rescued them by creating federal regulation.
Did you study under Howard Zinn[0] or something? Everywhere else was worse. The Pinkertons were awful but the USA has never been a weak enough state to allow private armies anywhere on its territory. It was probably the friendliest country on Earth for labour organising for the period in question.
My knowledge of the Dark Enlightenment is very limited at the moment, as I've just discovered it during preliminary research for a study of fringe movements in California related to technology. But during a cursory reading of Yarvin's I've found a few mentions of feudalism as a desired goal (obviously, under some weak autocracy), as well as mentions of traditional gender roles and a defense of slavery. I also believe that Peter Thiel funds Yarvin through a startup company of his. I will not list the false accounts he mentions (I noted down a few so far), because I don't want to discuss the subject here. A cursory glance, though, reveals the Dark Enlightenment to be a fascinating Californian fringe movement, with some truly novel interpretations of familiar ideologies (mostly fascism, but not quite, as fascism was directly influenced by Romanticism while DE rejects Romanticism except that restricted Ayn-Randian form of it, and fascism was very much nationalistic -- again, through romantic influence -- while DE isn't).
As to your description of the Gilded Age, I am not sure what our points of disagreement are. Yes, the Pinkertons weren't actual armies, nor were they entirely ignored by the government. My description was greatly simplified for brevity. As to the conditions of workers at the time, comparisons to other societies are irrelevant, because that economic growth (which was the result of immigration, land expansions and new resources) was not at all contingent on the exploitation that was taking place.
In general, comparisons are often less useful when the framing discussion (in this case, questions of policy) is normative. If a tyrant comes to an island where the population is starving, and feeds them a loaf of bread a day but forces them to do backbreaking work for him, then their position is better than the alternative, but no one would suggest that this is in any way desirable or even ethical. The same would be true for another tyrant who feeds his enslaved population two loaves of bread a day.
Comparisons are useful if the claim is made that no other policy would have been possible, which I don't think anyone is making. Sure, growth would have been slower (concentrated power is always a lot more efficient, as time and resources are not required to achieve compromise and accommodate other stakeholders), but preference of efficiency over other human goals is purely a matter of value.
The Bolsheviks instituted an autocratic form of communism, which is pretty much the complete opposite of feudalism (which is usually the result of a free market, although the terms are anachronistic as no one used the term "free market", when real feudalism was actually in place), and in any case, feudalism is certainly anathema to any ideology promoting equality (like communism).
"What will be your new belief about $GROUP's technical talent"
Hopefully the most reasonable one: that there's no such thing about the technical talent of "$GROUP" per se, though our society might presently be so organized that it's easier for members of $GROUP to excel at certain kinds of tasks.
Of course there is such a thing - cook up a tech talent score based on these test results (see my other post on that topic) and then measure a group's mean, sigma or credible intervals.
How society is organized may be a cause for a group having higher/lower technical talent (so might "intrinsic" features, i.e. things not mutable by "society"), but it doesn't mean that talent doesn't exist.
The thing is that the effect of such biases is hard to measure directly. Employers will get better employees than before, but may also miss more good employees than before, yet notice the former much sooner than the latter (which will have a delayed effect).
The best way to address it is to first measure for biases (run a demographic analysis on players), and second actively design for unselected populations with focus groups. I am sure some of this information has already been collected by game studios.
My first thought when reading the blurb on your site was 'oh great... they're going to make a CTF where you have to participate in a team, and all other members of your 'team' are actually AI programmed to be as distracting and abrasive as possible and you will have to work on a 25 year old PL/SQL system designed by a prima donna so you can show your ability to work in hostile environments.' I am sure that's not what you're doing, but that's what comes to mind when I hear 'real systems' and 'real challenges'...
On your "About Us" page, you say "We're especially interested in talking to you if you will be hiring 10+ engineers in the next year, as we may be able to help you out with that. Get in touch with any of us."
Does a company need to be hiring 10 or more engineers to make it worth it to talk to you?
I'm at a small company, that has been trying to hire engineers for the last couple of years, and not having much success; we've hired a couple, had a couple take other offers, lost a few to attrition.
We would love to be able to get some candidates who have already passed a technical screen; less time for us, scheduling an hour with a bunch of candidates who don't pan out, less time for them, only focusing on one more in-depth set of problems rather than a whole bunch of different phone screens.
But we're not hiring 10+ developers any time soon; that would double the worldwide size of our dev team. We're looking for maybe 3 or so at this point.
Would you consider working with smaller companies as well?
I obviously can't speak for the founders, but I suspect that if you're viewing this as a replacement for a technical phone screen, you might not be viewing it as intended.
Did you read Thomas' "The Hiring Post" which was at the top of HN a couple days ago?
I would love to improve our hiring process, and include more realistic work samples as part of it.
However, I don't feel that a full-fledged work-sample test is realistic, especially if every employer started requiring it. Imagine applying to a few different jobs, and each requiring you to do 20 hours of learning of their material followed by some open ended realistic problem, that could take many hours to complete. If you were looking around at a few different employers, this would be a full-time job itself; some candidates would skip it because they could find a job which required less effort up front easily enough, some would simply not have the time because they are working a full-time job while doing it.
On the other hand, if there is a shared work-sample test like this one, that will be used by a number of employers, it means that it can be more in depth than any single employer could do, while simultaneously wasting less of the candidate's time as they do the one test, rather than one per potential employer.
So yes, in a way I am viewing this as a replacement for a technical phone screen, because I am interested in improvements to that process but haven't come across any potential improvements which seem viable, but this one sounds like it could have potential.
The "read the resume, read their existing code samples, and do a phone screen" approach covers a lot of ground in a little time, so while there are a large number of problems with it, it's pretty hard to improve on without spending a lot of the candidate's time to get up to speed on a more realistically sized problem, and a lot of good candidates aren't willing to devote that much time unless they really, really want to work for that particular employer. We're a small company, working in a niche business, so it's hard to get that kind of dedication from a single candidate.
I'm totally happy to talk to anyone who hires engineers, is interested in what we're doing and/or wants to know more about how we made this work at my last gig. Please don't hesitate to reach out; you can't waste my time.
They just say, "especially," and I think a charitable reading would tell us that it's a call to get a good base of early adopters from which more-easily generalizable results can be gleaned.
This looks great. Looking forward to trying it. Good on all of you for trying to change the tech hiring status quo.
I know this whole thing is still in flux, but does it look like there are going to be any remote work opportunities in the early days? I'm probably in the minority but given my personal circumstances (living in a non-tech-rated US city and not considering moving) I'm really only available for remote.
What do you expect will be the effect of this on hiring women? Do you have any relevant data from microcorruption (what % players were women)? It'd be great if Erin could also chime in.
We do not have access to the µc data, any longer. That belongs to Matasano. I am now sad that I did not ask myself the same question and pay closer attention while I did.
However, this is what I am hoping and dreaming for: that work-sample testing will completely level the playing field. Regardless of gender, age, or origin, if you can do the work, you should get the job. I believe in this so strongly that I am dedicating the rest of my career to work sample testing and, eventually, training.
Getting women to play, though? I'm not so great at marketing, so still working on that. I am writing a blog post to go up later in the week that addresses some of the issues in hiring women, based on my personal experiences and those of my female peers. The tone of the post is as politically neutral as it can be. My hope is to draw out some productive dialog rather than piling on stink for flies. Specifically, vast majority of the people in my professional network, male, female or in between, are genuinely interested in addressing the "women-hiring problem," but they avoid any discussion of it because ... well, it's always covered in flies.
The challenge will be to make just enough of a stink that it draws attention and not flies.
> However, this is what I am hoping and dreaming for: that work-sample testing will completely level the playing field. Regardless of gender, age, or origin, if you can do the work, you should get the job.
That's a nice thought, though it rather presumes that the playing field is level outside of the immediate neighborhood surrounding the hiring process.
I think that neighborhood, though, is probably, while not without gender/race/etc. bias, overrated in terms of the proportion of that bias on the course between birth and getting a software job that it is perceived to contribute.
Believe me, that has crossed my mind. This is where the segue into training comes. We've discussed it. As a woman, my one experience of being allowed to play outside and beyond the hiring process is pretty consistent with the anecdotal evidence given by my peers. But correlation is not causation. For every one candidate, there are an uncounted number of pivots to consider, and mapping them all out is impossible.
I agree that bias is overrated as a contributing factor to this so-called problem. But it is one we focus on because we also believe that behaviors can be controlled for and habits can be rehabilitated. There are a lot of narratives where the circumstances are very different but the dynamic is the same. Someone whose parents want them to be a doctor or lawyer who aggressively derail them from pursuing the arts. Some very intelligent kid growing up in poverty with no access to resources who starts acting out and ends up with a criminal record. A woman who is forced to take HomeEc in high school instead of trigonometry (me).
Taken in isolation, these narratives can be explained away. As examples of a larger pattern, they become flaws in a system that can be engineered away. There's a meta-bias, and I'm still searching for a way to describe it that doesn't trigger an emotional response. It seems that until we can have that rational, engineering-focused discourse to identify the not level playing fields of the world, cataloging their characteristics, finding what things can be controlled for and/or eliminated,... sigh I don't know how to finish that sentence.
You say it's a nice thought. I believe it's a nice thought. More people hope for it than do not. What exactly is keeping all of us who genuinely believe that it's at least worth a try to apply a dialectic method to the problem from banding together and attempting it?
I think the desire to to figure out where a level playing field can be built within a neighborhood of aptitudes. And to be fair to a process of refinement for that goal, some problems can't be solved before you have them. Like a business plan, the initial draft is just a guess, and I think we can all agree that it's a heretofore unsolved problem, the gender thing.
Also true for large coding projects. At some point, you have to identify the components that will need to be built and portion them off into approachable tasks. In this case, the first task is agreeing on the 'heretofore unsolved problem' -- that hasn't happened yet.
Discerning invested participants from trolls might help.
>The tone of the post is as politically neutral as it can be. My hope is to draw out some productive dialog rather than piling on stink for flies.
I am sorry, but that is a line with a negative width: if you so much as hint that this may in part be the fault of males (as a whole or, even worse, individuals) pitchforks will be out. If you do not hint strongly enough that this is the fault of males (as individuals and as a group) pitchforks will be out on the other side. As a more personal point the sheer toxicity of those debates continues to shock me, even after spending more than a decade on the internet. Vi vs Emacs debates don't tend to end in treats of rape and/or genocide but "why aren't there more women in computing" almost always do.
This isn't to say that we can't draw any benefit from your writing, just be aware that of the things you can't say (http://www.paulgraham.com/say.html) this is the thing you can't say most.
Hi, I've taken over from Tom for hiring at Matasano. There's a couple of things that you need for "diverse" recruiting (e.g. hiring women in tech):
1) A way of evaluating candidates that avoids mirrortocracy style institutional -isms like sexism and ageism.
2) A way of convincing people who have bounced off the field due to -isms to even try.
BTW, Tom was great at Matasano on #1, and only moderately successful at #2. We get tons of people who aren't in the industry, but they tend to self-select to be young men. For an individual company, #2 is way harder. It's "easy" to fix yourself, but hard to fight the larger culture.
Starfighter looks to me like something that supports #1 directly (by allowing objective metrics), and enables #2. It's a way for people who aren't welcome into a field to dip their toe in without some roomful of young white men asking them illegal questions about their child-rearing plans. On the other hand, there's still tremendous pressure pushing people away, and it's a deep problem. Still, part of the puzzle and I (personally) highly endorse.
As to the numbers, I've been meaning to dig into the data, but Matasano's hiring as fast as we need to, so it's been hard to motivate myself to do so. Maybe once graduation season ends...
Yes. And as someone who also was stuck for many moons on later levels of that challenge: keep going. Research. Ask questions -- people will help. Honestly, I had no expectation of getting past the first microcorruption level but goddamn if it doesn't feel good when (not if) it happens.
if you're monitoring everything players do, doesn't that create an incentive for them to write their own tests locally and only interact with your servers when they have stuff already working? so they look like a genius instead of a guy who made a bunch of mistakes. "look i never make any syntax errors, hire me". but seriously whatever you monitor, players will have incentive to try to figure out and game it. and that incentive messes up the regular experience, it's an incentive to do something other than beat the level.
First, no part of our model involves monitoring how many "mistakes" someone made, and second, if you see what challenges like this are like (again, see Microcorruption), you know that there's no realistic way to do them that doesn't involve making zillions of little mistakes.
I bet that Thomas and Patrick could do something like have dynamically created tests, where there are multiple versions of an exploit, so that they test the same skill but in different ways. You might have a challenge that tests whether you can do SQL injection, but the tables may differ. Overrunning a buffer (as Patrick mentioned in his post about Microcorruption) could depend on different payloads.
I naiively believe that things could be varied enough that you would need to understand the concepts rather than paste answers, in which case "cheating" would mean "learning", and is basically something they say they want you to do. I'm not sure how hard it would be to get to that point rather than being vulnerable to Bob the Super Coder posting walkthrough screencasts, but I trust that the founders are already savvy enough to have been thinking about that.
Unfortunately, I suspect you overestimate the obfuscating power of dynamic content when the number of users is sufficiently large.
A certain MMO I play recently had a limited-time event built around figuring out the meaning of different clues (locations to go to for the actual meat of the event), and despite a fairly large number of variations, people had collectively figured out just about every possible clue->location mapping within a matter of hours.
That's not to say you can't prevent cheating, but that even with relatively little incentive (that whole clues thing gave only a single cosmetic item, and anecdotally I've seen very few people actually use theirs) users can and most likely will outpace any attempt to prevent it by means of varying the problem.
This is an important problem. As part of interviewing processes, I have several times tried offering "take home work challenges". I had to stop after, having run the experiment several times, I detected plagiarism in about 30-40% of the cases. The risk is of plagiarism is real for any problem that's well known enough that the problem and solution can appear online, and detecting plagiarism is not always easy.
I like this response. I think that one of the best things about a take-home challenge is that during the interview, you can then say, "The way you implemented this function was interesting. Let's talk about why you did it that way." If he knows what he's doing (or made a weird solution), you get a great glimpse into his thought process. If he plagiarized, then he's garbage.
English teachers do the same thing for detecting plagiarism. Bring the kid into your office and talk about the essay. If he's completely clueless, someone else wrote it for him.
It's a neat idea, but ultimately, worrying about cheating or whether someone has cheated feels like a distraction. It suggests a test that isn't repeatable. I'm only confident in my ability to detect obvious plagiarism. Subtle plagiarism can exist in varying forms, such as reading an analysis of the problem and solutions. Some people who plagiarize will pass a thorough Q&A about their code, because they fully understood the explanation of the solution, but are getting an unfair advantage over candidates who worked out a solution themselves from scratch.
Overall, I get more value from tests that are constructed so that I can learn positive things about the candidate - as many opportunities as possible for the candidate to distinguish themselves. If the only reason for a particular approach was to provide the opportunity for immoral candidates to weed themselves out by committing obvious plagiarism (negative data), then there's probably a better approach that tells me more about the candidate per unit time.
If I was going to continue, I would use a problem that is (1) more representative of the actual work being done by the team; less of a puzzle (2) custom designed for the team or company; not a preexisting or well known problem. (Even candidates who don't cheat can have an unfair advantage on well-known problems if they have coincidentally encountered it before! Another reason to use unique questions.)
> Some people who plagiarize will pass a thorough Q&A about their code, because they fully understood the explanation of the solution, but are getting an unfair advantage over candidates who worked out a solution themselves from scratch.
This reminds me of the debate over whether performance-enhancing drugs should be allowed in mathematics. Why do you think it's so important for the candidate to personally invent every aspect of their solution? What if you just told people that it's ok to use external resources to solve the problem?
A class might give exams in any of these ways:
- exams only happen in class, where everyone can notionally be supervised
- exams are take-home, but you can't read the textbook while you're taking one
- exams are take-home, and you're free to read the textbook
There's cheating under all of those models, including the first one which takes the form that it does specifically to prevent cheating. The implicit goal (for the students) of model 1 is to make sure they've internalized whatever is being taught. The implicit goal of model 3 is to make sure that, even if they haven't internalized the material, they're capable of applying it. The implicit goal of model 2 is to make sure they'll comply with arbitrary, unenforceable demands (in this context, usually called "the Honor Code"). That might make sense if you're hiring a cashier -- but is it really your first priority?
Do you feel similarly that the implicit goal of this model is "to make sure they'll comply with arbitrary, unenforceable demands" and still not to test internalization of the material?
This is to my (1) as my (2) is to my (3). The applicable standard is even called "the Honor Code". I don't see why you think I'll see a difference. It's quite clear that making sure (or emphasizing that) the students are The Right Sort Of People is an explicit goal of the Princeton policy; see the final sentence of the relevant section of the document you linked. ("STUDENTS MUST WRITE AND SIGN THE HONOR PLEDGE IN FULL ON THE COMPLETED EXAMINATION PAPERS", caps original.)
It's less important than it sounds for a bunch of reasons having both to do with the actual "game" we are building and with our business model. It's not like we spit out a number for every player, and everyone with a number better than X gets a job offer.
Your last post on hiring definitely found its legs and I am sure will cited for some time to come. You mentioned that you used to send a sampling of key text books to potential Matasano candidates and was wondering if you could share some references. I imagine Applied Crypto might be on there. Any any other important resources for mastering CTFs you might pass along would be obliged ;)
These days we mostly send The Web Application's Hacker's Handbook and a link to microcorruption. (We do somehow get candidates which haven't heard of microcorruption.) Generally, we continue to endorse Tom's Amazon reading list: http://www.amazon.com/An-Application-Security-Reading-List/l...
Do you have a wishlist of all the problem domains you'd like to tackle? (I'm thinking web, mobile, and desktop applications above the system level when I ask this, but any other areas are wonderful. Databases. etc.)
We prefer to keep things a pleasant surprise, but we'll be emailing anyone who signs up with a suggested reading list if they want to start brushing up on old skills or acquiring new ones to get ready for launch day. The reading list will give a good deal of direction.
Thanks, my original question was what actually is a flag. Is it an atomic object that can actually be captured.would be great if it was something unique like say Nixon's are.
By this I mean, the article uses some creepy language as only a company enthusiastic about its power to broker reputation can.
>>We can tell you exactly what happened when your candidates tried to implement a REST API.
What, are you going to keep a record of the time I fiddled with it for an afternoon and then use that to disqualify me from a job?
You say elsewhere about MicroCorruption:
>Microcorruption player identities were totally private. There's no way for a recruiter to look someone up, unless they used a very-identifiable username.
Sounds like a good starting point, but what if I don't want my report card stored and used to compare with other people? What if I don't want to feel like my performance is owned by you?
>>We assess for skill first, passively as players play our games and then actively. Our founders — talented technologists — personally reconstruct candidates’ solutions and evaluate them.
If I don't have any control over how and when my information is going to be monitored, I'm forced into an ugly position where I have to treat the whole thing as part of my career and perform competitively. The sense of relaxed exploration is killed, the idea of treating it like a college course or project isn't viable, and honestly I begin to resent the project.
Alternatively, I play on an anonymous account, and then if I decide to use my information about me in a professional setting, I'm best served by creating a new account and just redoing everything.
Edit: I should say that the idea really excites me and I'd love to learn in this format.
Yes. You can participate anonymously or pseudonymously. I think we say in three places that we only give out contact details if candidates ask us to.
What, are you going to keep a record of the time I fiddled with it for an afternoon and then use that to disqualify me from a job?
Our incentive is to find reasons why you're hireable, not reasons why you're not.
What if I don't want to feel like my performance is owned by you?
Your performance is owned by you -- we won't have copyright to your code or anything. Signal we gather, though? Owned by us. That's the trade: we give you a really fun game to enjoy, but we control the universe you play in and, inside that universe, we control everything and see everything. If you do not like this trade, that's your prerogative and I respect your opinion. Nobody will force you to play Starfighter.
Edit: The following is a neutral appraisal, not a criticism of either the implicit or explicit variety.
If you feel that it is an accusation or a criticism, I would suggest that indicates something unrealistic in how you view business negotiations.
As patio said, this is a trade between interested parties. Generosity is therefore expected to be limited.
---
Fair enough. I just wanted it on record that this is a business transaction for you, not a generous educational endeavor.
Some people want to pretend that they're in it to make the world a better place when that's not truly their absolute top priority. They're probably asking "Why can't it be both?" when I say that this is business, not generosity.
One simply can't have multiple top priorities, particularly with profit and generosity; generosity is practically defined as something that doesn't profit. Something like one of your (patio's) blog posts are given generously in that there is no formal expectation/obligation on a reader to 'give back' some asset, for instance.
If your top priority is providing a service to the community (e.g., education), you need a sustainable support model, whether its as a business or through someone else's charitable donations.
If your top priority is a profit-making business, you need to provide a product that is (or, at a minimum, people perceive as) valuable.
Perhaps you can't have two top priorities (I actually am not sure I agree that this is the case, I don't see that it is actually impossible for two priorities to be equal in priority and above all others), but even if you can't, its quite possible for "business" to be a means and "education" to be the goal, and vice versa. They aren't incompatible.
Exactly. Tesla Motors comes to mind as a company for which "electrification of the world transportation" is the goal, and "profitable business" is only means to an end.
I understand the fear though; most of the companies we interact with (and I'd wager, 90% of startups we see here) are what I call toilet-paper companies - they'd gladly switch from whatever it is they're doing to manufacturing toilet paper if that would render more profit or increase chances of getting acquihired; their top priority is business, not the goal. I hope that Starfighter isn't such toilet-paper endeavour.
It is. And Elon Musk stated repeatedly that it's a secondary goal, means to an end. There's a lot of wiggle room between "profitable business" and a "toilet paper company".
Which is why I was careful to prepend "generous" to every instance of "education" in my response. Starfighter isn't generosity of knowledge, it's barter of it.
Perhaps what you're saying is true in that universities have a financing division, but my point is that this isn't a case of university. The focus is on how they're going to make money as middlemen (and dwarves will sing about their riches), not how their business is subservient to their idealistic educational aims.
Even if you are being generous with your time and effort, you can't carry out an effort with any kind of reach without some support model. Financing it through its own operations may not be the only choice, but its not a choice incompatible with generous motivation.
The thing is, making money off providing a useful service that benefits society as a whole isn't immoral, so I don't see why it's that big of a deal. Nobody is forcing anyone to participate.
EDIT: In fact, socially rewarding companies that do good things incentivizes others to adopt ethical strategies and might do more good than a vow-of-poverty educational service provider.
> The thing is, making money off providing a useful service that benefits society as a whole isn't immoral, so I don't see why it's that big of a deal. Nobody is forcing anyone to participate.
Of course it's not immoral but still, knowing whether "a useful service" or "making money off it" is a top priotity for the company is important. Most of the companies you and I interact with are of the second type, and I guess this is at the root of throwawaymaroon's worry.
I’m curious as to how much this will select for people with free time. Many of the CTFs I’ve played in required a quite non-trivial amount of investment of time (which I didn’t mind, as they’ve been quite enjoyable), but there are certainly people who can’t afford n dozen hours to solve problems due to family/work/other obligations.
I do think this is significantly better than the alternative, but I’m also concerned that it will just create a new class of people who do unfairly poorly in the hiring process.
We'll try to make Starfighter maximally useful for casting as wide a net as possible in the candidate pool, including folks who have demanding career/family/etc situations. I'm an ex-salaryman, believe me, I know the frustration.
That said, the perfect shouldn't be the enemy of the good here. The hiring process as it exists on March 9th 2015 is already insanely hostile to people who don't have scheduling flexibility to invest hundreds of hours to doing speculative work. "Hey, could you fly to a different continent for 3 days to do six rounds of in-person interviews?" is considered an attractive, reasonable proposition.
We can extract more signal than that gauntlet gives, in substantially less time, delivered in the candidate's own space and at their own pace. This is an unambiguous win for candidates with commitments.
How do you ensure people won't be required to still fly out for 3 days of interviews? How do you prevent this from being just another filter before the "real" interview begins?
Starfighter doesn't make any money unless they actually place someone. Their pitch selects for people that aren't doing well in the traditional hiring process. So if a company tries to use its ordinary process, just putting the candidates in at the front, we can expect they won't do especially well. If a few candidates get sent out to an interview, have a miserable time, and don't get the job, I'd bet Starfighter will fire the client. Even when you are just selling to clients, you need to occasionally fire a toxic one. In a two sided marketplace it is absolutely crucial. The screened, high quality, applicants looking for a new job (or at least willing to consider one) are too valuable a resource to waste on companies that aren't serious.
I think I deleted the line from the draft that actually got published, but an earlier draft said:
Companies will process Starfighter candidates expeditiously and with dignity. Why? Because we're the work-sample company, and each candidate's experience is a work-sample of your hiring process. I see no need to introduce the best engineers in the world to anyone but the companies that most have their act together.
(Plus, yeah, obvious confluence of interests there regarding incentive compatibility.)
Where is this hyperbole of 3 days of interviews coming from? Sure, someone had that experience. 99% of people don't.
Anyway, I'm happy to spend lots of time meeting and mingling with my future team and employer. Most people good enough to surpass these CTF's are going to be using the in-house interview time to be in the driver's seat of reverse-assessing their potential employer, realistically.
The interview for my newest job was spread out over two half-days (my choice versus one full-time day). I came back to visit and talk with people three times after I finished my interview and had an offer extended. My choice. I needed to really feel the culture. And ultimately that's what caused me to join the company.
I had 24 hours of interviews at a company in Massachusetts, and then they decided to start negotiating salary. I told them what my last job had paid, and then they stopped talking to me.
This was my immediate concern as well, though (to pre-answer the question Thomas already asked you) I don't have any good off-the-top-of-my-head suggestions as to how to stop it.
I think this sort of kick in the pants is exactly the sort of thing that has to at least be tried to sane-ify hiring in tech, but I'm oddly fascinated to watch the ways in which little bits of status-quo inertia and company politics may act to sabotage it.
It is very easy to imagine companies not really "getting it" and seeing it the way they see github contribs or stackoverflow scores or whatever other metric that some do use as a signal, but which very rarely actually gets someone in the door or even on a particularly fast track.
In an ideal world, any sort of testing like this plus a screen to make sure you're not a psycho, a narcissist or just a general pain in the ass on a personal level should be sufficient for virtually any programming job, but there are lots of obstacles without obvious technical fixes: HR dept politics, the hazing culture ("We all did this stupid interview thing, so should the new guy/girl!"), general distrust ("nobody ever got fired for hiring someone using the old method"), etc.
I will be watching this experiment with great interest (and hoping it succeeds).
My worry is that this service could be seen as yet another certification, and just adds to the list of things you "have to do."
I guess I can ask you directly: how much time do you spend interviewing people, not counting the work-samples you already require?
As for suggestions, it's really about the next year. Unless you've already been doing this, you have to find people, and place them. And that means working not only with the programmers, but also the companies involved. Part of that would mean coaching them on how to use your system. You want to destroy the current tech interview process? You have to replace it. You want to remove the tech portion? Then I think the best way right now is working directly with the companies you are placing people with. They need to play by your rules, and a year from now is when you'll have a better understanding of where you stand, as the person you've placed is hired, and still employed.
Obviously there is more than just one candidate and company, but unless that happens, I don't see a company just buying into your system without keeping it's own. So you need to teach them what it is you are providing, and what they should do to maximize interaction with the candidate. The interviews will still exist: I want to meet the person I'm working for before I show up. But instead of dealing with white boards and ping pong balls, we are having a conversation about my potential future with the company, and what I can bring to the table.
patio11 said it: "we’re here to destroy it, and create something new and better in its place."
A job interview has two sides, the applicant, and the company, and if you want to destroy the interview and rebuild it, you have to do it with both sides at the same table. Otherwise, you aren't destroying anything, just adding more red tape before anyone gets to the table.
Literally the first objective I had when I took over recruiting was to reduce the time demands on Matasano's hiring process, and the work-sample process we came up with slashed time demands by more than half. Candidates we passed on wrote us to say how much more they liked how we hired than other companies. That's because:
* We demanded the same or slightly less time in total than other conventional-interviewing companies did.
* The scheduling of our demands was totally flexible, unlike interviews, which are rigidly scheduled. If you're a morning person, and you have next Tuesday free, that's when you threw the 2-odd hours you needed at the challenge.
* Obviously, it's easier and less stressful to put effort in from your couch than in an alien office environment with people staring at you waiting for you to answer properly.
* The challenges themselves were fun. They were real work: we didn't have people literally slaying dragons (they were breaking a web app and a client/server app), but they were the distilled enjoyable essence of that work with most of the BS removed. In fact, if you didn't find the challenges fun, that was a huge signal that we were the wrong job for you.
* If you did decide we were wrong for you, you could stop at any time --- randomly, on a Tuesday night, with a slice of pizza in your mouth --- and have none of the weird social pressures that would make you sit through a string of pointless interviews. Which is an experience I have had, more than once.
Regarding certification: pure, chill-filtered hatred of certifications is what got me into the part of software security I wound up in. There is zero chance that we are going to build something with the market dynamics of a certification.
Great point. It's happened before. For example, at many companies, HackerRank-esque code tests were meant to replace the technical screen, but just ended up being an additional step you have to do.
More red tape happens because companies feel like they have the power to ask people to jump through whatever hoops they want. But you can reverse this. If you have the people companies want desperately, you can dictate what the companies can and cannot do (no 3-day interviews, for example). I think that's what Starfighter is trying to do.
I think you have an interesting concern. I don't worry that this will become an "extra certification", though.
I imagine that one might still get called for a 3 day interview, but I suspect that people using a work-test like Starfighter before asking for that will already have a MUCH better idea about your capabilities than they would have without it. Interviews then seem like they become more about assessing your fit with the company, and less about trying to weed out incompetence -- because they wouldn't invite you for an interview unless you'd already demonstrated your competence in this way.
That sounds a bit bad, if it were for only one company, but what excites me about Starfighter is that it sounds like something I'd want to do for fun __anyway__, and its signals about my competence (I hope! ;)) could then be given to multiple companies -- including ones I might never have thought to apply to.
I'd much rather hear, "Greetings Starfighter, .... we want to hire you to do more like that" than get spammed by random recruiters because my LinkedIn profile happens to have Python or Java on it.
At this point, my biggest fear is that I might not have the technical chops to complete the challenges.
>>At this point, my biggest fear is that I might not have the technical chops to complete the challenges.
I fully expect that I don't have the technical chops right now to complete the challenges, but I can barely contain my excitement to get started. Learning whatever it takes to overcome a defined challenge is so much more thrilling than learning a topic for the sake of knowing it.
Why is it unfair to hire people who spend large fractions of their free time honing their skill over those who don't? I mean, if it shows in the quality of their work...
Edit: I know this comment is unpopular, but every craft industry does work like this to some extent. As a programmer you have a portfolio, and all else being equal those who spend their free time coding have larger and more impressive portfolios. That's the career advice you always give someone on getting their feet into the industry: code more on your own.
Because free time is something disproportionately available to the privileged. I grew up in a household where money, even if it wasn't abundant, wasn't a huge concern, so I could spend my time outside of school learning math/computers. I've never had to keep a part-time job to put food on the table, spend anhours walking because I couldn't afford gas, or generally be forced to trade my time for money at a terrible ratio. There are people who have, many of whom are likely a better candidate than me, who might just not have the time or mental energy to get home and spend 6 hours hacking on microcontrollers.
I think I see what you mean: if this replaces interviews entirely then some people just won't be considered for the job because they have children. To me that sounds like a policy issue (it's illegal to discriminate based on irrelevant features to the job). I.e., not something that the designer of the CTF should be responsible for, but something a company could be very liable for if they aren't careful in how the apply the CTF.
Parent's point is that companies implicitly filter for this by asking candidates to participate in time-consuming multi-phase interview processes. It's already difficult to work contract-for-hire or take time off for several days of on-site interviews, and it gets much more complicated when you're not well-off or you also have to take care of your kids.
Of course. But you still hire more skilled people over less skilled people, and if free time contributes to skill it's not necessarily unfair.
In fact, I think a quality CTF could level the playing field, since each candidate could be judged on their relative abilities (how far they can progress through a game that adapts to their prior knowledge). Moreover, one could offset the time commitment by paying candidates to do the CTF. Certainly paying people a decent wage for 6 hours is worth the benefit of hiring a better long-term candidate. There are all kinds of ways to fix the problem.
That's definitely the ambition here, and it squares with our experience with the Crypto Challenges and Microcorruption: the "top tier" of people in both those events were, as a general rule, not very experienced in either of those problem domains.
I finished 4th in Microcorruption. While exploiting vulnerabilities is not my day job, I have been playing CTFs for a long time, and am familiar with the process. It is certainly something one can overspecialize in.
Looking at the first page of the Hall of Fame, I see big names like Alex Sotirov, Russ Cox, Ricky Zhou, Ludvig Strigeus, and many other familiar names/handles usually seen at CTF events. There are also many unknowns, which I suppose was Thomas's point.
There were "elite" participants, but they were numerically dominated by strong participants without the background. Which, if you're a hiring manager, is a very interesting and exploitable bit of data. Our odds of hiring Russ Cox or Alex Sotirov were not good. Our odds of being the first serious job for the next Alex Sotirov were better.
>I grew up in a household where money, even if it wasn't abundant, wasn't a huge concern, so I could spend my time outside of school learning math/computers.
This accumulated advantage you describe is present either way. The length of the interview or CTF process is not going to change this.
There are other companies that have tried to be the one-stop-shop for job hunting. It would suck if I worked on my Starfighter rep[1] and then I see a job I want but they want to use my HackerRank rep or StackOverflow rep or whatever elese instead.
[1] a term I just made up, and may not correspond to something in reality
I don't know what it means to be a one-stop-shop for job hunting. I'm flattered by the concern on this thread that we're somehow going to monopolize dev hiring so completely that every working developer will need an account with us in good standing, but (a) that's not going to happen and (b) we wouldn't let it happen in the alternative universe where it might.
The fear is not that Starfighter dominates; that would be a good case.
The fear is that there are a bunch of hiring fiefdoms, all waxing and waning, so that I have to spend a bunch of time on a bunch of them to stay relevant. I don't want to be laid off one day and then find out that these days employers are using StackOverflow scores or wherever and that I should have spent the past year working on that.
OTOH, if 1) I can easily find out that SoftwareCo is hiring through Foo-CTF without having built a Foo-CTF profile, and 2) I can spend, say, 5 hours building a profile at Foo-CTF and get a serious response at the end of that, that's good. It means instead of spending 5 hours doing the silly technical interview dance with SoftwareCo, I'm spending 5 hours doing the serious work-sample test with Foo-CTF. Foo-CTF's value-add is that they're experts at doing the work-sample test.
This fear is unplugged from the reality of recruiting incentives.
It is in the direct financial interests of companies to recruit you directly, not through us. To recruit through us, they have to pay for the privilege.
I was thinking this too. I did the cryptopals while in grad school and was able to complete it. I got 15 minutes in microcorruption before I got sucked into a wormhole of meetings.
One of the conventional complaints against CS-algorithm-heavy interviewing is that companies ask all these tough questions about distributed cache latency in red-black tree optimizations, then once they hire people, they're just bashing out PHP and MySQL. How do you make sure that the companies hiring your services have challenges that live up to the skills you're selecting for?
I've got nothing against CRUD apps, because they make the world go round, but it is unlikely that Starfighter will choose to send our candidates to firms making CRUD apps. The difference will be fairly obvious when you see our client roster. We're geeks with good taste for the kind of jobs a geek would actually want to have.
(Sorry for being coy here. Don't want to jinx contract signing.)
The problem is that the great hackers are already inundated with unattractive job offers. Offers that seem very attractive to people who are not them.
If you had a deal with Google/Apple/Dropbox/NSA that said "If we find someone who can complete this challenge, you agree you hire them at $500k/year + $100k signing bonus doing work on Skunkworks Project X" you would have an incredible flow of hackers.
No one has created a marketplace for world class experts and yet these positions and people exist.
The problem is that the great hackers are already inundated with unattractive job offers.
This is true for a subset of great hackers who are conveniently visible to the tech industry's antiquated, inefficient, insane, and exclusionary hiring practices. It is very much not true for many very talented engineers in the world.
We're going to arbitrage that inefficiency to zero.
>We're geeks with good taste for the kind of jobs a geek would actually want to have.
You're geeks who've assumed that the only job a geek wants to have involves finding system-level exploits.
It's all very clever to write a program that reads use it's own bytecode as the secret, but does that feed people? Get them to Mars? Heal people? Does it make application programming easier, less error prone, more accessible? Perhaps there exists some insights at the lowest level of program execution that are only revealed in a career like this (a la _A New Kind of Science_), but somehow I doubt it.
Typically the people who find these sorts of bugs have jobs penetration testing and auditing applications. Some of those applications are important to people's lives and wellbeing. And if you don't work for the NSA or similar, you're actively helping the application owners to find and remediate those bugs, which improves security for the creators and the users.
For example, if you're an application security analyst at Google and you find a security flaw that could result in users' personal information being leaked, aren't you making the world a better place by finding and repairing that bug?
> Does it make application programming easier, less error prone, more accessible?
If you want to write systems that improve the safety of application programming, it helps to have a lower-level understanding of what is happening.
As for Mars, if articles like this (http://spinroot.com/dcas/) are any indication, there will need to be a lot of systems programming and especially tool building that requires deep systems understanding.
It's so weird to me, because for me, it is absolutely the other way around. It's easy to break into computer systems. Put a single quote in the wrong place on enough web apps in a /19's worth of IP addresses and you will get in. What's interesting about security is the tour of functionality it takes you on. Security is an opportunity to crawl into the ventilation ducts, open the hatches, and directly tinker with the inner workings of hugely complex systems. It's a goal system that gives you a reason to do that, that points you in a coherent direction, that forces you to learn and retain otherwise random-seeming details, and rewards you with huge endorphin rushes to keep you going.
It doesn't even have to be competitive. It's just... fun.
Virtually everything I know about computer science I learned because of software security.
I don't think you need to care that much about security to benefit from a syllabus of exercises framed by security.
Well, see, that's totally coherent and makes a lot of sense. Everyone has had epiphanies from a deep dive or two, but usually it's bugs, not security, that "frames the discovery process".
I think you should copy/paste this onto your landing page.
Wow, that's such an awesome point. I still remember the rush from reading 2600 and trying to break into systems in middle/high school.
That joy and excitement is totally gone now in Bob's CRUD Shop(TM), so it's phenomenal to have a curated lab to play around with (that won't get you arrested).
I assume, given your involvement, that this isn't likely to be an issue, but how likely is it that your candidates will be able to find jobs outside of SF/NY, or in fact outside the US?
If the goal is to find the top talent that wouldn't don't normally show up on a firm's hiring radar, then I would expect that includes finding people who aren't in those cities (and don't want to be).
I couldn't find any mention of geography it in your blog post or on the starfighters.io website, but I've been burned many times by initiatives where the absence of geogrphical information is supposed to imply "USA only" (because they forget that the rest of the world exists), or if you're very lucky "USA and select European countries".
Actually, I guess there's the fact that you're hiring someone who you know is kind of addicted to an online game... and if you don't find anything more interesting for them to do, they'll probably go right back into it.
Wonderful, let me add this to the list of things I need to do in my spare time to remain a hire-able resource .... GitHub, Meetups, reading hacker news, preparing for puzzle interviews, , HackerRank, conquering a CTF. Jeez, can we say dance monkey dance.
> What are some things you'd worry about a company like this doing that would increase dance-monkey-dance factor?
Not the OP, but anything where a recruiter can go look my rating up, they will. And then that becomes a positive signal, increasing the speed of the rat race, because to be "hip", you have to do the N+1 things. Blech.
If you want to reduce the "dance" factor, have a sheltered tunnel from Starfighter to the first day on the job. Don't have it be a pipe to the tech interview, don't let it be a pipe to a recruiter's desk, don't let HR do anything but verify employment eligibility. Seriously.
Microcorruption player identities were totally private. There's no way for a recruiter to look someone up, unless they used a very-identifiable username. Privacy is important to us.
How would you imagine the bizarro-world third party recruiter that would emerge from something like Starfighter could interoperate with employers to shield you from the kind of BS you're concerned about? We're interested in ideas!
I would suggest Starfighter Recruiter, LLC have a contract with a VP, CTO. Something to the effect of...
"We provide you the names of people interested in your fine company, along with our certificate of credibility. You in turn agree that the interview process will be entirely on the intangibles[1] of interpersonal relationships, career goals, etc".
Along with that would be a rider that would involve Starfighter Recruiter LLC taking liability for providing a measurably crappy candidate, as well as disclaimers all around relating to intangibles.
Personally, I don't really want to futz around with long-winded negotiations and alpha male chest thumping. I want to demonstrate my capability, discuss my career goals and interests, and verify that fine hiring company is actually a reasonable place to work that meshes with me and my career aspirations.
[1] I.e., candidate is a loud-mouthed jackass that can't work with anyone, but can pass any technical challenge with flying colors.
I think if it leads to higher quality job leads than that would be great for many people. I find personally with response rates to cold drops of a resume being around 75%. The amount of time shifting through positions I don't want >> than pursuing those that I do.
The great employers don't require to much chest thumping and I've had some down right fun interviews. I question the culture of a company that uses a CTF game as a metric to hire people. People who play and succeed at CTFs tend to be hyper-competitive and thats not always a wonderful characteristic in a team member or company.
Thats fair, after reading some of your comments above. I believe you don't intend this to be more work for people. And, I believe hiring on both sides of the equation is a non trivial problem.
We used it at Matasano and had a diverse culture of people --- many with families --- with a variety of different personalities and personal styles. Beware of attribution error.
The underlying idea here isn't speculative. We used it at Matasano, and it was extraordinarily effective.
It's certainly the case that I have never personally have desired to do the ACM competitions, as they, IMO, reflect neither CS aptitude nor software engineering capability. I always chose to do my homework, sleep in, or fool around with my own interests.
But it did seem from the outside that those doing ACM competitions were the upper echelon of the school.
Perhaps a CTF situation would be similar. We can only see what unfolds.
Sorry, I think I'm a bit of a cynic and a bit burned out. There seems to be at least an expectation in the tech industry of having to maintain certain after hours activities to make one self marketable. Personally having a 50+ hour a week day job + side projects + studying + leveling up in order to find new employment is exhausting .... I'm just concerned adding another implied responsibility to a candidates plate seems overwhelming.
But, if someone loves to play CTFs and is seeking employment than its a great match. Me personally I'm a build things I want to see in the world and the job will come ... kind of person.
"expectation in the tech industry of having to maintain certain after hours activities to make one self marketable"
On hacker news you are getting a very small slice of the tech industry. And by reading HN you are subjected to repeated attempts to make you think there is a clear way to be marketable and to earn a living. Rest assured that there are people in tech that the HN crowd makes fun of that are earning fine livings and enjoying their jobs.
Also, rest assured that for the overwhelming majority of competent, smart tech workers, only a tiny percentage have heard of HN, and a tiny percentage of that slice view it as something to be taken seriously. For most, there is no difference between Hacker News and https://twitter.com/ViceHN.
I think you should be talking to you current employer about setting aside some of your current work time to training. One or two days a month for you study / explore technologies that will grow your skills and therefore the skills the company can draw on.
Director: Our operating expenses are through the roof. We've got a dozen developers and we're still behind schedule...
Manager: That's because we don't have any "good" developers. We need to find some of those 10x productive developers and get rid of some of the guys we have.
Director: So you're telling me we can replace 10 developers for the price of one good one?
Manager: That should be true, but the 10x guy is going to want a bit more money.
Directory: Let's do this!
(some time later at the interview)
Manager: Were looking for highly productive programmers and we're willing to pay top dollar. We see you've accomplished X, Y, Z so we want you to come and join our team.
Developer: Sorry, I can't...
Manager: How about $50k more salary...
Developer: Wow, ok, I can't really say no to that.
(some time later the first day)
Manager: Welcome to your first day, here is your team. Get as much information from them as possible because we'll be laying off 10 of them in 30 days. Good luck!
> What are some things a company like this could do to decrease dance-monkey-dance factor?
- Let players be paid by your clients to play for ~1 day.
- Introduce a non-game version that's a straight forward coding assessment.
- Make the game un-cheatable.
- Write into your contracts that you won't be the exclusive hiring funnel.
- Keep your clients confidential and don't let them mention your game in job ads (i.e., target people who just want to play a game).
Edit: I have to add, this does seem like a dance-monkey exercise and not something that an actual profession would have people do. My preferred hiring approach would just be contractor-for-a-day arrangements.
”My preferred hiring approach would just be contractor-for-a-day arrangements.”
I’m not sure that method would be suitable for people who already are employed. At least in Sweden, where I live, you can (AFAICT) get fired for working for your employer’s competitors. So if you don’t get the new job, you might lose your current job as well.
He raises a good point. I stopped reading NYT online regularly when I started spending more time reading HN. The fact is you have to pick your battles. And what you spend your time on.
Your question is good though. If I understand you to be saying "what can we do to get you to drop one of those other things or you nightly watching of reality tv?"
While I'm very happy to see attempts to get rid of the interview and move towards a work-sample test, this comment represents the whole sense of unease I have at this.
I think patio11 and tptacek hope this can replace those things, so that you can work towards a new gig on your own terms (instead of going through the usual song-and-dance).
There are lots of ways to acquire and prove skill and knowledge; ultimately, they are all monkey dances to at least some degree. Until we develop a way to read minds (and it'd be great to also do the reverse: give everyone the ability to insta-learn, Matrix-style), we will continue to have to use some kind of signaling mechanism. Right now, there are many of them, all with various levels of difficulty, entrance requirements, and resource requirements. Many are only available to those with various types of scarce resources: free time, supportive parents and family members, and money. Let's list some examples:
- get a degree in any subject (money, family support)
- get a degree in a more pertinent subject (money, family support)
- continually learn new theory/techniques/technologies on your own (time)
- write FLOSS code (time)
- get industry certifications (money)
- get achievements with HackerRank and other 'challenge' services (time)
- go to interviews, answer questions (time, possibly others)
Not to mention all of the 'administrative' things people do to communicate their capabilities (as another comment noted. Those take time, too.)
Something like this seems like it is taking a stab at replacing at least a few of these, but of course it still requires at least some extra resources (in this case, time.) The go-at-your-own-pace nature of it definitely works in its favor. Things like this evolve similar to technology, and its cycles of consolidation and deconsolidation; where once we may have used separate devices (for music, video, reading, etc.), we now use just a couple, or increasingly, one. Perhaps we could think of things like this as the next steps in 'skill and knowledge acquisition consolidation.' Hopefully, things like this will reduce the need to spread efforts so thinly, and save us more of those scarce resources. So while it may be another monkey dance, perhaps it will allow us to dance less, but in a more focused way. Or at least stay on a smaller number of simian dance floors.
If nothing else, it'll be another fun way to learn new things and experiment with theory/techniques/technology you ordinarily wouldn't.
I find this announcement fascinatingly incomprehensible; it clearly emerges from a slice of the software universe quite distant from mine. The use of the acronym "CTF" suggests that the authors are very familiar with this genre of game; but how exactly does it work? What does it have to do with programming? I guess they are making programming into a game (?) and this somehow has something to do with hiring?
A CTF is a game that one plays by programming. I wish I could point you to competing CTFs that you can sign up and play right now, but most disappear from the Internet because they are fiendishly difficult to keep running. Ask me (or any team that has shipped one) why if you're curious.
One example would be MicroCorruption. https://microcorruption.com Thomas and Erin helped build it back when they were still at Matasano. It's a hybrid of a game -- there's a narrative, flavor text, a progression of difficulty, levels, a leaderboard, etc -- and a programming assignment. The programming assignment happens to be some variant of "Here's assembly code; find an input which exploits the vulnerability we planted in it."
Starfighter CTFs will similarly be a game that one plays by programming. Similarities end there. We can do very interesting things with this.
So, for those of us who fit your archetype, that mostly build web apps in high-level languages like Python, haven't touched Assembly in a decade, but do not have a team of seasoned hackers hanging out with us in our living rooms, what are the odds that we're going to be able to use it to refine our skills in isolation?
I fully understand that if your company is looking for pen testers, I'm not the guy you'd hire. That's cool, but since one of the aims is developing and refining skill, how approachable is it intended to be?
This game is not for people who like you (or me) don't have much interest in assembly, systems programming or pen testing. (This group includes, by the way, a large group of very talented programmers who've shipped most of the software the world uses).
It is troubling that the language they use on the site implies that systems and security programming is what all programmers aspire to do, but that's just an unfortunate bias on the part of the authors. Personally, I think they'd do well to change it, but perhaps that's exactly how they feel and exactly the kind of messaging they need to attract the right people.
I find this to be an issue with many of the articles linked on this site. They refer to 'hackers' or 'programmers' .It usually means people doing some sort of web app, or related thing, and it usually doesnt apply to me at all though I know I'm a very good programmer. I make games.
I have a background in architecture, its a mature industry and there are architects, structural engineers, mechanical and electrical engineers, architectural technologists, quantity surveyors, project managers etc.
Perhaps as coding as a job matures over the decades this process will also take place.
Or, much more commonly, it's a popular game mode in any number of multiplayer games. You're using some really niche jargon without properly qualifying it.
When somebody asks for a definition of a word, it's not generally considered necessary to explain the term and then also go into detail regarding other, unrelated definitions of the term.
They're operational nightmares. Among other reasons, their designed intent is to allow untrusted users to execute arbitrary computer code on your infrastructure. The abuse problems are mammoth, the not-malicious-but-still-bugged submissions from users even worse. They have lots of moving parts, most of which are more complicated than standard CRUD app components.
Also, fundamentally, a CTF is a distraction from the core business everywhere it runs.
I guess you might be confusing the very unfortunate use of the acronym CTF for security-related programming games with the usual meaning of Capture The Flag, as in a game mode in First-Person Shooters where people work as a team to capture enemy's flag and bring it back to their base.
I recognized your confusion because not so long ago, I also had it. I started hearing about reputable security conferences having "CTF tournaments" and was all like "oh cool, they're playing shooters for prizes, go geeks!" and then was disappointed when I discovered that someone just stole the term CTF to call pentesting contest with it (it takes a lot of imagination to say that a file is "a flag"...). Not to say those games aren't fun - and I am very excited about Starfighter - it's just I don't like stealing names with well-established meanings and using them to something completely unrelated and not really fitting.
I retract my point about "stealing names". I guess that this application of term CTF just wasn't known widely outside INFOSEC field and that's why a lot of programmers (like me) get confused, as security conferences become more recognizable in the mainstream.
The wiki article has a short discussion of how that morphed into video game terminology, which is the etymology that I'd expect without having researched it.
I get what they're trying to do, but did this read like a rant from a crazy person to anyone else?
Just the first header block read like they were creating some kind of videogame company... and then finally at the bottom of that text block, "Oh, it has something to do with interviews..." and then halfway down the next section, "oh, it's that kind of CTF".
Anyway, seems cool if they can make applying for jobs more interesting.
I actually liked the style. For me it read as amazing, totally cool and definitely-want-to-participate-in thing - but that might be mostly because I know who patio11 and tptacek are and don't apply the typical --bullshit-filter=strong flag I do when reading other announcements.
Entirely insane, yes! After reading this article, I have no real idea what they are doing, but it appears that they are related to an entire genre of games I didn't know about.
I think it's primarily written to a different audience than yourself, then. I mean that in the best of ways. As an infosec sort, it came across pretty clearly what they intended. I think it's just a culture difference between information security, and the rest of IT.
It reminded of how pickup artists market their books/seminars[0], which gave me a little chuckle. Nevertheless, I'm excited about this, and signed up immediately.
> We’re not here to fix the technical interview: we’re here to destroy it, and create something new and better in its place.
That sounds pretty bold - some of us are kind of happy to go in and do an interview and talk about our experience and not spend a lot of time playing a game. I could see the game as more likely to work for people with lots of time and not as much experience or proficiency with the typical process. I could see it working very well indeed to find people who might otherwise have been ignored.
In any event, it sounds cool, and I wish you guys the best of luck with it!
If the only reason you'd mess around with Starfighter is to get a job, we've done something wrong. This isn't one of those sites where you solve discrete programming puzzles for badges, or in lieu of a whiteboard interview.
> If the only reason you'd mess around with Starfighter is to get a job, we've done something wrong
That was just my first impression.
At my age, I kind of know how I stack up: I've met really awesome people like Andrew Tridgell and... I'm not one of them. So I don't feel like I have anything to prove, and I much prefer to hack on open source ( http://journal.dedasys.com/2014/04/27/an-erlang-postgres-dri... ) or side projects where I make and create something rather than playing a game. So I really don't think I'd use it unless I had to as part of a job process. I don't think that's where you're going with it though; it's more of a way to attract interesting people who might otherwise not get noticed, right?
So I'm "not your target market", which is of course natural and to be expected. I think it's a great way to find people who might otherwise slip through the cracks, and have the time and inclination for puzzles.
We've been talking to people for a few months now about this and have heard that concern repeatedly. We're working on designing and releasing something that rewards competition for the kinds of people for whom that's a motivator, and rewards tinkering for the kinds of people who enjoy tinkering.
I'm the Dwarf Fortress kind of game player, when I ever play games, if that gives you a sense of the direction I'm pulling us.
If Starfighter is only rewarding to people like Andrew Tridgell, we've done something very wrong; count on us to adjust course quickly if that happens.
I'm super excited about it even though I'm happy where I am. I've been doing devops for a while now and I can feel my other dev skills slowly atrophying. This will be (hopefully) a great way to help keep those skills as current as possible.
"(a) For the same reasons you'd noodle with any other game"
I play games for fun and to relax.
"(b) For similar reasons to the ones that make you noodle with a programming language you doubt you'll ever use in production"
On your website, it says that these will actually be real problems for real companies. I would think it's something I would see in production?
"(d) Because for some of the technologies/concepts we work with, our dumb game will be the easiest way to get your hands dirty with them."
Isn't this what open source is all about? Pretty much every job I've ever had involved technologies I could just download or install myself and throw on a Linux box/VM.
I suppose it could involve proprietary technology that I would never see in the wild, but if this is the case, I don't think I could get enough experience with it on your game to prove to a company that I know enough to get a job.
Usually CTFs / wargames / whatever are the only way you ever get to work with stuff like radio or telephony protocols, credit cards, homegrown crypto...
Also you can probably expect things dealing with custom VMs, file formats or networking. Say for instance one of the challenges involves a machine with a broken TCP/IP stack, so you have to write your custom client that can talk to it, how often do you do that in real life? Yet it can translate into useful skills.
Thank you. Exactly this. Try to make a list of interesting stuff you'd like a chance to play around with, in a highly structured environment that handholds you just long enough, and that eliminates all the nonsense required to get dev environments working, or expensive subscription fees, or bankrolls, or laboratories... those are all places we want to be.
Part of the problem I had earlier today was with the words "fun" and "player". I think the words I was looking for were "participant" and "rewarding".
I think the gaming analogies make sense. If you invest time and effort in it, it sure will be more useful in your CV than "Level 80 Paladin" under "Other Achievements"; if not, no hiring manager will ever pass on an otherwise great candidate for not being a part of it.
And it allows outliers to get potentially amazing job offers from your partners.
It's exciting to think that with today's easy access to cheap cloud computing and good process isolation you can have instanced "raids" in the same way as an MMO so I can get my very own version of a challenge that can be as realistic as possible.
Different people have different definition of fun. Some play Minecraft just to build CPUs out of redstone.
> On your website, it says that these will actually be real problems for real companies. I would think it's something I would see in production?
The key word isn't "production", it's "you". I, for example, don't really expect to ever use Go or Rust in professional setting. Nor do I expect to use Prolog. Then there are real production systems today that use PDP machines (nuclear reactors) or COBOL (banks). I don't expect to work on them either, but sure as hell would like to play with them at some point.
I get the feeling that Starfighter is about giving geeks some hard-core tech game with a side effect of helping them find a job that is not boring.
1) Is this a Stripe-like CTF that happens over some caffeine-fueled weekend? Or is it more of an ongoing Project Euler-style drop-in-and-solve type process? I don't think I could handle the former, but the latter sounds quite enjoyable and something I might do in my free time just to learn. I did that with PE for a while since it was fun to earn completion points.
2) You mention "Let's Play" style videos. How do you make it so that the solutions aren't given away? Does each player have a customized CTF somehow?
I loved the Stripe CTF. We're not going to adopt the time-boxed model from them. Starfighter : our CTFs :: Blizzard :: Blizzard games. There's no reason for us to ever turn them off. We'll likely have a particular flagship property but we have the option of dropping a new game on the market any time we want.
You don't have to rush through Starfighter games. Some people will, of course, just like there is a metagame to be the first guild on the server to complete a new raid. Players are gonna play. But if you want to pick up our CTF a year after release and just casually spend a few hours learning a new skill, you'll be able to do that, and we'll fully support you in it.
How do you make it so that the solutions aren't given away?
The same way you do it with math problems. Mastery of the subject material is the easiest cheat code.
>The same way you do it with math problems. Mastery of the subject material is the easiest cheat code.
Elaborate? Math questions are the opposite of a good example, aren't they? Even with randomized values it's fairly easy to write generic solutions. People do this even for Project Euler. How much more so for something even cooler sounding?
(I'm very excited to see this project and it sounds like a lot of fun. I'd just be worried that it would end up being gamed, since the drive for cheating is so strong everywhere.)
The complexity of the problem solver generally increases faster than the complexity of the problem generator.
For example, a general solver for "what is #{x} + #{y}?" is relatively easy. A general solver for "what is the derivative of #{random_equasion_with_diffuculty(3)}?" requires a full computer algebra system. Take one more step into advanced mathematics, and things get more dicy -- automated theorem provers, for example, require human guiding and hints.
> The same way you do it with math problems. Mastery of the subject material is the easiest cheat code.
This, at best, sidesteps the question. If a solution is a string of characters, that string of characters will end up on the Internet.
If a solution is a series of tasks---try this string of characters, observe what happens, then try this other string of characters---people will still share the strings and instructions on the Internet. All this still holds if the strings are personalized in some way; then the general patterns will be shared.
I spent a few years at university as a teaching assistant for a programming course using an interactive IDE where this kind of thing was required: Run the provided unit tests, ONLY THEN write some code, ONLY THEN run these other tests, ONLY THEN make the appropriate modifications etc. The idea was to make cheating more expensive than actually thinking about the issues, and to some extent this succeeded in the narrow sense that cheating WAS more expensive, but it did not succeed in the more general sense: people STILL cheated because they really didn't want to think things through.
The higher the stakes are, the more not-so-great people will be attracted to the system and will chug through based on someone else's solutions. This will be a fun game for those who see it as a game, but I don't see how the using-this-for-recruiting part will not implode almost instantly.
Finally, the whole thing about keeping even the general subject areas a secret is a bit silly. The work the best tech companies in the world need ranges from medical visualization to databases to systems programming to pentesting (one of you wrote this will not be a pentest game!) to static analysis and verification to distributed systems to scientific computing to drawing pretty pictures in web browsers to... You will only cover small sliver of this. Everybody knows this, why be an ass about it?
I am surprised that Patrick is moving onto yet a different project.
After leaving his job he works as a freelance online marketing expert. Then quits that despite implying making a lot of money. Instead wants to create online marketing courses to reach bigger audience, but takes forever to produce any content and is now abandoning that track. Creates AppointmentReminder with some good initial success but reading between the lines that is going to be sold/abandoned as well.
Now moving onto yet another project. Seems you have created several great opportunities for yourself but cannot stick and focus on any one thing?
I am curious about a biz dev kind of thing, and it may be entirely too early to answer, but here goes:
"Contingent recruiting" has an extremely negative connotation to me. Quite literally, I have to constrain my bias when designing hiring pipelines to not throw recruiter backed resumes away. I understand that you are only taking the payment model, and are trying to undermine the business model, but have you run into any problems associating with the industry?
Or, more interestingly, have you encountered anyone on the consumer side (HR, C-Suite management, etc) that has pointed out a non-obvious advantage to typical contingent recruiting. I for one would love to hear why companies keep going back to that obviously terrible well.
Another attempt to apply data analysis to produce a concrete number that represents the merits and talents of a fellow human being.
How utterly irrational.
It's ideas like this that make me question why I bother being a programmer for anything more than leisure these days... except that I have no idea what else I could do to keep a roof over my family's head this late in my life.
You call yourselves, "engineers," but I've yet to work for any company that treated you like one or even invested in your future. You're offered a salary well above average for most working adults. They lure you with frivolous perks and stock options. They never send you to school for training, offer you a pension to keep you secure in your golden years, and I've never seen any unions you could join to demand these things from your employers. Your career is probably not going to be on the line if you merge a patch that degrades performance or introduces a timing error. But don't think they'll care about you when it comes to the bottom line. Someone else seeks to profit off of your talents and abilities and nothing else. If they can hire someone to automate you away, they will.
You may love your job, but your job never loves you back. Don't remember where I heard this, but it's always good to remember. It applies (in varying measures) to anyone from the founders and CEO to the lowliest employee.
If you could tell us how to apply data analysis to produce a concrete number that represents the merits and talents of a fellow human being, I would love to hear it; it would make my job a lot easier. I didn't know that was possible. Is this some kind of linear algebra-y machine learning sort of thing?
Most likely it is possible. Take this "Alex" you frequently speak highly of. If you gave Alex a different work sample test, do you believe he'd have done poorly on it? If not, then there is some common factor (call it H) both tests are measuring.
(I can confidently state that you do believe he would have performed similarly on a different test - if he didn't, this whole idea would be fundamentally invalid.)
Now the important question is to determine, for any particular subtest, how closely a person's score correlates with the hidden factor H. This is, for example, how IQ tests are created. You can take a look at the literature surrounding them. Key starting points would be principal component analysis, clustering, hidden markov models and the like.
There is no simple recipe - every data analysis problem is different in it's own way. But there are general themes. I'm happy to discuss in more detail, feel free to write to me if you are interested.
I could tell if GP was being sarcastic so I deleted my original reply.
The softer side of the equation is that the approach of filtering candidates for businesses allows them to minimize (to some market minimum) the cost for highly skilled individuals. Projects such as this just triggered a bone I'm presently picking over.
Old-guard corporations that hire the legally-liable kind of engineer seemed to have taken a much different approach if the stories I've been told by a retired mechanical engineer who worked for Chrysler are true. A completely irrational approach. They hired people based on their potential and made them into the engineers they needed.
The copy on the announcement sounds like this game will quantify everything about the performance of a participant in the game in order to sell them to a curated list of potential employers. What about this system incentives employers to invest in the career development of these people and strengthens our collective bargaining power as the people who build this stuff?
To be more constructive I might suggest turning down the hyperbole and use fewer adjectives. Keep the pitch to employers for employers. Don't be patronizing to inexperienced developers: inspecting the assembly output of a compiler is not elite and not difficult to explain to someone given the right context and framing. I understand your game is about competition but the "winner/failure" schism it can create is a big turn off for a lot of otherwise intelligent, creative, and capable people. It doesn't have to be about being, "the best," in order to be fun and rewarding.
IME, IQ tests (like other standardized tests, SAT's, GRE's etc) are easily gamed. I did a couple practice ones and my score jumped up by 20+ points the second time. Where is the evidence that it measures some hidden factor? Seems to me that (like other standardized tests) it simply measures the subject's level of training and preparation.
It depends which test is used and there are limits in magnitude to practice effects, i.e. you can't coach anyone to a perfect score in a test with a large bank of test questions. Psychometricians are aware of these critiques. Tests like the SAT, GRE, GMAT etc., lightly disguised IQ tests all, are useful for predicting academic performance among other things.
>Effects of practice on the Wechsler Adult Intelligence Scale-IV across 3- and 6-month intervals.
Estevis E1, Basso MR, Combs D.
A total of 54 participants (age M = 20.9; education M = 14.9; initial Full Scale IQ M = 111.6) were administered the Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV) at baseline and again either 3 or 6 months later. Scores on the Full Scale IQ, Verbal Comprehension, Working Memory, Perceptual Reasoning, Processing Speed, and General Ability Indices improved approximately 7, 5, 4, 5, 9, and 6 points, respectively, and increases were similar regardless of whether the re-examination occurred over 3- or 6-month intervals.
> Practice Effects for the Stanford–Binet Intelligence Scales
The Stanford-Binet Intelligence Scales—Fifth Edition (SB5) is a widely used
assessment tool for measuring intelligence (Roid, 2003). According to Roid, a key
advantage of this intelligence test’s most recent revision is that it includes improved lowend
items for better measurement of young children or adults having mental retardation.
Sbordone, Saul, and Purisch (2007) report that the range of the SB5 was expanded to
allow the assessment of very low and very high levels of cognitive ability. Roid and
Barram (2004) indicate that the practice effects on the SB5 were smaller than expected.
For example, the nonverbal IQ of the SB5 showed shifts of only 2 to 5 points as
compared to the 4 to 13 points on the Performance IQ of the Wechsler scales (i.e., the
WAIS-III and WISC-III). Roid and Barram add that the lower shift, and thus practice
effect, is even more notable given that the retest period for the SB5 was 5 to 8 days
versus 23 to 35 days on average for the Wechsler scales.
At an absolute minimum you should be collecting lots of statistics and using them to train a bayesian candidate filter to estimate hiring probability (the same architecture as a bayesian spam filter would work).
If you find yourself wanting something more advanced than that, random forests would be a good match here, as would clustering the candidates using DBNs or simply k-means and then using base rates.
Also, if you haven't looked at https://www.kaggle.com/competitions you really should -- they're similar to what you're doing, only for machine learning.
I was hoping the challenges would include machine learning. There is so much exciting overlap between ML and security. From detecting intrusion attempts to scraping and processing pastebins.
I think it is amazing what you guys did here: you got more points on a give me your e-mail, we are building something post, than Stripe got when announcing their company. Really speaks volume on Patio's writing.
Will there also be marketing challenges? Like: Every day patio11 clicks on one email. Your task is to write an engaging headline. Or: Here is some Google Analytics code. Make it show 50.000 visits any way you can. Very hard to cheat. (But so is getting Google security to file a bug report).
I really enjoyed the comments. Especially the ipod-hasnt-got-wifi criticism. You guys must know that what you are doing is terrible and you should feel bad.
Who knew decoding morse in some wave extracted from a corrupted image file in Cicada 3301 could also lead to a job. Will there be an ARG element to the CTF's? Like you pose as an agent from the NSA or something?
I haven't yet cultivated any answers to my own questions to have fomented any clear ideas or call to action. I'm presently mired in Andrew Keene's The Internet Is Not The Answer, Alexis O'Hanian's Without Their Permission, and my own hunches and experiences from my lived experience and those of people I've met. Perhaps there's an essay in it somewhere.
Unfortunately there's no clean, quantifiable solution to human problems like labor relations that I can see. High-tech companies want to filter out potential hires to get the most highly-skilled candidates for the least amount of money they can get away with. They ultimately want to make more money with less capital and labor costs in order to maximize profits for their founders and investors. Which seems rather contrary to other engineering disciplines where, it has been related to me, they took in the hires they came across and made them into the engineers they needed.
>Unfortunately, the technology industry is fundamentally unserious as to how it presently identifies and employs engineers.
Ok, so lets replace whiteboarding ancient algorithms and useless trick questions with a capture the flag game!! That'll make the industry seem less "unserious."
This is so unfathomably ridiculous. I can't believe you're marketing this as an interview replacement. If I get asked what my "Starfighter score" is before a job interview, I will be running as fast as I can in the other direction before they finish their sentence.
There will be people who become great at this game, and if (god help us) this gains momentum, they will get jobs.
I can only hope the rest who are developing applications and actually writing programs will come out on top, and employers will recognize that some people want to build things and not just improve their ranking in a game.
Nobody is going to ask you what your "Starfighter score" is.
Without opening up a pointless vein of second-guessing about Patrick's writing: I think the timing of this post has led people to believe that this right here is my answer to The Hiring Post.
No. The Hiring Post itself contains what I believe the answers to The Hiring Post are.
There are two halves of the recruiting problem: OUTREACH and QUALIFICATION.
Companies need to be smarter about qualification. We have a lot of thoughts on how they can be smarter and I am in full-on insane street preacher mode about those thoughts, which are not directly remunerative to our business.
Starfighter is about the OUTREACH side of the recruiting problem. Our goal is to be a credible pipeline of candidates who are not effectively identified by the resume/interview/github process. We are looking for buried talent, and to have fun doing it along the way.
I know HN all too well, and I think the instinct people will have reading this comment is to try to reconcile it with exactly the words Patrick chose to use in his blog post and then object to the perceived discrepancies. Please, no: that's a waste of all our time. Patrick speaks for Erin & I. I speak for Erin & Patrick.
Thanks for your well thought reply to my regrettably snarky comment. I agree with you on many points, particularly how companies need to be smarter about qualification.
But yes, this post does seem to directly correlate, in timing and in content, to Patrick's Hiring Post. I'm surprised to see you say this is not your response to that, especially considering that it addresses many of the questions we should ask ourselves as outlined in The Hiring Post (namely consistency, data, and the idea of one being a “natural”).
Also the post itself is linked in the statement:
>We’re not here to fix the technical interview: we’re here to destroy it, and create something new and better in its place.
I disagree with the fact that outreach is a problem in recruiting. There is an entire industry of companies and teams created for the very purpose of reaching out to software developers for their skills. How many of us reading this thread get multiple recruiter emails a week?
Are the recruiters effective? Maybe or maybe not, but I (who am apparently one of seemingly only few) just don't feel that a competitive game needs to add itself as another middleman in the realm of software recruiting with the intent of "destroying the technical interview."
Awesome, I've been waiting for something like this
That being said, I've got a couple questions
Will all the CTFs be security focused?
One of the most common problems coursera ran into was people copying the assignments of other people. How do you intend to prevent copying for the CTFs? I see that you mentioned you're not going to DMCA tutorials or Let's Plays and instead track their every iteration on the code. I guess you can get rid of outright copying that way but I'd love to hear your other strategies.
Other than that this seems like a really interesting way to do recruiting. I can't wait to see the actual CTFs.
Also you've got a tiny typo in the WHY IS STARFIGHTER THE RIGHT TEAM FOR THIS? section:
"I have a folder in Gmail saving messages from geeks who used by career advice or salary negotiation tips to their advantage."
I think the by should be my
I'm sure you've got your own ideas for your CTFs, but if you haven't heard of them before I would recommend you check out the Matasano Crypto Challenges (cryptopals.com). They ran them privately for awhile, you just emailed them and said you wanted in and they sent you the challenges, once you emailed the answers for one batch they gave you another, etc. I've tried various 'code gaming' things before, and the Matasano challenges were, in my opinion, the best ones I've come across. (I am exempting things like Stepic's bioinformatics challenges since their main goal is to expand your knowledge of bioinformatics rather than of programming, though they do get into some advanced data structures later on)
Besides cryptography, what are some other programming problem domains you'd enjoy playing with? We're particularly interested in problem domains that are hard for ordinary programmers to pick up on a whim. We want to make it possible to seriously engage with interesting problems while in your couch with the Daily Show running in the background.
I guess what sort of domains do people like to play around with in their own time? For games there's 3d rendering, physics and AI which seem fun (at least to me) but I'm not sure how to necessarily make fun challenges out of them.
For non-direct-games things fancier AI like deep learning, audio synthesis, making compilers/parsers/transpilers, math stuff like project euler, puzzle/board game solvers/bots (https://gist.github.com/christopherhesse/51e9baf0e3d440d8aff...) come to mind.
Some of the stuff in the 3rd stripe CTF was pretty interesting like making a search engine or dealing with distributed system consensus problems (raft).
Maybe using the coolest new framework is fun for some people, like a challenge could involve angular.js so you could get some experience using it without having to go full-on side project or put it in at work without having any real idea if you should.
Implementing things that you are familiar with but that you don't understand the internals of has been fun for me in the past (for instance like https://www.mikeash.com/pyblog/friday-qa-2015-02-20-lets-bui...). Maybe you always wondered how this thing works, and now you can figure it out and solve this challenge at the same time.
Sounds very competitive, achiever-focused. Do all organizations want programmers who are viciously determined to win? Does that lead to society-efficient solutions?
(To elaborate a little, Richard Bartle noticed four types of players of MUD. Achievers are just one.)
The 4th is killers, not griefers. Killers prefer playing specifically against other players. Killers are generallyl looking for the challenge/reward of fighting and winning against other players, rather than necessarily trying to grief someone. Though obviously getting killed in PVP can sometimes feel like griefing.
Very cool guys. As a long time HN reader, I have a lot of respect for what you guys are doing. I also spent the last two years of my life building CodeCombat, which started life with a very similar mission ("a game that finds developer hiring leads"). We were YC W14 and roundly failed at that business model. I just sent Patrick an email at his Kalzumeus email, but I'd be happy to chat with you guys about we learned and potentially connect you to others who can help you avoid the most common mistakes. My email is in my profile.
Hum, Thomas and Patrick working together... That by itself is worth keeping an eye on.
I didn't do so well on the microcorruption game. I'm curious to see what else they can come up with, hopefully both less specialized, more generic, but also more advanced...
May be interesting. But the claims made in the announcement are very hard to believe. The admission of not being ready to show that the claims have been achieved aren't helping my scepticism either.
Still... if it works, and is actually better than most other attempts at code-gaming this may be fun. Profitable? They will have to see.
But it worked in a very specific context (security) and a specific company (Matasano), no? I know Stripe has done these CTFs in the past (and mostly security related), but they only spin them up on occasion. If the promise of such a technique is so effective, then why wouldn't Stripe invest the money (they've got plenty, let's be honest) in doing this all of the time? Especially given that a recruiter who would place someone at Stripe is probably taking home $20-40k on a placement.
Also, what are the plans outside of security related (if any)? Can you do a CTF style process for non-security?
The answer to the "why hasn't stripe done this" question is simple. They aren't in the business of doing it. You might as well ask why stripe doesn't sell database systems. I mean, they probably have some expertise in building those out as well.
Stripe is going to hire a couple of hundred developers over the next decade (assuming things go well). It sounds to me like Starfighter's goals don't translate to a couple of hundred X 20-40k. They seem a touch more ambitious.
I'm not looking for a job and am unlikely to be, hopefully ever again.
Will this be structured so I can use it as a proxy for a programming mentor? I think it would be awesome as a replacement for MOOCs, which I enjoy but find inefficient for the time invested. Books are efficient but not as much fun.
You might want to talk about the diversity benefits of this approach. In particular, it would be attractive to lots of companies if you can make a credible case for this attracting traditionally hard-to-reach demographics, and then of course assessing them more fairly.
I get that this is work sample testing - but it seems to be for an extremely limited work sample.
Most / much of the success of a valuable developer is not in their specific technical talent, but in areas like expounding a vision, documenting, running a team, navigating politics and fund raising. (Bit vague here I admit, personal experience of success is problem for me)
Patio11 made a name for himself saying "tech is a necessary but not sufficient condition for business success - now learn about basic marketing"
Something similar seems to be applied here - how can a CTF program ever handle politics or investor problems or documentation issues.
Getting good technical capable people is important - but not sufficient.
Will be interested in seeing the CTF program however ... Good luck.
> expounding a vision, documenting, running a team, navigating politics and fund raising
It sounds like you're mistaking "developer" for "tech startup founder". Someone looking for a developer needs at most 2 of those skills (documenting for sure, navigating politics in an organization full of bureaucracy).
This whole interview process is broken. Even with this mind of ctf game, one should not be expected to race through peers with different kinds of tools.
Developers are expected to build resumes at work via daily routines, then at home via github, hackerrank, side projects. System administrators are expected to do daily system administration routine, know OSes, tools, cloud/iaas/paas providers inside out and code side projects and build a coding portfolio.
With years of experience, one expects to be respected and not lose time on games, tests, unwanted side projects. Interview process is built towards young gals/guys.
What race/competition dynamics we are going to have are for the benefit of the participants. Microcorruption has been running for over a year; the first finisher finished a few days after we released it. I'd have paid very close attention to someone who finished Microcorruption yesterday, if I was still hiring for Matasano.
I understand what you are trying to do, but that's another competition in the roots. We already have too many.
GitHub, BitBucket, HackerRank, TrueAbility, Certifications, Education, Jobs. It's becoming of a norm that if someone puts more personal time to show off, they are better engineers. Also that's another wrong position in IT. You can not have personal life. I do not want to be a marketeer, I'm an engineer.
If you ever ask any of us around here, perhaps most will tell you a scary story about the competition and the time constraints the competition brings on people.
Eventually with every racing/gaming/ranking tool out there it's becoming to be harder to get a good position in IT without investing in more and more personal time.
I suspect that the Starfighter crew's answer around this is twofold:
1) Of course there will always be other ways to get technical work. You needn't feel compelled to use their product unless you think the time input is worth the reward.
2) If they haven't made the process fun, that is something you want to spend your valuable free time on, they have failed (at least in acquiring you as a product).
That's not how this works. We don't host challenges for other companies. We're an outreach project. It costs companies money to find people through us; skipping us and going right to employers saves them money. I don't expect many employers to ask candidates to go to us to make them more expensive.
And, by the way, if you don't have a great resume and you're looking to work in a field directly related to the CTF we're running: making you more expensive is exactly what we're going to do. :)
So will there be profile scores? Will any company or person able to see how a participant is doing in StarFighter? If it is so, companies will use it as a differentiating factor in the hiring process. And that's what I'm talking about.
By the way nowadays shiny resumes are not considered relevant. It's the bottom line of screening. I have a great resume in my country and I am expensive here. But it would not matter for any SF startup.
I think you are misunderstanding the incentives that are setup via the Starfighter business model. You are thinking of them on the QUALIFICATION side of the pipeline. That is, you have entered a companies hiring pipeline and they are using Starfighter to determine whether you should stay in it or not.
The Starfighter team is trying to change the ACQUISITION side of the pipeline. They are stepping in to the space where traditional recruiters feed applicants into the pipeline. In marketing terms they are generating leads. Their value proposition to the employer is that the leads they generate will be of higher caliber than the traditional contingent recruiters. Their value proposition to you is that Starfighter is a) fun and/or b) at least better than dealing with a traditional recruiter. Your Starfighter experience replaces your resume/cover letter that gets you into the pipeline. Not your ability to navigate the pipeline once you get into it.
If your resume/traditional recruiters/responding to web ads/word of mouth is more valuable than Starfighter CTFs fun - hassle + employer payoff than you are quite simply not a good fit for the Starfigher Recruitment agency. Just like you wouldn't be a good fit for a DBA centric recruitment agency if you weren't a DBA.
If I'm disappointed about anything in the announcement, its that it is obvious to me anyway, that they are NOT trying to replace the current qualification steps in a hiring pipeline. Because to me, getting rid of that time wasting/unproductive process is the big value add, not shoveling more devs into the top of the pipeline (management at my company may feel differently).
No, performance and participant identities will be public only to the extent participants want them to be. Again back to our incentives: it does not in fact work to our benefit for any random employer in the world to be able to query our site to qualify a candidate. Those companies are free-riding off our work. :)
We need participants to make the game fun (for all of us). We can't do things that make participants wary of us.
This is fuzzy because I didn't let Patrick write about what the game was (that's entirely we're-about-to-ship-itis). It'll be less fuzzy soon.
I may seem to be against StarFighter, but no. I'm not. Actually I'm on the mailing list and waiting for the service to be offered. But I am not fully sure that I will be a good contender. I have not been a good gamer for a long long time.
I believe you fully understand my concerns. It's about the interviewing process, not the tools. Every tool has it's uses and quirks and every institution may use the tools in different ways, intended or unintended.
I really hope everyone doesn't feel like they have to be a "contender". It's not a competition to get our attention. It's much simpler: we have a basket of skills and concepts we want to let people play with. For people who engage with that stuff and find they really enjoy it and latch onto it, we happen to believe we'd have good ideas on how to match those people with jobs. That's it!
This "best of the best" stuff was meant to be empowering, but in a lot of ways it didn't come off that way.
How about a game to discover and formally describe new problems? The ability to find new problems is just as important if not more important than the ability to solve already known problems (especially those that have been solved by others) Obviously, the ability to do both (find new problems and solve them) is the most important of all. Focusing on solving known problems is NOT that interesting... Can you imagine building an environment that allows programmers to discover and formally describe absolutely novel problems in computer science? Just curious.
Is there any protection planned against people who would be paid to solve the puzzles on someone else's behalf, i.e. against cheating by hiring someone to "train" your account?
Honestly they don't seem to be a good fit. I mean they are going to get in, no issue, but they already know how to run a business (charge more) and they can pretty easily get access to investors.
The more interesting question is if they will ever end up teaching at YC.
This looks fantastic. One possible wrinkle: I predict that, although top-performing Starfighters will be a diverse lot, they will lack "diversity". I hope the Starfighter founders have a plan to push back against the carping complainers a venture such as this is likely to attract. (My suggestion is to point out that CTFs are the ultimate expression of "Shut up and show us the code." [1])
I don't understand your point. I've worked with mediocrity far too often in the programming world, and outright incompetence as well. These kinds of contests/games are designed to weed that out.
If you're talking about the "winners" having personality issues, then hopefully that will be caught in the interview process.
I think they were suggesting that the distribution of the top performers might not be "diverse" (for some meaning of that term), which would lead to criticism (like how people criticize companies for not having a diverse workforce, or conferences for not having a diverse speakers list).
I am a beginner programmer, but I am curious what this is all about so I have signed up. Hopefully I can learn something from this even if I am just am just a fighter but no star.
This is neat, but is it really that hard for people to tell who has the skills they're looking for and who doesn't? Every developer I've interviewed it's been really obvious to me what they know and what they're bullshitting about.
I guess there's value is all in assembling a pre-qualified pool for employers to recruit from, though.
But in that way this is kind of like the 2015 version of CNE. Except it's free, which is good. And hopefully less miserable.
If you read http://sockpuppet.org/blog/2015/03/06/the-hiring-post/, you will see that this approach is looking to find folks that were not obviously good, as you say, but they found someone that is a stellar crypto breaker.
Many on HN, including myself, are very excited about Starfighter. However, many HN readers are probably already employed or can easily find employment if they desire. You guys and gal know that maximizing outreach will be critical to your success. You aim to find buried talent, but that talent must first know Starfighter exists and understand it well enough to be enticed to try. I assume you want to both find talent whose ability is not accurately represented in a typical technical interview, but also those who have aptitude for these skills but whose resume wouldn't even be considered.
I grew up near a small town surrounded by family farms. I've really enjoyed manipulating code and finding bugs in games and other software since middle school, and I suspected I was the only one when I was younger. There were very few "nerds" at our school who focused on computers, and even those that did almost never discussed the discovery or exploitation of bugs. Although I stumbled upon a few communities online that shared my interests, I hadn't found one that just clicked. I happened upon HN while learning more about startups, and it was just right for me. I'm grateful I found HN, but I wonder why it took me so long to find the community I was looking for.
I didn't even know about CTF games until tptacek's recent hiring post, and it was another "How could I have been oblivious to this awesome thing for so long?" moment. There are many people who are a great fit for your clients, and they won't know about Starfighter unless you cast a wide or deep enough net. How do you plan to address this challenge?
I actually began working on the exact same thing within the last month or so in my spare time.
Having competed in the CCDC for 6 years in college, I found it pretty insane when I saw that employers don't use real hardware during interviews.
Looks like I picked the wrong side project now this team is working on it. However, interviews are broken enough that there is plenty of room in the space =]. Can't wait to see what you launch.
The next step could be to crowdsource the design of challenges or puzzles. Like a 'masters tier', where you
get special points for designing particularly clever puzzles. This could help remedy the constant demand for new puzzles due to writeup contamination.
BTW, I completed microcorruption, foobar and two stripe CTFs. It was lots of fun and I am really looking forward to Starfighter!
This theme (test your coders, don't interview them) is spreading about, I keep getting spammed by them as a way to "only hire top talent!" its an interesting proposition and I can't wait to see a bit of history behind it. At the very least it will weed out people who think they are programmers when they are not.
I am curious to see how this will work for those like yourself that make CRUD apps. I didn't go to school for programming, but self-taught. I'm afraid it'd be over my head?
The best line: We haven’t done any consulting in a while. We will continue doing no consulting, to the best of our inability.
I think that anyone motivated to research and work on these problems, will likely be a decent hire in most organisations.
Having done other CTFs and puzzles like Matsano / Stripe's, I have found the research part is quite enjoyable. I have little to no knowledge of crypto but researching and solving some of the Matsano challenges was extremely rewarding.
With that said I found the information is pretty thinly spread and it is often hard to find relevant information to the problem domain, particularly with regards to crypto, so I hope they include good starting points for problem research.
tptacek: ever heard of Cicada 3301? It's this mysterious group that has anonymously posted very involved cryptopuzzles. Not clear why, but one theory has been for recruiting people into intelligence agencies.
I love what you're doing but I agree with other commenters that presenting it as a game to programmers but a recruiting channel to companies might not align well. What do you do if a lot of your top players aren't interested in getting a job? There are probably exceptions, but do you think people who are good enough to beat your CTF have that much trouble getting a job?
I'm not sure how this solves the topcoder problem. That problem being is it attracts only certain kinds of software developers and while it is used as a recruiting tool it basically changes nothing when it comes to the interview and hiring process.
Even if you are recruited on topcoder you sometimes still need to submit your resume to some online resume eater "to get you in the system". And then you go on the same time wasting interviews as everyone else.
Personally, I don't play games of any sort but I like to write software and solve puzzles so I find topcoder SRMs more appealing than a programming game, but I'll definitely check it out.
Topcoder challenges were not at all relevant to the kinds of programming problems I needed solved, and were not credible enough for me to consider whether they were predictive.
Don't get me wrong, I would love for the current system to go away, since every interview I had got me one step closer to an imminent hear attack. But, I think the most we will get out of Starfighter is that it will become just another step in the process, MAYBE replacing the phone interview. Nothing is preventing a group of people from sitting in front of the computer, collaborating on a solution, there is absolutely no way to catch that. Also, none of the companies that are most in demand will abandon their dreaded on-site interview any time soon. It works for them, they need to weed out as many people as possible.
Reminds me of a local company who recruits employees exclusively by organizing coding contests: http://contest.catalysts.cc/en/
It works great for them, but I believe you attract a certain kind of developers with these contests; you attract people who enjoy puzzles, and people who enjoy measuring their abilities against others. It could be that those are the traits that make good employees; but I myself would never take part in such a competition.
Then again, I don't actually want to work for anyone else at all.
So, we launched a long running CTF platform a few days back: https://backdoor.sdslabs.co. I guess more in the space is merrier.
My prediction is that Starfighter will become something akin to HackerRank where companies host their own CTFs to recruit people. While it sounds fine, making CTF problems is significantly harder than making algo problems. I'm sure the team is right for this, so it might just work.
I hope they do release some work in the open (such as their sandbox environment).
I wonder if the authors are fans of Stargate Universe. The show's premise involves a key character being identified by success on a puzzle embedded in a video game.
I presume it is based on the film "The Last Starfighter" http://www.imdb.com/title/tt0087597/ - in which a teen perfects an arcade game, and is then selected as a pilot for the actual spacecraft.
From the blog post it sounds like you primarily plan to be used as a "funnel" for employers that engage you, where you are like a super-recruiter bringing candidates that you have qualified yourself.
Do you plan for there to be an option for employers that aren't asking you to refer candidates to still see some view of the performance of a player? Either by paying some smaller fee or by allowing players to have public profiles with some sort of scores?
There is a long-term PVP element to what we're doing, so we want as many players as we can get. We'll play with a number of different ways of matching talent to very smart employers, but our #1 overriding dominating design goal has to be to make players comfortable, so the game works.
patio11: Why name the company Starfighter, while the domain is startfighterS(.io)? Why not name the business Startfighters? (I understand the other domain was taken).
I fear that starfighter will overlook goood candidates simply because they are not good at gaming, which makes the second approach by ortask more interesting and maybe balanced.
Love the concept, can't wait to see it. I don't normally play computer games, but this sounds intriguing. One minor point: "Alt-tab over to your email and click the confirm link in the email we just sent you." If you use web-based email, that's probably Ctrl-tab :)
There's a small typo that caught my eye. In the 'Why is the Starfighter the Right Team' section, 4th paragraph, 2nd sentence "I have a folder in Gmail saving messages from geeks who used by career advice or salary negotiation ..." It should be my instead of by.
- Do you think there will be some threshold where people get past a certain stage and then start getting offers, with nothing being offered below that? (my only experience is basic OverTheWire challenges, so not sure if this will have similar progression of 'levels')
Based on my experience with the crypto challenges and Microcorruption: there will be a lot of players, and most of them will just be there to twiddle the knobs and see how things work.
There will be a subset of participants for whom it's worth their time and ours to have a conversation. Our job will be to identify that subset and start that conversation.
Starfighter is not a security company. We're going to test for a wide range of skill sets. These may include web app security but will not be limited to that.
You can reasonably assume "What skills does the market want to hire for?" is a good proxy for what we'll be assessing for. I love security, but that's 0.01% of the software market. We have... grander ambitions.
Is the second problem domain web perhaps? So people can learn that <script> tags can not be self-closing, and so they won't break the JS includes so bootstrap.min.js doesn't load and their burger menu in mobile-view actually works!
Part of the current interviewing process is Github and side-projects. I truly understand the reasons for Starfighter, but are you putting any real value behind it? I like CTF, but it's not quite the same as contributing to open source projects.
You're quite right that a CTF is not the same as open source projects... and I thought the point of this was exactly that. People who contribute to open source projects prove they have a certain set of skills. What if you need people with a different set? What if instead of being able to learn a large codebase, it is more important that they are able to look at a problem and design an algorithm to solve it in the first place? Rarely in an open source project will you be going in and radically changing the fundamentals.
"You will use real technology. You will build real systems. You will face the real problems faced by the world’s best programmers building the world’s most important pieces of software. You will conquer those problems. You will prove yourself equal to the very best. Becoming a top Starfighter player is a direct path to receiving lucrative job offers from the best tech companies in the world, because you’ll have proven beyond a shadow of a doubt that you can do the work these companies need done."
What an absolute waste of time. You can do all of those things as hobby projects which are yours and actually do something useful in the world besides exist as an achievement to be unlocked.
Show your future employer your github, not your starfighter score.
Isn't it about putting in the time ? How would kalzumeus know if candidate is doing it for the first time vs after 500 hrs of experience. I presume the latter would have a huge advantage
What sort of range of engineers will this be aiming to recruit? Only the very best (and most expensive)? Or a range including more junior / less well-paid positions?
The answer is at the bottom of the announcement: "I have no announcement to make about my involvement with Appointment Reminder at this time. We will, naturally, continue keeping all commitments to our customers."
Yes. I don't know how to describe how until you've actually played it. You know how World of Warcraft and Tetris are both games? Starfighter is a game, too. We're very much not Tetris. Or Topcoder.
There's a chuck of the movie where they're recruiting potential code breakers by putting a difficult puzzle in the newspaper and telling readers if they can solve it in under 5 minutes to apply to the job. Same sort of premise here.
"You will use real technology. You will build real systems. You will face the real problems faced by the world’s best programmers building the world’s most important pieces of software."
I'm wondering, what is your business model? Are the users doing free bug fixes and work for the employers while they are proving their skills?
No sane employer would want free bug fixes. Think of what that would do to their IP assignment issues alone.
Our business model is explained in the post, under the heading How Will Starfighter Make Money. Briefly: we'll find the best engineers in the world, then broker their introduction to companies who want to hire them. Companies will pay us a substantial amount of money for this service.
Don't get suckered into using a bot. They may be cheaper in the short run but you know you're going to get lifebanned. My prices are nothing compared to your future earnings.