A computer program automatically torturing applicants with endless puzzle tests is not a way to find talented qualified people with experience delivering working results that delight the user. It's a good way to find people that have a lot of free time to play games because they are unemployed.
In the years following my first job out of school (decades ago) I can't recall any work that I have gotten by going to these sites, or dealing with monkey tests. Work comes because of my reputation and experience which speaks for itself. At conferences people give me their card and tell me to call them if I am looking to 'move up', which generally means "pay more than the last guy". Any time one contract or job ends, I look through these cards. Most of the time I get several phone calls from people I have met of the sort: "Hey Bugsy, I heard rumors of ABC Corp having layoffs. You looking to get out? We have a position..."
It's bad enough when the interviewer wastes more than 10 minutes of time with puzzles. Having it be automated so it can waste hours and hours without any human feedback is extremely offensive. Whoever designed this system knows nothing about acquiring talent.
The note in the article that in the future the site is going to be augmented with "real world tests" that force the user to design entire sites or otherwise labor for free borders on criminal since they are forcing you to do real work and you're not getting paid for it, in violation of state and federal labor laws.
If you haven't already seen examples of someone's work before you contact them, maybe you shouldn't be hiring them. Or maybe you need recruiters who know what they are doing.
Again, I have no doubt that desperate people who are unemployed because of their incompetence or lack of skill will not have any problem devoting the hours needed to google answers, or to hire third parties to help them complete these tests. I am sure complementary businesses will now open up that sell test answers to desperate applicants for a fee.
You may have misunderstood my comment. It's not about having a brand. It's about having a professional reputation. Professionals who are of the "excellent" caliber that this service purports to represent usually develop a professional reputation naturally without even trying.
This happens even if one is fairly isolated doing "heads down coding" for much of the year. There are times when you meet with clients in meetings, or talk to customers, and talk to competitors at industry conferences, standards meetings and the like.
People who are competent recognize other people who are also competent just by talking to them. People either know what you are talking about and can respond intelligently, or they don't. When people don't know what you are talking about, you downgrade the conversation and talk about gardening, the weather, sports. This is just normal everyday human interaction. It's strange reading some of the replies here, it sounds like some of the people on this discussion board don't know about how that is done. They seem to not have face to face interactions with other people in their field.
In the cases where you connect with someone who is likewise competent, you both remember the encounter. You trade business cards. And if you have an opening at your job for the sort of work the person does, when the position is mentioned, you pull out his card and say "What about this guy? I met him at the XYZ conference. He invented the ABC protocol that is used in GHI devices, just like the thing we are looking to get into."
Connections made don't have to be with management. Typically it is another engineer, a peer relationship. If you know what you are doing, then you get recommended when there is an opening. That is how most jobs at good companies get filled. It's not "nepotism" as someone below claims (apparently he is lacking a dictionary.) I have never worked for a relative. If I do in the future, they will have tried to hired me not because I am related, but because I am the best. And should that happen I would turn them down since I wouldn't want to mix up family relationships with work, that can be a disaster. In any case, if you are good, even the most reclusive and introverted developer/designer/inventor is going to have some people he talks to. And if he is good some of those people are going to talk to others about his work. If that is not happening for someone, the likely reason is they are not all that great. There is no shame in being incompetent nowadays, but the article under discussion is about hiring "excellent" people, not incompetent ones, so it is relevant for the specific topic at hand.
I strongly, strongly agree with everything you're saying and second it wholeheartedly, but I want to add that there is a class of people in our field that can talk brilliantly about what they've worked on, who have resumes with key roles on shipping products on them, and who will absolutely flatline if called upon to code. There is a logic to some level of programming quiz. You really want to verify that the person you're hiring is capable of sitting down, focusing on a problem, and turning out some reasonable facsimile of good code.
That said: this site clearly doesn't look like the right way to do it.
I think we're extraordinarily careful to be respectful of people's time. We've learned through difficult experience to put practical problems in front of candidates, but to avoid Aspie-type alpha-geekery in those problems, and to be appreciative of the time people give us to go through those problems.
Running the puzzle gauntlet doesn't sound respectful to me. There are great programmers who get off on solving puzzles and this process may not weed them out, but there are lots of other great devs who will walk if they see something like this. Caveat emptor.
There's also the opposite.. there are a huge number of programmers who are great at what they do, but asked to be social, come off as awkward and incompetent. Unfortunately it is these talented individuals who often get passed over because they aren't good at "networking", as the grandparent suggests should simply come "naturally." Unfortunately he doesn't seem to understand that something that comes naturally for some--talking to other people--is difficult and hellish for others. And it doesn't mean they wouldn't be a good choice for the job. The world is heavily biased towards people who know how to "network," and maybe this company is trying to expose individuals who don't, but happen to be really good problem solvers.
> he doesn't seem to understand that something that comes naturally for some
I think that is an excuse. Speaking as the person you are claiming doesn't understand, I am highly introverted and awkward. I hate interacting with people in public, it is exhausting. I throw up after having to do public speaking. Yet, I do talk enthusiastically about my work and the field with other people, and I have learned over the years, as introverts do, to be sociable, just as I have learned over the years how to write compilers, design microchips, do web design, etc. (Little of which was covered in depth at all in the university classes I took at a pretty good engineering school.)
Thomas, this is really an excellent page (and a sharp design). Describing up front what the process is definitely respectful of the candidates time!
We've found some success by trying to make the problems really interesting; perhaps some people jettisoned but others came back and said they had a really great time trying to solve them.
Maybe for software you could ask a question like: describe a little piece of software you'd like to build, you could build in an evening and would be useful/entertaining for you or someone else. describe how you'd do it.
+1 to both. Like you've put up on your careers page about candidates doing a web-app challenge, we're going to launch real-world challenges pretty soon!
The vast majority of candidates I've personally rejected did manage to sound like they knew what they were talking about … until I asked for simple pseudocode. Most every coworker who's done interviewing has said the same, as have many commenters here. It's just too easy to talk a good game, so I can't see a basis for recommending anyone I haven't actually worked with.
> They seem to not have face to face interactions with other people in their field.
That has very much been my experience, and the whole meeting-strangers-at-trade-shows thing seems similarly atypical to me. I've never known anyone who does that who wasn't dedicated to sales or marketing or biz dev full-time. I've been looped into maybe a dozen phone conferences and three in-person meetings with customers and partners in the last four years, and there's one guy (from our first API integration) who I could reasonably expect remembers my name without a CRM search. Dunbar's number doesn't leave room for very many of the semi-celebrities you're describing, and they're all going to be extroverts anyway, so I just get recruited by former coworkers (and the obligatory headhunters).
There are people who talk a good game but can't actually code. Or so I've heard. Perhaps you're overconfident about being able to judge other programmers without actually asking them to write some code? How would you know? How do you explain fizzbuzz? [1]
I think OPs complaint is that the service purports to find "great" programmers where they really mean "adequately competent to perform most tasks and able to learn". Another case of marketing speak annoying the nerds.
Programmers in the category "adequately competent" tend to self-identify as "great", so it is at least a consistent use of language.
And is it any wonder? Threads like this are filled with comments saying things like "80% of programmers can't code their way out of a paper bag", so is it any wonder that the folks who can program adequately get big heads about it?
Absolutely! It's totally insulting to be asked to do busy-work just to prove your worthiness before even being interviewed.
A month or so ago I met some people from a company at a networking event. They were looking for people with my kind of skill set and seemed quite enthusiastic. I made it clear that I was only interested in freelancing but he said they were out to "get the best, whatever it takes, so freelancing is fine". So the guy I'm speaking to gives me his card and I email a few days later.
I get an email back, not from him but from HR so not a good start! And the email basically says "build us this trivial demo application just to prove you can program before we interview you" and to make it even more insulting links to tutorials in case I don't know what I'm doing.
I couldn't think of anything nice to say so I never replied :)
Yes, exactly so. And the company's HR department will interpret the lack of response as: "The screening worked, yet another incompetent guy never replied since obviously the only reason someone wouldn't beg and do tricks is because they are dumb! Wow we are really saving a lot of time with this screening!" What they fail to realize is that the "beg and do tricks" method screens out those of us who are competent.
All of the companies I have applied to have asked that I spend several hours solving programming problems before giving me an offer. And why shouldn't they? My experience as an interviewer has been that such great things as "decades of industry experience designing and implementing very complex systems", or a Ph.D, or a 4.0 GPA from a top school are only very loosely correlated with whether or not a candidate can code his way out of a paper bag. As such, it is reasonable to think that there is a profit to be made filtering out the >80% of candidates who cannot code their way out of paper bags.
I've never been particularly desperate or unemployed (though I did not have a job while I was in university), and I think that the few minutes necessary to solve any of these problems is a reasonable use of time if it can signal a potential employer that you are more likely to be competent than a huge majority of their applicants. You are correct to say that nepotism is a better way of getting places, but I don't think it is reasonable to expect everyone to do it.
How is it possible that someone with "decades of industry experience designing and implementing very complex systems" can't code their way out of a paper bag?
Also, if > 80% of the programmers can't code, who the hack has been employing them all these years?
I don't know how people manage to get "decades of industry experience designing and implementing very complex systems" without being able to program. I have only run into a couple such people, but I suspect that is because we get very few applicants with that kind of employment history.
I don't think >80% of programmers can't code. I think that programmers who can't code are much more likely to be applying for jobs than programmers who can code, which makes the pool of applicants look much worse than the pool of all programmers would look.
The 80% who can't code are hired by companies that don't test their skills, and there are lots of companies like that. Then they realize how much they suck, and lay them off whenever they get a chance. Most of these "bad" programmers write crappy (often insane) code that then good programmers have to deal with. Many of them delegate. I've even seen one guy outsource his work.
Most of the programmers looking for a job suck. If you're truly good and have a reputation, you don't need to interview (unless you want to work for a very specific niche where you don't have connections).
If you haven't already seen examples of someone's work before you contact them, maybe you shouldn't be hiring them.
Because obviously all great programmers are expected to have built their online cred before applying for a job, right? I'm sure candidates with background in, say, HFT love sharing their work on their blog, or enjoy hacking on their pet projects after toiling away at their 60+ hour work weeks.
I could not agree more. I think initial product was developed with focus on Indian IT market where filtering right candidates is big pain. Hereafter whatever I am saying applies to India. In India For 10 jobs you might get 1000-2000 applications, and you can not interview them all, scanning all those CVs is just impossible. This kind of system can work when you are dealing with large number of raw talent. But then chances are majority of candidates are just good for solving puzzles as they prepare months and months just solving sample puzzles. Plus so many exploitable loopholes as you suggested. Regarding complementary businesses, that's already there. As matter of fact there are large number of institutes to help anyone to crack each and every aspect of IT job interview.
End results, I often meet people with 5-10 years of experience who can not write a functional block of code without ctrl-c+v which makes me feel sad.
There are some people who do these kinds of problems for fun. As a recruiter, I know plenty of companies who would be interested in interviewing people like that, and this is one way to reach those people.
In the years following my first job out of school (decades ago) I can't recall any work that I have gotten by going to these sites, or dealing with monkey tests. Work comes because of my reputation and experience which speaks for itself. At conferences people give me their card and tell me to call them if I am looking to 'move up', which generally means "pay more than the last guy". Any time one contract or job ends, I look through these cards. Most of the time I get several phone calls from people I have met of the sort: "Hey Bugsy, I heard rumors of ABC Corp having layoffs. You looking to get out? We have a position..."
It's bad enough when the interviewer wastes more than 10 minutes of time with puzzles. Having it be automated so it can waste hours and hours without any human feedback is extremely offensive. Whoever designed this system knows nothing about acquiring talent.
The note in the article that in the future the site is going to be augmented with "real world tests" that force the user to design entire sites or otherwise labor for free borders on criminal since they are forcing you to do real work and you're not getting paid for it, in violation of state and federal labor laws.
If you haven't already seen examples of someone's work before you contact them, maybe you shouldn't be hiring them. Or maybe you need recruiters who know what they are doing.
Again, I have no doubt that desperate people who are unemployed because of their incompetence or lack of skill will not have any problem devoting the hours needed to google answers, or to hire third parties to help them complete these tests. I am sure complementary businesses will now open up that sell test answers to desperate applicants for a fee.