At the core of the problem is that this approach can be used to burn a lot of a candidate's time without an equivalent investment from the company.
I've mentioned this before - I applied (maybe 5 years ago) to a company that first asked me to take a Java test (about 1 hour of work). Then the recruiter called me and sent me a take home project that should take "5-7 hours". I did this, crickets chirped for a month, though I did check in with the recruiter for a while (I gave up eventually). Finally the recruiter called me with a one-line brush off "we've decided not to move forward at this time…"
Granted, this was a bad experience, and it won't always be like this. But I'm pretty close to saying "never" to these exams.
I thought Gayle's suggestions were good. She goes so far as to suggest a 90% passage rate - that you should not be giving these tests to people who are unlikely to pass then, using them to confirm, not screen.
This only applies if the take-home interview is obligatory.
I agree that obligatory take-home interviews are unfair (I feel the same about trial periods). They work (they're probably more accurate than a standard interview), but they take too much time from the applicant (and will thus scare away many of the best people).
As an option, however, I totally disagree. There's a significant percentage of good programmers who are ill served by standard interviews. Those people want this.
You know, while I agree that these problems are mitigated somewhat by making the exam optional, I don't think this alone solves all the problems associated with this approach. I still think that asking candidates to spend, say, 5-7 hours on something is a big deal, even if you give them a choice to take a different path.
I'm not inherently opposed to technical tests, either in person or take home exams. I believe that many exam-based institutions adhere to a code (probably unwritten) that grants certain rights to the examinee. Think about exams in college, where stakes can actually be quite high.
Think about exams at a university. Typically the subject matter and nature of exam questions is available in advance. The grader is highly competent in the field. An associated study path is available for the exam. The exam will be graded and scored, and those scores will be communicated to the student within a set time frame. Feedback will be provided to the student. The exam is part of achieving a lasting credential, such as credit for a course on a transcript.
None of these things exist in technical exams, and in many ways, this increases the stress. Merely making an exam "optional" doesn't erase all these problems. I think this is the core problem with technical exams, they contain all of the stress for the student, but have none of those rights that I believe exist in universities and other exam-based institutions for a good reason.
Yes, that is a very good point. Part of being good at hiring is becoming able to work with different kinds of people with different needs and preferences.
I've had a similar experience. A company I was trying to intern at asked me to write a rails app for them before they would think about giving me an interview. I wrote the app (which all-in took maybe 10 hours) and sent the recruiter a link to the heroku instance the app was running on and the repo on github.
I never got a response. I think three months later I might have gotten a one-liner saying they had gone in "a different direction" or some bullshit like that.
It's insulting and ludicrous for companies to treat prospective employees so disrespectfully.
This x100. I hate these interviews as one of the first exercises. More than once over the years I have spent 5+ hours doing a coding exercise for a company only to get a one sentence response, telling me that my "code fell short". If I pressed them further they might say my code was "hard to read" or "not what they were looking for". Vague, unhelpful, and pitiful responses. 0 interaction. No chance for a rebuttal.
Its a very lopsided assessment. Normally an in interview you can judge the company while the company judges you. That way in an hour interview you both can probably tell its not a good fit. If I'm applying and coding in python and within 15 minutes of a live coding exercise they tell me I shouldn't use list comprehensions because they are hard to read, I'd be out of there faster than I could measure. If I just spent 5 hours coding to get the same response? I'd get a little upset.
When I give critical feedback I will typically go through line by line and tell the candidate what is wrong and what is good. This is done with the candidate, since nobody is 100% correct so my judgement might be wrong and they can correct me. Many, many companies skip this crucial part and as a candidate you don't know if they are going to do it or not, so why waste your time?
Unfortunately, a big part of this problem is that companies face serious liability risks if they give useful feedback.
My experience with tech interviews is that they are actually exams, taken under stressful conditions, with none of the courtesies normally extended to a student.
For instance, in college, or grad school, there is a process for taking an exam. There is typically an affiliated study path, you receive feedback on your performance by a set deadline, and at a good university, someone highly competent grades your exam.
In spite of this, people often describe exams like the bar or their medical boards as the most stressful academic experience they've ever had. As programmers, we have to go up to the whiteboard regularly to take a test, or complete a take-home exam and send it off to who knows who, but we often don't know the subject that will be tested, the competence or credentials of our examiners, whether it will reeve a fair assessment, or even any assessment at all (do they just throw the thing in the trash and say "we decided to go in a new direction"? Truly I have no idea.
That's a huge problem, and it is actually outright harming our industry.
This particular article is about hiring women, but I'm absolutely certain that plenty of men are also deterred from pursuing new tech jobs because they can't stomach the idea of another round of technical testing (with none of the factors I listed above that makes it more fair for the examinee), whether that is white boarding data structure or doing take home projects. I think a lot of people may look at this and decide to just enter a different industry altogether, and I really can't blame them.
I'm on a tangent here, but I really think that tech needs to heal itself, and we're a long way form it.
That is exactly right. Its terrifying. If if you were the first engineer at 2 successful companies and have a strong github it doesn't always seem to factor in.
I would cram for hours before my blackboard interview making sure I knew all sorts of algorithms, just in case. My friend and I would quiz each other. I don't see how memorizing tree transformations relate to a Django application.
I like that they're making the project an optional track, but as a fairly extroverted interviewee I would much prefer talking through code with an interviewer. I feel like the 8 hours of programming I'd do on a take-home project could be assessed with a one-hour conversation walking through code I've already written. (It's especially frustrating when the projects are low-level, like creating a basic CRUD app that uses a lot of libraries, and I've implemented the solution so many times that most of the work on the project is just typing it out. I could just walk the interviewer through one of my many CRUD app side projects.)
Furthermore, collaborating in-person with another engineer tells me more about their engineering culture, and tells them more about whether I'm someone who can collaborate well with others. An hour of pair programming on one engineer's current real-job task would tell us both a lot about each other in a fraction of the time.
I've been giving applicants home assignments for years now, in 3 companies. I do it as a third step, only after a phone screening and at least one personal interview (that might include some very very light whiteboard coding, but will touch a lot of technical stuff), and only if I find the person to be right for the job.
So out of dozens of CVs, a handful will be invited to a personal interview, and maybe one or two people will actually get the home test. I get that it's time consuming so I try not to waste people's time if I'm not serious about them. On a couple of occasions when people said they're time limited, interviewing for a lot of places, etc - I've allowed them a one hour on site task that is of course simpler.
And it's very rare that people are rejected based solely on the code in the home test, it usually has more factors than that. But if you have two good candidates, it might help tip the balance in favor of one of them.
BTW I don't only review the code. I find that the quality of the documentation (not only comments - I ask for a short text describing the design choices, code structure, etc) is usually one of the best signals. Bad grammar and poor language, complicated descriptions of simple things, focusing on unimportant parts - are huge telltales of problems in a person's thinking. Clear, well versed, concise text usually indicates a smart and pragmatic person.
"At the core of the problem is that this approach can be used to burn a lot of a candidate's time without an equivalent investment from the company."
Maybe a tweak would be to only offer take home projects to candidates only after committing to bring them in for an on-site interview, then the interview consists mostly of reviewing the code from the take home.
(Personally, I think I'd still prefer the "coding in person" option.)
After three tries with this process, I've decided that "never" is too soon:
1x Take-home test didn't even get read (this was the nail in the coffin)
1x The interview failed because the salary wasn't competitive (should have asked beforehand about the salary, but didn't have the option; lesson learned)
1x Got an in person interview which led to an offer which was later rescinded (the trip was nice, so I'm not complaining)
Employers in this industry are just not responsible enough to even read through hours worth of work by prospective employees, so while a good idea on paper, the take-home test is a horrible idea in practice. Never again, indeed.
http://www.gayle.com/blog/2013/09/18/companies-who-give-cand...
At the core of the problem is that this approach can be used to burn a lot of a candidate's time without an equivalent investment from the company.
I've mentioned this before - I applied (maybe 5 years ago) to a company that first asked me to take a Java test (about 1 hour of work). Then the recruiter called me and sent me a take home project that should take "5-7 hours". I did this, crickets chirped for a month, though I did check in with the recruiter for a while (I gave up eventually). Finally the recruiter called me with a one-line brush off "we've decided not to move forward at this time…"
Granted, this was a bad experience, and it won't always be like this. But I'm pretty close to saying "never" to these exams.
I thought Gayle's suggestions were good. She goes so far as to suggest a 90% passage rate - that you should not be giving these tests to people who are unlikely to pass then, using them to confirm, not screen.